DSM-G600, DNS-3xx and NSA-220 Hack Forum

Unfortunately no one can be told what fun_plug is - you have to see it for yourself.

You are not logged in.

Announcement

#51 2008-07-16 02:47:38

bscott
Member
Registered: 2007-07-13
Posts: 48

Re: OMFG, Lost all my data! D-link is terrible!

bq041 wrote:

There is a very reliable way to do it.  Pull out your drive with the data on it, and start the unit only with the new one.  When you format is, you cannot possibly format the drive with data on it.

Of course that protects your primary drive in the short term, but getting the two drives to live together peacefully after that is problematic.  First off, making a RAID mirror is definitely out (in firmware 1.05 anyway), as we've established.  And secondly, I think it was you who explained that the 323 keeps track of the serial numbers of the last couple drives that have been inserted?  I'm trying to set up 3 drives, one primary and two swappable backup volumes, and not yet having much luck.  The second drive, formatted by itself, becomes Volume_1 until it is bumped to Volume_2 by re-insertion of the primary, and <something> happens (more often than not) to trigger the 323 to want to format again, or simply not recognize it as valid.

My DNS-323 ran a stable RAID1 mirror for a year until I began trying to upgrade.  I'd originally installed two 120gig drives which had been running mirrored inside my PC for 3 years before that - moving one into the 323, copying data over Ethernet, then moving the other and watching it pair them just fine, and that was before I even knew about customizing the innards.  But now - ffp and hacks aside, it just doesn't seem to cotton to drive swapping most of the time.  Maybe it's me?  (I could believe that, I've been working in I.T. and programming for almost 20 years now and have seen weirder...)  I don't think it's the drives, all 4 are new-ish ST3320620AS which pass all diagnostics when connected to my PC.

Offline

 

#52 2008-07-16 06:12:45

bq041
Member
From: USA
Registered: 2008-03-19
Posts: 709

Re: OMFG, Lost all my data! D-link is terrible!

bscott wrote:

Of course that protects your primary drive in the short term, but getting the two drives to live together peacefully after that is problematic.  First off, making a RAID mirror is definitely out (in firmware 1.05 anyway), as we've established.  And secondly, I think it was you who explained that the 323 keeps track of the serial numbers of the last couple drives that have been inserted?  I'm trying to set up 3 drives, one primary and two swappable backup volumes, and not yet having much luck.  The second drive, formatted by itself, becomes Volume_1 until it is bumped to Volume_2 by re-insertion of the primary, and <something> happens (more often than not) to trigger the 323 to want to format again, or simply not recognize it as valid.

My DNS-323 ran a stable RAID1 mirror for a year until I began trying to upgrade.  I'd originally installed two 120gig drives which had been running mirrored inside my PC for 3 years before that - moving one into the 323, copying data over Ethernet, then moving the other and watching it pair them just fine, and that was before I even knew about customizing the innards.  But now - ffp and hacks aside, it just doesn't seem to cotton to drive swapping most of the time.  Maybe it's me?  (I could believe that, I've been working in I.T. and programming for almost 20 years now and have seen weirder...)  I don't think it's the drives, all 4 are new-ish ST3320620AS which pass all diagnostics when connected to my PC.

First of all, your statement about getting 2 drive to live together after an individual format is completely untrue.  I have tested this many times.  If you are really concerned about it, and you have ffp, then just telnet in and set it up yourself.  My comment was in response to the comment that there is no reliable way to format a second drive without risking formatting your existing data drive.

As for RAID1, it is relatively easy to manually set-up a RAID1 after the fact.  I have created and removed RAID1 arrays on these units many, many times without a single bit of lost data.  If you check out this thread ( http://dns323.kood.org/forum/t2444-Wiza … -1.05.html ), you will see there is a script that upgrades RAID1 arrays from 1.03 to 1.04 or 1.05, without losing data.  It removes the array, re-partitions and formats the drive, copys the data, does the same for the second drive, then syncs the whole thing back up in RAID1.  A similar technique can easily be applied to all of what you are talking about. 

As for swapping drives in and out so you are using 3 drives, the system was never designed for this, so don't expect it to automatically work like that.  Yes, it was me who mentioned that it saves the serial numbers.  It also only does the comparison between the stored numbers and the drives when you log into the web admin interface.  As for the operation of the unit, it is unaffected by this.  If it bothers you, a very simple script can be made to automatically update the serial numbers when the unit is booted up.  Anyway, why are you trying to have swapable backup drives?  This kind of defeats the intent of backup.  You are by effect making the backup a primary data source and making the primary data source into the backup.  Why not just backup to the backup drive?  With the USB funplug, you can easily attach the backup to the USB to make the backup.  SATA to USB cases where I live can be gotten for under $20.  If you are trying to do the backup using RAID1, then you are not using RAID1 for intended purpose and you WILL have major problems, upto and including eventual loss of your data.


DNS-323     F/W: 1.04b84  H/W: A1  ffp: 0.5  Drives: 2X 400 GB Seagate SATA-300
DNS-323     F/W: 1.05b28  H/W: B1  ffp: 0.5  Drives: 2X 1 TB  WD SATA-300
DSM-G600   F/W: 1.02       H/W: B                Drive:  500 GB WD ATA

Offline

 

#53 2008-07-16 07:04:15

bscott
Member
Registered: 2007-07-13
Posts: 48

Re: OMFG, Lost all my data! D-link is terrible!

bq041 wrote:

First of all, your statement about getting 2 drive to live together after an individual format is completely untrue.  I have tested this many times.

So have I, a great many times over the past 6-7 weeks, believe me; I've gotten little else done at home!  In a way I'm glad to hear this isn't a general problem, as I continue to toy with the idea of buying a second DNS-323 despite my bad experiences...

bq041 wrote:

As for RAID1, it is relatively easy to manually set-up a RAID1 after the fact.  I have created and removed RAID1 arrays on these units many, many times without a single bit of lost data.  If you check out this thread ( http://dns323.kood.org/forum/t2444-Wiza … -1.05.html ), you will see there is a script that upgrades RAID1 arrays from 1.03 to 1.04 or 1.05, without losing data.

Well, that was my initial plan - until in this post I was advised not to use mdadm in this way...

bq041 wrote:

Anyway, why are you trying to have swapable backup drives?  This kind of defeats the intent of backup.  You are by effect making the backup a primary data source and making the primary data source into the backup.  Why not just backup to the backup drive?

I think you misunderstand me.  I've abandoned RAID long ago and merely want to have an offsite backup option.  My 323 exports Volume_1 to the local network, and a script periodically copies it (and NFS-mounted shares from my desktops) to Volume_2, which is otherwise not accessible to the outside world.  I wish to be able to swap Volume_2 drives, maybe every month or so.  I realize the 323 is not advertised for this particular use, but the problems I've experienced could (in my opinion) happen to someone using the device normally, say upgrading their storage or if they just make a mistake.

If I have to use a USB drive (I know they're cheap, I have half a dozen) for this sort of thing then there's probably no use for the DNS-323 at all, given that its RAID is now a nonissue for me.  A NAS should at least reduce cable clutter, power consumption and the complexity of your setup, IMHO.  And one of my goals in all this is to limit how complicated it gets - I don't want to have to re-learn everything from square one if it needs to be rebuilt in 2-3 years' time and I've forgotten half the details...

Anyway, thanks for your input!  I'll keep trying.  Or I will as soon as it finishes copying my wife's OSX root partition, which is a process that seems to slow asymptotically after the first few hours (turns out there's over half a million files - oops! should have checked that first... heh)

Offline

 

#54 2008-07-16 18:19:12

bq041
Member
From: USA
Registered: 2008-03-19
Posts: 709

Re: OMFG, Lost all my data! D-link is terrible!

Look at the dates.  The post you link to that where I advised not to do it unless you know something about it is a month before the post I sent you to.  And it is still true.  You also have to know how to properly modify the configuration files in addition to using mdadm in order for the unit to like it.

As far as your complexity issue, it is far less complex to use a USB drive attached to the DNS when you want to make a backup.  Since the DNS was not originally designed to use USB drives, there is no provision in it to do any type of integrity checks or config files for them, hence the unit always likes them.  These config file checks are the issues you are having with swapping drives in and out; the unit expects that when it sees a new drive, it will format it.  Would it not just be easier to plug in a usb drive once a month and backup the Volume_2 drive?  It can even be connected updated and disconnected with the power on.

If you are set on swapping the drives out on your left slot, then here is what needs to be done each and every time you swap them (before logging into the web admin).  You need to mount the flash on both locations of /dev/mtdblock0 and /dev/mtdblock1.  (Mount them using minix file system in the directory /sys/mtd1 and /sys/mtd2.)  You need to locate the physical serial number of the drive you are inserting (it will be on the lable of the drive, or you can run write_hd_info and read it from /tmp/scsi.log).  Then you will need to replace the serial number of the drive you removed with the serial number of the drive you are inserting in the hd_magic_num file on both locations of the flash.  You will also need to make sure that the hd_magic_num file stored in the 4th partition of the drive you inserting matches exactly the file from the drive you removed (/mnt/HD_b4/.systemfile/hd_magic_num).  Once this is done, you need to unmount the flash and run do_reboot.  This is what will make your system happy.

As soon as you log in to the web admin tool, several scripts and binaries run that make comparisons of the following 3 files in 4 loacations, as well as things such as hardware serial numbers extracted from the drives.  If all the information does not add up correctly, the admin tool will flag it and prompt for an action, which may be format, or swap drives.

Files: raidtab, raidtab2web, and hd_magic_num
Locations in flash: /dev/mtdblock0, /dev/mtdblock1
Locations in F/W 1.04: /mnt/HD_a4/.systemfile, /mnt/HD_b4/.systemfile
Locations in F/W 1.03: /mnt/HD_a2/.systemfile, /mnt/HD_b4/.systemfile


DNS-323     F/W: 1.04b84  H/W: A1  ffp: 0.5  Drives: 2X 400 GB Seagate SATA-300
DNS-323     F/W: 1.05b28  H/W: B1  ffp: 0.5  Drives: 2X 1 TB  WD SATA-300
DSM-G600   F/W: 1.02       H/W: B                Drive:  500 GB WD ATA

Offline

 

#55 2008-07-16 20:50:55

fordem
Member
Registered: 2007-01-26
Posts: 1938

Re: OMFG, Lost all my data! D-link is terrible!

A second reason to consider the use of USB drives is the lifespan of the SATA connector - they're not meant to be used for constant drive swapping.

Offline

 

#56 2008-07-17 06:10:16

bscott
Member
Registered: 2007-07-13
Posts: 48

Re: OMFG, Lost all my data! D-link is terrible!

bq041 wrote:

Look at the dates.  The post you link to that where I advised not to do it unless you know something about it is a month before the post I sent you to.  And it is still true.

I did notice that, I was merely pointing out that what you currently suggest is in line with what I'd originally intended to attempt (and thus, my plan-B approach is perhaps understandable in the circumstances)

I should have been keeping up with this forum throughout my travails, but instead I was avoiding it lest I drown these message boards with my complaints and questions...

bq041 wrote:

Would it not just be easier to plug in a usb drive once a month and backup the Volume_2 drive?  It can even be connected updated and disconnected with the power on.

Certainly using a USB drive is possible/practical; whether it's 'easier' is maybe a subject on which reasonable people can differ.  But I could back up to a USB drive without the DNS-323 too - and in fact I have done so for years.

Again: I know what I'm trying to do is not supported, and may not even make sense to everybody.  But in the process of attempting it I've encountered problems which I think could affect casual users too, problems that are preventable and which are worth considering as distinct issues.

bq041 wrote:

If you are set on swapping the drives out on your left slot, then here is what needs to be done each and every time you swap them (before logging into the web admin).

Thanks indeed for that detailed advice!  I feel it does somewhat validate my original claim, however ineptly worded it might have been, that adding a drive to the 323 can be problematic.  I'm glad to see there is a way to do it safely, but I stand by my point that it's boneheaded to write something which is even capable of - for instance - syncing a blank drive over a full one in the context of rebuilding a mirror, or formatting both drives while claiming in so many words to be formatting only one...

It seems as if your procedure for swapping drives could be automated with a script file, although I'm not sure I want to rely on something that intricate given my goal of a set-it-and-forget-it solution.  It's beyond the limits of what I'd expect myself to remember over time.  And yes of course I could document it, but I do that all day at work lately and it's not the sort of thing I feel like bringing home if I can avoid it, I hope to find something straightforward enough that it just makes sense to me, now and next year.

fordem wrote:

A second reason to consider the use of USB drives is the lifespan of the SATA connector - they're not meant to be used for constant drive swapping.

What is, these days?  Thanks for pointing that out, though I'm sure I've done more swapping in the past 7 weeks than I would ever have done in years of regular monthly swap-outs.   Too many USB enclosures (and their power bricks, where applicable) are just as failure-prone in my experience - if not moreso - as the SATA connector of a well-constructed unit like the 323 may be.  In fact I can't think of a storage medium I haven't had some kind of major problem with. 

You've both been very helpful, and I hope I can contribute positively to this forum in the future.

Offline

 

#57 2008-07-17 23:24:45

bq041
Member
From: USA
Registered: 2008-03-19
Posts: 709

Re: OMFG, Lost all my data! D-link is terrible!

bscott wrote:

It seems as if your procedure for swapping drives could be automated with a script file

It has.  Look in the script in post 15 of this thread: http://dns323.kood.org/forum/t2444-Wiza … -1.05.html

Granted this script is not in the context of what you are doing, but the subrouties do all of these.  The subroutines could easily be copied to another script and a fun_plug written that checks the files each boot up to see if they match the current drive, and if not, to make the corrections.  It would not be difficult at all.

It could also be written to hot swap a raid1 disk and automatically sync it (provided they were originally set up correctly), but this would go against the very fabric of what raid is all about.  (Using raid1 as a backup, I mean.)


DNS-323     F/W: 1.04b84  H/W: A1  ffp: 0.5  Drives: 2X 400 GB Seagate SATA-300
DNS-323     F/W: 1.05b28  H/W: B1  ffp: 0.5  Drives: 2X 1 TB  WD SATA-300
DSM-G600   F/W: 1.02       H/W: B                Drive:  500 GB WD ATA

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2010 PunBB