DSM-G600, DNS-3xx and NSA-220 Hack Forum

That will not necessarily work.  It is making several assumptions that may or may not be correct.  Just blindly adding a drive back without removing its superblock data can cause it to sync the wrong way (I have seen this happen).  It does appear from the report that the drive has been indeed removed, so you need to first find out why.  Shutting down incorrectly will not cause this.  Normally a failed drive will be shown as still in the array, but faulty.  This shows removed -- not typical.  Anyway run mount and check to make sure the syatem did not mount that drive by itself.  If it did, it will not add back until after you unmount it.  Once you know that it is not mounted, then run:

mdadm --zero-superblock /dev/sda2

This will remove the array information from that drive.  Since you had a problem with it, I would recommend reformatting it first, but it is up to you.  To do that:

mke2fs -m 0 /dev/sda2

Then you can run the command to add it.  Pleas not the correct syntax

mdadm /dev/md0 -a /dev/sda2

and immediately run

do_reboot

This will reboot it and it will sync following that.  You can also choose to allow sync to complese before rebooting, but you may get an amber light which will go away after the reboot.  Also, if you end up having to reboot during the sync, it will begin from scratch after the reboot.

All this also assumes you have not moved your drives around, which can cause this type of issue.

Last edited by bq041 (2008-07-16 06:29:25)

1) The reason I prefer the later syntax is because you can combine command very easily, such as: mdadm /dev/md0 -f /dev/sda2 -r /dev/sda2  As to the point of the syntax being correct, the manual I learned from (albeit is old) only covers the latter syntax, not the former.
2) The reason is that the DNS does not like it when you add the first scsi device to the array, since that is by default its primary disk.  Two things can potentially happen, you will get the amber light on one or both of the drives, and depending on how the unit originally booted up, the web admin will not like the setup and not allow you to do anything.  By rebooting immediately, the DNS will boot up in the proper sequence (as far as the stored config files are concerned) and begin the sync process whereby the DNS is happy during the whole thing.
3) N/A
4) This is most likely related only to the DNS.  It not only uses the superblocks of each drive, but also uses files called raidtab and raidtab2web to let the system know how to configure the raid, and which disk is the primary source of data.  If there is no superblock on the disk being added, it will automatically be added as a spare, but I have had happen 1 time while testing on the DNS, that I left the superblock intact before I added it where synced the wrong way.  It may have been a fluke, but why take the chance?  The only explaination I can give is that the information stored in both superblocks were consistant with each other, having come from the same array originally.  The DNS sets sda2 as the primary source of data automatically, and when it was added, the array "thought" it was the data source and synced the other one to it.  It could be due to the version or something broken in the version of mdadm in the DNS, I do not know.  As I said, it has only happened to me personally once.

4b) The other reason I zero out the superblock before anything else is, if you get a corrupted config file (raidtab or raidtab2web), the device can sporadically create the array or not create the array when it is booted up.  I have had multiple sucessive boots where the array formed on one and not on the other.  The bad part is since the files typically set the first scsi device to primary, if, for example, the case above is rebooted, there is the possibility of the array reforming and syncing to the first drive, which is his offline drive.  That's okay, as long as he has not added or changed data.  Albeit, in this situatioon, barring any reboots before attempting to add the drive back, it will probably work with no problem.  I just don't like to take the chance.

...and I must add: The only thing i did was to shut the box down from idle state with the power button, nothing else. I must admit, it was slow to react. It would not power off at first, so I had to keep pressing the front button until it eventually shutted down... damn "thunder and lightning"...

-D-


I add som follow up info:

/ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1]
md0 : active raid1 sda2[0] sdb2[1]
      486544512 blocks [2/2] [UU]

unused devices: <none>

and

/ # dmesg|grep sd
SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB)
SCSI device sda: drive cache: write back
SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB)
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda4
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB)
SCSI device sdb: drive cache: write back
SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB)
SCSI device sdb: drive cache: write back
sdb: sdb1 sdb2 sdb4
Attached scsi disk sdb at scsi1, channel 0, id 0, lun 0
Adding 530104k swap on /dev/sda1.  Priority:-1 extents:1
Adding 530104k swap on /dev/sdb1.  Priority:-2 extents:1
ext3: No journal on filesystem on sda4
ext3: No journal on filesystem on sdb4
ext3: No journal on filesystem on sda2
ext3: No journal on filesystem on sdb2
md: bind<sda2>
md: bind<sdb2>
md: kicking non-fresh sda2 from array!
md: unbind<sda2>
md: export_rdev(sda2)
ext3: No journal on filesystem on sda4
ext3: No journal on filesystem on sdb4
md: bind<sda2>
disk 0, wo:1, o:1, dev:sda2
disk 1, wo:0, o:1, dev:sdb2
disk 0, wo:0, o:1, dev:sda2
disk 1, wo:0, o:1, dev:sdb2

It still say: md: kicking non-fresh sda2 from array!, but then again: md: bind<sda2>... hmmm..

-D-

Last edited by deurges (2008-07-17 00:42:46)

From what I am reading about mdadm today, the non fresh sda might be refering to an out of date superblock.  If you look at the mdadm manual under the assemble section (used to assemble an already created array, such as at boot up), it briefly mentions that you have to force the array to start minus a disk if one of the superblocks is out of date.  Here is the excerpt.

For assemble:

-u, --uuid=
uuid of array to assemble. Devices which don't have this uuid are excluded

-m, --super-minor=
Minor number of device that array was created for. Devices which don't have this minor number are excluded. If you create an array as /dev/md1, then all superblocks will contain the minor number 1, even if the array is later assembled as /dev/md2.
Giving the literal word "dev" for --super-minor will cause mdadm to use the minor number of the md device that is being assembled. e.g. when assembling /dev/md0, will look for super blocks with a minor number of 0.


-f, --force
Assemble the array even if some superblocks appear out-of-date

-R, --run
Attempt to start the array even if fewer drives were given than are needed for a full array. Normally if not all drives are found and --scan is not used, then the array will be assembled but not started. With --run an attempt will be made to start it anyway.

-a, --auto{=no,yes,md,mdp,part}
See this option under Create and Build options.

-U, --update=
Update the superblock on each device while assembling the array. The argument given to this flag can be one of sparc2.2, summaries, or super-minor.
The sparc2.2 option will adjust the superblock of an array what was created on a Sparc machine running a patched 2.2 Linux kernel. This kernel got the alignment of part of the superblock wrong. You can use the --examine --sparc2.2 option to mdadm to see what effect this would have.

The super-minor option will update the prefered minor field on each superblock to match the minor number of the array being assembled. This is not needed on 2.6 and later kernels as they make this adjustment automatically.

The summaries option will correct the summaries in the superblock. That is the counts of total, working, active, failed, and spare devices.

This may be what is going on, and if it is, you would have wanted to zero the superblock before adding it back to the array.

bq041 wrote:

From what I am reading about mdadm today, the non fresh sda might be refering to an out of date superblock.  If you look at the mdadm manual under the assemble section (used to assemble an already created array, such as at boot up), it briefly mentions that you have to force the array to start minus a disk if one of the superblocks is out of date.  .....
This may be what is going on, and if it is, you would have wanted to zero the superblock before adding it back to the array.

Deurges is not assembling (mdadm --assemble) an array but hot adding a disk (mdadm --add) and this is a very different scenario.

In my experience with over a dozen of mdadm RAID arrays hot adding always works unless the added partition is smaller than the md device itself.

And once a disk is hot added all superblocks are in sync.

Last edited by skydreamer (2008-07-17 11:03:09)

bq041 wrote:

I was refering to the DNS starting up, not his hot add. .... I found they do not always react the way I would expect them to on a full blown linux machine.  .....

No worries, I think that poor deurges is probably getting confused with all our academic arguments.

I will hijack this threat a little if you do not mind since you might be able to shed some light on one peculiarity of the D-link's mdadm. When I create a new mdadm array on a linux machine and insert it into D-link it may take up to 10 minutes for the DNS-323 to boot up the very first time. I suppose it goes through its RAID1 motions but would you have an idea what exactly is it doing? The RAID1 is then recognized, as being in sync, all data is untouched and subsequent boots take the usual amount of time.

Skydreamer,
     This is because there are 3 files located in 4 locations that must be set-up DNS specific.  If you are using f/w 1.04 or 1.05, then you also need 2 more partitions.  This is so the DNS can determine if it needs to format or not, and also if the drive are corrctly inserted.

Files: hd_magic_num
         raidtab
         raidtab2web

Locations: /dev/mtdblock0/ (flash)  -- Mount this to /sys/mtd1 with minix
                /dev/mtdblock1/ (flash)  -- Mount this to /sys/mtd2 with minix
                /dev/sda4/.systemfile/ (f/w 1.04 and up)
                /dev/sdb4/.systemfile/ (f/w 1.04 and up)
                /dev/sda2/.systemfile/ (f/w 1.03)
                /dev/sdb2/.systemfile/ (f/w 1.03)

The raidtab and raidtab2web files must be respectively identical in all locations and updated accordingly for the particular setup.  (The 2 files are different, but all locations are the same.)  These files set the environment by telling the DNS how the drives are configured and the particulars about the array.  The modes are linear, raid1, raid0, and normal.  raidtab also sets up the raid masterdisk.  If you are interested I have written a few scripts for setting up and breaking raid1 arrays on the DNS.  They are on the forum.  I will eventually make them up with full functionality and no rebooting, but that will take a lot of time. 

I also made one to convert a f/w 1.03 raid1 array to 1/04.  It is more advanced that the make or break it ones because I added some safeguards to it.  In that particular script file, I have subroutines that create the 3 files for you automatically.  You may want to look at that part.  The link is here: http://dns323.kood.org/forum/t2444-Wiza … -1.05.html post #15.

The MOST important thing to remember is to UNMOUNT THE FLASH! when you are finished.  Corrupting this filesystem is a very bad thing.

hd_magic_num is special.  The instances on the flash are identical and follow the this format for a raid or multidrive setup:

<hard drive 0 magic number>
<hard drive 1 magic number>
<hard drive 0 serial number>
<hard drive 1 serial number>

For each of the drive locations, hd_magic_num follows this format:

<hard drive 0 magic number>
<hard drive 1 magic number>

I do not have the algorithm for the magic numbers, but I found if I generate random numbers when creating the array, it works with no problem.  If I remember they are between 1 and 5 digits.  Just make sure to update all 4 locations at the same time and the numbers match the respective drives.

Ask if you have any more questions.

Last edited by bq041 (2008-07-17 23:15:45)

bq041 wrote:

Skydreamer,
     ....
Ask if you have any more questions.

Cool, I actually like the DNS-323 little less after reading about the peculiar workings of its RAID subsystem.

Yep, I would have a million $ question: Why does the formatting stop at 94% when creating RAID1 on some large disks and the partition table does not utilize the entire disk?  I suppose that it is a bit remote to the original topic but this is how it all started....

bq041 wrote:

....

As for not all of the drive being utilized, you need to elaborate a little more. ....

Yep, I am quite familiar with linux handling of disks and fdisk and can confirm your findings. I have successfully grown partitions, physical and logical volumes, RAID arrays and ext3 FS from the command line although not in DNS-323. You can also make a partition bigger without loosing any data (i.e. increasing the value of the last cylinder) providing that either you have some free space after it or do not care about the adjacent partition.

As for the partitioning gap on the new disks see my previous threat, at the beginning of it there is a little write up on what happened after inserting brand new Seagate disks.

http://dns323.kood.org/forum/t2577-DNS- … linux.html

Thanks for your time on this bq041, you have doen some really thorough investigation on this topic, fair play to you!