DSM-G600, DNS-3xx and NSA-220 Hack Forum

SchroE46 · 2008-05-19 09:05:15

Thanks Chumby for the great tutorial, you did a great job. It is stuff like this and especially Fonz's FunPlug that keeps a community strong and allows end users to maximize the potential of their product(s). Companies need development and feedback outside of their circle to improve their products. Look at the iPhone.

I have been wanting to install the FunPlug for a period of time, but I did not set forth any time. Your tutorial allowed me to make time and it was really simple and it did not take much time at all. I went from a defective DNS-323 to a great NAS device. I originally set the DNS-323 as RAID1 & JBOD with remainder of space.

All was cool and I updated the firmware as it was released. But then with firmware 1.03, I started noticing access denied to certain files and incorrect file system sizes -I knew that something wasn't right. I upgraded to firmware 1.04 and a few weeks later, I get an E-Mail from my box stating that the left drive has failed. Sure enough, the status light in front of the left drive was pink (both drives are year or less old).

So I verified the files I had copied from several weeks earlier and formated both drives independently. I checked & upgraded the firmware to the latest & greatest (I hope) 1.05. The right drive now failed (no I did not switch them). I reformatted both drives again after upgrading the firmware.

I installed funplug 0.5 and followed your directions along with emartin24's helpful comment to use ffp instead of fun_plug.d. I now have everything up and working like a charm. I really like dns323_problems suggestion for the daily backup implementation.

I think your solution is way better than the RAID1 that dlink provides. And for all the confused people out there - RAID# is supposed to be fault tolerant and give you a sense of comfort that your data remains on multiple drives. But I do not trust dlink's shady RAID1 with my data. Plus I think there is a loss of performance using the RAID1 configuration vs Chumby's method. I will let someone else that has too much time to benchmark and provide the results.

Now I just need to figure out how to switch from telnet to ssh and make use of all the other goodies.

hastings69 · 2008-05-19 10:43:43

Thanks from me too.
You know the above thread by SchroE46 is echoed by me here.

I have had raid 1 since I purchased the unit in december. Thinking that it was the "safest" way to store data, but I know better now.
My DNS failed last week, and was repaired by DLINK.
BUT the raid software within the unit did not want to play ball. Thankfully I had backup of all data elsewhere as the RAID was broken and after mulitple issues I had to reformat the drives completely to get them to function properly again.

Having read this thread 3 weeks ago, I actually was after a backup but could not see how I could go from Raid 1 to this....maybe the failure was indeed a blessing.

Within about 30 mins I set up funplug 0.4 (funplug 0.5 appears to throw errors when unpacked so 0.4 appeared to do what I wanted) , and had the backup as described nicely in place, and whaddaknow, last night it copied it all over to volume 2

THANKS!! At least I feel that now my data is actually able to be restored if the unit fails!

I guess it is
1. Limitation of software raid array
2. Limitation of user - I am a newb I admit
3. I just want my data in place if either the DNS OR either disc fails, of which the RAID could not satisfy.....

Now also to working our SSH from telnet, (struggling), and enjoying at least a little better data safety. I am also toying with the time machine concept or backupclone.......ah too much to do.

halfsoul · 2008-05-20 06:28:47

Chumby,
I agree with previous two posters: directions are clear, concise, accurate & easy to follow. Have you thought about adding your instructions to the Wiki Howto page?

bq041 · 2008-05-20 14:47:44

Hastings-

If you had the unit funplugged, then you could have easily removed the array without having to format. All you would have had to do is delete the superblock on each drive using mdadm when in a telnet session. The drive would then become a stand-alone, no format required.

As for the limitation you refer to, I believe it is with the D-Link software. I too have had the unit come up as degraded (I have swapped drides between both my units) and have not had too much problem assembling the array manually. I try never to set RAID levels or format using the web utility. It seems to be the source of most of the problems. I do it all from a telnet session, and have no problems since then. The only dificult part of creating a RAID array manually is that the telnet session must run from someplace other than the hard drive, as both hard drives must be unmounted. Removing an array does not. The easiest was is just to pop out 1 disk and work on 1 at a time.

hastings69 · 2008-05-21 01:11:04

Hi bq,

Yes I was and am fun plugged, but hadn't the foggiest how I would have done that by myself. Certainly sounds easier than what I have was fumbling through. But I haven't lost anything due to being backed up onto another external.
DLINK suggested via email that I would be able to just plug the drives in after replacement and it would just work, but after it kept requesting several different messages of formating the drives I decided to call it quits and just start again and use this tutorial.

I really enjoy this unit especially given the minimal cost of setup, but I also think the RAID system is potentially a false sense of security, not to reignite the whole back up versus raid 1 argument, but because as a newb, if something goes amiss, I cant get myself out of trouble with RAID.

I would have tried harder and probaly asked if it was my only copy, it just cost me hours of time copying back and forward.

I just suck at the command line at the moment, and without this forum would be even further in a black hole. Getting the fun plug and backup for me is very little about understanding the finer details and much more about following a recipe in "another language" (cutting and pasting).

You would have saved me a Sunday, damn. Next time......
Cheers

Chumby · 2008-05-21 08:38:45

Howdy SchroE46, hastings69 and halfsoul... appreciate the comments! Like I said though this is only a summary of what I have learnt from others here

I have update the tutorial to reflect the "ffp" string change in 0.5. If anyone is using fp 0.5 can you let me know if the changes made in the tutorial still make sense?

Cheers,
Chumby

rcblackwell · 2008-05-21 15:02:29

bq041 wrote:
If you had the unit funplugged, then you could have easily removed the array without having to format. All you would have had to do is delete the superblock on each drive using mdadm when in a telnet session. The drive would then become a stand-alone, no format required.

I've found this thread and Chumby's post very interesting. So much so that I'd like to break the RAID1 arrary on my DNS323 then set up the two drives as described in the first post. Problem is I have no idea how to use mdadm to safely achieve the task. Would someone here provide the detail necessary to break the array while preserving data on Volume_1

With appreciation

Bob Blackwell
Pickering. ON

bq041 · 2008-05-22 15:41:24

I will post the instructions when I get home from work. I do not have my DNS right in front of me right now and do not want to give bad info. Do you by any chance have an external USB drive that takes SATA? This is not required, but makes thing a whole lot easier.

rcblackwell · 2008-05-22 16:48:13

bq041 wrote:
Do you by any chance have an external USB drive that takes SATA?

Not at the present however I can probably get my hands on one.

bq041 · 2008-05-23 02:37:07

Okay, This will be the first of a couple posts. This one is just going to take you through breaking the array and setting up for 2 standard drives. Later I will post how to create an array manually without having any usb devices.

1) you must have telnet access, and it must have been on the DNS long enough to have fully synced both drives
2) shutdown the DNS
3) pop out the left drive and start the box up
4) after you have established telnet access, push in the drive (yes, with the power on)
5) give the unit a minute and enter:

Code:

# mdadm --zero-superblock /dev/sdb2

6) shutdown the DNS
7) pop out the right drive and start up the box
8) repeat steps 4-5 exactally
9) reboot the DNS

You now will have 2 hard drives in standard mode. If you need to format them to get the unicode function (when upgrading the F/W from 1.03 or below), you can telnet in and format the second drive after unmounting it. Then re-mount it and copy your data back on, shutdown the unit, swap drives and repeat. I do not trust the web utility to format the correct drive, I do it manually.

Last edited by bq041 (2008-05-23 02:38:02)

rcblackwell · 2008-05-24 02:55:04

bq041 wrote:
Okay, This will be the first of a couple posts. This one is just going to take you through breaking the array and setting up for 2 standard drives.

I'll give the instruction a whirl over the weekend then advise. Much appreciated.

Bob Blackwell
Pickering, ON

hastings69 · 2008-05-24 03:28:35

Hey Chumby,
Does this propagate deletions? ie delete folder vol a, next night delete folder vol B?
Regards
H69

knireis · 2008-05-24 06:04:23

hastings69 wrote:
Hey Chumby,
Does this propagate deletions? ie delete folder vol a, next night delete folder vol B?
Regards
H69

yes, it will

shadow349 · 2008-05-25 03:14:32

ken wrote:
Hang on... Raid 1 is unreliable, so copy drive 1 to drive 2 every so often as a backup.

Tell me again why I bought a DNS-323 instead of a couple of cheap USB drive boxes?

Is there anyone else who thinks this thread is a little strange?

Well, RAID-1 protects you from disk failure.
rsync between drives protects you from a user failure (eg, "rm * .c" - notice the space between the asterisk and the '.c').

I do both with my DNS323; I am using RAID-1 and rsync to an external USB drive three times a day.

hastings69 · 2008-05-25 05:37:52

Hi Everyone,
Sorry to post about this again but I have found my backup didnt run last night. Here is the crontab output. I am FFP 0.5. I did rsync data to vol B 3 nights ago (success), then had to reinstall FFP (stuffed up SSH permissions and decided to start again from scratch) and the backup hasn't run since. Hence why I was hoping that this might just resync all the files on A and B (the delete question above). I have noted that there is no log folder or file in FFP, was I meant to create this? I can only see a FFPlog within the root.
THe unit has the correct time. Do I have to have volume B completely empty again as I deleted manually some files off it, do I reformat? Once it backs up will it recognise the folders again it had previously done before the changes or will it just create another fresh copy on the vol B? Hope this makes sense.

root@DNS323:~# crontab -l
32 2 * * * /usr/sbin/rtc -s
30 2 2 * * /usr/sbin/rtc -c
59 1 * * * /usr/sbin/daylight &
5 2 * * * /mnt/HD_a2/ffp/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >/mnt/HD_a2/ffp/log/rsync.last.log 2>&1

Cheers & thx
H69

rcblackwell · 2008-05-25 15:16:15

hastings69 wrote:
I have noted that there is no log folder or file in FFP, was I meant to create this?

I had the same issue until a log folder was created. Now all is working very well.

Bob Blackwell
Pickering. ON

rcblackwell · 2008-05-25 15:20:13

bq041 wrote:
You now will have 2 hard drives in standard mode.

SUCCESS! I now have two separate drives.

bq041 wrote:
If you need to format them to get the unicode function (when upgrading the F/W from 1.03 or below), you can telnet in and format the second drive after unmounting it. Then re-mount it and copy your data back on, shutdown the unit, swap drives and repeat. I do not trust the web utility to format the correct drive, I do it manually.

Thanks for the tip regarding unicode. I won't need to reformat the drives as I did that after installing FW 1.04.

With appreciation

Bob Blackwell
Pickering, ON

shadow349 · 2008-05-25 15:24:29

What happens when you log in as root and run this command from the shell:

/mnt/HD_a2/ffp/bin/rsync -av --delete /mnt/HD_a2 /mnt/HD_b2 -n

Note the '-n' at the end ... this will force rsync to only do a 'dry-run'; it will figure out what needs to be copied, but not actually do the copy. Also note that the 'a' flag is shorthand for the '-rlptgoD' flags.

Are any error messages generated?

If not, remove the '-n' and try again to let it actually do the copy.

Also, you might want to put two greater than signs before your log file in your crontab:

/mnt/HD_a2/ffp/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >> /mnt/HD_a2/ffp/log/rsync.last.log 2>&1

This way, the log file will be appended to on each invocation instead of replaced (so you can see how previous nights jobs finished). For example, assume that the jobs failed three nights ago. There might have been an important error message in the log telling you why. On the next night, the log file was overwritten, so you don't know why the failure happened.

Finally, consider adding "time" or "date" before your command:

( date ; /mnt/HD_a2/ffp/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 ; date) >>/mnt/HD_a2/ffp/log/rsync.last.log 2>&1

or

time /mnt/HD_a2/ffp/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >>/mnt/HD_a2/ffp/log/rsync.last.log 2>&1

In the first case, a timestamp will be put in the log file just before the rsync starts and just after it ends - makes it easier to determine dates in the log file.

The second case will not put the clock-time in the file, but will generate a report at the end of the rsync stating how long it took the process to run (both wall-clock time and cpu time). This won't tell you the date it was run, will at least tell you how long it took to run; which might be useful in tracking down an issue.

Last edited by shadow349 (2008-05-25 15:27:25)

rcblackwell · 2008-05-25 15:25:34

Chumby wrote:
COPY ALL DATA FROM VOLUME_1 (DATA DISK - RHS) TO VOLUME_2 (BACKUP DISK - LHS) ONCE A DAY

- Credit: mig, dkl, fonz & others who have contributed through this thread
- Last Update: 21 May 2008

Chumby et all,

Your efforts in this area are greatly appreciated. My thank you's go out to all who supported me during my transition from RAID to two separate drives using rsync.

Bob Blackwell
Pickering, ON

Chumby · 2008-05-26 00:27:02

rcblackwell...

Glad to see you have it up and going! Happy backups! Thanks for helping him out guys.

Cheers,
Chumby

bq041 · 2008-05-26 05:54:37

rcblackwell wrote:
SUCCESS! I now have two separate drives.

Good, I'm glad you got it. It wasn't too difficult, was it?

I know I said I would post how to manually build an array, but 2 things happened. One, I was out of town all weekend and had no time to work on it. Two, I am still sorting out the issue of the magic numbers with F/W 1.05 (this is actualy a topic in another post). The array works great and 350GB of data only took 120 minutes to sync. The only thing is that the web utility keeps saying the drives are in backwards and to swap them. If you bypass that screen, the status page recognizes a health working RAID 1 array. I message think this has to do with the extra partition that had been added for config files in the newer F/W. I was not using this partition, as I originally did RAID 1 under F/W 1.03. Anyway, I'm back home now and should get a chance in the next couple of days to rework it.

hastings69 · 2008-05-26 12:52:38

Thanks Shadow,
Did the dry run, no errors, and then did the real thing, no errors. Fantastic, I guess it just now needs to run automatically.

/bin/echo "5 2 * * * time /mnt/HD_a2/ffp/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >>/mnt/HD_a2/ffp/log/rsync.last.log 2>&1 " >> $CRONTXT

Is that correct? Should it run now with this cut and paste command. Another noob Q - I can't find a log file within FFP, is it because I am looking in the wrong place, or because it hasn't been created yet?

And this is my crontab output - it doesn't reflect the time stamp,

32 2 * * * /usr/sbin/rtc -s
30 2 2 * * /usr/sbin/rtc -c
59 1 * * * /usr/sbin/daylight &
5 2 * * * /mnt/HD_a2/ffp/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >/mnt/HD_a2/ffp/log/rsync.last.log 2>&1

I'll check if it adds a file tonight at 2:05am..... Thanks again Shadow, you're a champ.
regards
H69

rcblackwell · 2008-05-27 03:03:55

shadow349 wrote:
... Also, you might want to put two greater than signs before your log file in your crontab:

/mnt/HD_a2/ffp/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >> /mnt/HD_a2/ffp/log/rsync.last.log 2>&1

This way, the log file will be appended to on each invocation instead of replaced (so you can see how previous nights jobs finished).

An excellent idea! Is there anyway to limit the size of the log file?

Bob Blackwell
Pickering, ON

Chumby · 2008-05-27 03:17:53

Thanks shadow349. Agree - good idea to have the log file appending after every backup.

I have updated the tutorial to reflect this!

Cheers
Chumby.

bq041 · 2008-05-27 06:47:28

Write an if/then subroutine in the script that checks the file size and if it exceeds a certain amount to replace it instead of append it.

DSM-G600, DNS-3xx and NSA-220 Hack Forum

Announcement

#101 2008-05-19 09:05:15

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#102 2008-05-19 10:43:43

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#103 2008-05-20 06:28:47

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#104 2008-05-20 14:47:44

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#105 2008-05-21 01:11:04

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#106 2008-05-21 08:38:45

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#107 2008-05-21 15:02:29

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

bq041 wrote:

#108 2008-05-22 15:41:24

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#109 2008-05-22 16:48:13

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

bq041 wrote:

#110 2008-05-23 02:37:07

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Code:

#111 2008-05-24 02:55:04

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

bq041 wrote:

#112 2008-05-24 03:28:35

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#113 2008-05-24 06:04:23

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

hastings69 wrote:

#114 2008-05-25 03:14:32

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

ken wrote:

#115 2008-05-25 05:37:52

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#116 2008-05-25 15:16:15

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

hastings69 wrote:

#117 2008-05-25 15:20:13

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

bq041 wrote:

bq041 wrote:

#118 2008-05-25 15:24:29

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#119 2008-05-25 15:25:34

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Chumby wrote:

#120 2008-05-26 00:27:02

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#121 2008-05-26 05:54:37

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

rcblackwell wrote:

#122 2008-05-26 12:52:38

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#123 2008-05-27 03:03:55

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

shadow349 wrote:

#124 2008-05-27 03:17:53

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#125 2008-05-27 06:47:28

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Board footer