DSM-G600, DNS-3xx and NSA-220 Hack Forum

Chumby · 2008-01-16 11:05:22

Good work, DaveN and Mig...

All updated now! I also tried to close the confusion for knireis...

Hope the tutorial is going well for many users here... any other improvements I can make - let me know and I will update!

Cheers,
Chumby.

knireis · 2008-01-25 14:35:18

Well, got it to work also and I'm quite happy with it. Any idea what the transferspeed is from 1 disc to the other in MB/sec ?

Chumby · 2008-01-30 13:52:34

Thats good to see knireis.... glad things are working OK.

Not sure about the transfer speeds.... probably all depends on rsync and the types of drives speeds that you have installed. I set my copy to go at night so that the speed isn't really a worry anyway....its all done by the morning...

Cheers
Chumby

jduncan · 2008-01-31 11:50:25

Hi Guys,

This tutorial is very helpful. However, I require an even safer backup option. I want to keep my primary DNS-323 in RAID1 mode and have purchased a second DNS-323 for backup purposes. The second DNS-323 is also set to RAID1 mode and I want to schedule a nightly rsync backup from the primary DNS-323 to the secondary DNS-323.

Please could you provide step by step instructions on how to achieve this setup? If possible I would like the second DNS-323 to use a user account that is password protected as I don't want any users to be able to access the backup unit.

Many thanks

James

PS: Once I have this setup I would like to perform another backup from the primary DNS-323 to an Internet backup service so data is also stored offsite.

blbrown · 2008-01-31 16:41:08

jduncan wrote:
Hi Guys,

This tutorial is very helpful. However, I require an even safer backup option. I want to keep my primary DNS-323 in RAID1 mode and have purchased a second DNS-323 for backup purposes. The second DNS-323 is also set to RAID1 mode and I want to schedule a nightly rsync backup from the primary DNS-323 to the secondary DNS-323.

Please could you provide step by step instructions on how to achieve this setup? If possible I would like the second DNS-323 to use a user account that is password protected as I don't want any users to be able to access the backup unit.

Many thanks

James

PS: Once I have this setup I would like to perform another backup from the primary DNS-323 to an Internet backup service so data is also stored offsite.

Try BackupNetClone (http://backupnetclone.sourceforge.net/), which could automate the syncing between the two DNS-323 units (even if they're not in the same network). Unfortunately, I'm _still_ working on writing up the instructions into something that's easy to follow. Writing step-by-step instructions is very time-consuming, and you'll find that no one is going to spell it out exactly for you, since it would take a lot of their time in order to solve your unique situation. On the other hand, I'm guessing there are several people in this forum who would consult for you for a fee.

As for password-protecting, that should be easy enough with the standard DNS-323 webpage. If you really wanted to lock it down, you could enable only SSH access to the 2nd DNS-323, then you could disable samba, telnet, ftp, and all other methods of accessing the unit (using fun_plug add-ons). I think this would keep it secure even if the unit were reset back to factory defaults. What it won't prevent, however, is someone simply removing the HDDs and putting them into a standard Linux PC to access the data (assuming the RAID format doesn't cause issues).

Dexter Tones · 2008-02-08 03:30:14

I followed the tutorial and everything went smoothly. However, I was looking at the rsync FAQ and it says that one should "expect rsync to consume about 100 bytes per file" to "hold the list of files being transferred." I don't have many files on my DNS right now, but given the amount of RAM it has, I fear that getting an "out of memory" error may become an issue in the future (when I have many more files stored on the DNS). Is there a workaround for this at all?

Last edited by Dexter Tones (2008-02-08 03:33:17)

blbrown · 2008-02-08 16:04:11

Dexter Tones wrote:
I followed the tutorial and everything went smoothly. However, I was looking at the rsync FAQ and it says that one should "expect rsync to consume about 100 bytes per file" to "hold the list of files being transferred." I don't have many files on my DNS right now, but given the amount of RAM it has, I fear that getting an "out of memory" error may become an issue in the future (when I have many more files stored on the DNS). Is there a workaround for this at all?

Yes, I have noticed this issue as well. There are two work-arounds:

1. Do some scripting magic to walk your directory structure and run rsync on each one to make sure the file list doesn't get too long.

2. Wait until rsync (http://samba.anu.edu.au/rsync/) version 3.0 is complete and someone (fonz?) compiles it for the DNS-323. Version 3 addresses this specific issue and it's currently in pre-release which gives me hope it will be complete some time soon.

That said, I have BNC running on a directory structure with ~26000 files (on a Windows PC backup client). I've had no problems so far with the DNS-323 and BNC nicely tells me this file count in the email status (on the line that says "Backup location total file/directory count: xxx (xxxGB)"). On my DNS-323 that backs itself up (it's both the backup client and the backup server) I'm up to ~9000 files without problem so far.

At 100 bytes per file, you should have no problem using rsync with 320,000+ files, which should only consume half the physical memory of the DNS-323. And I'm guessing swap space would be used automatically if you used up the physical memory, though this would be stressing the unit I think.

DaveN · 2008-02-09 04:07:07

With regards to the transfer speed from 1 disk to the other:

I transferred approximately 250 GB (1800 files) from one drive to the other (both are WD5000AAKS 500GB 7200RPM SATAII 16MB Buffer) via an rsync cron job yesterday. The second drive was clean (freshly formatted). The transfer took just under 14 hours to complete - a transfer rate of 5Mb/s. I'm using the 1.04 firmware, fonz' 0.3 funplug (only telnet and the crontab update enabled) and the following command in cron:
/mnt/HD_a2/fun_plug.d/bin/rsync -av --log-file=/mnt/HD_a2/fun_plug.d/log/rsync.log --delete /mnt/HD_a2 /mnt/HD_b2 >/mnt/HD_a2/fun_plug.d/log/rsync.cron.log 2>&1

The exact details as report by rsync are as follows:
sent 257141580847 bytes received 33928 bytes 5205664.67 bytes/sec
total size is 257109830583 speedup is 1.00

The good thing about rsync is that tonight when it syncs (and transfers 0 bytes) it'll only take a few seconds to compare all the timestamps instead of taking another half day.

[Edit - the next day] Syncing the same set of files on the second night took 17 seconds! Only the log files had changes and were copied, the comparison of the other 1800 files took almost no time at all.
sent 204955 bytes received 64 bytes 12425.39 bytes/sec
total size is 257109993663 speedup is 1254078.86

Cheers,
Dave

Last edited by DaveN (2008-02-09 12:19:43)

knireis · 2008-02-12 21:25:27

Is it possible to add a second line with a new job after the first one in this example?:

# add the Rsync job to execute at 2:05 am - also create a logfile
/bin/echo "5 2 * * * /mnt/HD_a2/fun_plug.d/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >/mnt/HD_a2/fun_plug.d/log/rsync.last.log 2>&1" >> $CRONTXT

I don't want to sync everything om disc one but only some separate folders.

Or is it possible to explude a folder?

kring · 2008-02-13 19:17:47

Does this work with Firmware version 1.04?

saturn2001 · 2008-02-14 02:33:30

I have just purchase my dns-323 and follow the procedures at the start of this thread but i did install the new 1.04 firmware and have tried to get it to work without any success. The rsync log file came back with a error that the file or directory is not found. I will put revert back to firm 1.03 today and see if that changes anything.

kring · 2008-02-14 04:23:37

Mine has 1.04 as well, I went ahead and gave it a try and will check the log tomorrow after it runs tonight.

Chumby · 2008-02-14 05:25:43

Hi Guys,

I updated my firmware to 1.04 so time ago and everythign is working fine. Didn't miss a beat.

Cheers,
Chumby

saturn2001 · 2008-02-14 09:56:51

I upgraded my 323 right out of the box to 1.04 several days ago and following the steps here. Today I have downgraded back to v1.03 and it has now worked without any problems, not sure why. I know i didn't type anything in wrong because i did it about 4 times starting from formating the drives each time using fun_plug 0.3.
Chumby what steps did you use when you upgraded to 1.04 (the file i got was dated 1-10-08 off dlink's usa site) Did you start from scratch and reload everything formating the drives or just upgraded firmware.
Thanks

Chumby · 2008-02-14 10:43:45

Hi Saturn,

I have had this working 100% with the 1.03 firmware no worries. I simply downloaded the latest firmware (at the time - 1.04 dated 01/10/2008) and the backup just kept chugging along.

Does this shed any light?

Cheers,
Chumby

kring · 2008-02-15 01:03:10

Just to confirm, I installed 1.04 out of the box with no hard drives installed. then popped in the HDD's and ran through the install.. everything appears to be working, I still have another hour or two to finish syncing but I can tell that from yesterday the 2nd drive was empty and now I see the files there and it has a few hundred gig's already copied.

This is a great feature, I feel much safer this way as opposed to mirror.

thanks, this is fantastic, very easy to do too.

Chumby · 2008-02-28 13:34:50

Thats great to hear, kring!

kymlp · 2008-03-05 13:26:55

I'm thinking of doing the Vol 1 to Vol 2 copy as a backup ... I wonder why not use the "schedule download" feature of the 1.03fw? using Local Disk as the source? Is there a problem with this approach?

rap · 2008-03-06 20:35:10

I see that rsync version 3.00 has been released.

Can someone more knowledgeable than me say if this will work?

RAP

Kilokahn · 2008-03-07 08:22:21

Wouldn't it be just as easy to have a 3rd duplicate hard drive (I bought 4 unique drives) and just swap out when necessary? Lets say I want to weekly backup, I just take out the drive in bay 2 then swap it with another drive? Couldn't I just do that weekly to avoid data loss?

I don't know, I have never had a problem with data loss until recently (single hard drive failure) and purchased this as a backup solution.

Thanks!

Chumby · 2008-03-12 22:58:14

Yea...I swap my drive over every 3-4 weeks. Just eject the one out of the LHS bay and nsert the new one. Works a treat.

This means I have copies of my data (more or less going).

Cheers
Martin

halfsoul · 2008-03-13 22:27:16

True, but that solution has three limiations compared to the process decribed in this thread.
1. You get once every 2-3 weeks backups instead of daily (or however you physically swap drives)
2. It's a manaul backup operation instead of automated
3. Much more consumption/wear&tear (Vol B is running all the time)

my 2cents

timofej · 2008-03-25 00:24:16

Great tutorial Chumby! Many thanks! Even I who doesn't know anything about Unix and scripts have succeeded to make it work. :-D

I've got one question though. Is there an easy way to change the time of the backup or do I have to go all the steps over again? Can I just edit editcron.sh in my text editor and then repeat STEP THREE, pos. 4) from the tutorial?

Biscotte · 2008-03-25 14:15:10

Now that dns-323 firmware 1.04 is officially out and not been updated for a month or so (!), I am ready to go for this backup arrangement. Like you Chumby I will be rotating out hard discs to an offsite location.
I have read the tutorial and the whole thread. Very good and thanks in advance.
I still have a question.

Back in early January 2008 you said requirements:
Software
======
- Firmware 1.03
- Fonz's fun_plug 0.3 (link below)

I know the DNS 1.04 firmware is OK for update, but is Fonz's 0.3 still required?
Fonz now has kindly produced a version 0.4. Do I stick with Fonz Funplug 0.3 or updated FF 0.4?

This install looks like it will keep my gums going for a few days.
But one thing at a time. First the 1.04 firmware - pull the discs, throw the bones, dispel evil spirits, deep breath, stand on left leg, everything crossed - you know the routine.
And is rsync 3.0 worth installing then how would I do it and . . . and . . .and

Thanks in advance

Biscotte

unmesh · 2008-03-26 01:50:12

Chumby wrote:
Yea...I swap my drive over every 3-4 weeks. Just eject the one out of the LHS bay and nsert the new one. Works a treat.

This means I have copies of my data (more or less going).

Cheers
Martin

I'm new to the world of NAS devices but recently purchased a DNS-323 and have it running with just one drive. Unfortunately, at least as per this thread, the drive is in the left bay, not the right one.

Any ideas as to how to implement the fundamental premise of this thread with the bays swapped will be greatly appreciated.

Thanks.

DSM-G600, DNS-3xx and NSA-220 Hack Forum

Announcement

#51 2008-01-16 11:05:22

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#52 2008-01-25 14:35:18

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#53 2008-01-30 13:52:34

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#54 2008-01-31 11:50:25

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#55 2008-01-31 16:41:08

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

jduncan wrote:

#56 2008-02-08 03:30:14

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#57 2008-02-08 16:04:11

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Dexter Tones wrote:

#58 2008-02-09 04:07:07

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#59 2008-02-12 21:25:27

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#60 2008-02-13 19:17:47

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#61 2008-02-14 02:33:30

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#62 2008-02-14 04:23:37

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#63 2008-02-14 05:25:43

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#64 2008-02-14 09:56:51

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#65 2008-02-14 10:43:45

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#66 2008-02-15 01:03:10

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#67 2008-02-28 13:34:50

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#68 2008-03-05 13:26:55

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#69 2008-03-06 20:35:10

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#70 2008-03-07 08:22:21

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#71 2008-03-12 22:58:14

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#72 2008-03-13 22:27:16

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#73 2008-03-25 00:24:16

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#74 2008-03-25 14:15:10

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

#75 2008-03-26 01:50:12

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Chumby wrote:

Board footer