DSM-G600, DNS-3xx and NSA-220 Hack Forum

Unfortunately no one can be told what fun_plug is - you have to see it for yourself.

You are not logged in.

Announcement

#1 2007-10-26 07:56:05

Chumby
Member
From: Tasmania, Australia
Registered: 2007-08-01
Posts: 96
Website

Tutorial: Backup Everything from Vol A to Vol B once a night

COPY ALL DATA FROM VOLUME_1 (DATA DISK - RHS) TO VOLUME_2 (BACKUP DISK - LHS) ONCE A DAY.  BACKUP ANY ACCEDENTIALLY DELETED FILES.

- Credit: mig, dkl, halfsoul, fonz & others who continue to help through this thread
- Last Update: 24 May 2009

Why?
====
Automatically copying all data from one drive to the other can be used as part of a backup strategy.  Some reports indicate that RAID1 is not altogether reliable, so this may be a better alternative. I have all my working data on Volume_1 and rotate backup disks in Volume_2, every month.  I keep one disk away off site.  All your data from Volume_1 will be copied to Volume_2 (in the folder rsync_save) during the first night and then on the second night, only the new data is copied across - your first copy will take some time, but from then on things will be fast!  Any files deleted by accident will be stored in a back directory on Volume_2 (in the rsync_save_backup directory) so that you can still retrieve them if you need.

I was tired of trying software backups through my home network.  This is by far the easiest and most efficient backup method I could find.

Software
======
- Firmware 1.03 or 1.04
- fonz's fun_plug 0.5 (earlier versions will work with modifications but this tutorial is for 0.5)

Hardware
======
- 1 * dns323
- 2 * Hard Drives (a third can be used if you want to rotate backup disks in Volume_2 in order to keep one off site)

Procedure
=======

Skip steps ONE and TWO if you have already installed and formatted the disks and fonz's fun_plug.

STEP ONE – Install Drives and set the Correct Time
-----------------------------------------------------------

1) Install two disks into your dns323 (RHS for data and LHS for backup) and make sure they are formatted as standard drives, Volume_1 and Volume_2, via the dns323 admin page (mine is http://10.1.1.5).  Do not set them up in any type of RAID configuration.  Insert disk 1 (RHS drive bay) first and then disk 2 (LHS drive bay second). 

If you have a third disk for off site backup purposes, you may also wish to format it now. Take out the RHS disk and insert the new backup disk. Format it in the same way.  It is best to format a disk on its own in the dns323, without another disk being in the other bay which contains data.

2) In the dns323 admin, make sure you set the correct time (menu --> tools --> time).  My time is set from the time of my local computer.

STEP TWO – Install fun_plug and the Telnet Client, puTTY
----------------------------------------------------------------------

1) Install a fun_plug version as per instructions given by fonz.

2) Download a telnet client, puTTY (link below), install it on your PC and telnet into the dns323. You should something similar to:

          / #

You have successfully connected to the dns323.

STEP THREE – Create a Scheduled Backup
-------------------------------------------------

There are three programs already installed which you will use:

   1) vi - text editor in the BusyBox set of standard Linux single executable utilities (link below)
   2) rsync – the backup program (links below)
   3) cron – the scheduler (link below)

The first thing you need to do is create a file called 'editcron.sh' which is used to add your daily backup task to the scheduler, cron.

This file is created and saved in the 'start' directory inside the ffp directory.  This ensures the file will run on start up of the dns323.

1) After you have telneted into the dns323 and can see the # prompt, type in the following line of code.

Code:

vi /mnt/HD_a2/ffp/start/editcron.sh

Once that is done, the new file 'editcron.sh' has been created, vi has started and you can edit the new file. 

2) Press 'a' which takes you to vi edit mode.  Type in the following lines of code:


Code:

#!/bin/sh

CRONTXT=/mnt/HD_a2/crontab.txt

# start with existing crontab
/bin/crontab -l > $CRONTXT

# 1. add the Rsync job to execute at 2:05 am 2. Create a logfile (append to any existing log file) 3. Create a backup directory on Vol_B for deleted files
# note: use a single > after /mnt/HD_b2 if you want to a create a new log file each night with no append

/bin/echo "5 2 * * * /mnt/HD_a2/ffp/bin/rsync -av --delete  --backup --backup-dir=../backup_rsync_save/`date +%Y-%m-%d` /mnt/HD_a2/* /mnt/HD_b2/rsync_save >> /mnt/HD_a2/ffp/log/rsync.last.log 2>&1" >> $CRONTXT

# install the new crontab
/bin/crontab $CRONTXT

# clean up
/bin/rm $CRONTXT

This code adds a rsync task to the scheduler.

In this example, everything from Volume_1 (/mnt/HD_a2) will be copied over to Volume_2 (/mnt/HD_b2) at 2.05am every night.  The '2' and the '5' come from the beginning of the rsync task above.  You can use any time that suits (24hr format) eg. 6.30 pm would be '30 18 * * *'

The following diagram shows you what time options are available for rsync.

# +---------------- minute (0 - 59)
# |  +------------- hour (0 - 23)
# |  |  +---------- day of month (1 - 31)
# |  |  |  +------- month (1 - 12)
# |  |  |  |  +---- day of week (0 - 6) (Sunday=0 or 7)
# |  |  |  |  |
  *  *  *  *  *  command to be executed

3) Once you have typed in the above code, press 'esc' to exit vi edit mode and then ':wq' to save and exit vi completely.  You should be taken back to the # prompt and the file 'editcron.sh' will be created and saved in the 'start' directory inside the ffp directory.

4) For the fun_plug script to execute the 'editcron.sh' file, the 'editcron.sh' file must be executable.  Type in the following command at the # prompt:

Code:

chmod a+x /mnt/HD_a2/ffp/start/editcron.sh

A log file, rsync.last.log,  will also be created in the 'log' directory inside the main ffp directory.  It will tell you which files were copied across successfully.  If there is no log directory already created  (ie: mnt/HD_a2/ffp/log/) you will need to create one yourself for the script to work properly.  Use the following code to create the log directory

Code:

cd /mnt/HD_a2/ffp
mkdir log

STEP FOUR – Verify the Backup Task has been Added
-------------------------------------------------------------------

Once the above is complete, double check that the rsync task has been added to the scheduler.

1) Reboot the dns323.

2) Telnet in and from the # prompt, type crontab –l (lowercase L).  If everything has worked as it should you will see your rsync line as a scheduled task:

eg. in my example above:  5 2 * * *  etc...

There should be three other scheduled tasks also listed, which has already been set up by the dns323.  You don't need to worry about these.

When your scheduled time arrives, both HD lights on the dns323 start flashing away. Everything is working fine!  Enjoy hassle free backups from now on!  You can check you backup log for details of what was copied.  Regularly keep an eye on the backup_rsync_save folder on Volume_B so that it doesn't grow too large.

MAKING AN OFF SITE BACKUP
====================

If you have a new backup disk it will need formatting first. 

Eject both your backup and data disks out of Volume_1 and Volume_2 and insert the new backup disk into the RHS drive bay.  Login to the dns323 admin page, and a format drive option should appear.  Click OK to format the drive.  WARNING: Do not put the new drive in the LHS bay and select format (with your data drive still in the RHS drive bay), as your data drive may be formatted. (see troubleshooting below)

Once formatted, turn off the dns323, take out the newly formatted backup drive and place it in the LHS drive bay.  Put your drive with all the data on it back in the RHS bay.

Turn on the dns323, the main light will blink and hopefully the drives will be recognised as Volume_1 (RHS - data) and Volume_2 (LHS - backup).  If you have trouble seeing the drives, go into the dns323 admin, perform a system reset, and load up your default settings that you have previously saved.

Your new data drive, Volume_2 (LHS) is now ready and on the next scheduled rsync, the data will be copied across.

Take your original backup drive away for safe keeping off site ;)


RESTORING DATA or UPGRADING DRIVES
===========================

If your data drive (RHS) drive fails and you loose your data OR you wish to add in a bigger drive follow the following procedure:

Take out all drives, and place your new drive in the RHS drive bay.  Format the new drive. Install funplug as instructions above.

Place you backup drive back in the LHS drive bay, telnet into your DNS and type in the following commands.

Code:

cd /mnt/HD_b2/rsync_save
cp -a *.* /mnt/HD_a2

The -a preserves links, attributes, and recurses directories. If you do -af, then it will do all of -a plus forcing overwrite (only do this if you're sure)

If it's working correctly, it will sit there for quite a while until the operation finishes.  You will hear the drives copying away.

The fun_plug should also be restored onto thwe new drive.

Reboot your DNS 323.


TROUBLESHOOTING
=============

1) Losing Drive Mappings
-----------------------------

You may lose your drive mappings after setting up the backup schedule.  To rectify this problem, go into the dns323 admin page (presuming you can find it using the dns323 IP address - eg. http://10.1.1.5) and save your settings as a config file.  Next select 'restore to factory settings' (nothing will be wiped). On reboot, re-enter the dns323 admin interface and load your previously saved config file.  You should now by able to map your drives (Volume_1 & Volume_2) again using your Windows PC.

This works fine however you may need to set the correct time again on the dns323, so the scheduled task knows when to start.

2) Incorrect Time
--------------------

There are issues with time adjustments on the dns323.  This is another issue altogether.  If you want to ensure your time on the dns323 is as correct as possible, visit the reduce clock drift Wiki item (link below). You may also search the forums for Fonz’s timzone.sh script for help. For the purpose of a nightly backup, the exact time is  not critical.

3) Formatting Drives
------------------------

Always format a new backup drive in the RHS bay with your data disk out of the dns323.  If there are two drives in the dns323 and you attempt to format the drive in the LHS bay, the drive in the RHS bay may be formatted instead. 

For more information on this issue, read the thread:
http://dns323.kood.org/forum/viewtopic. … 85&p=2

Also when formatting drives, it may stall on 94%.  Read the following thread for possible solutions:

http://dns323.kood.org/forum/t767.html
http://dns323.kood.org/forum/t517-forma … -stop.html

LINKS
====

fun_plug

fun_plug 0.5
--------------
http://www.inreto.de/dns323/fun-plug/0.5/

Telnet Client (puTTY)
------------------------
http://www.chiark.greenend.org.uk/~sgta … nload.html (Download)

vi
--
http://en.wikipedia.org/wiki/Vi

an alternative tp vi is joe.

rsync
-------
http://samba.anu.edu.au/rsync/
http://en.wikipedia.org/wiki/Rsync

cron
------
http://en.wikipedia.org/wiki/Cron

Clock Drift
------------
http://dns323.kood.org/howto:reduce_clock_drift

Last edited by Chumby (2009-05-27 01:20:42)

Offline

 

#2 2007-10-26 10:57:18

fonz
Member / Developer
From: Berlin
Registered: 2007-02-06
Posts: 1716
Website

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Thanks for writing this nice summary, but I think there's sth wrong with the following lines. Shouldn't you be echo'ing the line into $CRONTXT file?

Chumby wrote:

Code:

...
# add the Rsync job to execute at 2:05 am – also create a log file
5 2 * * * /mnt/HD_a2/fun_plug.d/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >/mnt/HD_a2/fun_plug.d/log/rsync.last.log 2>&1
...

Offline

 

#3 2007-10-26 12:03:18

Chumby
Member
From: Tasmania, Australia
Registered: 2007-08-01
Posts: 96
Website

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Thanks for picking that up fonz.

I have made the edit you suggest in the tutorial above and all should be OK now.  Let me know if you still see any problems and I will re-edit again.  If others have this working fine, or have other suggestions, let me know.

Happy backups!

Cheers
Chumby

Last edited by Chumby (2007-10-26 15:48:28)

Offline

 

#4 2007-10-26 15:54:19

hennemtk
Member
From: Germany
Registered: 2007-07-16
Posts: 23

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Thanks very much, Chumby, for this tutorial!

I'll give it a try when I come back from my vacation (and after having installed a second drive ;-) ).

I think it will be quite useful to me.

Regards

Thomas

Offline

 

#5 2007-10-26 22:39:48

dan.crouthamel
Member
Registered: 2007-08-23
Posts: 43

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Nice job! cool

Offline

 

#6 2007-10-28 01:11:20

Chumby
Member
From: Tasmania, Australia
Registered: 2007-08-01
Posts: 96
Website

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Thanks for that hennemtk and dan.crouthamel - hope you find it useful!

Chumby.

Offline

 

#7 2007-10-28 14:44:43

Emacs
Member
Registered: 2007-06-24
Posts: 110

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Very cool tutorial!

Thanks, Chumby!

Cheers,

Emacs

Offline

 

#8 2007-10-28 23:42:48

fickle
Member
From: Melbourne, Australia
Registered: 2007-09-10
Posts: 249

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

can we make a sticky of this for future reference and to ensure it does not get deleted? this topic is one we see quite often and thus suggest we keep  it

Offline

 

#9 2007-11-03 00:24:10

Chumby
Member
From: Tasmania, Australia
Registered: 2007-08-01
Posts: 96
Website

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

That sounds like a good idea fickle.... not sure if it can be made a wiki entry?

Chumby

Offline

 

#10 2007-11-07 00:07:50

WildSioux
Member
Registered: 2007-03-05
Posts: 71

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Nice tutorial Chumby...I am tagging this thread for future reference.  I want to switch both my parents and my DNS-323 unitsfrom Raid 1 to a two drive setup with this script for backups.  I was mistaken when I set it up under the Raid 1 hoping for a good backup solution.  That was not the case when I lost a lot of files when I accidentally did a RM before realizing what I did and stopped it saving most of the files.  I need to do this soon.

Offline

 

#11 2007-11-07 03:56:52

fickle
Member
From: Melbourne, Australia
Registered: 2007-09-10
Posts: 249

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

is there a script / command that can copy one folder within the 323 to another folder in the 323? either full copy of the folder & its contents and or a partial based on last modified/changed/added files?

i only ask this as i have a 323 running under raid 1 thus no 2nd disk

cheers

Offline

 

#12 2007-11-07 13:06:23

dkl
Member
From: Toulouse
Registered: 2007-06-16
Posts: 104

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Of course, you can copy/backup from one folder to another one on the same volume, using rsync: you just have to modify the "editcron.sh" and change the src and dst directories of the rsync command to fit your needs.  But I don't understand the advantage of doing so if you already have a RAID1 configuration: you will then have four (4) copies of your data (2 on each disk). Is it really what you want? Or maybe I misunderstand you? The only advantage I see, in your case, is that you are protected against an "unwanted" file deletion/modification/corruption. But only if you detect this immediately, because your "nightly" backup will delete/modify/corrupt also the copy... (except if you use the "backup" function of rsync, which is nice too).

Considering the fact that you already have RAID1 (so all your data are duplicated), if you have files that are very important for you, and you want to be sure not to delete or modify them, it is simpler to protect them by putting them in "read-only" mode. This will also do the job. Or if you really need to backup them somewhere else, it's safer to do this on another volume (an external hard drive, for example), but not on the same disk.

Last edited by dkl (2007-11-07 13:07:48)

Offline

 

#13 2007-11-14 19:40:36

Logo
New member
Registered: 2007-11-14
Posts: 2

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

This is great, just got it up and working. Thanks...

I was just wondering if it's possible to schedule backups every other day (or longer)? I don't really modify that much data on a daily basis and am concerned about excessive drive wear (it took about 8 hours to transfer all the data).

I'm just about competent enough to follow this excellent tutorial, but have no clue when it comes to the relevant code. Thanks again...

Offline

 

#14 2007-11-15 07:44:36

Chumby
Member
From: Tasmania, Australia
Registered: 2007-08-01
Posts: 96
Website

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Hi Logo,

The first time you do a back up it takes ages because all the data has to be written across.  From here on in though it is really quick, because rsync copies things that are now different.

Cheers
Chumby

Offline

 

#15 2007-11-15 10:27:38

Emacs
Member
Registered: 2007-06-24
Posts: 110

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Logo wrote:

I was just wondering if it's possible to schedule backups every other day (or longer)?

Cron let's you specify in great detail when you want to run a specific task. Check the cron description here http://en.wikipedia.org/wiki/Cron

Apart from that I'm running the script for two weeks now and it is absolutely great. As said before, the first run takes some hours on my 500GB disks, but after that only changes are carried over. So, if you don't modify your files, the script will not do anything but comparing during the night.

Cheers,

Emacs

Offline

 

#16 2007-11-15 18:44:27

Logo
New member
Registered: 2007-11-14
Posts: 2

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

I didn't realize that it only transfers modified data, awesome.

Thanks again...

Offline

 

#17 2007-11-15 23:56:26

WildSioux
Member
Registered: 2007-03-05
Posts: 71

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Chumby,  I finally got around to installing this backup script today.  I am not sure if it works yet since it will run tomorrow morning at 3:05 in the morning.  But I did do the test you provided and it checked out ok.  I will have to check tomorrow to see if the files copied. 

One thing to note on my install.  And I missed this part since I see it is in there.  But I used telnet <IP>, after doing the vi edit mode, and pasting the code for adding the rsync task to the scheduler.  I hit "esc" and nothing happened.  Well I thought nothing happened, but it wouldn't give me the # back to continue.  So I searched for a way to exit "vi."  I found that "ZZ" works as well as what you typed "wq" <> I missed that.

Logo wrote:

I didn't realize that it only transfers modified data, awesome.

Thanks again...

That is nice to know!

Thanks, I will let you know if it works tomorrow.

Offline

 

#18 2007-11-19 13:35:18

gareth
Member
Registered: 2007-06-28
Posts: 50

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Chumby, thanks for the tutorial. I've implemented as stated and it works okay, except that on the second drive it creates a folder called HD_a2 under Volume_2, and then in the HD_a2 directory it lists all of the copied directories.

i.e. Drive 1:

Volume_1\
Volume_1\Media
Volume_1\Downloads
Volume_1\eBooks

Drive 2 after rsync looks like:

Volume_2\
Volume_2\HD_a2\
Volume_2\HD_a2\Media
Volume_2\HD_a2\Downloads
Volume_2\HD_a2\eBooks

Is this correct?

Thanks,
Gareth.

Offline

 

#19 2007-11-19 15:57:16

WildSioux
Member
Registered: 2007-03-05
Posts: 71

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

gareth wrote:

Chumby, thanks for the tutorial. I've implemented as stated and it works okay, except that on the second drive it creates a folder called HD_a2 under Volume_2, and then in the HD_a2 directory it lists all of the copied directories.

ie. Drive 1:

Volume_1\.
Volume_1\Media
Volume_1\Downloads
Volume_1\eBooks

Drive 2 after rsync looks like:

Volume_2\
Volume_2\HD_a2\
Volume_2\HD_a2\Media
Volume_2\HD_a2\Downloads
Volume_2\HD_a2\eBooks

Is this correct?

Thanks,
Gareth.

Yes, this is what I have so it must be correct.  I got mine working the other day and it works great.  Thanks again!

Offline

 

#20 2007-11-19 21:55:58

Chumby
Member
From: Tasmania, Australia
Registered: 2007-08-01
Posts: 96
Website

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Hi Guys,

Yes that what it does - creates a folder on the backup drive and puts all the contents of the data drive in there.

Great to see it has been working well for you guys.

Chumby.

Offline

 

#21 2007-11-19 21:59:26

gareth
Member
Registered: 2007-06-28
Posts: 50

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Great stuff Chumby, thanks for the confirmation.
Regards,
Gareth.

Offline

 

#22 2007-11-22 20:45:20

WildSioux
Member
Registered: 2007-03-05
Posts: 71

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Chumby, I just did this process to my parents box after reformatting to a two disk system instead of the raid.  I did everything to the T from what I did on mine.  But when I typed in "crontab -l" I got something different:

32 2 *** /usr/sbin/rtc -s
30 2 2 ** /usr/sbin/rtc -c
59 1 *** /usr/sbin/daylight

Since the line about the /mnt/HD_a2/...rsync isn't in there is it safe to say that it didn't work correctly?  I don't understand since I did everything the same from mine.  Please advise.
Thanks

Offline

 

#23 2007-11-22 22:14:51

Chumby
Member
From: Tasmania, Australia
Registered: 2007-08-01
Posts: 96
Website

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Yes... correct. The copying will not happen as rsync is not scheduled.

I presume you have rebooted the box and can telnet in OK?

Cheers
Chumby.

Offline

 

#24 2007-11-23 01:18:55

WildSioux
Member
Registered: 2007-03-05
Posts: 71

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

Chumby wrote:

Yes... correct. The copying will not happen as rsync is not scheduled.

I presume you have rebooted the box and can telnet in OK?

Cheers
Chumby.

Duh, I am a dumbass...I rebooted it and it says the correct job now.  Sorry for the irrelevant post.  I was just busy with everything else and missed the most important step.

Offline

 

#25 2007-11-29 09:12:30

Chumby
Member
From: Tasmania, Australia
Registered: 2007-08-01
Posts: 96
Website

Re: Tutorial: Backup Everything from Vol A to Vol B once a night

That's good to see, WildSioux!

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2010 PunBB