DSM-G600, DNS-3xx and NSA-220 Hack Forum

marinalink · 2008-05-31 23:56:41

Hi,

my DNS is configured with two disk, one with 500GB (sda) and the second with 750GB (sdb). Since I don't need high availability but I want to ensure that my date is somehow secure, I choose not to use RAID1 but two independent disks.

The strategy is to run each night a backup from sda (500GB) onto sdb (750GB). This gives you two advantages:
1) if there is a broken disk, you still have a version which is maximum one day old
2) if you erase by coincident a file, you will be able (if it was created before the last backup) to recover it from the second (backup) disk, in my case sdb.

A very good tutorial for using rsnapshot is given on http://rsnapshot.org/ under HOWTO. Please read it, it gives you a lot of information on the way rsnapshot works.

For those of you who just want to know how I did it, I attached my rsnapshot.conf which gathers all configuration data. In a nutshell I run every night a backup (called "daily"), these backups are kept for 7 days, after that it is copied to "weekly", these weekly backups are kept for 4 weeks. Every month a backup called "monthly" is created, which is kept for 3 month.
This strategy is configured in rsnapshot.conf:

Code:

#########################################
#           BACKUP INTERVALS            #
# Must be unique and in ascending order #
# i.e. hourly, daily, weekly, etc.      #
#########################################

interval        daily   7
interval        weekly  4
interval        monthly 3

My backups are stored on sdb in the directory .snapshots:

Code:

###########################
# SNAPSHOT ROOT DIRECTORY #
###########################

# All snapshots will be stored under this root directory.
#
snapshot_root   /mnt/HD_b2/.snapshots/

And finaly you configure at the end of the file your backups you want to perform:

Code:

###
# MY BACKUP on DNS323
backup  /mnt/HD_a2/     localhost/

To ensure that the backups are performed every night you should add them to the crontab:

Code:

10 3 1 * *    /ffp/bin/rsnapshot monthly
20 3 * * 6    /ffp/bin/rsnapshot weekly
30 3 * * *    /ffp/bin/rsnapshot daily

Important: let the monthly script run before the weekly and those before the daily backup! (see also the HOWTO of rsnapshot)

To ensure that the backups will be performed even after a reboot, I wrote a script which adds the above lines automatically after every boot into the crontab. Please find the script attached to this post. Just copy it to /ffp/start, run "chmod a+x crontab_add_scripts.sh" and execute "/ffp/start/crontab_add_scripts.sh start". But before, edit it according to your purpose!

If everything runs smoothly you will find your most recent backup in "/mnt/HD_b2/.snapshots/daily.0", the weekly backups in "/mnt/HD_b2/.snapshots/weekly.*" and the monthly backup in "/mnt/HD_b2/.snapshots/monthly.*".

Enjoy the backup tool, it is realy simple to use!
marinalink

Last edited by marinalink (2008-05-31 23:59:07)

Attachments:

rsnapshot.conf, Size: 7,131 bytes, Downloads: 569

marinalink · 2008-05-31 23:58:23

And here is the crontab_add_scripts.sh...

Enjoy!
marinalink

Last edited by marinalink (2008-05-31 23:58:34)

Attachments:

crontab_add_scripts.sh, Size: 851 bytes, Downloads: 457

rcblackwell · 2008-06-03 04:00:53

marinalink wrote:
Hi,

my DNS is configured with two disk, one with 500GB (sda) and the second with 750GB (sdb). Since I don't need high availability but I want to ensure that my date is somehow secure, I choose not to use RAID1 but two independent disks.

The strategy is to run each night a backup from sda (500GB) onto sdb (750GB). This gives you two advantages:
1) if there is a broken disk, you still have a version which is maximum one day old
2) if you erase by coincident a file, you will be able (if it was created before the last backup) to recover it from the second (backup) disk, in my case sdb.

A very good tutorial for using rsnapshot is given on http://rsnapshot.org/ under HOWTO. Please read it, it gives you a lot of information on the way rsnapshot works.

marinalink,

Thank you for the excellent little toturial. After reading your post and the How To on the rsnapshot web I disable the rsync job that I had running, installed perl 5.8... and rsnapshot. Thanks for the install script and config files.

Although everything is working well I'd like to make a few changes related to access levels of the storage directory. The "How To" do this is well documented however I'm not sure if the .snapshot directory can be removed then rebuilt without impacting the way rsnapshot runs. Is there some sort of record or log file that keeps track of the files that have been backed up? I guess what I'm really asking is how do I start the rsanpshot process all over again? I want rsnapshot to assume no files have been backed up therefore produce a full backup the next time the daily cron job is run.

Bob Blackwell
Pickering, ON

marinalink · 2008-06-03 17:08:35

Log file:
Concerning the log files, just have a look into the .rsnapshot file. There is one line which defines where you want to store the log file and there is another one which defines the verbosity level.

New backup:
To start from scratch for the backups, i.e. no heritage of old backups, you just delete all directories from .snapshots (i.e. daily.0,..., weekly.0,..., monthly.0,...) or you just remove .snapshots and create it again.

This should do the job. As far as I know, rsnapshot just checks if there are directories in .snapshot and syncs to the most recent one (normaly daily.0), but I'm not sure. To be sure, just delete all directories as stated in the paragraph above. There are no log files which give rsnapshot an indication what was backuped up. It is fully transparent, rsnapshot just checks, what is in the directories of .snapshot. So no magic around (which is good!).

marinalink

Last edited by marinalink (2008-06-03 17:10:20)

micmac · 2008-07-05 18:05:19

thanks marinalink. I grabbed your funplug package and used your rsnapshot.conf (I'm familiar with rsnapshot). Up and running.

EnricoM · 2008-07-25 08:41:22

Thanks Marinalink, up and running with your 'How to'. I love this backup tool!

knireis · 2008-07-25 08:51:37

What are the advantages or disadvantages of this method compared to the 'rsync job everynight' of which there is a tutorial on this site?

EnricoM · 2008-07-25 09:29:00

With a normal rsync job every night, you will only have 1 back up copy from the day before. If you have corrupted/lost data in your original, it will be copied to you backup within a day, so if you discover your data loss/corruption after 2 days, you will not be able to recover this data anaymore from your back up. Of course you could create multiple back up sets, but that will require a lot of disk space.

With rsnapshot you will automatically create multiple backup copies (6 backup copies for the last 6 days, 3 backup copies for the last 3 weeks and 3 backup copies for the last 3 months). So you can recover dataloss over a much longer period of time. The second advantage of rsnapshot is that the disk space used by these backups is slightly more than that of 1 full backup because unchanged files in the backup sets are stored as hard links and only the incrementals consume extra hard disk space.
You can also change the configuration of rsnapshot to create hourly backup's or to keep more or less monthly backup's.

Last edited by EnricoM (2008-07-25 09:56:03)

knireis · 2008-07-25 13:35:57

Ok, i'm ready to try. One more question, there is one directory on /mnt/HD_a2/ which i don't want to include in the backup process. How do i enter this? By just putting and exclude in: backup /mnt/HD_a2/ localhost/ ?

EnricoM · 2008-07-25 14:18:34

There is an 'exclude' statement in /ffp/etc/rsnaphot.conf.

just enter

Code:

exclude folder_to_be_excluded/

Beware not to use a leading '/' before folder_to_be_excluded and to put a <TAB> between exclude and folder_to_be_excluded. Dont forget 'the '/' at the end of folder_to_be_deleted.

@marinalink: there is a small syntax error in your config file, therefore it doesn't exclude the .systemfile folder.
The correct syntax should be:

Code:

exclude .systemfile/

Last edited by EnricoM (2008-07-25 14:23:54)

knireis · 2008-07-25 15:03:48

As i understand it, the rsnapshots run 10 minutes after each other, what if one job takes longer than 10 minutes, does that give any problems?

EnricoM · 2008-07-25 16:23:26

It should not be a problem, however, I have scheduled the 3 commands 1 hour after each other, also to avoid possible problems (I am not very familiar with rsnapshot).

When you run rsnapshot for the first time, it will create a sub-folder daily.0 (which will take a long time depending on the amount of data to be copied, in my case 6 hours for 110 GB). The second time daily.0 will be copied to daily.1 etc (From now on this will be fast because only small changes will be copied). After 1 week daily.6 will be created and these folders will continue to shift (first in first out). So rsnapshot daily will create folders daily.0 up to daily.6
rsnapshot monthly will copy daily.6 to monthly.0 (because hardlinks are used, this goes very fast, in my case a few seconds for 110 GB). If daily.6 does not exist, it displays a message (saying something like nothing to copy).

If you want to check what will happen to your files without changing anything on your file systems you can type (dry run):

Code:

rsnapshot -t daily

knireis · 2008-07-25 19:36:33

I guess you're right, only the daily backup can take a long time to complete, and it is scheduled last so no problems there.
Thanks

knireis · 2008-07-25 21:48:39

A bit off topic, i checked with crontab -l if the rsnapshot is planned to be active. It is, but i see also more scheduled crontab jobs. Any idea what they are?

root@CH3SNAS:~# crontab -l
32 2 * * * /usr/sbin/rtc -s
30 2 2 * * /usr/sbin/rtc -c
59 1 * * * /usr/sbin/daylight &
10 2 1 * * /ffp/bin/rsnapshot monthly
20 2 * * 6 /ffp/bin/rsnapshot weekly
30 2 * * * /ffp/bin/rsnapshot daily

DSM-G600, DNS-3xx and NSA-220 Hack Forum

Announcement

#1 2008-05-31 23:56:41

Short introduction: how to use rsnapshot for backups

Code:

Code:

Code:

Code:

#2 2008-05-31 23:58:23

Re: Short introduction: how to use rsnapshot for backups

#3 2008-06-03 04:00:53

Re: Short introduction: how to use rsnapshot for backups

marinalink wrote:

#4 2008-06-03 17:08:35

Re: Short introduction: how to use rsnapshot for backups

#5 2008-07-05 18:05:19

Re: Short introduction: how to use rsnapshot for backups

#6 2008-07-25 08:41:22

Re: Short introduction: how to use rsnapshot for backups

#7 2008-07-25 08:51:37

Re: Short introduction: how to use rsnapshot for backups

#8 2008-07-25 09:29:00

Re: Short introduction: how to use rsnapshot for backups

#9 2008-07-25 13:35:57

Re: Short introduction: how to use rsnapshot for backups

#10 2008-07-25 14:18:34

Re: Short introduction: how to use rsnapshot for backups

Code:

Code:

#11 2008-07-25 15:03:48

Re: Short introduction: how to use rsnapshot for backups

#12 2008-07-25 16:23:26

Re: Short introduction: how to use rsnapshot for backups

Code:

#13 2008-07-25 19:36:33

Re: Short introduction: how to use rsnapshot for backups

#14 2008-07-25 21:48:39

Re: Short introduction: how to use rsnapshot for backups

Board footer