DSM-G600, DNS-3xx and NSA-220 Hack Forum

From the TMTOWTDI files, here is the method that I use to back up my RAID-1 configuration to an external USB drive. It could also be used to run a disk-to-disk synchronization if you don't run RAID.

This is a quick port of what I have been using to backup my home server for many years. Now that I have retired my orginal server in lieu of my DNS-323, I wanted to make sure that I am still getting the same data protection. I am running this from Debian etch chroot and am only backing up the chrooted filesystem. There is no reason to believe that this wouldn't work under the fun_plug, though.

Features

- Runs from cron
- Records start and end times of process
- Logs all files copies and changes
- Prevents multiple concurrent invocations of program via lockfile
- Will email if there is an unexpected failure (it does not consider a "file vanished during copy" a failure)
- You can exclude directories from the sync via config file (eg, I don't copy tmp, cache, sys, proc, dev, rtorrent download dir)
- If the device is not mounted, the script will mount it, run the sync, and then unmount it.
- If the device is mounted when the script is run, it will sync, and then remount the backup device read-only.

Requirements

- rsync
- mailx
- The directory pointed to by BACKUP_MOUNT must exist
- The device pointed to by BACKUP_DEVICE must exist
- The file pointed to by EXCLUDE must exist (even if it is empty)

Installation

- Save the attached file to your DNS-323. Make note of where you saved it and what it is called.
  I'll assume it is saved to "/usr/local/bin/rsync-backup.sh"
- Make the file executable: chmod 775 /usr/local/bin/rsync-backup.sh
- Edit the file and update the following for your environment:
    LOCKFILE            What file do you want to use for your lockfile
    BACKUP_DEVICE    What is the device you want to backup
    BACKUP_MOUNT    Where should the backup device mount to
    EXCLUDE            A file with a list of directories that should not be sync
                    This must exist, even if it is empty
    BACKUP_LOG        Where do you want to write the backup log to
    ERRORS_TO        Who should get the email if there is an error
- Review the other variables
    DF
    DATE
    ECHO
    GREP
    ...

    These will vary by environment. When you find where the appropriate binary is on your system, update these variables.
- For testing, update RSYNC_FLAG to:
    RSYNC_FLAG="-a -v -n"    
- Edit the EXCLUDE file. Mine looks like:
    /mnt/
    /dev/
    /proc/
    /tmp/
    /opt/backup/
    /opt/bittorrent/
    /var/cache/
    /var/log/
    /var/tmp/
- Run the process manually, as root:
    /usr/local/bin/rsync-backup.sh >> /var/log/rsync-backup.log 2>&1

    Note that the log file (/var/log/rsync-backup.log) can be wherever you want it to be.
    You should probably make it the same as 'BACKUP_LOG'.
    This will not actually do any copying (thanks to the '-n' flag in RSYNC_FLAG)
- Check the log. If it runs correctly, edit rsync-backup and remove the '-n' from RSYNC_FLAG.
- If the job does not run correctly, figure out why or post for help.
- Add the job to cron. The schedule I use is:
    0 6,13,19 * * * /usr/local/bin/rsync-backup.sh >> /var/log/rsync-backup.log 2>&1

Performance

I have only run this copy from RAID-1 to external USB. I get about 12GB/hour. I'm not sure what disk-to-disk would be. With about 100k files, it takes around 2.5 minutes to run if there are only a few small files to update (usually Maildir updates). If there are major changes (like I just transferred a downloaded TV series to my "videos" share), it is limited by the transfer rate.

Other thoughts

If you want an exact duplicate on the backup drive, uncomment the second RSYNC_FLAG entry in the script ... this adds the "--delete" flag which will get rid of any file on the backup that does not exist in the source.

My original script used hard-linking and rsync to take snapshots three times a day. In essence, I could go back in time several months to recover earlier versions of any file. There is additional overhead in doing this and I didn't want to tax the unit any more than it already is. My compromise is that I do not include the '--delete' flag for the rsync by default, so when the data is copied to the backup, files that are deleted on the source are not currently removed from the backup. If anyone is interested, I could post the relevant bits of that script to see how that was accomplished.

---

Attachments:
rsync-backup.sh, Size: 3,039 bytes, Downloads: 251