DSM-G600, DNS-3xx and NSA-220 Hack Forum

Hey Chumby, thank you for this tutorial.

I tried it out and it works quite well. I only found a problem with the ownership of the folders and files on the backup drive. Using your suggested options "-rlptDv --delete" will cause all newly copied folders and files to belong to root:root on the backup drive. That is NOT what I want and probably a lot of other people agree with me and didn't even notice this behaviour so far.

I think that a much better way would be to use option "-av --delete" because this will also keep the ownership of the copied folders and files. This is a backup that we are talking about and backups are not about changing ownership, aren't they?


By the way, I like being informed about the results of the backup and since there is sendmail in ffp, you might as well add an option to send an email notification with the results.


My sync script looks like this at the moment:

#!/bin/sh
#this script synchronizes disk 1 (right slot) with disk 2 (left slot)
#it can be used by cron to automatically sync disks over night

DISK1=/mnt/HD_a2
DISK2=/mnt/HD_b2
SYNCOPT="-av --delete"
LOGFILE=/USB/var/log/sync-ab.log
PASS=`cat $DISK1/mailcred`

#write the current date and time to the log
date > $LOGFILE

#sync all the folders that we like
/ffp/bin/rsync $SYNCOPT $DISK1/Folder1 $DISK2 >> $LOGFILE
/ffp/bin/rsync $SYNCOPT $DISK1/Folder2 $DISK2 >> $LOGFILE

#send email notification
sendmail -U mailusername -P $PASS -t receiver@domain -s "Backup Details" \
servername sender@domain $LOGFILE

Of course you will have to put in the correct data for mailusername etc.

"sendmail --help" provides answers about the different options.

The script reads the password of the email account from the file "mailcred" which has permissions of 600.

Hi silversurfer,

this looks nice. I've been desperately trying to get my CH3SNAS to send mail. However, my sendmail reacts strangely:

root@NAS:/mnt/HD_a2/usbstorage/ffp# sendmail --help
sendmail: invalid option -- -
Invalid option ?
root@NAS:/mnt/HD_a2/usbstorage/ffp#

When I tried to use your code to test it, I got this:

root@NAS:~# sendmail -U test@example.com -P password -t test2@example.com -s "Backup Details" post.example.com test@example.com "Test"
sendmail: invalid option -- U
Invalid option ?
root@NAS:~#

What am I doing wrong? Why doesn't sendmail (it *does* exist in my bin directory) know its options?

Do I have to configure sendmail anyplace else before using it?

Greets
Ben

yojoe007 wrote:

blbrown wrote:

Dexter Tones wrote:

I followed the tutorial and everything went smoothly. However, I was looking at the rsync FAQ and it says that one should "expect rsync to consume about 100 bytes per file" to "hold the list of files being transferred." I don't have many files on my DNS right now, but given the amount of RAM it has, I fear that getting an "out of memory" error may become an issue in the future (when I have many more files stored on the DNS). Is there a workaround for this at all?

Yes, I have noticed this issue as well.  There are two work-arounds:

1. Do some scripting magic to walk your directory structure and run rsync on each one to make sure the file list doesn't get too long.

2. Wait until rsync (http://samba.anu.edu.au/rsync/) version 3.0 is complete and someone (fonz?) compiles it for the DNS-323.  Version 3 addresses this specific issue and it's currently in pre-release which gives me hope it will be complete some time soon.

That said, I have BNC running on a directory structure with ~26000 files (on a Windows PC backup client).  I've had no problems so far with the DNS-323 and BNC nicely tells me this file count in the email status (on the line that says "Backup location total file/directory count: xxx (xxxGB)").  On my DNS-323 that backs itself up (it's both the backup client and the backup server) I'm up to ~9000 files without problem so far.

At 100 bytes per file, you should have no problem using rsync with 320,000+ files, which should only consume half the physical memory of the DNS-323.  And I'm guessing swap space would be used automatically if you used up the physical memory, though this would be stressing the unit I think.

Rsync is now up to version 3.04, with 3.05 in pre-release test - was this problem addressed? Can I install ffp 0.4 and manual install rsync 3.04, or as is written above does it need to be "compiled" (I don't know what this means) to work correctly? And since ffp .05 includes rsync, does anyone know which version - as in, is it version 3.0 or above? I couldn't find that information anywhere, and have looked as best I can.

I am concerned about the physical memory being used up as I use my DNS for backing up thousands of digital photos from my job.

Thanks, James

Check post #158 in this thread.  Fonz has compiled a version of rsync since 3.0, which should help with memory usage in rsync.  I haven't bothered going to this version yet, but I trust fonz that it should work fine with ffp 0.5.

nurunet wrote:

I've been desperately trying to get my CH3SNAS to send mail. However, my sendmail reacts strangely:

root@NAS:/mnt/HD_a2/usbstorage/ffp# sendmail --help
sendmail: invalid option -- -
Invalid option ?
root@NAS:/mnt/HD_a2/usbstorage/ffp#

What am I doing wrong? Why doesn't sendmail (it *does* exist in my bin directory) know its options?

Do I have to configure sendmail anyplace else before using it?

Greets
Ben

Maybe you have a different version of busybox? Running "which sendmail" tells me that it is located in "/ffp/bin". Actually it is part of busybox. Running "sendmail --help" says:

BusyBox v1.11.1 (2008-07-13 20:04:00 CEST) multi-call binary

Usage: sendmail [-w timeout] [-U user] [-P password] [-X]
-t to [-t to]... [-n] [-s subject] [-c charset] server[:port] from [body] [attachment ...]

Send an email.

Options:
        -w timeout      Set timeout on network operations
        -U username     Authenticate with specified username/password
        -P password
        -t address      Recipient(s). May be repeated
        -X              Use openssl connection helper for secured servers
        -n              Request delivery notification to sender
        -s subject      Subject
        -c charset      Assumed charset for body and subject [utf-8]

You can see that it is busybox version 1.11.1. It is part of the current release of ffp 0.5.
Do a "busybox --help" to see what version you have got.

At the moment I have a problem with the mail notification anyway. If I run the script manually as root it sends the mail and if cron runs the script I get nothing. Cron seems to run as root, so that shouldn't be the cause of the problem. I need to investigate when I find the time.



@WildSioux

Have a look in the Wiki -> http://en.wikipedia.org/wiki/Cron  The fields in front of the command are explained there.

nurunet wrote:

BusyBox v1.12.1 (2008-09-29 20:38:04 CEST) multi-call binary

sendmail is pretty new in busybox, and because I don't know how well it works, I removed it from busybox 1.12. There's esmtp as a more mature replacement. Use 'man sendmail' to get help.

nurunet wrote:

This explains a lot. For esmtp: I tried it, but it is too powerful for me, I guess. Couldn't get it to work.

It's not too difficult. There's good documentation with examples available at:
http://esmtp.sourceforge.net/manual.html

PS: And the best thing about esmtp is that it supports authentication (which the busybox sendmail cannot do).

Chumby or anyone else...

Is there a way to set this script up to backup once a week instead of nightly?

If not, are there any other backup scripts for this box that would do that?

Thanks!

# +---------------- minute (0 - 59)
# |  +------------- hour (0 - 23)
# |  |  +---------- day of month (1 - 31)
# |  |  |  +------- month (1 - 12)
# |  |  |  |  +---- day of week (0 - 6) (Sunday=0 or 7)
# |  |  |  |  |
  *  *  *  *  *  command to be executed

Example:

30 3 * * 6 command to be executed

This would run the command every Saturday at 03:30 o'clock.

Chumby wrote:

Hey Silver... thanks for that ownership note.... have updated the tutorial to reflect the changes you suggest

So if I change the rsync params to preserve ownership, then will it change files that already have been copied?  Or should I create a fresh sync?

By the way, I found out why my cron wouldn't want to email me about the results. I'm so silly, I forgot to provide the path to sendmail.

This is what it looks like right now. I added another two lines to backup my ffp to the disks too, just in case my USB stick stops working.

#!/bin/sh
#this script synchronizes disk 1 (right slot) with disk 2 (left slot)
#it can be used by cron to automatically sync disks over night

DISK1=/mnt/HD_a2
DISK2=/mnt/HD_b2
USB=/mnt/USB
SYNCOPT="-av --delete"
LOGFILE=/USB/var/log/sync-ab.log

#write the current date and time to the log
date > $LOGFILE

#backup ffp from USB to DISK1
/ffp/bin/rsync $SYNCOPT $USB/ $DISK1/USB_Backup >> $LOGFILE

#sync all the folders that we like
/ffp/bin/rsync $SYNCOPT $DISK1/Frank $DISK2 >> $LOGFILE
/ffp/bin/rsync $SYNCOPT $DISK1/share $DISK2 >> $LOGFILE
/ffp/bin/rsync $SYNCOPT $DISK1/USB_Backup $DISK2 >> $LOGFILE

#send email notification
/ffp/bin/sendmail -t receiver@domain -s "Rsync run" \
mailserver sender@domain $LOGFILE

@nurunet: The last word on this is from silversurfer two posts up:

silversurfer wrote:

Well, I did a little test.

1. created a new file with ownership user:group
2. ran my sync script with -av once
3. checked that the file was on the target disk - it was there with user:group and the log file said that it was copied
4. changed ownership on target disk manually to root:root
5. ran the sync script again
6. checked the ownership of the file on the target disk - it was back to user:group

From this experience I would say that running rsync with the -av parameter will correct any ownership issues. Rsync didn't show anything in the log after step 5 because it didn't copy anything but as long as it fixes the ownership, that behaviour is fine for me.

I've been unable to get the script to work until this morning. I spent a good part of yesterday mucking around with this, this morning I deleted fun-plug completely and started from scratch, everything seemed fine but the backup still failed. As usual the fix was right in front on my nose. The file editcron.sh contains a reference for rsync to write to a log [/mnt/HD_a2/ffp/log/rsync.last.log]. I went to check the log, that file and folder didn't exist. I created the folder manually and ran the script again (changed the time in editcron.sh). All is well, the script ran as scheduled! Can anyone tell me if there fun-plug .05 install included the ffp/log folder? Might be worth adding to the instructions?

Yo!Adrian, eh?
Toronto

Hi Halfsoul and yo_adrian_eh ... have updated the tutorial to show how to create the log directory if not already there (I can't remember if it is or isn't there by default... seems not )

Cheers
Chumby