Unfortunately no one can be told what fun_plug is - you have to see it for yourself.
You are not logged in.
Hey Chumby, thank you for this tutorial.
I tried it out and it works quite well. I only found a problem with the ownership of the folders and files on the backup drive. Using your suggested options "-rlptDv --delete" will cause all newly copied folders and files to belong to root:root on the backup drive. That is NOT what I want and probably a lot of other people agree with me and didn't even notice this behaviour so far.
I think that a much better way would be to use option "-av --delete" because this will also keep the ownership of the copied folders and files. This is a backup that we are talking about and backups are not about changing ownership, aren't they?
By the way, I like being informed about the results of the backup and since there is sendmail in ffp, you might as well add an option to send an email notification with the results.
My sync script looks like this at the moment:
#!/bin/sh #this script synchronizes disk 1 (right slot) with disk 2 (left slot) #it can be used by cron to automatically sync disks over night DISK1=/mnt/HD_a2 DISK2=/mnt/HD_b2 SYNCOPT="-av --delete" LOGFILE=/USB/var/log/sync-ab.log PASS=`cat $DISK1/mailcred` #write the current date and time to the log date > $LOGFILE #sync all the folders that we like /ffp/bin/rsync $SYNCOPT $DISK1/Folder1 $DISK2 >> $LOGFILE /ffp/bin/rsync $SYNCOPT $DISK1/Folder2 $DISK2 >> $LOGFILE #send email notification sendmail -U mailusername -P $PASS -t receiver@domain -s "Backup Details" \ servername sender@domain $LOGFILE
Of course you will have to put in the correct data for mailusername etc.
"sendmail --help" provides answers about the different options.
The script reads the password of the email account from the file "mailcred" which has permissions of 600.
Offline
Does anyone know how to change this script to backup once a week instead of nightly?
Offline
Hi silversurfer,
this looks nice. I've been desperately trying to get my CH3SNAS to send mail. However, my sendmail reacts strangely:
root@NAS:/mnt/HD_a2/usbstorage/ffp# sendmail --help
sendmail: invalid option -- -
Invalid option ?
root@NAS:/mnt/HD_a2/usbstorage/ffp#
When I tried to use your code to test it, I got this:
root@NAS:~# sendmail -U test@example.com -P password -t test2@example.com -s "Backup Details" post.example.com test@example.com "Test"
sendmail: invalid option -- U
Invalid option ?
root@NAS:~#
What am I doing wrong? Why doesn't sendmail (it *does* exist in my bin directory) know its options?
Do I have to configure sendmail anyplace else before using it?
Greets
Ben
Offline
blbrown wrote:
Dexter Tones wrote:
I followed the tutorial and everything went smoothly. However, I was looking at the rsync FAQ and it says that one should "expect rsync to consume about 100 bytes per file" to "hold the list of files being transferred." I don't have many files on my DNS right now, but given the amount of RAM it has, I fear that getting an "out of memory" error may become an issue in the future (when I have many more files stored on the DNS). Is there a workaround for this at all?
Yes, I have noticed this issue as well. There are two work-arounds:
1. Do some scripting magic to walk your directory structure and run rsync on each one to make sure the file list doesn't get too long.
2. Wait until rsync (http://samba.anu.edu.au/rsync/) version 3.0 is complete and someone (fonz?) compiles it for the DNS-323. Version 3 addresses this specific issue and it's currently in pre-release which gives me hope it will be complete some time soon.
That said, I have BNC running on a directory structure with ~26000 files (on a Windows PC backup client). I've had no problems so far with the DNS-323 and BNC nicely tells me this file count in the email status (on the line that says "Backup location total file/directory count: xxx (xxxGB)"). On my DNS-323 that backs itself up (it's both the backup client and the backup server) I'm up to ~9000 files without problem so far.
At 100 bytes per file, you should have no problem using rsync with 320,000+ files, which should only consume half the physical memory of the DNS-323. And I'm guessing swap space would be used automatically if you used up the physical memory, though this would be stressing the unit I think.
Rsync is now up to version 3.04, with 3.05 in pre-release test - was this problem addressed? Can I install ffp 0.4 and manual install rsync 3.04, or as is written above does it need to be "compiled" (I don't know what this means) to work correctly? And since ffp .05 includes rsync, does anyone know which version - as in, is it version 3.0 or above? I couldn't find that information anywhere, and have looked as best I can.
I am concerned about the physical memory being used up as I use my DNS for backing up thousands of digital photos from my job.
Thanks, James
Offline
yojoe007 wrote:
blbrown wrote:
Dexter Tones wrote:
I followed the tutorial and everything went smoothly. However, I was looking at the rsync FAQ and it says that one should "expect rsync to consume about 100 bytes per file" to "hold the list of files being transferred." I don't have many files on my DNS right now, but given the amount of RAM it has, I fear that getting an "out of memory" error may become an issue in the future (when I have many more files stored on the DNS). Is there a workaround for this at all?
Yes, I have noticed this issue as well. There are two work-arounds:
1. Do some scripting magic to walk your directory structure and run rsync on each one to make sure the file list doesn't get too long.
2. Wait until rsync (http://samba.anu.edu.au/rsync/) version 3.0 is complete and someone (fonz?) compiles it for the DNS-323. Version 3 addresses this specific issue and it's currently in pre-release which gives me hope it will be complete some time soon.
That said, I have BNC running on a directory structure with ~26000 files (on a Windows PC backup client). I've had no problems so far with the DNS-323 and BNC nicely tells me this file count in the email status (on the line that says "Backup location total file/directory count: xxx (xxxGB)"). On my DNS-323 that backs itself up (it's both the backup client and the backup server) I'm up to ~9000 files without problem so far.
At 100 bytes per file, you should have no problem using rsync with 320,000+ files, which should only consume half the physical memory of the DNS-323. And I'm guessing swap space would be used automatically if you used up the physical memory, though this would be stressing the unit I think.Rsync is now up to version 3.04, with 3.05 in pre-release test - was this problem addressed? Can I install ffp 0.4 and manual install rsync 3.04, or as is written above does it need to be "compiled" (I don't know what this means) to work correctly? And since ffp .05 includes rsync, does anyone know which version - as in, is it version 3.0 or above? I couldn't find that information anywhere, and have looked as best I can.
I am concerned about the physical memory being used up as I use my DNS for backing up thousands of digital photos from my job.
Thanks, James
Check post #158 in this thread. Fonz has compiled a version of rsync since 3.0, which should help with memory usage in rsync. I haven't bothered going to this version yet, but I trust fonz that it should work fine with ffp 0.5.
Offline
yojoe007 wrote:
And since ffp .05 includes rsync, does anyone know which version
Offline
nurunet wrote:
I've been desperately trying to get my CH3SNAS to send mail. However, my sendmail reacts strangely:
root@NAS:/mnt/HD_a2/usbstorage/ffp# sendmail --help
sendmail: invalid option -- -
Invalid option ?
root@NAS:/mnt/HD_a2/usbstorage/ffp#What am I doing wrong? Why doesn't sendmail (it *does* exist in my bin directory) know its options?
Do I have to configure sendmail anyplace else before using it?
Greets
Ben
Maybe you have a different version of busybox? Running "which sendmail" tells me that it is located in "/ffp/bin". Actually it is part of busybox. Running "sendmail --help" says:
BusyBox v1.11.1 (2008-07-13 20:04:00 CEST) multi-call binary Usage: sendmail [-w timeout] [-U user] [-P password] [-X] -t to [-t to]... [-n] [-s subject] [-c charset] server[:port] from [body] [attachment ...] Send an email. Options: -w timeout Set timeout on network operations -U username Authenticate with specified username/password -P password -t address Recipient(s). May be repeated -X Use openssl connection helper for secured servers -n Request delivery notification to sender -s subject Subject -c charset Assumed charset for body and subject [utf-8]
You can see that it is busybox version 1.11.1. It is part of the current release of ffp 0.5.
Do a "busybox --help" to see what version you have got.
At the moment I have a problem with the mail notification anyway. If I run the script manually as root it sends the mail and if cron runs the script I get nothing. Cron seems to run as root, so that shouldn't be the cause of the problem. I need to investigate when I find the time.
@WildSioux
Have a look in the Wiki -> http://en.wikipedia.org/wiki/Cron The fields in front of the command are explained there.
Offline
root@NAS:~# which sendmail
/ffp/sbin/sendmail
root@NAS:~# busybox --help
BusyBox v1.12.1 (2008-09-29 20:38:04 CEST) multi-call binary
Copyright (C) 1998-2008 Erik Andersen, Rob Landley, Denys Vlasenko
and others. Licensed under GPLv2.
See source distribution for full notice.
Usage: busybox [function] [arguments]...
or: function [arguments]...
BusyBox is a multi-call binary that combines many common Unix
utilities into a single executable. Most people will create a
link to busybox for each function they wish to use and BusyBox
will act like whatever it was invoked as!
Currently defined functions:
[, [[, adduser, adjtimex, arp, arping, ash, awk, basename, bbconfig,
bunzip2, bzcat, bzip2, cal, cat, catv, chgrp, chmod, chown, chroot,
chrt, chvt, cksum, clear, cmp, comm, cp, crond, crontab, cryptpw,
cut, date, dc, dd, deallocvt, deluser, depmod, df, dirname, dmesg,
dos2unix, du, echo, ed, egrep, eject, env, ether-wake, expand,
expr, fakeidentd, false, fdisk, fgrep, find, fold, free, freeramdisk,
fsck.minix, ftpget, ftpput, fuser, getopt, getty, grep, gunzip,
gzip, halt, hdparm, head, hexdump, hostid, hostname, hwclock,
id, ifconfig, ifdown, ifenslave, ifup, inetd, init, insmod, install,
ip, ipaddr, ipcalc, ipcrm, ipcs, iplink, iproute, iprule, iptunnel,
kill, killall, killall5, klogd, last, length, ln, logger, logname,
logread, losetup, ls, lsmod, lzmacat, md5sum, mdev, mesg, microcom,
mkdir, mkfifo, mkfs.minix, mknod, mkswap, mktemp, modprobe, more,
mount, mountpoint, mv, nameif, nc, netstat, nice, nmeter, nohup,
nslookup, od, openvt, pgrep, pidof, ping, pivot_root, pkill, poweroff,
printenv, printf, ps, pwd, raidautorun, rdate, rdev, readahead,
readlink, readprofile, realpath, reboot, renice, reset, resize,
rm, rmdir, rmmod, route, run-parts, runlevel, rx, script, sed,
seq, setconsole, setlogcons, setsid, sh, sha1sum, slattach, sleep,
sort, split, stat, strings, stty, su, sulogin, sum, swapoff, swapon,
switch_root, sync, sysctl, syslogd, tac, tail, tar, taskset, tee,
telnet, telnetd, test, time, top, touch, tr, traceroute, true,
tty, ttysize, udhcpc, umount, uname, uncompress, unexpand, uniq,
unix2dos, unlzma, unzip, uptime, usleep, uudecode, uuencode, vconfig,
vi, vlock, watch, watchdog, wc, wget, which, who, whoami, xargs,
yes, zcat, zcip
root@NAS:~# sendmail --help
sendmail: invalid option -- -
Invalid option ?
root@NAS:~#
Seems like some things changed here, although I do have ffp 0.5. As I do have sendmail installed too, I still wonder why it won't take the --help argument.
Offline
nurunet wrote:
BusyBox v1.12.1 (2008-09-29 20:38:04 CEST) multi-call binary
sendmail is pretty new in busybox, and because I don't know how well it works, I removed it from busybox 1.12. There's esmtp as a more mature replacement. Use 'man sendmail' to get help.
Offline
This explains a lot. For esmtp: I tried it, but it is too powerful for me, I guess. Couldn't get it to work.
Offline
nurunet wrote:
This explains a lot. For esmtp: I tried it, but it is too powerful for me, I guess. Couldn't get it to work.
It's not too difficult. There's good documentation with examples available at:
http://esmtp.sourceforge.net/manual.html
PS: And the best thing about esmtp is that it supports authentication (which the busybox sendmail cannot do).
Offline
Thanks for your help, fonz, but I read that. Tried it, to no avail. See http://dns323.kood.org/forum/t3081-Send … esmtp.html for details.
I just wished someone got this to work and posted some kind of tutorial, it simply exceeds my Linux skills.
Have a nice evening!
Ben
Offline
Chumby or anyone else...
Is there a way to set this script up to backup once a week instead of nightly?
If not, are there any other backup scripts for this box that would do that?
Thanks!
Offline
WildSioux wrote:
Chumby or anyone else...
Is there a way to set this script up to backup once a week instead of nightly?
If not, are there any other backup scripts for this box that would do that?
Thanks!
Offline
# +---------------- minute (0 - 59) # | +------------- hour (0 - 23) # | | +---------- day of month (1 - 31) # | | | +------- month (1 - 12) # | | | | +---- day of week (0 - 6) (Sunday=0 or 7) # | | | | | * * * * * command to be executed
Example:
30 3 * * 6 command to be executed
This would run the command every Saturday at 03:30 o'clock.
Offline
Chumby wrote:
Hey Silver... thanks for that ownership note.... have updated the tutorial to reflect the changes you suggest
So if I change the rsync params to preserve ownership, then will it change files that already have been copied? Or should I create a fresh sync?
Offline
halfsoul wrote:
So if I change the rsync params to preserve ownership, then will it change files that already have been copied? Or should I create a fresh sync?
Well, I did a little test.
1. created a new file with ownership user:group
2. ran my sync script with -av once
3. checked that the file was on the target disk - it was there with user:group and the log file said that it was copied
4. changed ownership on target disk manually to root:root
5. ran the sync script again
6. checked the ownership of the file on the target disk - it was back to user:group
From this experience I would say that running rsync with the -av parameter will correct any ownership issues. Rsync didn't show anything in the log after step 5 because it didn't copy anything but as long as it fixes the ownership, that behaviour is fine for me.
Offline
By the way, I found out why my cron wouldn't want to email me about the results. I'm so silly, I forgot to provide the path to sendmail.
This is what it looks like right now. I added another two lines to backup my ffp to the disks too, just in case my USB stick stops working.
#!/bin/sh #this script synchronizes disk 1 (right slot) with disk 2 (left slot) #it can be used by cron to automatically sync disks over night DISK1=/mnt/HD_a2 DISK2=/mnt/HD_b2 USB=/mnt/USB SYNCOPT="-av --delete" LOGFILE=/USB/var/log/sync-ab.log #write the current date and time to the log date > $LOGFILE #backup ffp from USB to DISK1 /ffp/bin/rsync $SYNCOPT $USB/ $DISK1/USB_Backup >> $LOGFILE #sync all the folders that we like /ffp/bin/rsync $SYNCOPT $DISK1/Frank $DISK2 >> $LOGFILE /ffp/bin/rsync $SYNCOPT $DISK1/share $DISK2 >> $LOGFILE /ffp/bin/rsync $SYNCOPT $DISK1/USB_Backup $DISK2 >> $LOGFILE #send email notification /ffp/bin/sendmail -t receiver@domain -s "Rsync run" \ mailserver sender@domain $LOGFILE
Offline
Chumby wrote:
2) Telnet in and from the # prompt, type crontab –l (lowercase L). If everything has worked as it should you will see your rsync line as a scheduled task: (funplug 0.5 = ffp directory)
Code:
5 2 * * * /mnt/HD_a2/fun_plug.d/bin/rsync -rlptDv --delete /mnt/HD_a2 /mnt/HD_b2 >> /mnt/HD_a2/fun_plug.d/log/rsync.last.log 2>&1
This still shows the old parameters.
One question. I changed the parameters now. Will it have to copy everything again or can rsync just change the attributes?
Cheers
Ben
Offline
@nurunet: The last word on this is from silversurfer two posts up:
silversurfer wrote:
Well, I did a little test.
1. created a new file with ownership user:group
2. ran my sync script with -av once
3. checked that the file was on the target disk - it was there with user:group and the log file said that it was copied
4. changed ownership on target disk manually to root:root
5. ran the sync script again
6. checked the ownership of the file on the target disk - it was back to user:group
From this experience I would say that running rsync with the -av parameter will correct any ownership issues. Rsync didn't show anything in the log after step 5 because it didn't copy anything but as long as it fixes the ownership, that behaviour is fine for me.
Offline
Hey!
I hadn't understood, whether or not rsync would correct ownership by modifying the existing file or simply overwriting it with the new version - which would take far longer. So I changed the user and group manually and recursively for all files in the "HD_a2" directory on "HD_a1". Then I ran the new rsync task. Everything went smooth and fast.
Greetz
Ben
Offline
I've been unable to get the script to work until this morning. I spent a good part of yesterday mucking around with this, this morning I deleted fun-plug completely and started from scratch, everything seemed fine but the backup still failed. As usual the fix was right in front on my nose. The file editcron.sh contains a reference for rsync to write to a log [/mnt/HD_a2/ffp/log/rsync.last.log]. I went to check the log, that file and folder didn't exist. I created the folder manually and ran the script again (changed the time in editcron.sh). All is well, the script ran as scheduled! Can anyone tell me if there fun-plug .05 install included the ffp/log folder? Might be worth adding to the instructions?
Yo!Adrian, eh?
Toronto
Offline
@yo_adrian_eh: You're not the first person to have that same issue. crontab seems to fail without generating an error when the destination directory for a log file doesn't exist. I agree that it would be nice to add to the instructions.
@newbie73: Assuming you've searched the forums and can't find a solution, perhaps you could start a new thread for your question, since it seems to be off-topic.
Offline
Hi Halfsoul and yo_adrian_eh ... have updated the tutorial to show how to create the log directory if not already there (I can't remember if it is or isn't there by default... seems not )
Cheers
Chumby
Offline