Unfortunately no one can be told what fun_plug is - you have to see it for yourself.
You are not logged in.
I just set up and ran BackupPC first for a full backup of about 9 GB and then a few hours later I did an incremental backup of the same share.
To maximize speed, I have been running BackupPC itself on my Linux server where it is only consuming a total of about 20-30% of the CPU power. I am then storing the backup data on the dns-323 which I am mounting via NFS. The dns-323 is very lightly loaded during a backup (with the nfs processes taking up only a few percent of cpu power)
Since I am backing up the server itself, the rate limiting step should be the NFS file transfer speed as constrained by the underlying 100Mbps ethernet link (since neither the linux server nor the dns-323 are computationally constrained).
Nevertheless, I got average net transfer speeds of .46 MBps compressed (.87 MBps uncompressed) - now since all the compression is happening on the server side I am effectively getting less than half a megabyte per second backup speed.
Now when I do a raw NFS large file copy, the copy proceeds at about 8.2 MBps or almost 20 times as fast as the backup speed.
I guess I am perplexed about what is causing this slowdown by a factor of 20.
I can accept a slowdown by a factor of 2-3 due to a combination of overhead, perhaps file checking, and small files but not a factor of 20. Also, note that this was on a full backup so there shouldn't have been any need to read/compare with previous backups.
Even more disturbingly, the subsequent incremental backup took almost as long. Given that the incremental changed files accounted for only about 4% of the total backup the effective backup speed was .01MBps compressed (and .04MBps uncompressed) -- which is ridiculous.
Again, it seems like neither the Linux server nor the dns-323 were either cpu or bandwidth constrained.
Any thoughts?
Offline
Just a suggestion ....
You already have an idea of the transfer speeds for "large file copy" - how about bench marking "small file copy"? It might be very enlightening to compare the time taken to transfer a single 2GB file to that taken to transfer 1000 x 2MB files.
Offline
Bingo!
Turns out the problem is indeed small file transfer speed over NFS particularly since I had mistakenly thought I was using 'async' but had forgotten to properly change my /etc/exports file. DOH!
Here is some data for copying a directory of 10000 random 4KB files using 'cp -a'
Local copy on Linux Server:
- Native ext3-to-ext3: 30.3 MB/sec
- Local NFS with async: 3.3 MB/sec ( 10x slowdown penalty for NFS vs. native)
- Local NFS with sync: 0.127 MB/sec (additional 25x slowdown for sync)
Local copy on dns-323:
- Native ext3-to-ext3: 3.9 MB/sec (about 8x slower than my Linux server)
- Local NFS with async: 0.38 MB/sec (about 10x slowdown vs. native)
- Local NFS with sync: 0.042 MB/sec (additional 10x slowdown for sync)
Linux server to dns-323:
- NFS with async: 0.89 MB/sec (about 10x slower than large file transfer)
- NFS with sync: 0.06 MB/sec (about 15x slower than async)
Moral of the story is to use 'asnc' if you are doing multiple small file transfers (as with BackupPC) since that alone will give you a 10-30x improvement over 'sync'.
HOWEVER, for small files, nfs is still about 10x slower than either native ext3-ext3 transfer of small files or NFS transfer of large files. (The consistency of this 10x slowdown at least makes me more confident that the issue isn't my dns-323 or the nfs drivers there)
I guess my only remaining question is whether there are any simple NFS tunings that will allow me to decrease the small file transfer penalty to less than 10x.
Offline
Note NFS for small files is however 3x slower than rsync/rsyncd but the same speed as rsync/ssh:
Linux server to dns-323:
- Rsync/rsyncd: 2.3MB/sec (about 3x faster than NFS)
- Rsync/ssh: 0.89 MB/sec (essentially the same as NFS)
Offline