Unfortunately no one can be told what fun_plug is - you have to see it for yourself.
You are not logged in.
Pages: 1
Hi,
to whom it may concern: I was trying to back up my HDs of the DNS323 on an external HD connected via USB. I thought rsync might be handy since a interrupted backup could be restarted without having to copy all the already transfered data. To my surprise, I had to realize that the data transfer rate was only around 4MB/s! Therefore I started some investigations about the data rates and the processor load of the DNS323. Here are my results:
I tested the transfer rates with one big file (665MB). I know that this only represents the maximum and a large number of smaller files will be much slower. However, it gives some very interesting insghts (network was Gigabt Ethernet):
TEST1: HD_a2 -> local USB external HD via rsync -avuHx 665MB/164s = 4.1MB/s
NAS:
PID PPID USER STAT VSZ %MEM %CPU COMMAND
24716 26061 root S 2468 4% 47% rsync -avuHx --progress /mnt/HD_a2/Videos/Others/Skifahren Sölden 2010.mov .
24718 24717 root R 2508 4% 46% rsync -avuHx --progress /mnt/HD_a2/Videos/Others/Skifahren Sölden 2010.mov .
2493 1 root SW 0 0% 1% [usb-storage]
14375 5 root SW 0 0% 1% [pdflush]
TEST2: HD_a2 -> local USB external HD via cp -a 665MB/40s = 16.6MB/s
NAS:
PID PPID USER STAT VSZ %MEM %CPU COMMAND
24773 26061 root R 1332 2% 55% cp -a /mnt/HD_a2/Videos/Others/Skifahren Sölden 2010.mov .
2493 1 root SW 0 0% 8% [usb-storage]
15856 5 root DW 0 0% 1% [pdflush]
TEST3: NAS -> Laptop (interal HD) via rsyncd 665MB/85s = 7.8MB/s
NAS:
PID PPID USER STAT VSZ %MEM %CPU COMMAND
26374 26348 1000 R 2476 4% 94% /ffp/bin/rsync --daemon --config=/ffp/etc/rsyncd.conf
TEST4: NAS -> Laptop (USB external HD) via NFS (cp -a) 665MB/47s = 14.1MB/s
NAS:
PID PPID USER STAT VSZ %MEM %CPU COMMAND
1996 1 root R 5876 9% 89% /ffp/sbin/unfsd -e /ffp/etc/exports
Laptop:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10298 root 20 0 4892 988 348 D 6 0.1 0:01.89 cp
For reference only, I also checked if the data transfer rates were limited by my USB port on my Laptop or the USB HD itself, but this was not the case. I could transfer with more than 25MB/s:
TEST5: Laptop (internal HD) -> Laptop (USB external HD) via cp -a 665MB/26s = 25.6MB/s
Laptop:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10603 root 20 0 4372 412 280 D 8 0.0 0:02.08 cp
10096 root 15 -5 0 0 0 D 4 0.0 0:01.72 usb-storage
This means that:
1) local transfersbackups shall be done via cp instead of rsync (cp is a factor of 4 faster than rsync; rsync needs much more processing power and therefore limites the data transfer rates to 4MB/s)
2) Network transfers shall be done via NFS and not via rsync server (Note: I only could test unfs from ffp, since I didn't dare to install FW 1.08 with its somehow broken NFS server).
In general it can be concluded, that the DNS 323 is clearly limited in its transfer rates by its processor (ARM 500MHz). Therefore it would be interesting to test the NFS server from FW 1.08 if it is faster or if it has the same speed as unfs.
Cheers,
Andreas
Offline
I had similar rsync and cp -a numbers, going from one HD to the other on the DNS. I sync one disk to the other.
Keep in mind that rsync really shines on subsequent invocations, after you have most of the data already on the destination. If you run your rsync command twice, it will take almost no time on the second invocation as the file is not copied if it is already on the destination. If you run cp -a twice, it will copy the file both times.
Perhaps it is best to use cp -a to make an initial backup, and then use rsync after that? I tried it with one file and it seems to work...
Offline
Fully understood, but it is not clear for me, why rsync really needs 4 times more processing power when copying just one file? It doesn't have to make any checks if it is an update or not.
Just another test I did:
TEST6: NAS -> Laptop (internal) via Samba 665MB/64s = 10.4MB/s
NAS:
PID PPID USER STAT VSZ %MEM %CPU COMMAND
6855 6828 nobody R 5196 8% 62% /usr/sbin/samba/smbd -D
That means, NFS is a bit faster for the user. Why the ARM cpu is not fully loaded (just 62%) is a mytery for me, since the Gigabit Network should have still some reserves and the Laptop was not the bottle neck either.
Offline
rsync is designed to be a low-bandwidth copy utility that also does a lot of data integrity checking. So, it probably is not trying to push large blocks of data around, which would be better for a local copy. Since it is *usually* going over a network, it probably is working with that in mind (smaller blocks). In addition, it is probably computing a checksum or making some other extra effort to improve data integrity, again assuming it is being used over a network. There might be an option to turn off some of this if you know you are running local.
For your other test, I would think that samba has a little more overhead than NFS. Also, the CPU isn't being fully used by smbd because it is waiting on the I/O, both disk and network, is the bottleneck. Rsync would chew up more CPU due to checksum computations.
Offline
oxygen wrote:
or use cp -au (-u for update, overwrites only if target is older or missing)
-au is not supported by ffp, only -a
Offline
I am getting from HD_a2 to HD_b2 14.5MB/s while copying a 3GB iso file.
Offline
I have exactly the same issue. the rsync transfer is very slow, cp is much faster, that's really an issue. I'm wondering whether installing or not a full debian could solve or not this problem...
Offline
Pages: 1