Unfortunately no one can be told what fun_plug is - you have to see it for yourself.
You are not logged in.
I am trying to transfer large files from my Fedora 8 Linux machine to my DNS-322 (firmware 1.05) over a 10/100 Ethernet.
Using rsync, I get ~1.8MB/sec
Using ftp, I get ~11MB/sec
I know that rsync has some more overhead but not 5x of course.
The fact that ftp transfers at 11MB/sec on a 100kbps link indicates to me that the network layer itself is working.
The linux machine is lightly loaded and has a 2.5GHz Intel CPU with plenty of memory.
I am using the following rsync command:
rsync -vaxH --stats --partial --progress -e "ssh -l logname" .....
Any ideas?
Offline
Interesting -- I never realized the ssh part was so costly (or that the DNS-323 was so slow).
Indeed running ps or top shows that ssh is using 85-90% of cpu and rsync 10-15%.
So is my experience of remote rsync transfers being limited to about 1.8MB/sec typical????
Would it be significantly faster to use another (less secure) remote shell such as rsh?
If so are binaries available?
Also, I'm surprised it is really that slow. On my (old) Linux machine with a P4 2.53 GHz processor (no mult-core, no mult-thread), the cpu usage for sshd is about 0.7%.
So is the DNS-323 really 15 times slower than an old P4 machine? I know you can't go by clock speeds but the DNS-323 does run at 500MHz.
Offline
You're running on a LAN that you presumably control - is encryption really necessary? Just for example what would stop someone from gaining direct access to the data on either the host or the DNS-323?
Last edited by fordem (2008-09-18 19:22:54)
Offline
As a workaround, I just tried doing an rsync locally with the dn-323 partition mounted on my Linux machines as a local SAMBA share.
The transfer speed is now about 8.5MB/sec (vs. 1.8MB/sec for ssh and 11MB/sec for ftp) with smbd taking up about 50% of the CPU.
This is all on a 100Mbps Ethernet link.
Rsync is still slower than ftp (by about 30%) -- is this due to protocol ahead? if so it would seem to be a pretty heavy tax.
Note I'm really impressed though that ftp gives 11MB/sec on a 100Mbps link. On my last ftp, I got 89mpbs!!!
(Note while samba mounting is a workaround, I still don't really like it because samba mounting doesn't preserve all the unix ownership & permissions)
Offline
No - encryption isn't necessary - I would be happy to use 'rsh' - just wondering if the binaries are available?
Offline
Just realized that the "best" solution is probably to set up rsyncd on the dns-323 box.
This works like a charm and gives me transfer speeds of 10-11 MB/sec which is great on a 100mbps link.
Just wondering though why a skelaton rsyncd.conf file is not included in the fun_plug. It was a pita (but good learning experience) to write my own...
Thanks!
Offline
puterboy wrote:
Just wondering though why a skelaton rsyncd.conf file is not included in the fun_plug.
man rsyncd.conf
....
EXAMPLES
A simple rsyncd.conf file that allow anonymous rsync to a ftp area at /home/ftp would be:
[ftp]
path = /home/ftp
comment = ftp export area
A more sophisticated example would be:
...Offline
Fair enough. In the spirit of giving something back, here is a more "sophisticated" version of rsyncd.conf that I am using. The protections I am using may be redundant or unnecessary but it seems to work:
secrets file = /ffp/etc/rsyncd.secrets
strict modes = true
pid file = /var/run/rsyncd.pid
# Either create this log directory or place elsewhere or just comment out
log file = /mnt/HD_a2/log/rsyncd.log
use chroot = false
max connections = 5
max verbosity = 2
list = false
timeout = 600
# Refuse (and warn) about xattrs since not supported on dns-323
refuse options = xattrs
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz *.rar
#The following lines are overkill but they basically redundantly shut
#down access except as allowed in the following modules
auth users =
hosts deny = *
read only = true
write only = true
uid = nobody
gid = 501
[root]
comment = Root location on DNS-323 with root permissions
# Allow access to local hosts
hosts allow = 192.168.1.0/255
path = /mnt/HD_a2/
auth users = root
uid = root
gid = root
read only = false
write only = false
[users]
comment = Root location on DNS-323 with user permissions
# Allow access to local hosts
hosts allow = 192.168.1.0/255
path = /mnt/HD_a2/
auth users = root, user
uid = user
gid = users
read only = false
write only = false
Offline
puterboy wrote:
Just realized that the "best" solution is probably to set up rsyncd on the dns-323 box.
This works like a charm and gives me transfer speeds of 10-11 MB/sec which is great on a 100mbps link.
Just wondering though why a skelaton rsyncd.conf file is not included in the fun_plug. It was a pita (but good learning experience) to write my own...
Thanks!
Hey Puterboy
Do you mind sharing the config files - I have setup rsyncd on one of my dns's and am using rsync ( client mode ) on the other to transfer files. So far have only been able to get about 2MB/sec. When I use rsync on my mac with the two dns's mounted as filesystems I get 12MB/sec. Would dearly like to have a solution where I don't need a computer between the devices to transfer ( at a reasonable speed ).
Thanks.
Offline