DSM-G600, DNS-3xx and NSA-220 Hack Forum

hp1 · 2010-07-08 07:54:45

I've searched but I haven't found a similar setup to what I'm trying to accomplish. I'm using a Dlink323 as a backup for a time capsule as well as a linux file server. I'm mounting the time capsule via smb and the service via NFS, and using rsync to manage the file replication

Using rsync over SMB to the time capsule I get between 1.2 - 1.5 Mbps this is connected to a gig switch and the time capsule also connected at gig speeds.

Using rsync over NFS to the server I get between 500-800kps. The server is connected to a gig switch and I can copy files at 8-9Mbps, so it isn't a bottleneck.

So my question is, are these speeds normal? Is there anything else I can tune on the dlink? Is anyone else using it as a backup device and have any pointers?

thanks in advance for any suggestions or advice.

chiron · 2010-07-14 10:52:46

I've also done some tests:

I have 2 hard drives of 1To and firmware 1.08, and nfs shares activated through the web interface.

I've opened an ssh connection to the nas and done the following:

- rsync of a directory between the discs using mounting points:performance horrible, about 1 mo/s
- a single cp, performance ok. unfortunately the -v switch is not supported. So I can only guess, but it's much faster.

Accessing the nfs shares does the same. So it seems that rsync has a problem.

So do anybody have a clue to be able to use the DNS-323 as a backup solution? I'm going to try other solutions (freefilesync for example)

karlrado · 2010-07-14 20:21:15

root@Toaster:/data# du -sh Audiobooks/
482M Audiobooks/
root@Toaster:/data# time rsync -a Audiobooks/ /backup/tmp

real 2m58.254s
user 1m15.460s
sys 0m47.120s

or about 2.8 megabytes per second.
I am running rsync on the DNS-323, not on some other computer that has mounted my DNS-323 disks.

The folder contains 66 files in about the 6-12 MB range each.

/data is a symlink to /mnt/HD_a2
/backup is a symlink to /mnt/HD_b2

I am using the Optware tools - no fonz fun plug. FW 1.7

There are a couple of things about rsync to consider:

1) It is designed for remote synchronization, hence its name: rsync. Sure, it works on local filesystems, but it's not particularly tuned to do that.

2) After you've done an initial rsync of your files, rsync is pretty good at subsequent synchronizations. That is where is shines.

FWIW, I use rsnapshot, which is really just a wrapper around rsync. I've got about a half TB of data on my main data disk. I use rsnapshot to copy it over to the other disk. My data does not change much, so when rsnapshot runs every four hours, it doesn't need to do much and finishes in a few minutes. Sure, if I dump 100 GB on my data disk, the next sync takes a long time. But I don't really care; I'm not sitting there waiting for it.

Yeah, the DNS-323 isn't super-speedy. But I'm making it work for what I need it to do.

If I were updating a few hundred GB of data a day on the DNS-323, I don't think that this approach would cut it.

Loose Gravel · 2010-07-19 16:33:53

I wouldn't use rsync to sync via nfs or smb at all. Rsync isn't built for that.
Syncing a large updated file would require to
a) read large file locally
b) calculate crc's locally (cut file in chunks, calculate crc per chunk)
c) read large file via network (your loose your speed here)
d) calculate crc's locally
e) find difference in crc's
f) write difference via network (often whole file has to be written - you loose your speed here also)

Try to run rsync on one machine and rsync daemon on the other.
Updating goes like this:
a) read large file locally on machine 1, calculate crc's
a) read large file locally on machine 2, calculate crc's (this in done in parallel on two machines)
b) send crc's via network from machine 2 to machine 1 (only a few bytes)
c) find difference in crc's
d) send delta to machine 2 via network (delta only)
e) write difference to file locally.

Initial filecopy requires copying all files + Overhead, so rsync is slower (because of the overhead) on the first copy. Where rsync really shines is updating.

PS: I use the dns-323 as a time capsule running rsnycd (deamon), and running rsync on my working machines (windows, linux)

skydreamer · 2010-07-19 22:31:49

I am using rsync for synching data between a linux box(master) and DNS-323(mirror) and the speed is much higher than what is being reported here. The files are being written to DNS-323 mounted on linux over nfs:
sent 8646116632 bytes received 11542 bytes 6359785.34 bytes/sec

rsync is running on the linux machine.

chiron · 2010-07-23 17:49:16

Loose Gravel wrote:
I wouldn't use rsync to sync via nfs or smb at all. Rsync isn't built for that.

Well, what the "R" of rsync stands for? I think it's remote, isn't it? ;-) So it should be able to do this job remotely...
The post of skydreamer makes me think that I've been playing too much with this DNS-323 and funplugs. the top command shows a high usage of the processor.

The nfs shares do also cause the nas to hang unexpectedly, and require an A/C unplug / replug reset. Having to do that on a linux system is not confortable...

I've installed recently a real ubuntu server on an unused laptop, and configured on it: ftp server, upnp server, print server, and even scan server (which is not possible on the DNS-323). I'm going to reformat / reset this nas to factory defaults, disable all gadgets and dedicate it only to storage of backups. Shall it also fail in this duty, I will take the disks away, put them in usb boxes, and use the NAS as a paperweight.

Loose Gravel · 2010-07-26 13:44:58

chiron wrote:
Well, what the "R" of rsync stands for? I think it's remote, isn't it? ;-) So it should be able to do this job remotely...

Yes, R ist for "remote" :-)
But R ist not for "network share".

Reread my post. The first example is "network share" (e.g. via nfs ). rsync is running on the local machine only. rsync doesn't know that one directory is local and the other is remote (via network share). So files are considered as local files, which is not the case, which gives bad performance. rsync IS ABLE to do this job, but with bad performance, as you have correctly noted.

The second example is truly "remote". You have one instance of rsync running locally on your machine and a second one running locally on the other machine ("remote" from your point of view). Two (!) running rsyncs. Files are NOT copied via fns or samba, but via an optimized rsync protocol. No nfs or samba needed! This is what rsync is BUILT FOR.

Last edited by Loose Gravel (2010-07-26 13:45:53)

DSM-G600, DNS-3xx and NSA-220 Hack Forum

Announcement

#1 2010-07-08 07:54:45

Rsync over NFS & SMB Terrible Performance?

#2 2010-07-14 10:52:46

Re: Rsync over NFS & SMB Terrible Performance?

#3 2010-07-14 20:21:15

Re: Rsync over NFS & SMB Terrible Performance?

#4 2010-07-19 16:33:53

Re: Rsync over NFS & SMB Terrible Performance?

#5 2010-07-19 22:31:49

Re: Rsync over NFS & SMB Terrible Performance?

#6 2010-07-23 17:49:16

Re: Rsync over NFS & SMB Terrible Performance?

Loose Gravel wrote:

#7 2010-07-26 13:44:58

Re: Rsync over NFS & SMB Terrible Performance?

chiron wrote:

Board footer