Unfortunately no one can be told what fun_plug is - you have to see it for yourself.
You are not logged in.
Hello All
Abstract:
======
I did some stupid things. Basically all my files in my directory tree are all over the place and disorganized. Wondering if anyone has advice or knows some good utilities for cleaning up the mess (deleting redundancy, identifying files not moved, etc.).
Source of the problem:
===============
I mounted the DNS-323 with smbmount and started running many of mv commands in parallel (ie mv <directory/file ...> <directory> &) to start setting up the DNS-323 as my backup. Issues with smbmount forced the mv command to fail randomly. Thanks to this forum and the associated wiki I managed to get the device mounted using NAS. smbmount is not an issue anymore, the current problem is restarting all the failed mv commands.
Because I ran all the mv commands in parallel I am not sure what commands I ran, so I decided to use rsync. Now again I didn't learn from my past mistakes. I ran many rsync commands in batch mode (ie batch rsync <source> <dest> <EOT>) so that the files on the NAS would reflect the same as my computer. The stupid thing is that A) I did it all in parallel so I lost track of what I was copying B) I screwed up some rsync commands so the <dest> is not the intended destination (I was really tired at that point) C) I didn't rsync all the files, so I don't know what files on the local system I can delete.
The other stupid thing is the directory tree on the NAS is not the same as the one on the local system. The local system had many HD's so I have many mount points, and because of disk space issues many files of common purpose were spanned across multiple folders.
Prognosis:
=======
- I have files on the original system that I want moved to the NAS, but I don't know what ones;
- I have many duplicate files on the NAS that are in random locations;
- I have files that are on both the local system and the NAS, I want to delete the duplicates on the local system
- because of the smbmount failures some duplicates aren't the same hence simple diffs of ls outputs is not good enough
Thoughts on the solution
================
Currently I am running a bunch of scripts that are getting md5sums of files on the NAS (and eventually the local machine). I am hoping that I can use them so compare files and identify the problems and resolutions.
I spent some time thinking about how I would use the resulting md5sums. The hope was to use some simple code to parse the results and print out where the mistakes happened. However, after thinking about the algorithms in the code, I am starting to think that simple code might not solve this problem.
I don't want to spend a bunch of time coding a program to resolve the problem. Yet, at the same time the mess might be too complicated for a mere mortal to solve with just the basic Unix commands (mv rm cp diff md5sum ... etc.). So I really don't know what I should do.
My Request
=======
1) Does anyone have advice on the best approach to fix this mess?
i) I have already learned not to fix problems when tired
2) Are there any utilities that can help clear up this mess?
Offline
PS I understand that this might not be the best forum, If people know some better forums please suggest.
Thanks
Offline
There are some dupe finders that work on the DNS-323. I have used them. I will have a look later in the day and post back.
Offline
ok, see this thread http://dns323.kood.org/forum/viewtopic.php?id=2771
One additional thing that I did (over and above running them on the dns-323 itself), was to do the file comparing via a PC, so I:
enabled unfsd.sh in ffp/start
then mounted the dns-323 from a pc running ubuntu with fslint (see http://www.pixelbeat.org/fslint/) installed (sudo apt-get install fslint)
and ran fslint. I think from memory you can tell it to scan multiple places - so if you have the pc hdd mounted and the dns-323 mounted you should be able to do a compare.
Alternatively, pull the hdd from the dns-323 and insert into the PC with ubuntu (or your favourite linux distrib) and compare from there. fslint has a gui which allows click and delete.
You can also use fdupes. See the post quoted above.
Both use md5 checksums for file compare, but I think that fslint uses a more advanced algorithm (i.e. not only md5).
If you run them on the dns-323 itself remember the temp space issue I mentioned in the post - was easy enough to get round.
These will not spot directory differences - they are file level comparers, but hopefully this will help you tidy things up a bit. They certainly helped me.
Offline
Hi luusac
Thanks a tonne for the recommended SW. Haven't started using it seriously yet, but just playing around on smaller directories I think fslint will be very helpful. It is very close to what I was trying to accomplish with all my md5sums. So that will save me a lot of work sorting through it all.
On the DNS I have things mounted NFS now, and it seems to be fairly stable so far. Thanks for the help!!!
Take Care
Offline
you are welcome, it saved me an awful lot of time when I was trying to merge backups taken from different machines at different times over the years - I had more duplicates than you can shake a stick at!
Offline