DSM-G600, DNS-3xx and NSA-220 Hack Forum

Progress.

New added function.

Script will cycle after a set pause period unless you put a file in the watch directory (default name 'cycle').  It will check for this file every XXX seconds and initiate a cycle of jobs if found.

i.e.
You set the standard cycle time to be every hour.  You set the cycle check time to be every 5 minutes.  All jobs in the script will run 1 hour after the last cycle unless it finds a cycle file.  It looks for the cycle file every 5 minutes to save you having to wait for the hour cycle to pic up new .torrent files or move completed items.

Outstanding items.

Check and rewrite if need the max running processes function.
Need to review the code for any obvious optimisations.
Need to write install script.
Need to write instructions.
Need to package.


New amendment to do.

Each cycle it will set the bandwidth even if it does not need setting.  Need to add a check if possible to change the bandwidth only if it needs changing.

Notes.

I am currently thinking of leaving out the max processes function.  It was added for the WD World book which seems to have very little processing ability and so could only manage 1 torrent without slowdown.  I have yet to see the DNS-323 slowdown with 10+ torrents downloading.  I may leave it in but disable it and release later if there is a demand for it.

RB

More of a place holder before bed than an update.....

Need to also write the code to manage the log file generated. Initial thought is to count the number of lines after a certian file size is reached and to -10 lines from the top.  Check to be done with every cycle.

RB

Hi BikNorton,

I was thinking along the same lines what with the uShare and the Trans / Clutch util scripts both doing the same sort of checking.

Maybe having a /mnt/HD_a2/triggers directory for checking for trigger files.
Having a /mnt/HD_a2/triggers/trigger_files directory for 'blank' trigger files so the user just has to copy from here to the trigger directory.
Having a /mnt/HD_a2/triggers/scripts directory for the scripts to trigger.
I don't think a single recycle script will be possible at this point as the scripts I have put together all have the ability for different timings for recycling and certianly this trans/clutch script does a lot more than just wait for a new torrent.  Setting bandwidth and moving completed files need to be checked for.  Saying that we could have a /mnt/HD_a2/triggers/perm_triggers folder containing trigger files with a one line entry with 'full-cycle=600 trigger-check=60".

When the start script runs for a 'job' that has been triggered it checks /mnt/HD_a2/triggers/trigger_files and touches and trigger file is it one is missing (people are bound to move rather than copy by mistake once or twice).

The trigger check script will monitor the trigger directory and fire scripts from the /mnt/HD_a2/triggers/scripts.  WHy I prefer putting them here is as it will make the system pretty independent from the ffp stuff and so changes Fonz may choose to implement should have minimal effect and it will also help not to clutter up the /ffp/start directory.  You would only need a trigger system start script in the /ffp/start folder to get the whole thing going.  There is no reason the scripts in the trigger_scripts diectory could not just call the scripts in the /ffp/scripts folder if required.

The Transmission & Clutch util script has been running for a couple of days now and I have found and removed a couple of bugs.  I may, if I have time, today put a quick installer together and package so people can take a look.  It will be a beta with all risks involved.  It seems to work fine on my NAS so should be fine for others.

RB

Ok,

First bug reported by me.

If you run the script multiple times it may shut down transmission as it reports there is nothing to do.

If this happens to you then dont panic just kill any instances you have of the script running and then start it again once with the script in ffp/start and it will pickup the torrents already in transmission (if there are any) and start transmission for you.

This is unlikely to happen and happened to me because as well as the start script for this in the ffp/start directory and I also had a line starting it in the transmission start script.

With the install you should not have this issue.

RB

Should this script work with latest Transmission?

This doesn't work for me, log says localhost:9091 responded: "success" but the torrent I have in torrent folder does not begin to download. If I add the torrent manually, it does remove it after done seeding.

Here's what I have edited:
################
# Configuration
################

SAMBA_CHOWN=nobody

# Funplug base directory Path.
BASE_SYS=/ffp

# Transmission start / stop script directories.
TRANS_SCRIPT=/start

# Share base directory
BASE_SHARE=/mnt/HD_a2

# Transmission install directory.
TRANSMISSION_HOME=$BASE_SHARE/.transmission-deamon

# path to transmission-remote
# if you built transmission from source, you'll need to change this
TRANSMISSION_REMOTE=$BASE_SYS/bin/transmission-remote

# command to start transmission when it is not running
TRANSMISSION_START=$BASE_SYS$TRANS_SCRIPT'/transmission.sh start'

# command to stop transmission when it is running
TRANSMISSION_STOP=$BASE_SYS$TRANS_SCRIPT'/transmission.sh stop'

# transmission working directory
# note that if you're using the transmission startup script (/etc/init.d/transmission.sh),
# this should should be the same as in it
WORK_DIR=$BASE_SHARE/Torrents

# the directory to watch for new .torrent files
# leave blank if you don't want to use one
WATCH_DIR=$BASE_SHARE/Torrents

# the directory to put completed downloads into
# leave blank if you want to keep seeding completed downloads in the working directory
COMPLETEDIR=

# whether or not to remove completed downloads automatically at all
# if set to 0, torrents are seeded indefinitely until you remove them manually
# if set to 0, the following two settings won't have any effect on anything
REMOVE_COMPLETED=1

# the seeding ratio you'll want to be seeding the downloaded files to before removing them
SEED_UNTIL_RATIO=1.5

# how many torrents should be downloading at the same time
# recommended to keep this at 1, since mbwe does have so little memory and so slow cpu
MAX_SIMULTANEOUS=5

# how often the watch directory should be checked, in seconds
# by default 5 minutes (=300 seconds)
FREQUENCY=300

# whether or not to quit transmission, if there are no torrents in the queue.
# if quit, it will be restarted when something is found in the watch directory
# if you're planning on adding torrents also by other means (with clutch, for example)
# than using the watch directory, set this to 0
QUIT_TRANSMISSION_WHEN_INACTIVE=0

# bandwidth scheduling
# list schedules in following format
# SCHEDULE_DOWNLOAD[[hour to start]]=[download speed in kb]
# SCHEDULE_UPLOAD[[hour to start]]=[upload speed in kb]
# use 24h clock hours in the hours
# 0 as speed means unlimited
# the following examples limit the bandwidth during daytime ( 8 - 23 ) and set them unlimited for night
# note that your upload speed shouldn't be set to unlimited, but to
# about 90% of your total upload bandwidth in order to keep it from limiting your download speeds also...
SCHEDULE_DOWNLOAD[8]=0
SCHEDULE_UPLOAD[8]=0
SCHEDULE_DOWNLOAD[23]=0
SCHEDULE_UPLOAD[23]=0

# logging options
LOG_ENABLED=true
LOGGER="$BASE_SYS/bin/logger -t trans_utils"

############################################
# end of configuration
# you shouldn't need to edit anything below
############################################

Last edited by headman (2008-11-17 07:36:16)

To be clear with everyone, I made very little modification to the original torrent_helper to produce trans_utils.sh.  I provided a trans_utils_control.sh modelled after the /ffp/start/samba.sh and kept it very simple.  Any issues with the scripts that aren't related to the
configuration variables and the control script are most probably existing in the original script too.

There does appear to be some interesting issues with the MAX_SIMULTANEOUS setting.  The WEB UI reports that multiple torrents are downloading, but when you execute transmission-remote -l, the right number of torrents are Downloading!  Not sure why this mismatch, but it kind of seems like its an issue with transmission-daemon and transmission-remote.

BishopPell, Thanks for your script. I've seen it fill my download list automatically - super! What I do see is that all files are put in the download queue at once (webUI). I set the max. simultaneous downloads to 2, since my modem chokes easily on torrent-traffic. But the webUI shows all as downloading, and transmission-remote -l gives at least five active torrents (only three are idle). Is something not quite OK with the max. simultaneous check?

Also, I see nothing in the logfile. It is there, but there's only a "localhost:9091 responded: "success"" every now and then. Does the systemlog need enabling for this to work? It might reveal a little more what the script is doing and where the max. simultaneous boundary kicks in (if ever).

::EDIT::

Never mind about the logfiles - I found them already. They were in /var/log, NOT in /ffp/var/log. But anyway, the messages only show "Nov 23 15:26:47 CH3SNAS user.notice trans_utils[26370]: Checking for schedule, torrents, etc...". No mention of a max. simultaneous, which should be there.

Any idea?

::EDIT::

Another few hours revealed some "max. simultaneous" messages in the log, so there must be something else going on. I think I know what it is: The running list is made using:

RUNNING_TORRENTS_LIST=`echo "$TORRENT_LIST" | grep Downloading`

But in transmission-remote -l I see many torrents having "Down & Up". This is probably new phrasing in 1.40, so I will try doing this tonight:

RUNNING_TORRENTS_LIST=`echo "$TORRENT_LIST" | grep Down`

I will keep you updated on progress.

::EDIT::

Couldn't resist...

I searched the sourcecode of 1.34 and 1.40 for the part where the status is set, and found it in "remote.c". In 1.34 these statusmessages are possible:

case TR_STATUS_CHECK_WAIT: return "Will Verify";
        case TR_STATUS_CHECK:      return "Verifying";
        case TR_STATUS_DOWNLOAD:   return "Downloading";
        case TR_STATUS_SEED:       return "Seeding";
        case TR_STATUS_STOPPED:    return "Stopped";
        default:                   return "Error";

For Transmission 1.40 this status presentation received some attention, and the following statuses are possible:

case TR_STATUS_CHECK_WAIT:
        case TR_STATUS_CHECK: {
            const char * str = status == TR_STATUS_CHECK_WAIT
                             ? "Will Verify"
                             : "Verifying";
            double percent;
            if( tr_bencDictFindDouble( t, "recheckProgress", &percent ) )
                tr_snprintf( buf, buflen, "%s (%.0f%%)", str, percent*100.0 );
            else
                tr_strlcpy( buf, str, buflen );

            break;
        }

        case TR_STATUS_DOWNLOAD:
        case TR_STATUS_SEED: {
            int64_t fromUs = 0;
            int64_t toUs = 0;
            tr_bencDictFindInt( t, "peersGettingFromUs", &fromUs );
            tr_bencDictFindInt( t, "peersSendingToUs", &toUs );
            if( fromUs && toUs )
                tr_strlcpy( buf, "Up & Down", buflen );
            else if( toUs )
                tr_strlcpy( buf, "Downloading", buflen );
            else if( fromUs ) {
                int64_t leftUntilDone = 0;
                tr_bencDictFindInt( t, "leftUntilDone", &leftUntilDone );
                if( leftUntilDone > 0 )
                    tr_strlcpy( buf, "Uploading", buflen );
                else
                    tr_strlcpy( buf, "Seeding", buflen );
            } else
                tr_strlcpy( buf, "Idle", buflen );
            break;
        }

So there is a new status "Up & Down" as well as a status "Uploading", but I think the above fix (grep "Down") will do the trick. Results due tonight!

::EDIT::

Another day, another edit. Please notify me if you are getting tired of my ramblings ;-)

As any of you (probably more capable bash scripters) would have noticed immediately: the 'column header' line (the one with ID Done ETA Up Down Ratio Status Name) also contains the phrase 'Down'. So the script log was filled with attempts of trying to stop the torrent 'ID Ratio Up ...'. Please be gentle and don't laugh too loud (chuckles allowed).

Tonight I will try another grep attempt, this time specifying the exact names like this:

RUNNING_TORRENTS_LIST=`echo "$TORRENT_LIST" | grep -E 'Up \& Down|Downloading|Uploading|Seeding'`

Please note I add the Uploading status as well, since I use this script to cap the number of active torrents, not just the downloading ones. My modem is of a fickle type, choking easily when too many connection (attempts) are made.

This line I've also edited, again with 'Uploading' and 'Seeding' statuses included.

get-torrent-filename() { echo `echo $1 | awk 'BEGIN{FS="(Up & Down|Downloading|Uploading|Seeding|Idle)[ ]+"};{print $NF}' 2>/dev/null`; }

When (if) I get it working right with Transmission 1.40 I will attach the resulting script here. You can do a diff with the 'base version' from BishopPell to see what has been changed, and you can keep what works for you. But now to get it working...

::EDIT::

It seems to work now. I'll attach the resulting script here, for anyone to use / modify or whatever. I've altered functionality slightly so the script caps active torrents, not just downloading ones. This is because my modem really doesn't like many torrents and the frequent connections that go with them, even uploading / seeding ones. You can change it back to just capping downloads easily - it's in the comments sections of the script.

I also added a slight delay (5 seconds) in the automatic-torrent-starting section. When started, all torrents are Idle until a connection is made. The while-loop however goes so fast that it starts all torrents since none of them were active quick enough. The next run of the script will fix this, but since my modem is really sensitive to many connections, I added a "sleep 5000".

Last edited by DFreeze (2008-11-28 13:38:29)

---

Attachments:
trans_utils.sh, Size: 12,968 bytes, Downloads: 460

DFreeze wrote:

It seems to work now. I'll attach the resulting script here, for anyone to use / modify or whatever. I've altered functionality slightly so the script caps active torrents, not just downloading ones. This is because my modem really doesn't like many torrents and the frequent connections that go with them, even uploading / seeding ones. You can change it back to just capping downloads easily - it's in the comments sections of the script.

I also added a slight delay (5 seconds) in the automatic-torrent-starting section. When started, all torrents are Idle until a connection is made. The while-loop however goes so fast that it starts all torrents since none of them were active quick enough. The next run of the script will fix this, but since my modem is really sensitive to many connections, I added a "sleep 5000".

Not working for me and it does not generate any log except in /ffp/var/log/trans_utils.sh.log:
transmission-remote: (localhost:9091) Couldn't connect to server
transmission-remote: (localhost:9091) Couldn't connect to server
transmission-remote: (localhost:9091) Couldn't connect to server
transmission-remote: (localhost:9091) Couldn't connect to server
Stopping transmission-daemon

I have .torrent files in /mnt/HD_a2/Torrents/new and when completed move them to /mnt/HD_a2/Torrents/done and these I changed to your script. The old script worked fine.

It's working now thanks to your fixes! Great job The logging still does not work tho

Last edited by headman (2008-11-29 07:58:37)

It did not occur to me to check inside the message, Linux newbie in the house! :-)

If there's nothing downloading, it deletes the torrents I have in the folder that it checks for new torrents and it never starts the transmission client if it's been stopped. Any ideas why this happens? I had to change these lines to keep it running:

REMOVE_COMPLETED=0
QUIT_TRANSMISSION_WHEN_INACTIVE=0

DFreeze wrote:

Don't know what happens - obviously the script has problems starting transmission back up. I would have to test if transmission is started automatically over here, to verify if the script is correct. Will try that and report back.

Is the path to transmission.sh (/ffp/start/transmission.sh) set correctly?

I have not changed it:
# command to start transmission when it is not running
TRANSMISSION_START=$BASE_SYS$TRANS_SCRIPT'/transmission.sh start'

# command to stop transmission when it is running
TRANSMISSION_STOP=$BASE_SYS$TRANS_SCRIPT'/transmission.sh stop'

And it is weird because it can run the stop command automatically. Any ideas why it deletes the new torrents and does not start to download them?