DSM-G600, DNS-3xx and NSA-220 Hack Forum

Unfortunately no one can be told what fun_plug is - you have to see it for yourself.

You are not logged in.

#176 2009-11-23 10:52:58

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

index monkey wrote:

First KyleK Thanks automatic is cool, and works great with my torrents.

Now I'm trying to get automatic to download .nzb instead of .torrent from the following feed http://www.tvnzb.com/tvnzb_new.rss
Its working but there are two issues:

1 - all the download files are given a .torrent extension (not really a problem, I have written a script to change the extension back to .nzb)
2 - The rss feed has a link to the file "http://www.tvnzb.com/nzb/31746" which in turn is the "my.show.s01.e01.nzb" file. automatic is saving the file as "31746.torrent" so when its queued up in nzbget I have no idea what it is untill I check the automatic log or its downloaded!

Is there a way to have the file named as "my.show.s01.e01.nzb" rather than "31746.torrent"?

I've fixed some bugs in the code that prevented Automatic from behaving just as you describe above. Please check out version 0.4.3 and let me know if it works properly. I'll then put it up in the first post.

Last edited by KyleK (2009-11-23 11:10:55)

Offline

 

#177 2009-11-23 11:46:49

index monkey
Member
From: UK
Registered: 2007-06-14
Posts: 112

Re: [REL] Automatic for funplug-0.5

Hi KyleK,

The correct name is now given to the downloaded file, both filename and extension smile
however the regex filters are no longer applying so its queueing up all the files from the feed.

The logfile shows the following:

automatic.log wrote:

1 feed URLs
[init_regex] PCRE compilation failed at offset 6: unmatched parentheses (pattern: vostfr))my.show)
[init_regex] PCRE compilation failed at offset 9: missing ) (pattern: (?!.*(AC3)
[init_regex] PCRE compilation failed at offset 6: unmatched parentheses (pattern: vostfr))my.other.show)
[init_regex] PCRE compilation failed at offset 9: missing ) (pattern: (?!.*(AC3)

automatic.conf has the following filters:

automatic.conf wrote:

patterns =  {
                     "(?!.*(AC3|HR|x264|720p|French|Hebrew|Portuguese|Brazilian|German|vostfr))my.show"
                     "(?!.*(AC3|HR|x264|720p|French|Hebrew|Portuguese|Brazilian|German|vostfr))my.other.show"
                  }

Thanks again

Last edited by index monkey (2009-11-23 13:46:40)


DNS-323, HW B1, 2 x 2TB WD green, fw 1.08, fun_plug 0.5, transmission, automatic, nzbget newsreader & rsync time machine backup.

Offline

 

#178 2009-11-23 16:43:04

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

Meh, of course I forgot something hmm

Here is the fixed version.

Offline

 

#179 2009-11-23 16:52:59

index monkey
Member
From: UK
Registered: 2007-06-14
Posts: 112

Re: [REL] Automatic for funplug-0.5

Awesome! works prefectly.
A ninja flew down from heaven and sent me all my tv programs, and all I had to do was edit a text file smile

Thanks very much.


DNS-323, HW B1, 2 x 2TB WD green, fw 1.08, fun_plug 0.5, transmission, automatic, nzbget newsreader & rsync time machine backup.

Offline

 

#180 2009-12-04 18:50:05

crs2027
Member
Registered: 2008-10-14
Posts: 46

Re: [REL] Automatic for funplug-0.5

I'm having a bit of trouble understanding how to use this program,

I guess I go to a site that hosts torrents.... isohunt for example.

Search for my fav. TV show?

click the RSS icon, use the page address (copy/paste) in to the config?

restart automatic?

Even if it added the "RSS" how can I decide where it should start.. like series 3, episode 2 and automatically continue on up to today and then in the future?

I've never really messed around with RSS stuff so I might need a bit of an explanation on how to actually use "automatic" smile

Another Q:  (if/when I get it running) I would like to run automatic just *once* during the night for 3 days at a certain time.... the config suggests that it remains running in the background and is triggered from a timer.  Can I disable that and just run it through crontab when I would like it to be run?  that would be very helpful!

Many thanks to KyleK for all your work on transmission and automatic.

and TIA for replies smile

Offline

 

#181 2009-12-04 20:03:33

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

You got the RSS part right. You need an RSS feed that contains direct links to torrent files. Isohunt has that, eztv does it for TV shows only.

Now in order to tell Automatic what to download, you need to specify patterns. Automatic uses the regular expression syntax to define patterns.

So, for example, you want Automatic to catch all torrents from the show "That Show", you would specify a pattern in the config file:

Code:

patterns = { 
            "That.Show"
}

The dot (.) is a placeholder for any character (e.g. a whitespace, or an actual dot).
Now, if you only want your shows in XviD format, you can specify that as well:

Code:

patterns = { 
            "That.Show.*XviD"
}

The expression '.*' means "any number of any characters". So 'xvid' doesn't have to stand directly after 'Show', but anywhere later in the name.
You can specify multiple patterns by putting each on on a separate line and in double quotes. Make sure there is a set of curly braces at the beginning and the end:

Code:

patterns = { 
            "That.Show.*XviD"
            "Some.Other.Show.*720p"
}

You cannot specify which season to download (well, actually, you can, by making the pattern more specifiy), but you cannot tell Automatic to download only episodes 3 onwards of a show (thinking of it, that's not true either. I believe you can create a regular expression that suits these needs as well, but it would be rather complicated smile

To answer your last question: Yes, you can use Automatic with crontab. Simply specify the parameter '-o' on its commandline. It tells Automatic to directly quit after running just once.

Be aware though that most RSS feeds only contain a certain amount of items (e.g. the last 50 episodes). With the abundance of TV shows available, Automatic might not actually 'see' any new episodes when you run it only at a specific time.

Alternatively, you can tell Automatic to add torrents to Transmission in a paused state. They'll be added, but won't start downloading automatically. You'll have to start them by hand.

Offline

 

#182 2009-12-05 17:10:19

crackerjo1
New member
Registered: 2009-12-05
Posts: 4

Re: [REL] Automatic for funplug-0.5

So I might be terribly confused, but I could not find a "getting started" readme file or anything, so maybe someone could help me.  I installed 0.4.3-2 and set up my conf file and started automatic with:

Code:

/mnt/HD_a2/ffp/start/automatic.sh start

and got the message:

Code:

Starting /ffp/bin/automatic -c /ffp/etc/automatic.conf

So I assume it is working.  However I don't see any new torrents being added or anything. 

I have my url set to:

Code:

url = "http://rss.thepiratebay.org/205"

and my patterns as:

Code:

patterns =  { "The.Big.Bang.Theory.S03*.*HDTV.XviD"

            }

Am I missing some important step?  Am I missing some important documentation that talks about this important step and how dumb I am?

Thanks to anyone for the help, it's greatly appreciated.  My conf file is attached in case that might help.


Attachments:
Attachment Icon automatic.conf, Size: 2,819 bytes, Downloads: 462

Offline

 

#183 2009-12-05 17:34:11

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

No, you did everything correctly. Automatic doesn't add anything to Transmission because the RSS feed you mentioned simply doesn't contain any entry for The Big Bang Theory.
As I mentioned 2 posts above yours, RSS feeds usually only contain a fixed number of "latest news", or in this case, latest torrents.
The Pirate Bay feed apparently lists the 50 newest torrents.


One little thing about your regular expression though: "S03*.*" means "S0(the number 3 zero or more times)(anything else)"
As a result, that expression would add torrents from every season to Transmission. If you only want files from S03, get rid of the first asterisk:

Code:

patterns =  { "The.Big.Bang.Theory.S03.*HDTV.XviD" }

Offline

 

#184 2009-12-05 18:16:50

crackerjo1
New member
Registered: 2009-12-05
Posts: 4

Re: [REL] Automatic for funplug-0.5

Thanks for the quick response.  I thought of that, however I looked at the feed in google reader and there are hundreds of  in it, with many Big Bang Theory entries.  Is this just Google being Google or are there actually that many entries available?

And thanks for correcting my pattern

Offline

 

#185 2009-12-05 18:25:42

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

I believe Google reader saves unread items for you, that's why you see more  than are actually in the file.
See, the way RSS works is this: Your tool (in this case Google Reader) literally downloads the provided URL after a given interval (like 30 minutes). The file it receives is an XML file, which usually contains once <channel>, and inside this channel, there are a number of <item> entities.

You can try it yourself, on the NAS:

Code:

$ wget -O 205.xml http://rss.thepiratebay.org/205

Then open the XML in the browser of your choice.

Offline

 

#186 2009-12-05 18:37:40

tanstaafl1963
Member
Registered: 2009-08-22
Posts: 58

Re: [REL] Automatic for funplug-0.5

parser error : Start tag expected, '<' not found

I wonder if there's just something wrong with  my url feed source.  I have several shows in an RSS feed from showrss.kamorra.  Yesterday ALL of season 2 for Burn Notice appeared in the feed, and then in automatic.   Not wanted, because I had them all.  And now, I'm getting the message below from my automatic.log file.  NOTHING has changed in the .conf file in months.

*------------------- automatic.conf setting ---------
url={

"http://showrss.karmorra.info/rss.php?user_id=5476&hd=null&proper=null"

=*------------------- automatic.conf setting



Cat of the automatic.log file.

/mnt/HD_a2/logs # cat automatic.log
1 feed URLs
Read 35 patterns from config file
Restored 70 old entries
Entity: line 1: parser error : Start tag expected, '<' not found
brick
^
Error: Unable to parse input data!

-----------------------

I have just stopped both transmission and automatic, restarted transmission then automatic and the same issue occurs.  I'm assuming this error has something to do with why my 3 shows that should download, Monk and 2 eps of Dollhouse are not downloading.. 

Any help would be appreciated.


DNS-323 FW 1.09
2 2-TB Drives
Automatic  0.7.0 |  Transmission 2.42

Offline

 

#187 2009-12-05 18:49:46

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

That error indicates that the RSS feed Automatic downloaded was not properly formatted (i.e., not a proper XML file).
Considering the fact that a complete show showed up in Transmission over night, and now the problem with the feed, I would assume that the problem is on their end.

When I tried to download the XML from your link for the first time, I got an error, but about 5 minutes later it worked.

Let me know if the problem persists.

Offline

 

#188 2009-12-05 18:55:42

tanstaafl1963
Member
Registered: 2009-08-22
Posts: 58

Re: [REL] Automatic for funplug-0.5

KyleK wrote:

That error indicates that the RSS feed Automatic downloaded was not properly formatted (i.e., not a proper XML file).

The odd thing is when I added that url to my google reader, it came up fine. It initially showed the gazillion Burn Notice episodes, but when I refreshed, it came back with the Monk and 2 Dollhouse.

KyleK wrote:

When I tried to download the XML from your link for the first time, I got an error, but about 5 minutes later it worked.

Did you use the wget you mentioned previously?  When I tried that it is taking several minutes...  I guess it does sound like it's their side....

Thanks for the quick response...


DNS-323 FW 1.09
2 2-TB Drives
Automatic  0.7.0 |  Transmission 2.42

Offline

 

#189 2009-12-05 18:55:51

crackerjo1
New member
Registered: 2009-12-05
Posts: 4

Re: [REL] Automatic for funplug-0.5

Thanks Kyle, that makes sense.  Google just stores way too much information for me to comprehend haha.  Looks like I learned something today.

One last question, is there a way to exclude terms from the pattern?  Most of these feeds seem to have more than one of the same episode so I want to exclude the one with 720p for example.

Offline

 

#190 2009-12-05 18:58:54

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

Yes. Please take a look at the INFO 2 paragraph in the very first post of this thread. It is explained there how to exclude torrents with certain terms.

Offline

 

#191 2009-12-05 19:09:06

crackerjo1
New member
Registered: 2009-12-05
Posts: 4

Re: [REL] Automatic for funplug-0.5

Sorry.. Another case of RTFM.  Thanks again

Offline

 

#192 2009-12-06 01:36:10

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

Automatic 0.4.4 is out. It only fixes a couple of issues, nothing special.

Just a heads-up: Next on the list is logging output messages to a file while running in daemon mode (already done), and I'm trying to make Automatic work with private torrent sites that require cookies on their RSS feeds. Fingers crossed! smile

Offline

 

#193 2010-01-07 18:49:02

nyunyu
Member
Registered: 2009-05-03
Posts: 44
Website

Re: [REL] Automatic for funplug-0.5

I want to download and install this however this is what I get instead.

/mnt/HD_a2 # funpkg -i Automatic-0.4.4.tgz
FATAL: Automatic-0.4.4.tgz: File not found


Can you give me a new link for this package?

Thank you.


DNS323 + 2x1TB WD GREEN + Fun Plug 0.5 (Fonz) + Transmission 1.92 + Automatic 0.6 = GREAT!
Looking for Auto Direct Download script if possible smile

Offline

 

#194 2010-01-10 00:35:19

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

@nyunyu: Have you actually downloaded the .tgz package from the first post of this thread? You must be in the same folder as that file when you issue the funpkg command, otherwise it will fail (because it can't find the package it's supposed to install).

Offline

 

#195 2010-01-10 06:12:10

nyunyu
Member
Registered: 2009-05-03
Posts: 44
Website

Re: [REL] Automatic for funplug-0.5

KyleK wrote:

@nyunyu: Have you actually downloaded the .tgz package from the first post of this thread? You must be in the same folder as that file when you issue the funpkg command, otherwise it will fail (because it can't find the package it's supposed to install).

duh! I didn't bother that link because it shows as ffp0.5 and I already have that. Sorry for that man.
Installed and success.

Anyone want to share their conf file? especially in "patterns" part. I just don't have the clue how to set this part.

Are these correct?
patterns =  {
              "(?!.*720p)Greys.Anatomy"
              "(?!.*HDTV)Greys.Anatomy"
              "(?!.*HDTV)Heroes"
              "(?!.*720p)House"
              "(?!.*HDTV)House"
              "(?!.*HDTV)My.Name.is.Earl.S04"
              "(?!.*HDTV)Reaper"
              "(?!.*XviD)The.Girls.Next.Door"
            }

Last edited by nyunyu (2010-01-10 10:35:20)


DNS323 + 2x1TB WD GREEN + Fun Plug 0.5 (Fonz) + Transmission 1.92 + Automatic 0.6 = GREAT!
Looking for Auto Direct Download script if possible smile

Offline

 

#196 2010-01-10 20:46:17

nyunyu
Member
Registered: 2009-05-03
Posts: 44
Website

Re: [REL] Automatic for funplug-0.5

KyleK, how to check if Automatic is running? I checked the torrent folder and there was nothing inside.
Also, is there a plan to develop a GUI for automatic? Like maybe if we could go to NAS_IP_ADDRESS:PORT and shows a page where all details like status, schduled, etc..


DNS323 + 2x1TB WD GREEN + Fun Plug 0.5 (Fonz) + Transmission 1.92 + Automatic 0.6 = GREAT!
Looking for Auto Direct Download script if possible smile

Offline

 

#197 2010-01-11 00:27:44

Muppet
Member
Registered: 2008-05-21
Posts: 44

Re: [REL] Automatic for funplug-0.5

Code:

patterns =  { 
              "Heroes.*S0[3-9]E.*720p"
              "House.*S0[6-9]E.*720p"
            }

1st gets everything with Heroes in it from series 3-9 at 720p. Same for house except starting at season 6. The pattern you use depends on the name of the torrent but that style works with scene releases.

.. and scroll up just a bit on this very page where Kyle has already explained this stuff.

Type "top" to see if its running.
I don't leave it running all the time because it inevitably crashes after several days running. Better to regularly schedule it via cron using the -o flag. eg..

Code:

8 * * * * /mnt/HD_a2/ffp/bin/automatic -o -c /ffp/etc/automatic.conf

Last edited by Muppet (2010-01-11 00:32:53)

Offline

 

#198 2010-01-11 09:17:12

nyunyu
Member
Registered: 2009-05-03
Posts: 44
Website

Re: [REL] Automatic for funplug-0.5

Muppet wrote:

Code:

patterns =  { 
              "Heroes.*S0[3-9]E.*720p"
              "House.*S0[6-9]E.*720p"
            }

1st gets everything with Heroes in it from series 3-9 at 720p. Same for house except starting at season 6. The pattern you use depends on the name of the torrent but that style works with scene releases.

.. and scroll up just a bit on this very page where Kyle has already explained this stuff.

Type "top" to see if its running.
I don't leave it running all the time because it inevitably crashes after several days running. Better to regularly schedule it via cron using the -o flag. eg..

Code:

8 * * * * /mnt/HD_a2/ffp/bin/automatic -o -c /ffp/etc/automatic.conf

I checked and automatic started. But somehow there is no activity.
I just confused right now. Am I doing the right step or did I missing something?
Also, is it possible to use 2 rss urls?


DNS323 + 2x1TB WD GREEN + Fun Plug 0.5 (Fonz) + Transmission 1.92 + Automatic 0.6 = GREAT!
Looking for Auto Direct Download script if possible smile

Offline

 

#199 2010-01-11 09:20:18

Muppet
Member
Registered: 2008-05-21
Posts: 44

Re: [REL] Automatic for funplug-0.5

You can do multiple URLs the same way as the patterns:

Code:

url = {
       "http://www.blahblahblah"
       "http://bingbongbing"
       "http://wahwahwah"
      }

Offline

 

#200 2010-01-11 10:04:44

nyunyu
Member
Registered: 2009-05-03
Posts: 44
Website

Re: [REL] Automatic for funplug-0.5

Ok, all working now. I noticed that the reason automatic shows no activities was because of the rss link I used.
I was using this link: http://www.tvnzb.com/tvnzb_new.rss and it seems automatic can't retrieve the torrent from that link.


DNS323 + 2x1TB WD GREEN + Fun Plug 0.5 (Fonz) + Transmission 1.92 + Automatic 0.6 = GREAT!
Looking for Auto Direct Download script if possible smile

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2010 PunBB