DSM-G600, DNS-3xx and NSA-220 Hack Forum

KyleK · 2009-11-23 10:52:58

index monkey wrote:
First KyleK Thanks automatic is cool, and works great with my torrents.

Now I'm trying to get automatic to download .nzb instead of .torrent from the following feed http://www.tvnzb.com/tvnzb_new.rss
Its working but there are two issues:

1 - all the download files are given a .torrent extension (not really a problem, I have written a script to change the extension back to .nzb)
2 - The rss feed has a link to the file "http://www.tvnzb.com/nzb/31746" which in turn is the "my.show.s01.e01.nzb" file. automatic is saving the file as "31746.torrent" so when its queued up in nzbget I have no idea what it is untill I check the automatic log or its downloaded!

Is there a way to have the file named as "my.show.s01.e01.nzb" rather than "31746.torrent"?

I've fixed some bugs in the code that prevented Automatic from behaving just as you describe above. Please check out version 0.4.3 and let me know if it works properly. I'll then put it up in the first post.

Last edited by KyleK (2009-11-23 11:10:55)

index monkey · 2009-11-23 11:46:49

Hi KyleK,

The correct name is now given to the downloaded file, both filename and extension
however the regex filters are no longer applying so its queueing up all the files from the feed.

The logfile shows the following:

automatic.log wrote:
1 feed URLs
[init_regex] PCRE compilation failed at offset 6: unmatched parentheses (pattern: vostfr))my.show)
[init_regex] PCRE compilation failed at offset 9: missing ) (pattern: (?!.*(AC3)
[init_regex] PCRE compilation failed at offset 6: unmatched parentheses (pattern: vostfr))my.other.show)
[init_regex] PCRE compilation failed at offset 9: missing ) (pattern: (?!.*(AC3)

automatic.conf has the following filters:

automatic.conf wrote:
patterns = {
"(?!.*(AC3|HR|x264|720p|French|Hebrew|Portuguese|Brazilian|German|vostfr))my.show"
"(?!.*(AC3|HR|x264|720p|French|Hebrew|Portuguese|Brazilian|German|vostfr))my.other.show"
}

Thanks again

Last edited by index monkey (2009-11-23 13:46:40)

KyleK · 2009-11-23 16:43:04

Meh, of course I forgot something

Here is the fixed version.

index monkey · 2009-11-23 16:52:59

Awesome! works prefectly.
A ninja flew down from heaven and sent me all my tv programs, and all I had to do was edit a text file

Thanks very much.

crs2027 · 2009-12-04 18:50:05

I'm having a bit of trouble understanding how to use this program,

I guess I go to a site that hosts torrents.... isohunt for example.

Search for my fav. TV show?

click the RSS icon, use the page address (copy/paste) in to the config?

restart automatic?

Even if it added the "RSS" how can I decide where it should start.. like series 3, episode 2 and automatically continue on up to today and then in the future?

I've never really messed around with RSS stuff so I might need a bit of an explanation on how to actually use "automatic"

Another Q: (if/when I get it running) I would like to run automatic just *once* during the night for 3 days at a certain time.... the config suggests that it remains running in the background and is triggered from a timer. Can I disable that and just run it through crontab when I would like it to be run? that would be very helpful!

Many thanks to KyleK for all your work on transmission and automatic.

and TIA for replies

KyleK · 2009-12-04 20:03:33

You got the RSS part right. You need an RSS feed that contains direct links to torrent files. Isohunt has that, eztv does it for TV shows only.

Now in order to tell Automatic what to download, you need to specify patterns. Automatic uses the regular expression syntax to define patterns.

So, for example, you want Automatic to catch all torrents from the show "That Show", you would specify a pattern in the config file:

Code:

patterns = { 
            "That.Show"
}

The dot (.) is a placeholder for any character (e.g. a whitespace, or an actual dot).
Now, if you only want your shows in XviD format, you can specify that as well:

Code:

patterns = { 
            "That.Show.*XviD"
}

The expression '.*' means "any number of any characters". So 'xvid' doesn't have to stand directly after 'Show', but anywhere later in the name.
You can specify multiple patterns by putting each on on a separate line and in double quotes. Make sure there is a set of curly braces at the beginning and the end:

Code:

patterns = { 
            "That.Show.*XviD"
            "Some.Other.Show.*720p"
}

You cannot specify which season to download (well, actually, you can, by making the pattern more specifiy), but you cannot tell Automatic to download only episodes 3 onwards of a show (thinking of it, that's not true either. I believe you can create a regular expression that suits these needs as well, but it would be rather complicated

To answer your last question: Yes, you can use Automatic with crontab. Simply specify the parameter '-o' on its commandline. It tells Automatic to directly quit after running just once.

Be aware though that most RSS feeds only contain a certain amount of items (e.g. the last 50 episodes). With the abundance of TV shows available, Automatic might not actually 'see' any new episodes when you run it only at a specific time.

Alternatively, you can tell Automatic to add torrents to Transmission in a paused state. They'll be added, but won't start downloading automatically. You'll have to start them by hand.

crackerjo1 · 2009-12-05 17:10:19

So I might be terribly confused, but I could not find a "getting started" readme file or anything, so maybe someone could help me. I installed 0.4.3-2 and set up my conf file and started automatic with:

Code:

/mnt/HD_a2/ffp/start/automatic.sh start

and got the message:

Code:

Starting /ffp/bin/automatic -c /ffp/etc/automatic.conf

So I assume it is working. However I don't see any new torrents being added or anything.

I have my url set to:

Code:

url = "http://rss.thepiratebay.org/205"

and my patterns as:

Code:

patterns =  { "The.Big.Bang.Theory.S03*.*HDTV.XviD"

            }

Am I missing some important step? Am I missing some important documentation that talks about this important step and how dumb I am?

Thanks to anyone for the help, it's greatly appreciated. My conf file is attached in case that might help.

Attachments:

automatic.conf, Size: 2,819 bytes, Downloads: 514

KyleK · 2009-12-05 17:34:11

No, you did everything correctly. Automatic doesn't add anything to Transmission because the RSS feed you mentioned simply doesn't contain any entry for The Big Bang Theory.
As I mentioned 2 posts above yours, RSS feeds usually only contain a fixed number of "latest news", or in this case, latest torrents.
The Pirate Bay feed apparently lists the 50 newest torrents.

One little thing about your regular expression though: "S03*.*" means "S0(the number 3 zero or more times)(anything else)"
As a result, that expression would add torrents from every season to Transmission. If you only want files from S03, get rid of the first asterisk:

Code:

patterns =  { "The.Big.Bang.Theory.S03.*HDTV.XviD" }

crackerjo1 · 2009-12-05 18:16:50

Thanks for the quick response. I thought of that, however I looked at the feed in google reader and there are hundreds of in it, with many Big Bang Theory entries. Is this just Google being Google or are there actually that many entries available?

And thanks for correcting my pattern

KyleK · 2009-12-05 18:25:42

I believe Google reader saves unread items for you, that's why you see more than are actually in the file.
See, the way RSS works is this: Your tool (in this case Google Reader) literally downloads the provided URL after a given interval (like 30 minutes). The file it receives is an XML file, which usually contains once <channel>, and inside this channel, there are a number of <item> entities.

You can try it yourself, on the NAS:

Code:

$ wget -O 205.xml http://rss.thepiratebay.org/205

Then open the XML in the browser of your choice.

tanstaafl1963 · 2009-12-05 18:37:40

parser error : Start tag expected, '<' not found

I wonder if there's just something wrong with my url feed source. I have several shows in an RSS feed from showrss.kamorra. Yesterday ALL of season 2 for Burn Notice appeared in the feed, and then in automatic. Not wanted, because I had them all. And now, I'm getting the message below from my automatic.log file. NOTHING has changed in the .conf file in months.

*------------------- automatic.conf setting ---------
url={

"http://showrss.karmorra.info/rss.php?user_id=5476&hd=null&proper=null"

=*------------------- automatic.conf setting

Cat of the automatic.log file.

/mnt/HD_a2/logs # cat automatic.log
1 feed URLs
Read 35 patterns from config file
Restored 70 old entries
Entity: line 1: parser error : Start tag expected, '<' not found
brick
^
Error: Unable to parse input data!

-----------------------

I have just stopped both transmission and automatic, restarted transmission then automatic and the same issue occurs. I'm assuming this error has something to do with why my 3 shows that should download, Monk and 2 eps of Dollhouse are not downloading..

Any help would be appreciated.

KyleK · 2009-12-05 18:49:46

That error indicates that the RSS feed Automatic downloaded was not properly formatted (i.e., not a proper XML file).
Considering the fact that a complete show showed up in Transmission over night, and now the problem with the feed, I would assume that the problem is on their end.

When I tried to download the XML from your link for the first time, I got an error, but about 5 minutes later it worked.

Let me know if the problem persists.

tanstaafl1963 · 2009-12-05 18:55:42

KyleK wrote:
That error indicates that the RSS feed Automatic downloaded was not properly formatted (i.e., not a proper XML file).

The odd thing is when I added that url to my google reader, it came up fine. It initially showed the gazillion Burn Notice episodes, but when I refreshed, it came back with the Monk and 2 Dollhouse.

KyleK wrote:
When I tried to download the XML from your link for the first time, I got an error, but about 5 minutes later it worked.

Did you use the wget you mentioned previously? When I tried that it is taking several minutes... I guess it does sound like it's their side....

Thanks for the quick response...

crackerjo1 · 2009-12-05 18:55:51

Thanks Kyle, that makes sense. Google just stores way too much information for me to comprehend haha. Looks like I learned something today.

One last question, is there a way to exclude terms from the pattern? Most of these feeds seem to have more than one of the same episode so I want to exclude the one with 720p for example.

KyleK · 2009-12-05 18:58:54

Yes. Please take a look at the INFO 2 paragraph in the very first post of this thread. It is explained there how to exclude torrents with certain terms.

crackerjo1 · 2009-12-05 19:09:06

Sorry.. Another case of RTFM. Thanks again

KyleK · 2009-12-06 01:36:10

Automatic 0.4.4 is out. It only fixes a couple of issues, nothing special.

Just a heads-up: Next on the list is logging output messages to a file while running in daemon mode (already done), and I'm trying to make Automatic work with private torrent sites that require cookies on their RSS feeds. Fingers crossed!

nyunyu · 2010-01-07 18:49:02

I want to download and install this however this is what I get instead.

/mnt/HD_a2 # funpkg -i Automatic-0.4.4.tgz
FATAL: Automatic-0.4.4.tgz: File not found

Can you give me a new link for this package?

Thank you.

KyleK · 2010-01-10 00:35:19

@nyunyu: Have you actually downloaded the .tgz package from the first post of this thread? You must be in the same folder as that file when you issue the funpkg command, otherwise it will fail (because it can't find the package it's supposed to install).

nyunyu · 2010-01-10 06:12:10

KyleK wrote:
@nyunyu: Have you actually downloaded the .tgz package from the first post of this thread? You must be in the same folder as that file when you issue the funpkg command, otherwise it will fail (because it can't find the package it's supposed to install).

duh! I didn't bother that link because it shows as ffp0.5 and I already have that. Sorry for that man.
Installed and success.

Anyone want to share their conf file? especially in "patterns" part. I just don't have the clue how to set this part.

Are these correct?
patterns = {
"(?!.*720p)Greys.Anatomy"
"(?!.*HDTV)Greys.Anatomy"
"(?!.*HDTV)Heroes"
"(?!.*720p)House"
"(?!.*HDTV)House"
"(?!.*HDTV)My.Name.is.Earl.S04"
"(?!.*HDTV)Reaper"
"(?!.*XviD)The.Girls.Next.Door"
}

Last edited by nyunyu (2010-01-10 10:35:20)

nyunyu · 2010-01-10 20:46:17

KyleK, how to check if Automatic is running? I checked the torrent folder and there was nothing inside.
Also, is there a plan to develop a GUI for automatic? Like maybe if we could go to NAS_IP_ADDRESS:PORT and shows a page where all details like status, schduled, etc..

Muppet · 2010-01-11 00:27:44

Code:

patterns =  { 
              "Heroes.*S0[3-9]E.*720p"
              "House.*S0[6-9]E.*720p"
            }

1st gets everything with Heroes in it from series 3-9 at 720p. Same for house except starting at season 6. The pattern you use depends on the name of the torrent but that style works with scene releases.

.. and scroll up just a bit on this very page where Kyle has already explained this stuff.

Type "top" to see if its running.
I don't leave it running all the time because it inevitably crashes after several days running. Better to regularly schedule it via cron using the -o flag. eg..

Code:

8 * * * * /mnt/HD_a2/ffp/bin/automatic -o -c /ffp/etc/automatic.conf

Last edited by Muppet (2010-01-11 00:32:53)

nyunyu · 2010-01-11 09:17:12

Muppet wrote:
Code:
patterns =  { 
              "Heroes.*S0[3-9]E.*720p"
              "House.*S0[6-9]E.*720p"
            }
1st gets everything with Heroes in it from series 3-9 at 720p. Same for house except starting at season 6. The pattern you use depends on the name of the torrent but that style works with scene releases.

.. and scroll up just a bit on this very page where Kyle has already explained this stuff.

Type "top" to see if its running.
I don't leave it running all the time because it inevitably crashes after several days running. Better to regularly schedule it via cron using the -o flag. eg..
Code:
8 * * * * /mnt/HD_a2/ffp/bin/automatic -o -c /ffp/etc/automatic.conf

I checked and automatic started. But somehow there is no activity.
I just confused right now. Am I doing the right step or did I missing something?
Also, is it possible to use 2 rss urls?

Muppet · 2010-01-11 09:20:18

You can do multiple URLs the same way as the patterns:

Code:

url = {
       "http://www.blahblahblah"
       "http://bingbongbing"
       "http://wahwahwah"
      }

nyunyu · 2010-01-11 10:04:44

Ok, all working now. I noticed that the reason automatic shows no activities was because of the rss link I used.
I was using this link: http://www.tvnzb.com/tvnzb_new.rss and it seems automatic can't retrieve the torrent from that link.

DSM-G600, DNS-3xx and NSA-220 Hack Forum

Announcement

#176 2009-11-23 10:52:58

Re: [REL] Automatic for funplug-0.5

index monkey wrote:

#177 2009-11-23 11:46:49

Re: [REL] Automatic for funplug-0.5

automatic.log wrote:

automatic.conf wrote:

#178 2009-11-23 16:43:04

Re: [REL] Automatic for funplug-0.5

#179 2009-11-23 16:52:59

Re: [REL] Automatic for funplug-0.5

#180 2009-12-04 18:50:05

Re: [REL] Automatic for funplug-0.5

#181 2009-12-04 20:03:33

Re: [REL] Automatic for funplug-0.5

Code:

Code:

Code:

#182 2009-12-05 17:10:19

Re: [REL] Automatic for funplug-0.5

Code:

Code:

Code:

Code:

#183 2009-12-05 17:34:11

Re: [REL] Automatic for funplug-0.5

Code:

#184 2009-12-05 18:16:50

Re: [REL] Automatic for funplug-0.5

#185 2009-12-05 18:25:42

Re: [REL] Automatic for funplug-0.5

Code:

#186 2009-12-05 18:37:40

Re: [REL] Automatic for funplug-0.5

#187 2009-12-05 18:49:46

Re: [REL] Automatic for funplug-0.5

#188 2009-12-05 18:55:42

Re: [REL] Automatic for funplug-0.5

KyleK wrote:

KyleK wrote:

#189 2009-12-05 18:55:51

Re: [REL] Automatic for funplug-0.5

#190 2009-12-05 18:58:54

Re: [REL] Automatic for funplug-0.5

#191 2009-12-05 19:09:06

Re: [REL] Automatic for funplug-0.5

#192 2009-12-06 01:36:10

Re: [REL] Automatic for funplug-0.5

#193 2010-01-07 18:49:02

Re: [REL] Automatic for funplug-0.5

#194 2010-01-10 00:35:19

Re: [REL] Automatic for funplug-0.5

#195 2010-01-10 06:12:10

Re: [REL] Automatic for funplug-0.5

KyleK wrote:

#196 2010-01-10 20:46:17

Re: [REL] Automatic for funplug-0.5

#197 2010-01-11 00:27:44

Re: [REL] Automatic for funplug-0.5

Code:

Code:

#198 2010-01-11 09:17:12

Re: [REL] Automatic for funplug-0.5

Muppet wrote:

Code:

Code:

#199 2010-01-11 09:20:18

Re: [REL] Automatic for funplug-0.5

Code:

#200 2010-01-11 10:04:44

Re: [REL] Automatic for funplug-0.5

Board footer