DSM-G600, DNS-3xx and NSA-220 Hack Forum

Unfortunately no one can be told what fun_plug is - you have to see it for yourself.

You are not logged in.

Announcement

#401 2010-12-23 22:49:28

FunFiler
Member
Registered: 2010-05-23
Posts: 577

Re: [REL] Automatic for funplug-0.5

Can you shorten it match on "tokkal"? I have no idea if it makes sense as I don't understand the language.


3 * (DNS-323 with 2 * 2TB) = 12TB Running FW v1.08 & FFP v0.5
Useful Links: Transmission, Transmission Remote, Automatic

Offline

 

#402 2010-12-27 17:00:45

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

Uuh, I'm pretty sure Automatic currently does not support Unicode characters. I'll have to check if libxml and libpcre support UTF8, but Automatic itself (when parsing the config file) certainly does not.

I didn't really think of this when developing Automatic hmm I'll have to see what needs to be done to add the proper support.

Offline

 

#403 2010-12-27 18:03:46

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

Daro wrote:

Hi,

i browsed through the whole thread but i didnt find solution for my problem. How can i filter this:

Code:

<title>Így jártam anyátokkal - 6x12 [HDTV-720p - Eng - CTU](stevent02)</title>

as you see i have problem with non english characters. I tried with unicode setting like \u00cd as "Í" but pcre doesnt support it.

Can someone help me how can i use the non english chars?

Do you have a public RSS feed for me that contains UTF8 characters so I can test around?

Offline

 

#404 2010-12-28 19:13:30

Daro
Member
Registered: 2010-12-23
Posts: 21

Re: [REL] Automatic for funplug-0.5

Sorry but looks like that site hasnt public RSS feed what i use, but if you want i can send you my link in private, just for testing.

Offline

 

#405 2010-12-31 03:54:34

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

@Daro
Please try this release and tell me if it works now.
Assuming the RSS feed is in UTF8 encoding, make sure your automatic.conf is stored as UTF8 file as well, otherwise the regular expressions won't match.

Offline

 

#406 2010-12-31 09:04:40

Daro
Member
Registered: 2010-12-23
Posts: 21

Re: [REL] Automatic for funplug-0.5

Here is the result after this

Code:

 /ffp/bin/automatic -f -c /ffp/etc/automatic.conf -v4

Code:

../src/config_parser.c, 561: reading multiline param
../src/config_parser.c, 575: multiline param: param_good=0
../src/config_parser.c, 601: [parse_config_file] option: filter
../src/config_parser.c, 602: [parse_config_file] param:  pattern => "(?!.*Web)Dr\..Csont.*720p"
           folder => "/mnt/HD/HD_a2/Downloads/Sorozatok/Bones S06/"
          (105 byte)
../src/config_parser.c, 603: [parse_config_file] -----------------
../src/config_parser.c, 320: filter= pattern => "(?!.*Web)Dr\..Csont.*720p"
           folder => "/mnt/HD/HD_a2/Downloads/Sorozatok/Bones S06/"
          (type: 2)
../src/config_parser.c, 561: reading multiline param
../src/config_parser.c, 575: multiline param: param_good=0
../src/config_parser.c, 601: [parse_config_file] option: filter                gy.jĂĄrtam.anyĂĄtokkal.*720p"
           folder => "/mnt/HD/HD_a2/Downloads/Sorozatok/How I met your mother S06/"
          (149 byte)
../src/config_parser.c, 603: [parse_config_file] ----------gy.jĂĄrtam.anyĂĄtokkal.*720p"
           folder => "/mnt/HD/HD_a2/Downloads/Sorozatok/How I met your mother S06/"
          (type: 2)
Segmentation fault

I attached 2 files, the automatic.conf and the rss.php what i think an xml, what should get the automatic as input. But as i see the the automatic is failed when parsing the config file, so it doesnt reach the section when reading the xml and compare with filters.
I cleared the passkey because its a private torrentsite. If you need the real links then we can talk in private.

Thx
Daro


Attachments:
Attachment Icon test files.zip, Size: 3,209 bytes, Downloads: 344

Offline

 

#407 2011-01-01 23:59:30

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

@Daro

Here's another test build. I tested this with the files you provided and the regular expression matched just fine, so this should work with the feed as well.

Let me know if so, then this will be v0.6.5.

Offline

 

#408 2011-01-02 10:30:27

Daro
Member
Registered: 2010-12-23
Posts: 21

Re: [REL] Automatic for funplug-0.5

KyleK wrote:

@Daro

Here's another test build. I tested this with the files you provided and the regular expression matched just fine, so this should work with the feed as well.

Let me know if so, then this will be v0.6.5.

Thx, it looks like work, but i dont know why print out the verbose wrong, but its miner problem, the point is its working.

Thank you very much:)

You left the daemon start/stop script, because its a test version? You will present from this a full version?

I will testing on it more torrents with Hungarian letters:)

Daro

Last edited by Daro (2011-01-02 11:01:40)

Offline

 

#409 2011-01-21 18:34:16

Daro
Member
Registered: 2010-12-23
Posts: 21

Re: [REL] Automatic for funplug-0.5

Hi,

i am here again.
The new test version worked great until now. Now it has a problem

Code:

Entity: line 130: parser error : xmlParseEntityRef: no name
<title>Law & Order: Los Angeles - 1x08 [Web-Dl-720p - Eng - POD](aljasPOD)</titl

The interesting thing is the rss doesnt contain that torrent what the parser write. I dont know where it gets this xml, and why doesnt download and use the right one. If i am trying in utorrent it works fine, but on dns always listing that law and orders episodes. Maybe this xml saved somewhere and automatic cant overwrite it?

Last edited by Daro (2011-01-21 18:35:26)

Offline

 

#410 2011-01-21 19:11:20

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

Daro wrote:

Hi,

i am here again.
The new test version worked great until now. Now it has a problem

Code:

Entity: line 130: parser error : xmlParseEntityRef: no name
<title>Law & Order: Los Angeles - 1x08 [Web-Dl-720p - Eng - POD](aljasPOD)</titl

The interesting thing is the rss doesnt contain that torrent what the parser write. I dont know where it gets this xml, and why doesnt download and use the right one. If i am trying in utorrent it works fine, but on dns always listing that law and orders episodes. Maybe this xml saved somewhere and automatic cant overwrite it?

This is a problem with your feed: As you can see, the episode name contains an unescaped ampersand, which confuses the XML parser.

Offline

 

#411 2011-01-21 21:12:19

Daro
Member
Registered: 2010-12-23
Posts: 21

Re: [REL] Automatic for funplug-0.5

KyleK wrote:

This is a problem with your feed: As you can see, the episode name contains an unescaped ampersand, which confuses the XML parser.

Thats ok, but the feed isnt contain anymore that. Thats why i dont understand it. Its an old one.. I dont know where the automatic get it.

Offline

 

#412 2011-01-21 21:32:55

KyleK
Member
From: Dresden, Germany
Registered: 2007-12-05
Posts: 1178

Re: [REL] Automatic for funplug-0.5

Automatic doesn't store any XML, only links to the torrents it downloaded previously (in the automatic.state file).

Please check if Automatic uses the correct configuration file, and not an old one.
Also, can you send me the logfile where this error occurs?

Offline

 

#413 2011-01-21 23:09:43

Daro
Member
Registered: 2010-12-23
Posts: 21

Re: [REL] Automatic for funplug-0.5

KyleK wrote:

Automatic doesn't store any XML, only links to the torrents it downloaded previously (in the automatic.state file).

Please check if Automatic uses the correct configuration file, and not an old one.
Also, can you send me the logfile where this error occurs?

Oops, i mislook, the webpage show only the half xml, if i check the source then there are more and the Law & order still there. So looks like the "&" sign is the problem...

The configuration file isnt changed, so i dont think so it is the problem.

I send the xml, the log and the conf file


Attachments:
Attachment Icon Test.rar, Size: 11,954 bytes, Downloads: 334

Offline

 

#414 2011-01-22 10:22:19

Daro
Member
Registered: 2010-12-23
Posts: 21

Re: [REL] Automatic for funplug-0.5

Ok, looks like when those torrents with  "&" sign out from rss feed then all back to normal and automatic recognize the other torrent what filter define (The mentalist).

Offline

 

#415 2011-01-31 01:48:11

wtfok
Member
Registered: 2011-01-25
Posts: 31

Re: [REL] Automatic for funplug-0.5

I just wanted to congratulate you with the achievement of programming a brilliant piece of software like this. I'm actually going to buy another NAS and dedicate it to this.

I have one question though: uTorrent has this function for setting an episode number to download from. How can one fix this with regular expressions easily, as some torrents are named [ss]x[ee], and some S[ss]E[ee]?
Also, I would give you a hand in writing a filter/pattern-generator in PHP for some basic functions? Like: fill in series-name, episode to download from, etc, providing the correct text to paste into a conf file?

Offline

 

#416 2011-01-31 10:09:39

FunFiler
Member
Registered: 2010-05-23
Posts: 577

Re: [REL] Automatic for funplug-0.5

Just write multiple filters to handle new shows. RegEx is EXTREMELY powerful.
starting Example

"[\w|\.][0-9][0-9]x[0-9][0-9][\w|\.]"
and
"[\w|\.]s[0-9][0-9]e[0-9][0-9][\w|\.]"

Last edited by FunFiler (2011-01-31 11:13:31)


3 * (DNS-323 with 2 * 2TB) = 12TB Running FW v1.08 & FFP v0.5
Useful Links: Transmission, Transmission Remote, Automatic

Offline

 

#417 2011-02-01 09:39:39

wtfok
Member
Registered: 2011-01-25
Posts: 31

Re: [REL] Automatic for funplug-0.5

This post is a mess; encountered some problems, and in the end, found some solutions to imo very common problems

FunFiler wrote:

Just write multiple filters to handle new shows. RegEx is EXTREMELY powerful.
starting Example

"[\w|\.][0-9][0-9]x[0-9][0-9][\w|\.]"
and
"[\w|\.]s[0-9][0-9]e[0-9][0-9][\w|\.]"

That's quite a complex world, but I'll learn it someday, I guess tongue

Is there an easy way to test if my filters actually work?

KyleK wrote:

You can test your regular expressions here: http://www.regular-expressions.info/jav … ample.html

"Download all torrent files with 'Chuck' AND '720p' in any order, somewhere in the filename, but do not download files with 'season' in them".

Code:

(?=.*720p)(?!.*season)Chuck

At least, I had to check "Chuck"; "chuck" did not work. I guess the "not case sensitive" feature is achieved by lower-casing all found .torrent files in the first place?

Last edited by wtfok (2011-02-01 11:54:36)

Offline

 

#418 2011-02-01 14:44:09

FunFiler
Member
Registered: 2010-05-23
Posts: 577

Re: [REL] Automatic for funplug-0.5

Even easier:

filter = { pattern => "(?!.*(720p|season))^Chuck\." }

You can build on the exclusion list by using |
I use ^ to indicate "starting with" to avoid it being elsewhere in the string
I also use \. at the end to avoid "Chucking  ..." or some other variation.

Makes a nice clean conf file

Code:

...
filter = { pattern => "(?!.*(720p|micro|repack))^CBC.InSecurity\." }
filter = { pattern => "(?!.*(720p|micro|repack))^Chase\." }
filter = { pattern => "(?!.*(720p|micro|repack))^Chuck\." }
filter = { pattern => "(?!.*(720p|micro|repack))^Cops\." }
filter = { pattern => "(?!.*(720p|micro|repack))^Covert.Affairs\." }
...

Last edited by FunFiler (2011-02-01 14:49:24)


3 * (DNS-323 with 2 * 2TB) = 12TB Running FW v1.08 & FFP v0.5
Useful Links: Transmission, Transmission Remote, Automatic

Offline

 

#419 2011-02-01 14:49:41

wtfok
Member
Registered: 2011-01-25
Posts: 31

Re: [REL] Automatic for funplug-0.5

FunFiler wrote:

Even easier:

filter = { pattern => "(?!.*(720p|season))^Chuck\." }

You can build on the exclusion list by using |
I use ^ to indicate "starting with" to avoid it being elsewhere in the string
I also use \. at the end to avoid "Chucking  ..." or some other variation.

Thanks for the help, you really have a point there for the exclusion list. But doesn't | stand for "or"?
Anyhow, I don't think ^ is such a good idea, because some torrents do have a prefix or whatever, and then it would not work. Also, the first character after "Chuck" isn't always a dot, so I don't think that's a good idea either. Maybe \s is an option there? Read something about that in this thread...

Edit: did test "(?!.*(720p|season))^Chuck\.", wouldn't work. What does work, though, is "(?!.*(720p+season))^Chuck\.". My conclusion is that | is indeed OR and + is AND.

Building on that thought, I think this one is quite optimal: (?!.*(720p+season))^Chuck\W. It matches everything except text after the word "Chuck". I'm now finding out how to check everything before that word: nothing OR everything except an alphabetical character.

Last edited by wtfok (2011-02-01 15:37:42)

Offline

 

#420 2011-02-04 07:50:06

wtfok
Member
Registered: 2011-01-25
Posts: 31

Re: [REL] Automatic for funplug-0.5

Hi, got a little problem. My logfile says something I don't understand:


[01/04/02 19:36:05] ../src/automatic.c, 514: Daemon started
../src/automatic.c, 535: 1 feed URLs
../src/automatic.c, 536: Read 8 filters from config file
../src/web.c, 523: Upload to 'http://localhost:9091/transmission/rpc' failed: Couldn't connect to server
../src/state.c, 100: [load_state] Error: Unable to open statefile '/home/root/.config/automatic.state' for reading: No such file or directory
../src/web.c, 431: [getHTTPData] 'http://rss.thepiratebay.org/208': Couldn't resolve host name
../src/web.c, 431: [getHTTPData] 'http://rss.thepiratebay.org/208': Couldn't connect to server
[01/04/05 01:08:34] ../src/automatic.c, 389: [0] Found new download: The.Big.Bang.Theory.S04E14.720p.HDTV.x264-CTU [ALEX] (http://torrents.thepiratebay.org/614865 … .x264-CTU_[ALEX].6148653.TPB.torrent)
../src/torrent.c, 133: sendHTTPData() failed! (Response Code: 401)
[01/04/05 01:38:37] ../src/automatic.c, 389: [0] Found new download: The.Big.Bang.Theory.S04E14.720p.HDTV.x264-CTU [ALEX] (http://torrents.thepiratebay.org/614865 … .x264-CTU_[ALEX].6148653.TPB.torrent)
../src/torrent.c, 133: sendHTTPData() failed! (Response Code: 401)
[01/04/05 02:08:39] ../src/automatic.c, 389: [0] Found new download: The.Big.Bang.Theory.S04E14.720p.HDTV.x264-CTU [ALEX] (http://torrents.thepiratebay.org/614865 … .x264-CTU_[ALEX].6148653.TPB.torrent)
../src/torrent.c, 133: sendHTTPData() failed! (Response Code: 401)
[01/04/05 02:38:42] ../src/automatic.c, 389: [0] Found new download: The.Big.Bang.Theory.S04E14.720p.HDTV.x264-CTU [ALEX] (http://torrents.thepiratebay.org/614865 … .x264-CTU_[ALEX].6148653.TPB.torrent)
../src/torrent.c, 133: sendHTTPData() failed! (Response Code: 401)
[01/04/05 03:08:44] ../src/automatic.c, 389: [0] Found new download: The.Big.Bang.Theory.S04E14.720p.HDTV.x264-CTU [ALEX] (http://torrents.thepiratebay.org/614865 … .x264-CTU_[ALEX].6148653.TPB.torrent)
../src/torrent.c, 133: sendHTTPData() failed! (Response Code: 401)

So "sendHTTPData() failed". I don't know what that means and what I can do about that?

Offline

 

#421 2011-02-04 12:33:42

FunFiler
Member
Registered: 2010-05-23
Posts: 577

Re: [REL] Automatic for funplug-0.5

In my case, I want the "|" which is indeed OR, but use whatever works for you.

I suspect the errors you see indicate a DNS server configuration error.


3 * (DNS-323 with 2 * 2TB) = 12TB Running FW v1.08 & FFP v0.5
Useful Links: Transmission, Transmission Remote, Automatic

Offline

 

#422 2011-02-05 12:14:23

wtfok
Member
Registered: 2011-01-25
Posts: 31

Re: [REL] Automatic for funplug-0.5

Hm, now that I look at this, I'm actually wrong about the filters. The goal was to DO download 720p versions, but not with "season" in them. Guess I'll have to puzzle again tongue
These torrent-file URLs given by the RSS feed are by the way starting with http:// etcetera. That would mean that ^ is not correct in the patterns right?

About the DNS servers, I did set this up in the NAS, and Transmission is working (trackers are resolved). So is there something within Automatic I should consider setting up?

Last edited by wtfok (2011-02-05 12:23:37)

Offline

 

#423 2011-02-05 14:36:41

FunFiler
Member
Registered: 2010-05-23
Posts: 577

Re: [REL] Automatic for funplug-0.5

Automatic parses the XML Document returned from the feed URL, so no, it doesn't start with "http://....". The link to the actual torrent will, but that isn't parsed. As I've mentioned, the ones I've posted above are ones that I actually use and have been using since the day I came across Automatic. If you don't want to limit with "^", then take it out smile

I download the non 720p versions to save space. The example for Chuck is
filter = { pattern => "(?!.*(720p|micro|repack))^Chuck\." }

If you want only the 720p version, then you can modify and try
filter = { pattern => "(?!.*(season|micro|repack))^Chuck.*720p" }

This excludes anything with "season", "micro" and "repack" (I don't personally want these last 2, you can remove if you want)

As for your other issue, "Couldn't resolve host name" tells me you have a DNS issue unless of course that host does not exist. It could also be a gateway, firewall, ISP issue or the server was down or too busy. Looking at the URLs you are using, are you authenticating? Try a simple (different) torrent site first. You can also increase the level of detail that gets logged.

Last edited by FunFiler (2011-02-05 15:17:22)


3 * (DNS-323 with 2 * 2TB) = 12TB Running FW v1.08 & FFP v0.5
Useful Links: Transmission, Transmission Remote, Automatic

Offline

 

#424 2011-02-05 15:54:59

wtfok
Member
Registered: 2011-01-25
Posts: 31

Re: [REL] Automatic for funplug-0.5

Alright, I've set up my filters differently again, according to your advice smile

About the resolving issue: I did perform a ping command from the nas:

Code:

root@wtfok-Server:~# ping tracker.thepiratebay.org:80
PING tracker.thepiratebay.org:80 (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.275 ms
64 bytes from 127.0.0.1: seq=1 ttl=64 time=0.200 ms
64 bytes from 127.0.0.1: seq=2 ttl=64 time=0.201 ms
64 bytes from 127.0.0.1: seq=3 ttl=64 time=0.197 ms
64 bytes from 127.0.0.1: seq=4 ttl=64 time=0.200 ms
64 bytes from 127.0.0.1: seq=5 ttl=64 time=0.199 ms
64 bytes from 127.0.0.1: seq=6 ttl=64 time=0.199 ms

--- tracker.thepiratebay.org:80 ping statistics ---
7 packets transmitted, 7 packets received, 0% packet loss
round-trip min/avg/max = 0.197/0.210/0.275 ms

Ought to be working, right? Also, directly uploading a .torrent file is just working fine. I double checked the filled in DNS servers, which are 8.8.8.8 and 8.8.4.4, and are still filled in. Ah well, I'll check monday, when the new Chuck comes out smile

Offline

 

#425 2011-02-05 21:23:34

FunFiler
Member
Registered: 2010-05-23
Posts: 577

Re: [REL] Automatic for funplug-0.5

Bump up your log level "-v 3". If it still doesn't work Monday, post your config and log files.

Are you using a URL that requires authentication?


3 * (DNS-323 with 2 * 2TB) = 12TB Running FW v1.08 & FFP v0.5
Useful Links: Transmission, Transmission Remote, Automatic

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2010 PunBB