SABnzbd Forums


https://237678.vipkidinterhk.tech/

nzb-matrix.eu NZBs have syntax error

https://237678.vipkidinterhk.tech/viewtopic.php?t=12992

Page 1 of 2

nzb-matrix.eu NZBs have syntax error

Posted: January 22nd, 2013, 6:43 am
by bf161
Hi,

thanks for the great software :) I recently did a fresh install of sab + sickbeard and found the following when trying to open NZBs from nzb-matrix.eu i get syntax errors each time. Other indexers are fine, just seems to be nzbmatrix.eu sourced NZBs.

example from sab log

Code: Select all

2013-01-22 19:26:10,510::INFO::[urlgrabber:116] Grabbing URL http://nzb-matrix.eu/getnzb/6e281a0316221bdbd5ffb88b6ae94191.nzb&i=17897&r=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
2013-01-22 19:26:13,678::INFO::[misc:781] Creating directories: C:\XXXXXXXXXX\Suits.S02E11.720p.HDTV.x264-IMMERSE
2013-01-22 19:26:13,680::WARNING::[nzbstuff:714] Invalid NZB file Suits.S02E11.720p.HDTV.x264-IMMERSE.nzb, skipping (reason=syntax error, line=1)
however if i get the NZB from another source, say nzbclub, it works fine. Opening the two xmls, all i can see is the following difference:

top few lines of working nzb from nzbclub

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.0//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.0.dtd">
<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">
<file poster="canuck@suck (ilikeboys)" date="1357704394" subject="[121381]-[FULL]-[#a.b.teevee]-[ Justified.S04E01.HDTV.x264-2HD ]-[01/29] - "justified.s04e01.hdtv.x264-2hd.nfo" yEnc (1/1)">
<groups>
<group>alt.binaries.teevee</group>
</groups>
<segments>
<segment bytes="8130" number="1">[email protected]</segment>
top few lines of not working nzb from nzb-matrix.eu

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.1//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.1.dtd">
<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">

<head>
 <meta type="category">TV > SD</meta>
 <meta type="name">Justified.S04E01.HDTV.x264-2HD</meta>
</head>

<file poster="canuck@suck (ilikeboys)" date="1357704394" subject="[121381]-[FULL]-[#a.b.teevee]-[ Justified.S04E01.HDTV.x264-2HD ]-[01/29] - "justified.s04e01.hdtv.x264-2hd.nfo" yEnc (1/1)">
 <groups>
  <group>alt.binaries.teevee</group>
can anyone offer a suggestion ? does sab not support the dtd ver 1.1 ?

thanks.

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 22nd, 2013, 7:18 am
by shypike
The <meta> fields are supported in 0.7.9.
Possibly SABnzbd stumbles over the part:

Code: Select all

<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.1//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.1.dtd">
I don't think any other site supporting 1.1 bothered to actually change the DTD version string.
I'll check.
Can you email an example to [email protected] ?

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 22nd, 2013, 8:34 am
by bf161
hi shypike, nzb's sent to [email protected]. thanks.

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 22nd, 2013, 2:43 pm
by shypike
Which SABnzbd and operating system are you using?
The examples that you sent load fine for me.
Even the category is picked up.

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 22nd, 2013, 8:44 pm
by bf161
hi shypike
hmm, odd and must be something on my side... i will re-install everything on my side then. i am using win7 64bit + sabnzbd version 0.7.9.
thanks

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 23rd, 2013, 11:31 am
by bf161
shypike
i looked into this some more after i saw the same issue in and older version of sabnzbd and after a re-install of the program. I think what might be happening is that the sabnzbd http (and by proxy of that Curl) is not re-assembling chunked http correctly, or at least something not quite right where sabNZB is sending the HTTP get for the NZB using http 1.0, but nzb-matrix is replying with chunked encoded http/1.1 (not sure that is kosher). If this is the case, it would mean the first line of the downloaded file is the chunk size 0x2000 which causes "syntax error line=1" in the parser if it makes it through.

Code: Select all

Via: 1.1 qt-mfc2:80
X-Powered-By: PHP/5.3.10-1ubuntu3.4
X-DNZB-Name: Suits.S02E11.720p.HDTV.x264-IMMERSE
X-DNZB-Category: TV > HD
X-DNZB-MoreInfo:
X-DNZB-NFO: http://nzb-matrix.eu/api?t=getnfo&id=6e281a0316221bdbd5ffb88b6ae94191&raw=1&i=17897&r=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

2000
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.1//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.1.dtd">
<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">

<head>
...
interestingly passing https instead of http for the nzb-matrix hosted NZB file works around it, make of that what you will.

thx.

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 23rd, 2013, 1:44 pm
by shypike
It also worked fine when I used the nzb-matrix.eu URL that you sent me.
Just tried it again using curl, no problem.
On which OS are you testing this?

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 23rd, 2013, 9:02 pm
by bf161
hi shypike
this is win7 64bit (intel). I also found that native curl will work fine too, it sends actual http 1.1 and is happy to handle the chunked response. when i sniff the traffic from sabnzb, it's sending http 1.0 - not sure this is due to different curl lib or something like that.
thanks.

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 24th, 2013, 1:47 am
by shypike
SABnzbd doesn 't use curl, but the internal webserver Cherrypy.
I also tested SABnzbd's binary on Win7-64., no issues.
Doesn't SABnzbd get a proper file for you?

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 24th, 2013, 4:29 am
by bf161
shypike wrote:SABnzbd doesn 't use curl, but the internal webserver Cherrypy.
I also tested SABnzbd's binary on Win7-64., no issues.
Doesn't SABnzbd get a proper file for you?
my mistake, actually on looking closer it seems just using pythons native urllib to grab it, not cherrypy per-se. Also, not sure what you mean by "gets a proper file" - it just fails with the error in my post, nothing else . If i download it with curl (which can eat the chunked encoding without any issue) and then host it locally, it works fine:

root@zzz:/var/www# curl "http://nzb-matrix.eu/getnzb/4248c1a26b7 ... xxxxxxxxxx" > nzb/test.nzb

then add http://my.local.webserver.ip/nzb/test.nzb the the sabNZBd webUI - works fine and starts downloading the file.

here is a nzb download from sabNZBd, it's http 1.0

Code: Select all

Hypertext Transfer Protocol
    GET /nzb/test.nzb HTTP/1.0\r\n
        Request Method: GET
        Request URI: /nzb/test.nzb
        Request Version: HTTP/1.0
    Host: 192.168.x.x\r\n
    User-Agent: SABnzbd+/0.7.9\r\n
    Accept-encoding: gzip\r\n
    \r\n
Looking into class URLGrabber (urlgrabber.py) there is no way it will ever handle chunked encoding natively and code is using urllib which in python 2.x seems only http 1.0 capable. Further, nzb-matrix.eu (or other, see note) is being dodgy by sending chunked encoding to a http 1.0 client (which it shouldn't).

Note: No idea why i get chunked encoding back from nzb-matrix.eu (do you ? can you check it with wireshark?), maybe its due to my location ending up on a different server (resolves to 77.72.149.7 for me) or something i just thought of is that the country i am posting from does have some sort of "great firewall" and may be messing with it.

either way, i stepped through urlgrabber, and sure enough, the chunk sizes are still in the output file 'fn'

Code: Select all

>>> opener = urllib.FancyURLopener({})
>>> opener.prompt_user_passwd = None
>>> opener.addheaders = []
>>> opener.addheader('User-Agent', 'SABnzbd+123123')
>>> opener.addheader('Accept-encoding','gzip')
>>> url="http://nzb-matrix.eu/getnzb/4248c1a26b7a7167b3e81642b70b23c0.nzb&i=17897&r=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
>>> fn, header = opener.retrieve(url)
>>> print header
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection: Keep-Alive
Content-Disposition: attachment; filename="Californication_S06E02_HDTV_XviD-AFG.nzb"
Content-Type: application/x-nzb
Transfer-Encoding: chunked              <<<<<<<<<<<<<<<<
Date: Thu, 24 Jan 2013 08:45:01 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: Apache/2.2.22 (Ubuntu)
Set-Cookie: PHPSESSID=im2up9kj8tuomfbgsh2imi7fe7; path=/
Via: 1.1 qt-mfc4:80
X-Powered-By: PHP/5.3.10-1ubuntu3.4
X-DNZB-Name: Californication S06E02 HDTV XviD-AFG
X-DNZB-Category: TV > SD
X-DNZB-MoreInfo:
X-DNZB-NFO: http://nzb-matrix.eu/api?t=getnfo&id=4248c1a26b7a7167b3e81642b70b23c0&raw=1&i=17897&r=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

>>> f=open(fn).readlines()
>>> print f[0:10]
['2000\r\n', '<?xml version="1.0" encoding="UTF-8"?>\n', '<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.1//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.1.dtd">\n', '<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">\n', '\n', '<head>\n', ' <meta type="category">TV > SD</meta>\n', ' <meta type="name">Californication S06E02 HDTV XviD-AFG</meta>\n', '</head>\n', '\n']
>>>
now switching to http 1.1 compliant urllib2

Code: Select all

>>> import urllib2
>>> o = urllib2.Request(url)
>>> o.add_header('User-Agent', "SABnzbd")
>>> o.add_header('Accept-encoding','gzip')
>>> resp = urllib2.urlopen(o)
>>> print resp.info()
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection: Keep-Alive
Content-Disposition: attachment; filename="Californication_S06E02_HDTV_XviD-AFG.nzb"
Content-Type: application/x-nzb
Transfer-Encoding: chunked            <<<<<<<<<<
Date: Thu, 24 Jan 2013 09:16:58 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: Apache/2.2.22 (Ubuntu)
Set-Cookie: PHPSESSID=e9d4v8msg0lik9d9lq18hq3hq2; path=/
Via: 1.1 qt-mfc4:80
X-Powered-By: PHP/5.3.10-1ubuntu3.4
X-DNZB-Name: Californication S06E02 HDTV XviD-AFG
X-DNZB-Category: TV > SD
X-DNZB-MoreInfo:
X-DNZB-NFO: http://nzb-matrix.eu/api?t=getnfo&id=4248c1a26b7a7167b3e81642b70b23c0&raw=1&i=17897&r=xxxxxxxxxxxxxxxxxxxxxxx

>>> buf = resp.read()
>>> print buf[0:1000]
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.1//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.1.dtd">
<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">
<head>
 <meta type="category">TV > SD</meta>
no more chunk junk in the output.

so all in all, urllib is http 1.0 only, for some reason either nzb-matrix.eu is ignoring it's http 1.0 request and sending back chunked data, and/or something inline is messing with it. Simple solution is to use https, else sabNZB could use urllib2 and thus become http 1.1 capable.

thanks.

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 24th, 2013, 7:19 am
by shypike
You're right about SABnzbd using urllib.
I was mixing things up with the api nzb upload, which does use CherryPy.
I"m willing to switch to urllib2, but I won't be able to test the difference,
because it works for me the way it's implemented currently.
Which is a riddle itself.

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 25th, 2013, 12:31 am
by kevindd992002
Does this mean that nzb-matrix.eu is really not a good API provider for SAB/CP/SB?

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 25th, 2013, 1:25 am
by shypike
kevindd992002 wrote:Does this mean that nzb-matrix.eu is really not a good API provider for SAB/CP/SB?
Doesn't it work for you ?

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 25th, 2013, 1:47 am
by kevindd992002
Lots of errors in the logs of SB.

Re: nzb-matrix.eu NZBs have syntax error

Posted: January 25th, 2013, 11:48 am
by shypike
kevindd992002 wrote:Lots of errors in the logs of SB.
And SABnzbd warnings?
All times are UTC-05:00
Page 1 of 2

Powered by phpBB® Forum Software © phpBB Limited