shypike wrote:SABnzbd doesn 't use curl, but the internal webserver Cherrypy.
I also tested SABnzbd's binary on Win7-64., no issues.
Doesn't SABnzbd get a proper file for you?
my mistake, actually on looking closer it seems just using pythons native urllib to grab it, not cherrypy per-se. Also, not sure what you mean by "gets a proper file" - it just fails with the error in my post, nothing else . If i download it with curl (which can eat the chunked encoding without any issue) and then host it locally, it works fine:
root@zzz:/var/www# curl "
http://nzb-matrix.eu/getnzb/4248c1a26b7 ... xxxxxxxxxx" > nzb/test.nzb
then add
http://my.local.webserver.ip/nzb/test.nzb the the sabNZBd webUI - works fine and starts downloading the file.
here is a nzb download from sabNZBd, it's http 1.0
Code: Select all
Hypertext Transfer Protocol
GET /nzb/test.nzb HTTP/1.0\r\n
Request Method: GET
Request URI: /nzb/test.nzb
Request Version: HTTP/1.0
Host: 192.168.x.x\r\n
User-Agent: SABnzbd+/0.7.9\r\n
Accept-encoding: gzip\r\n
\r\n
Looking into class URLGrabber (urlgrabber.py) there is no way it will ever handle chunked encoding natively and code is using urllib which in python 2.x seems only http 1.0 capable. Further, nzb-matrix.eu (or other, see note) is being dodgy by sending chunked encoding to a http 1.0 client (which it shouldn't).
Note: No idea why i get chunked encoding back from nzb-matrix.eu (do you ? can you check it with wireshark?), maybe its due to my location ending up on a different server (resolves to 77.72.149.7 for me) or something i just thought of is that the country i am posting from does have some sort of "great firewall" and may be messing with it.
either way, i stepped through urlgrabber, and sure enough, the chunk sizes are still in the output file 'fn'
Code: Select all
>>> opener = urllib.FancyURLopener({})
>>> opener.prompt_user_passwd = None
>>> opener.addheaders = []
>>> opener.addheader('User-Agent', 'SABnzbd+123123')
>>> opener.addheader('Accept-encoding','gzip')
>>> url="http://nzb-matrix.eu/getnzb/4248c1a26b7a7167b3e81642b70b23c0.nzb&i=17897&r=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
>>> fn, header = opener.retrieve(url)
>>> print header
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection: Keep-Alive
Content-Disposition: attachment; filename="Californication_S06E02_HDTV_XviD-AFG.nzb"
Content-Type: application/x-nzb
Transfer-Encoding: chunked <<<<<<<<<<<<<<<<
Date: Thu, 24 Jan 2013 08:45:01 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: Apache/2.2.22 (Ubuntu)
Set-Cookie: PHPSESSID=im2up9kj8tuomfbgsh2imi7fe7; path=/
Via: 1.1 qt-mfc4:80
X-Powered-By: PHP/5.3.10-1ubuntu3.4
X-DNZB-Name: Californication S06E02 HDTV XviD-AFG
X-DNZB-Category: TV > SD
X-DNZB-MoreInfo:
X-DNZB-NFO: http://nzb-matrix.eu/api?t=getnfo&id=4248c1a26b7a7167b3e81642b70b23c0&raw=1&i=17897&r=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>> f=open(fn).readlines()
>>> print f[0:10]
['2000\r\n', '<?xml version="1.0" encoding="UTF-8"?>\n', '<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.1//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.1.dtd">\n', '<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">\n', '\n', '<head>\n', ' <meta type="category">TV > SD</meta>\n', ' <meta type="name">Californication S06E02 HDTV XviD-AFG</meta>\n', '</head>\n', '\n']
>>>
now switching to http 1.1 compliant urllib2
Code: Select all
>>> import urllib2
>>> o = urllib2.Request(url)
>>> o.add_header('User-Agent', "SABnzbd")
>>> o.add_header('Accept-encoding','gzip')
>>> resp = urllib2.urlopen(o)
>>> print resp.info()
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection: Keep-Alive
Content-Disposition: attachment; filename="Californication_S06E02_HDTV_XviD-AFG.nzb"
Content-Type: application/x-nzb
Transfer-Encoding: chunked <<<<<<<<<<
Date: Thu, 24 Jan 2013 09:16:58 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: Apache/2.2.22 (Ubuntu)
Set-Cookie: PHPSESSID=e9d4v8msg0lik9d9lq18hq3hq2; path=/
Via: 1.1 qt-mfc4:80
X-Powered-By: PHP/5.3.10-1ubuntu3.4
X-DNZB-Name: Californication S06E02 HDTV XviD-AFG
X-DNZB-Category: TV > SD
X-DNZB-MoreInfo:
X-DNZB-NFO: http://nzb-matrix.eu/api?t=getnfo&id=4248c1a26b7a7167b3e81642b70b23c0&raw=1&i=17897&r=xxxxxxxxxxxxxxxxxxxxxxx
>>> buf = resp.read()
>>> print buf[0:1000]
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.1//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.1.dtd">
<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">
<head>
<meta type="category">TV > SD</meta>
no more chunk junk in the output.
so all in all, urllib is http 1.0 only, for some reason either nzb-matrix.eu is ignoring it's http 1.0 request and sending back chunked data, and/or something inline is messing with it. Simple solution is to use https, else sabNZB could use urllib2 and thus become http 1.1 capable.
thanks.
I recently did a fresh install of sab + sickbeard and found the following when trying to open NZBs from nzb-matrix.eu i get syntax errors each time. Other indexers are fine, just seems to be nzbmatrix.eu sourced NZBs.
