0.6.5 fails to upload NZB's

[dbrooks]

Hi there,

I'm the package maintainer for SABnzbd+ on FreeBSD. First, thanks for reminding me that 0.6.7 is out. I submitted an update for 0.6.6, but it hasn't yet been committed. We'll likely go right to 0.6.7 barring a quick release.

In regards to problems you're having, I'm aware they exist inside the port, however I don't know what options are available to me. The only thing I could feasibly think of would be providing the patched CherryPy as another port. Everything worked fine in SABnzbd+ 0.5.x with a stock version of CherryPy 3.2.0 AFAIK, what about SABnzbd+ 0.6.x changes that?

I also tried to overwrite all the 0.6.5 files with 0.6.7 files, but the problem with this is, the installed files are not in one place. They actually are in three different folders:
/usr/local/lib/python2.7/site-packages/sabnzbd/
/usr/local/share/sabnzbdplus/
/usr/local/bin/SABnzbd.py

This is by FreeBSD port convention. If you want to run it inside its own localized directory, just unpack the tarball.

This means these folders are missing:
/cherrypy
/email
/licenses

Like I mentioned above, I'll look into retaining the cherrypy directory, but I'm not sure about the implications in adhering to the FreeBSD port structure. It may need to be broken out into it's own port -- the problem there is that there's already a www/py-cherrypy port. I'll need to see what can be done.

A better question in my mind is why is a patched version of Cherrypy required for SABnzbd+ to perform this one task? What about not having a patched version of Cherrypy causes SABnzbd+ to silently ignore a specific action?

- Can't that be worked on within SABnzbd+ itself and save package maintainers like myself and whoever maintains the Ubuntu package time?
- Do the SABnzbd+ maintainers have a patch file I can apply to a stock version of CherryPy to turn it into a version that works? Surely they do. Where may I find it?

The best workaround I can provide you right now is to simply set up a watch folder, like you've done. That's probably why I haven't noticed this specific issue, as I'm usually inserting nzb's either via watch folder or by specifying a URL -- all of which works fine.

[Kriss]

OK. Thank you, for the fast response.
I usually use other programs to automatically send the NZB's to SABnzbd. I just wanted to upload the nzb this one time and found it's not working.
It's not a big problem, because SABnzbd is just working fine

Hope you guys find a solution

[shypike]

Dbrooks
The main reason is that the CherryPy people bring out versions that are highly incompatible.
The previous official release lacks required features,
The current official release contains a major bug, making https fail most of the time.
I have reported this bug, but given the glacial speed of the CherryPy development, I have little hope.
I'm not familiar with Unix packet creation, but the maintainer of the Ubuntu port seems to have cracked this issue.

Kriss
I'm not sure what you are doing, but just unpacking the tar.gz file that we distribute
does result in a working version. You need to place it in user space and not into the
standard locations. So you need to tweak the startup script.
Installing the package is still required because that ensures that you will have
the required modules, like yEnc and Cheetah.

[jcfp]

In the Ubuntu package, sabnzbd's python stuff is installed as a private module, because it isn't intended for use with other programs anyway. The main executable sits in /usr/bin, patched so it knows where to look for everything else. Hence, to use the included rather than system cherrypy all that needs to be done is copy it to the same directory that holds the private module. A quick explanation in the packaging documentation with a link to that cherrypy bug shypike is referring to is enough to justify that approach as far as the distros are concerned.

The resulting structure is this:
/usr/bin/sabnzbdplus
/usr/share/sabnzbdplus/sabnzbd/__init__.py
/usr/share/sabnzbdplus/cherrypy/__init__.py

[sander]

@jcfp

I have a question about the special cheetah that you say that is needed:

On a fresh Ubuntu 11.04, with universe & multiverse enabled and after "sudo apt-get install python-cheetah par2 unrar", I was able to run SABnzbd from source (so from http://sourceforge.net/projects/sabnzbdplus/. FYI sabnzbdplus from the Ubuntu repository is NOT installed), and succesfully download NZBs with it.

In the unpacked SABnzbd-0.6.7, I don't see an included cheetah (or other stuff), so I would say SABnzbd is using the plain python-cheetah I installed with the apt-get-command.

So my question is: is the special cheetah really needed? Or am I misunderstanding your post?

[shypike]

No special Cheetah is needed, Cheetah's backward-compatibility is excellent.

We're talking about CherryPy, where the standard releases are either inadequate
or having fatal bugs. Both in combination with SABnzbd, that is.

[sander]

No special Cheetah is needed, Cheetah's backward-compatibility is excellent.

We're talking about CherryPy, where the standard releases are either inadequate
or having fatal bugs. Both in combination with SABnzbd, that is.

Ah. Oh. Sorry for misreading the post. I was mislead by the need to install cheetah.

I indeed see a directory /SABnzbd-0.6.7/cherrypy/, so that's the special cherrypy stuff?

[ja30278]

Ran into this bug on FreeBSD 9. For some reason, file.value is set to None (not sure where the bug orginates, the code in python's cgi.py looks like it should do the right thing). A trivial workaround patch which fixes the issue:

--- interface.py.old 2011-09-05 09:00:02.000000000 -0400
+++ interface.py 2011-09-05 09:32:28.000000000 -0400
@@ -333,7 +333,9 @@
if msg: return msg

nzbfile = kwargs.get('nzbfile')
- if nzbfile is not None and nzbfile.filename and nzbfile.value:
+ if nzbfile is not None and nzbfile.filename:
+ if not nzbfile.value and nzbfile.file:
+ nzbfile.value = nzbfile.file.read()
sabnzbd.add_nzbfile(nzbfile, kwargs.get('pp'), kwargs.get('script'),
kwargs.get('cat'), kwargs.get('priority', NORMAL_PRIORITY))
raise dcRaiser(self.__root, kwargs)

[shypike]

That's a rather weird result.
CherryPy is supposed to deliver a tuple consisting of the name (nzbfile.filename) and the content (nzbfile.value).

Does this happen for each NZB upload to SABnzbd or is there any variation (like size)?
Which CherryPy are you using?
Which browser? Does the browser run locally or remote?

[ja30278]

Doesn't seem to vary with nzb size. I've tested with recent versions of Chrome and Firefox. The nzbfile object that cherrypy is delivering looks like it comes through pretty much untouched from the CGI module, which overrides __getattr__ to make .value equivalent to self.file.seek(0); self.file.read(); self.file.seek(0), which makes it make even _less_ sense that an explicit read would succeed.

Could be some sort of race..in which the file isn't done uploading yet (but seems kinda unlikely...the largest nzb I was attempting to upload was way under 1M, on a gigabit local network)

[shypike]

Interesting.
I had been looking at how to prevent receiving the NZB as a huge in-memory blob.
The NZB parser is perfectly capable of parsing from a file instead of a memory object.
Thanks for the tip.

[ja30278]

Yea, I originally had a slightly more intrusive patch that also changed the add_nzbfile function in __init__.py to avoid slurping the file content out of one tmpfile just to write() it back out, but this was the smaller workaround. As an aside, I didn't see an obvious link for sending patches. Is there such a thing?

edit: just saw that this was a sourceforge project

[shypike]

No, we're on github for code-hosting (previously on Launchpad).
SF is just for distributing binaries.
Or were you talking about CherrPy's development?
Its authors aren't the most responsive people...

[dbrooks]

FYI, when the FreeBSD port maintainers commit my diff for news/sabnzbdplus-0.6.9, this problem should be fixed (or, at least, worked around).

My fix was adding the following line directly before "import cherrypy" in SABnzbd.py:

Code: Select all

sys.path.insert(0,'/usr/local/share/sabnzbdplus')

This effectively shoves '/usr/local/share/sabnzbdplus' into the first element of the sys.path list, which coincidentally is where the python "import" function looks when hunting down modules. It looks sequentially, so adding it to the front causes python to look there first for a module called "cherrypy", which it finds. This will happen regardless of whether or not a vanilla version of CherryPy is installed elsewhere on the system.

It's a workaround, but it works. All those using SABnzbd+ on FreeBSD out of ports would do well to upgrade to 0.6.9 when it's committed.

Granted, I haven't been following the entirety of this thread, but I still think making this work with vanilla CherryPy vs. including a patched version would be the ideal long term solution.

[jcfp]

Granted, I haven't been following the entirety of this thread, but I still think making this work with vanilla CherryPy vs. including a patched version would be the ideal long term solution.

I guess everybody here agrees on that, this bug was reported to the cherrypy project months ago by shypike himself.