Problem starting after upgrading to 0.7.20 (ubuntu 12.04)

[mr604]

sabnzbdplus 7.2.0 - ubuntu 12.04 -

Installed from jcfp repos. Tried upgrading failed to upgrade b/c of start issues

Code: Select all

Traceback (most recent call last):
  File "/usr/bin/sabnzbdplus", line 1873, in <module>
    main()
  File "/usr/bin/sabnzbdplus", line 1292, in main
    from util.pystone import pystones
ImportError: No module named util.pystone

So it's breaking around here:

Code: Select all

        # measure and log Pystone performance, and - if possible - CPU type
        try:
            # First try pystone from Python test libary
            from test.pystone import pystones
        except:
            # otherwise use the one provided by SABnzbd
            from util.pystone import pystones
        pystonetime,pystoneperformance = pystones(1000)
        logging.debug('CPU Pystone available performance is %s',int(pystoneperformance))

I poked at the installed pystone and it's there.

Code: Select all

# python --version 
Python 2.7.3
# file /usr/lib/python2.7/test/pystone.py
/usr/lib/python2.7/test/pystone.py: a /usr/bin/python2.7 script, ASCII text executable

# python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test.pystone import pystones
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pystone

Not sure why this is failing out. Not sure where to check the baked in util.pystone on the install...

Please advise.

m!

[sander]

Ouch. That is my fault. I will follow up on thuis

[mr604]

Wonderful. Thanks for looking into this.

m!

[sander]

Now a response while sitting at my computer, not phone:

First of all: I programmed that piece of code, so that's why I said it's my fault it goes wrong on your system. I have to put in an extra try/except.

Now an analysis:

I thought pystone.py was installed by default if you have python installed. Not so on your system? Can you do this:

Code: Select all

sudo updatedb
locate -i pystone.py

FWIW on my system:

Code: Select all

$ what-provides /usr/lib/python2.7/test/pystone.py
libpython2.7-stdlib:i386: /usr/lib/python2.7/test/pystone.py

Furthermore, I expected that the PPA package would provide everything, including the sabnzbd/util directory including mailslot.py and pystone.py. But I can't find it:

Code: Select all

sander@haring:~$ dpkg -c  /var/cache/apt/archives/sabnzbdplus_0.7.20-0ubuntu1~jcfp1~trusty_all.deb | grep -i -e mailslot -e pystone
sander@haring:~$

... searching for 'mailslot' and 'pystone' does not reveal anything. Weird. @jcfp: is that how it should be?

There is a utils directory, but no util directory:

Code: Select all

sander@haring:~$ dpkg -c  /var/cache/apt/archives/sabnzbdplus_0.7.20-0ubuntu1~jcfp1~trusty_all.deb | grep -i util
drwxr-xr-x root/root         0 2014-11-22 12:41 ./usr/share/sabnzbdplus/sabnzbd/utils/
-rw-r--r-- root/root     16730 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/rarfile.py
-rw-r--r-- root/root      2419 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/upload.py
-rw-r--r-- root/root     29511 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/listquote.py
-rw-r--r-- root/root      3449 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/json.py
-rw-r--r-- root/root      4329 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/pathbrowser.py
-rw-r--r-- root/root        14 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/__init__.py
-rw-r--r-- root/root      4838 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/servertests.py
-rw-r--r-- root/root     19598 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/kronos.py
-rw-r--r-- root/root      9951 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/systrayiconthread.py
-rw-r--r-- root/root      5833 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/ssmtplib.py
-rw-r--r-- root/root      2892 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/certgen.py
-rw-r--r-- root/root     11573 2014-11-21 20:41 ./usr/share/sabnzbdplus/sabnzbd/utils/rsslib.py
sander@haring:~$

[jcfp]

Cannot reproduce this, tried the 12.04 from http://virtualboxes.org/images/ubuntu/ and it all works (importing pystone, same from within sab, with or without os updates).

... searching for 'mailslot' and 'pystone' does not reveal anything. Weird. @jcfp: is that how it should be?

Mailslot contains only windoze stuff; the copy of pystone.py isn't needed as it is always supplied by the python packages on which sab depends. So yes, 'from util.pystone import pystones' failing is no surprise: the pystone file is not included in the sab package, because that code path shouldn't ever be used in the first place. In this case, for precise:

Code: Select all

Version: 0.7.20-0ubuntu1~jcfp1~precise
Depends: python (>= 2.7.1-0ubuntu2)

satisfied by python (2.7.3-0ubuntu2.2), which in turn:

Code: Select all

Depends: python2.7 (>= 2.7.3)

satisfied by python2.7 (2.7.3-0ubuntu3.5), providing (as confirmed by the output supplied from the topic starter...):

/usr/lib/python2.7/test/pystone.py

The real question is why importing the (system-wide) pystone module fails for TS, with what seems to be a normally updated python2.7 from ubuntu's (security) updates:

Code: Select all

# python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test.pystone import pystones
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pystone

Suggestions, anyone?


PS: disabling sab's debug logging is a viable workaround, in case you needed one.

[mr604]

Good morning, gents. As Sander requested, here is the output of the commands.

Code: Select all

$ sudo updatedb
$ locate -i pystone.py
/usr/lib/python2.7/test/pystone.py
/usr/lib/python2.7/test/pystone.pyc
/usr/lib/python3.2/test/pystone.py
/usr/local/lib/python2.7/dist-packages/test/pystone.py

Code: Select all

$ what-provides /usr/lib/python2.7/test/pystone.py
python2.7: /usr/lib/python2.7/test/pystone.py

The real question is why importing the (system-wide) pystone module fails for TS, with what seems to be a normally updated python2.7 from ubuntu's (security) updates:

I can confirm this is the version of python installed with the system.

I think something in the python search path is messed up here?? I'm really not sure why...

Code: Select all

$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for dir in sys.path:
...     print dir
... 

/usr/local/lib/python2.7/dist-packages/pyserial-2.6-py2.7.egg
/usr/local/lib/python2.7/dist-packages/simplejson-3.3.0-py2.7-linux-x86_64.egg
/usr/local/lib/python2.7/dist-packages/gdata-2.0.18-py2.7.egg
/usr/local/lib/python2.7/dist-packages/todoist-0.0.1-py2.7.egg
/usr/local/lib/python2.7/dist-packages/python_twitter-1.1-py2.7.egg
/usr/local/lib/python2.7/dist-packages/requests_oauthlib-0.4.0-py2.7.egg
/usr/local/lib/python2.7/dist-packages/oauthlib-0.6.0-py2.7.egg
/usr/local/lib/python2.7/dist-packages/gmusicapi-3.1.1_dev-py2.7.egg
/usr/local/lib/python2.7/dist-packages/appdirs-1.3.0-py2.7.egg
/usr/local/lib/python2.7/dist-packages/mock-1.0.1-py2.7.egg
/usr/local/lib/python2.7/dist-packages/oauth2client-1.2-py2.7.egg
/usr/local/lib/python2.7/dist-packages/proboscis-1.2.6.0-py2.7.egg
/usr/local/lib/python2.7/dist-packages/validictory-0.9.3-py2.7.egg
/usr/local/lib/python2.7/dist-packages/httplib2-0.9-py2.7.egg
/usr/local/lib/python2.7/dist-packages/pip-1.5.6-py2.7.egg
/usr/local/lib/python2.7/dist-packages/walter-0.1-py2.7.egg
/usr/local/lib/python2.7/dist-packages/geeknote-0.2a-py2.7.egg
/usr/local/lib/python2.7/dist-packages/thrift-0.9.1-py2.7-linux-x86_64.egg
/usr/local/lib/python2.7/dist-packages/markdown2-2.3.0-py2.7.egg
/usr/local/lib/python2.7/dist-packages/html2text-2014.9.25-py2.7.egg
/usr/local/lib/python2.7/dist-packages/evernote-1.25.0-py2.7.egg
/usr/local/lib/python2.7/dist-packages/oauth2-1.5.211-py2.7.egg
/usr/lib/python2.7
/usr/lib/python2.7/plat-linux2
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages/PIL
/usr/lib/python2.7/dist-packages/gst-0.10
/usr/lib/python2.7/dist-packages/gtk-2.0
/usr/lib/pymodules/python2.7
/usr/lib/python2.7/dist-packages/ubuntu-sso-client
>>> 

I think it's the 'test' directory in the /usr/lib/python2.7 path.

Code: Select all

$ python -m test.regrtest  
/usr/bin/python: No module named test.regrtest
$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/usr/lib/python2.7/test')
>>> import pystone
>>> 

This is weird....

Code: Select all

>>> import regrtest
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/test/regrtest.py", line 213, in <module>
    from test import test_support
ImportError: cannot import name test_support

I tried re-installing python2.7 to no avail. I can't purge the package because damn near everything depends on it in Ubuntu.

Removing debug logging works for me, so I have sabnzbd running again. I'm putting this issue down for the moment.
If you want me to poke at something on this machine, just let me know.


Thanks

m!

[jcfp]

I suspect there's something in those /usr/local/lib/python2.7/dist-packages/* paths that's causing your issues.

What if you run python like this (no output is good...):

Code: Select all

PYTHONPATH=/usr/lib/python2.7 python -c 'from test.pystone import pystones'

[aSystemOverload]

Hi,
Don't know if related, but in the last two days SABnzbd has stopped working, I can't access the webfront. I've tried reinstalling using:

Code: Select all

tar xzf SABnzbd-0.7.20-src.tar.gz
mv SABnzbd-0.7.20 /usr/local/sabnzbd
chown -R sabnzbd:nzb /usr/local/sabnzbd

Then the usual:

Code: Select all

/etc/init.d/sabnzbdplus start

Trying to access via 192.168.1.14:8080/sabnzbd, which has worked for a number of months till now. I'm running Debian Jessie. If you can point me the direction of any specific logs that would be useful to sort this out.

[mr604]

jcfp: Negative, Ghost Rider.

Code: Select all

$ PYTHONPATH=/usr/lib/python2.7 python -c 'from test.pystone import pystones'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named pystone

I suspect that may be the case, as well. That being said, I'm not sure that trick did what you hoped it would do:

Code: Select all

PYTHONPATH=/usr/lib/python2.7 python -c 'import sys; print sys.path'
['', '/usr/local/lib/python2.7/dist-packages/pyserial-2.6-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/simplejson-3.3.0-py2.7-linux-x86_64.egg', '/usr/local/lib/python2.7/dist-packages/gdata-2.0.18-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/todoist-0.0.1-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/python_twitter-1.1-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/requests_oauthlib-0.4.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/oauthlib-0.6.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/gmusicapi-3.1.1_dev-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/appdirs-1.3.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/mock-1.0.1-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/oauth2client-1.2-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/proboscis-1.2.6.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/validictory-0.9.3-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/httplib2-0.9-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/pip-1.5.6-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/walter-0.1-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/geeknote-0.2a-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/thrift-0.9.1-py2.7-linux-x86_64.egg', '/usr/local/lib/python2.7/dist-packages/markdown2-2.3.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/html2text-2014.9.25-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/evernote-1.25.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/oauth2-1.5.211-py2.7.egg', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']

That is it, however. It is completely something in the search path.

Code: Select all

$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path=['/usr/lib/python2.7']
>>> print sys.path
['/usr/lib/python2.7']
>>> import test.pystone
>>> 

That being said I am more confused now, than I was before.

I futzed around with removing every path and re-adding them back in. I had mixed results.

Stranger still...

Code: Select all

python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.sort()
>>> import test.pystone
>>> 

So I went through by hand and did this:

Code: Select all

>>> sys.path.reverse()                                                                                                                                                           
>>> print sys.path.pop()                                                                                                                                                         
/usr/lib/pymodules/python2.7
>>> sys.path.reverse()                                                                                                                                                           >>> import test.pystone                                                                                                                                                          Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pystone

Until I got to an empty list...So, I did this:

Code: Select all

>>> print sys.path.pop()                                                                                                                                                         
/usr/lib/python2.7
>>> print sys.path
[]
>>> sys.path.append('/usr/lib/python2.7')                                                                                                                                  
>>> print sys.path
['/usr/lib/python2.7']
>>> import test.pystone                                                                                                                                                          
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pystone

Am I taking crazy pills? Sanity check:

Code: Select all

>>> import test.pystone
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pystone
>>> sys.path=['/usr/lib/python2.7']
>>> import test.pystone
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pystone

Yep. I'm taking crazy pills. I am really confused. If I try and call test.pystone before zeroing out the sys.path it WON'T find it ever.
I can't do this any more I have a bad headache.

aSystemOverload: You might want to look into installing with the Debian/Ubuntu packages and repos instead of the tarball. Maybe: https://forums.sabnzbd.org/viewtopic.php?f=16&t=9844 ?


Cheers

m!

[mr604]

Okay. So I had one more kick at the can, and figured it out. I hobbled together this crummy shell script to iterate through and remove each library and test.

Code: Select all

#!/bin/bash

FUCK=" /usr/local/lib/python2.7/dist-packages/pyserial-2.6-py2.7.egg /usr/local/lib/python2.7/dist-packages/simplejson-3.3.0-py2.7-linux-x86_64.egg /usr/local/lib/python2.7/dist-packages/gdata-2.0.18-py2.7.egg /usr/local/lib/python2.7/dist-packages/todoist-0.0.1-py2.7.egg /usr/local/lib/python2.7/dist-packages/python_twitter-1.1-py2.7.egg /usr/local/lib/python2.7/dist-packages/requests_oauthlib-0.4.0-py2.7.egg /usr/local/lib/python2.7/dist-packages/oauthlib-0.6.0-py2.7.egg /usr/local/lib/python2.7/dist-packages/gmusicapi-3.1.1_dev-py2.7.egg /usr/local/lib/python2.7/dist-packages/appdirs-1.3.0-py2.7.egg /usr/local/lib/python2.7/dist-packages/mock-1.0.1-py2.7.egg /usr/local/lib/python2.7/dist-packages/oauth2client-1.2-py2.7.egg /usr/local/lib/python2.7/dist-packages/proboscis-1.2.6.0-py2.7.egg /usr/local/lib/python2.7/dist-packages/validictory-0.9.3-py2.7.egg /usr/local/lib/python2.7/dist-packages/httplib2-0.9-py2.7.egg /usr/local/lib/python2.7/dist-packages/pip-1.5.6-py2.7.egg /usr/local/lib/python2.7/dist-packages/walter-0.1-py2.7.egg /usr/local/lib/python2.7/dist-packages/geeknote-0.2a-py2.7.egg /usr/local/lib/python2.7/dist-packages/thrift-0.9.1-py2.7-linux-x86_64.egg /usr/local/lib/python2.7/dist-packages/markdown2-2.3.0-py2.7.egg /usr/local/lib/python2.7/dist-packages/html2text-2014.9.25-py2.7.egg /usr/local/lib/python2.7/dist-packages/oauth2-1.5.211-py2.7.egg /usr/lib/python2.7 /usr/lib/python2.7/plat-linux2 /usr/lib/python2.7/lib-tk /usr/lib/python2.7/lib-old /usr/lib/python2.7/lib-dynload /usr/local/lib/python2.7/dist-packages /usr/local/lib/python2.7/dist-packages /usr/lib/python2.7/dist-packages /usr/lib/python2.7/dist-packages/PIL /usr/lib/python2.7/dist-packages/gst-0.10 /usr/lib/python2.7/dist-packages/gtk-2.0 /usr/lib/pymodules/python2.7 /usr/lib/python2.7/dist-packages/ubuntu-sso-client"

for ITEM in $FUCK
do
    echo $ITEM
    python -c "import sys; sys.path.remove('$ITEM'); import test.pystone"
done

/usr/local/lib/python2.7/dist-packages/html2text-2014.9.25-py2.7.egg

This jerk right here had:

Code: Select all

$ cd /usr/local/lib/python2.7/dist-packages/html2text-2014.9.25-py2.7.egg
$ ls
EGG-INFO  html2text  test

A test directory earlier in the search path.

pip uninstall that package and works like a charm.

/me sighs....


m!

[jcfp]

I'm not sure that trick did what you hoped it would do:

Apparently not, it was supposed to bring that path to the front of the line. Anyway, glad you got it figured out in the end!

[sander]

@jcfp:

What do you think:
- should I change the code so that a missing pystone.py does not result in the error posted?
- are you willing to include pystone.py in your PPA?
- or just leave it this way because the problem was caused by an incorrect setup (it was, wasn't it)?

[jcfp]

- should I change the code so that a missing pystone.py does not result in the error posted?
- are you willing to include pystone.py in your PPA?
- or just leave it this way because the problem was caused by an incorrect setup (it was, wasn't it)?

No, no, kind of. The whole thing comes down to 'pollution' on the python path. The potential issues resulting from that aren't restricted to any specific module.

Edit: you can easily reproduce this kind of thing by creating bogus modules in the directory that's is prepended to the python path in the packaged versions of sab:

Code: Select all

pkill nzbd # stop sab
sudo mkdir /usr/share/sabnzbdplus/test
sudo touch /usr/share/sabnzbdplus/test/__init__.py
# assuming permissions allow read access for normal users...
sabnzbdplus -l2

When you rename the polluting directory from test to logging, you get in the way of another module and thus earn yourself a slightly different traceback, but the end result is exactly the same.