MovieGrabber - Completely Automated Movie Downloader

[rascalli]

Trying to start it as deamon with : python MovieGrabber.py -d (on QNAP 419P NAS)

Unfortunately it is not working .. any idea's ??

[sl1000]

Don't tell me you want a qpkg out of this too

[splnut]

Ill try this again tonight when i get a sec


Just downloaded the source on Windows 7 64 bit:

Code: Select all

C:\Users\Chad\Desktop\1>python MovieGrabber.py
Traceback (most recent call last):
  File "MovieGrabber.py", line 82, in <module>
    import cherrypy
  File "C:\Python27\lib\site-packages\cherrypy\__init__.py", line 68, in <module
>
    from cherrypy import _cpdispatch as dispatch
  File "C:\Python27\lib\site-packages\cherrypy\_cpdispatch.py", line 200, in <mo
dule>
    punctuation_to_underscores = str.maketrans(
AttributeError: type object 'str' has no attribute 'maketrans'

Downloaded the Binary and when I tried to edit the IMDB settings I get this

500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last):
File "cherrypy\_cprequest.pyc", line 645, in respond
File "cherrypy\lib\encoding.pyc", line 188, in __call__
File "cherrypy\_cpdispatch.pyc", line 29, in __call__
File "MovieGrabber.py", line 3085, in save_config_imdb
TypeError: unsupported operand type(s) for -: 'Decimal' and 'float'
Powered by CherryPy 3.2.0

Installed it on unraid and got

Code: Select all

Traceback (most recent call last):
  File "/mnt/disk/sdc1/Downloader1/1/lib/sitepackages/cherrypy/process/servers.py", line 187, in _start_http_thread
    self.httpserver.start()
  File "/mnt/disk/sdc1/Downloader1/1/lib/sitepackages/cherrypy/wsgiserver/__init__.py", line 1753, in start
    raise socket.error(msg)
error: No socket could be created

[16/Aug/2011:20:52:10] ENGINE Bus STOPPING
[16/Aug/2011:20:52:10] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9191)) already shut down
[16/Aug/2011:20:52:10] ENGINE Stopped thread 'Monitor'.
[16/Aug/2011:20:52:10] ENGINE Stopped thread '_TimeoutMonitor'.
[16/Aug/2011:20:52:10] ENGINE Bus STOPPED
[16/Aug/2011:20:52:10] ENGINE Bus EXITING
[16/Aug/2011:20:52:10] ENGINE Bus EXITED
root@Tower:/mnt/disk/sdc1/Downloader1/1#

[war59312]

Sweet, thanks for the updates. Looking good now.

Nice clean log:

Code: Select all

rob@RobsUbuntuServer:~/Desktop/MovieGrabber$ python MovieGrabber.py
[16/Aug/2011:23:54:14] ENGINE Listening for SIGHUP.
[16/Aug/2011:23:54:14] ENGINE Listening for SIGTERM.
[16/Aug/2011:23:54:14] ENGINE Listening for SIGUSR1.
[16/Aug/2011:23:54:14] ENGINE Bus STARTING
[16/Aug/2011:23:54:14] ENGINE Started monitor thread 'Monitor'.
[16/Aug/2011:23:54:14] ENGINE Started monitor thread '_TimeoutMonitor'.
failed to create drawable
[16/Aug/2011:23:54:14] ENGINE Serving on 127.0.0.1:9191
[16/Aug/2011:23:54:14] ENGINE Bus STARTED
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET / HTTP/1.1" 200 1856 "" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET /stylesheets/default.css HTTP/1.1" 200 4619 "http://127.0.0.1:9191/" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET /stylesheets/defaultcolors.css HTTP/1.1" 200 1479 "http://127.0.0.1:9191/" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET /stylesheets/colorschemes/darkblue.css HTTP/1.1" 200 356 "http://127.0.0.1:9191/" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET /favicon.ico HTTP/1.1" 200 144033 "" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
^C[16/Aug/2011:23:54:26] ENGINE Keyboard Interrupt: shutting down bus
[16/Aug/2011:23:54:26] ENGINE Bus STOPPING
[16/Aug/2011:23:54:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9191)) shut down
[16/Aug/2011:23:54:26] ENGINE Stopped thread 'Monitor'.
[16/Aug/2011:23:54:26] ENGINE Stopped thread '_TimeoutMonitor'.
[16/Aug/2011:23:54:26] ENGINE Bus STOPPED
[16/Aug/2011:23:54:26] ENGINE Bus EXITING
[16/Aug/2011:23:54:26] ENGINE Bus EXITED
[16/Aug/2011:23:54:26] ENGINE Waiting for child threads to terminate...

Ok notice it uses 127.0.0.1 instead of my private IP though. Which means by default you can NOT access it from any other computers on the network. Bummer!

So I simply changed server.socket_host line in webconfig.ini to my private IP address.

So it really should use the correct private IP address instead of 127.0.0.1.

Update: Whoops looks like you already knew that, see line 27 in script. My bad!

[rascalli]

@sl1000 , at this moment not yet ;-) ... would be nice to have this mixed with Couchpotato .. then you have an all in one movie app.

Somehow I do not see it searching for movies.
I set some actors I like ... waited some hours .. but nothing in the logs.

Can I run a manual search ? Or any other idea's ?

[indomiti]

Hey binhex, i tried the "debug MovieGrabber.py" but it does not start properly.. This is the output i get when i try to start it

Code: Select all

File "debug MovieGrabber.py", line 1433
break
SyntaxError: 'break' outside loop

[indomiti]

@sl1000 , at this moment not yet ;-) ... would be nice to have this mixed with Couchpotato .. then you have an all in one movie app.

Somehow I do not see it searching for movies.
I set some actors I like ... waited some hours .. but nothing in the logs.

Can I run a manual search ? Or any other idea's ?

Do you have the "Run now" button in the Home menu/tab? You should have three buttons there, Run Now, Shutdown and Restart. If you don't you have not configured it properly.
If you have then you can click it, if you want it to run automatically you need to set your Schedule..

What this script does it goes to nzbmatrix or whatever site you have and do a search in the section etc that you have configured it to do. It will get 50 results maybe and it just checks if those 50 movies match any of your imdb filters, if it does it downloads it, if not it does nothing. Next time it runs a couple hours later it does the same thing, but maybe this time a new movie has been uploaded that has your criteria for downloading. It does not scan through all old movies on the entire site after the actors you specified.

[binhex]

Hey binhex, i tried the "debug MovieGrabber.py" but it does not start properly.. This is the output i get when i try to start it

Code: Select all

File "debug MovieGrabber.py", line 1433
break
SyntaxError: 'break' outside loop

my bad, forgot to remove the breaks, try this one:-

http://www.mediafire.com/?gk2qn9swd5hpskc

[binhex]

Trying to start it as deamon with : python MovieGrabber.py -d (on QNAP 419P NAS)

Unfortunately it is not working .. any idea's ??

i havent yet added in any ability to run this as a daemon process, i will take a look and see how hard this is to do.

Somehow I do not see it searching for movies.
I set some actors I like ... waited some hours .. but nothing in the logs.

Can I run a manual search ? Or any other idea's ?

ok take a look at FAQ A5, if you have filled in all the pre-requisites then you should see the "Run Now" link available on the Home tab, if you dont see this then you havent filled in all the details.

if you can see the run now button and clicking on it still does nothing then check that you arent specifying a unc path in the folders section (unc not currently supported). if your still getting issues then remove anything sensitive from your config.ini file (can be found in the config folder) and pm me the file and i will take a look for you

[binhex]

Sweet, thanks for the updates. Looking good now.

Nice clean log:

Code: Select all

rob@RobsUbuntuServer:~/Desktop/MovieGrabber$ python MovieGrabber.py
[16/Aug/2011:23:54:14] ENGINE Listening for SIGHUP.
[16/Aug/2011:23:54:14] ENGINE Listening for SIGTERM.
[16/Aug/2011:23:54:14] ENGINE Listening for SIGUSR1.
[16/Aug/2011:23:54:14] ENGINE Bus STARTING
[16/Aug/2011:23:54:14] ENGINE Started monitor thread 'Monitor'.
[16/Aug/2011:23:54:14] ENGINE Started monitor thread '_TimeoutMonitor'.
failed to create drawable
[16/Aug/2011:23:54:14] ENGINE Serving on 127.0.0.1:9191
[16/Aug/2011:23:54:14] ENGINE Bus STARTED
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET / HTTP/1.1" 200 1856 "" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET /stylesheets/default.css HTTP/1.1" 200 4619 "http://127.0.0.1:9191/" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET /stylesheets/defaultcolors.css HTTP/1.1" 200 1479 "http://127.0.0.1:9191/" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET /stylesheets/colorschemes/darkblue.css HTTP/1.1" 200 356 "http://127.0.0.1:9191/" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
127.0.0.1 - - [16/Aug/2011:23:54:16] "GET /favicon.ico HTTP/1.1" 200 144033 "" "Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0"
^C[16/Aug/2011:23:54:26] ENGINE Keyboard Interrupt: shutting down bus
[16/Aug/2011:23:54:26] ENGINE Bus STOPPING
[16/Aug/2011:23:54:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9191)) shut down
[16/Aug/2011:23:54:26] ENGINE Stopped thread 'Monitor'.
[16/Aug/2011:23:54:26] ENGINE Stopped thread '_TimeoutMonitor'.
[16/Aug/2011:23:54:26] ENGINE Bus STOPPED
[16/Aug/2011:23:54:26] ENGINE Bus EXITING
[16/Aug/2011:23:54:26] ENGINE Bus EXITED
[16/Aug/2011:23:54:26] ENGINE Waiting for child threads to terminate...

Ok notice it uses 127.0.0.1 instead of my private IP though. Which means by default you can NOT access it from any other computers on the network. Bummer!

So I simply changed server.socket_host line in webconfig.ini to my private IP address.

So it really should use the correct private IP address instead of 127.0.0.1.

Update: Whoops looks like you already knew that, see line 27 in script. My bad!

hi war59312, glad we are making progress :-), as you correctly identified the default host ip is the loopback address, the reason for this is two fold, firstly my app currently does not have any sort of authentication built in (its on my list) and thus i didnt want to expose this by default to the internet (or any other machine on your home lan), and secondly that i currently dont have any way of detecting the private ip of the host, so for the time being i will be sticking with the loopback ip as the default.

if you wish to change the ip you can either manually do this by editing the webconfig.ini file found in the config folder, or through the webgui on the configuration/general (restart required).

[indomiti]

Ok! Running the debug version shows this error when trying to download poster:

Code: Select all

17/08/2011 10:24:43 Error In Poster Download Function
Traceback (most recent call last):
  File "debug MovieGrabber.py", line 1422, in PosterDownload
    poster_image.load()
  File "MovieGrabber/lib/sitepackages/PIL/ImageFile.py", line 164, in load
    self.load_prepare()
  File "MovieGrabber/lib/sitepackages/PIL/ImageFile.py", line 231, in load_prepare
    self.im = Image.core.new(self.mode, self.size)
  File "MovieGrabber/lib/sitepackages/PIL/Image.py", line 37, in __getattr__
    raise ImportError("The _imaging C module is not installed")
ImportError: The _imaging C module is not installed
17/08/2011 10:24:43 Error In NZBMatrix Search Page Function
Traceback (most recent call last):
  File "debug MovieGrabber.py", line 2301, in RunDownloader
    PosterDownload()
  File "debug MovieGrabber.py", line 1467, in PosterDownload
    ErrorLogging("Poster Download Function")
  File "debug MovieGrabber.py", line 1422, in PosterDownload
    poster_image.load()
  File "MovieGrabber/lib/sitepackages/PIL/ImageFile.py", line 164, in load
    self.load_prepare()
  File "MovieGrabber/lib/sitepackages/PIL/ImageFile.py", line 231, in load_prepare
    self.im = Image.core.new(self.mode, self.size)
  File "MovieGrabber/lib/sitepackages/PIL/Image.py", line 37, in __getattr__
    raise ImportError("The _imaging C module is not installed")
ImportError: The _imaging C module is not installed

[binhex]

Ok! Running the debug version shows this error when trying to download poster:

Code: Select all

17/08/2011 10:24:43 Error In Poster Download Function
Traceback (most recent call last):
  File "debug MovieGrabber.py", line 1422, in PosterDownload
    poster_image.load()
  File "MovieGrabber/lib/sitepackages/PIL/ImageFile.py", line 164, in load
    self.load_prepare()
  File "MovieGrabber/lib/sitepackages/PIL/ImageFile.py", line 231, in load_prepare
    self.im = Image.core.new(self.mode, self.size)
  File "MovieGrabber/lib/sitepackages/PIL/Image.py", line 37, in __getattr__
    raise ImportError("The _imaging C module is not installed")
ImportError: The _imaging C module is not installed
17/08/2011 10:24:43 Error In NZBMatrix Search Page Function
Traceback (most recent call last):
  File "debug MovieGrabber.py", line 2301, in RunDownloader
    PosterDownload()
  File "debug MovieGrabber.py", line 1467, in PosterDownload
    ErrorLogging("Poster Download Function")
  File "debug MovieGrabber.py", line 1422, in PosterDownload
    poster_image.load()
  File "MovieGrabber/lib/sitepackages/PIL/ImageFile.py", line 164, in load
    self.load_prepare()
  File "MovieGrabber/lib/sitepackages/PIL/ImageFile.py", line 231, in load_prepare
    self.im = Image.core.new(self.mode, self.size)
  File "MovieGrabber/lib/sitepackages/PIL/Image.py", line 37, in __getattr__
    raise ImportError("The _imaging C module is not installed")
ImportError: The _imaging C module is not installed

right! that gives me something to work with, ive googled the importerror your getting and found the following article which matches the issue, it looks like i may need to bundle the missing file with the zip and hopefully this will fix the issue:-

http://effbot.org/zone/pil-imaging-not-installed.htm

it looks like the missing file is _imaging.so or _imagingmodule.so on a linux platform, im currently taking a break from programming today but shall take a look at this tommorrow for you, thanks for the traceback :-)

EDIT this is also interesting, i know your running this on linux but perhaps the issue is still around the fact your running python 2.6.x, quote from website:-

This problem is most common on Windows, where extensions built for a given minor version of Python (e.g. 2.1) only works with that version, or revisions thereof (e.g. 2.1.2, but not 2.2).

the version of PIL bundled with MovieGrabber is for python 2.7.x, worth a go updating your version of python?

[binhex]

Ill try this again tonight when i get a sec


Just downloaded the source on Windows 7 64 bit:

Code: Select all

C:\Users\Chad\Desktop\1>python MovieGrabber.py
Traceback (most recent call last):
  File "MovieGrabber.py", line 82, in <module>
    import cherrypy
  File "C:\Python27\lib\site-packages\cherrypy\__init__.py", line 68, in <module
>
    from cherrypy import _cpdispatch as dispatch
  File "C:\Python27\lib\site-packages\cherrypy\_cpdispatch.py", line 200, in <mo
dule>
    punctuation_to_underscores = str.maketrans(
AttributeError: type object 'str' has no attribute 'maketrans'

Downloaded the Binary and when I tried to edit the IMDB settings I get this

500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last):
File "cherrypy\_cprequest.pyc", line 645, in respond
File "cherrypy\lib\encoding.pyc", line 188, in __call__
File "cherrypy\_cpdispatch.pyc", line 29, in __call__
File "MovieGrabber.py", line 3085, in save_config_imdb
TypeError: unsupported operand type(s) for -: 'Decimal' and 'float'
Powered by CherryPy 3.2.0

Installed it on unraid and got

Code: Select all

Traceback (most recent call last):
  File "/mnt/disk/sdc1/Downloader1/1/lib/sitepackages/cherrypy/process/servers.py", line 187, in _start_http_thread
    self.httpserver.start()
  File "/mnt/disk/sdc1/Downloader1/1/lib/sitepackages/cherrypy/wsgiserver/__init__.py", line 1753, in start
    raise socket.error(msg)
error: No socket could be created

[16/Aug/2011:20:52:10] ENGINE Bus STOPPING
[16/Aug/2011:20:52:10] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 9191)) already shut down
[16/Aug/2011:20:52:10] ENGINE Stopped thread 'Monitor'.
[16/Aug/2011:20:52:10] ENGINE Stopped thread '_TimeoutMonitor'.
[16/Aug/2011:20:52:10] ENGINE Bus STOPPED
[16/Aug/2011:20:52:10] ENGINE Bus EXITING
[16/Aug/2011:20:52:10] ENGINE Bus EXITED
root@Tower:/mnt/disk/sdc1/Downloader1/1#

hmm ok this is really odd, ive just tested both running the compiled binary version and the source code version on a completely clean windows 7 64bit vm and both work straight out of the box! i just dont see the errors your getting.

ok few things to check:-

are you running the windows binary as a local admin of the machine? - if not then please make sure you are running as an admin
are you completely deleting ALL files and folders from previous builds? - if not then please do so and then extract the latest version and try it
what version of python have you got installed? - ideally you should be running python 2.7.x, python 3.x is not compatible.

ive done some google searches:-

problem 1.
type object 'str' has no attribute 'maketrans'

check this link out, points at wrong version of python on your host http://www.cherrypy.org/ticket/1066

problem 2.
unsupported operand type(s) for -: 'Decimal' and 'float'

this looks to be some math calculation of a decimal and float value, i will take a look at this and see if i can trace it.

problem 3.
No socket could be created

this sounds like cherrypy couldnt bind to the loopback ip 127.0.0.1 which is wierd!, possibly a firewall restriction?, you can try changing this in the webconfig.ini file in the config folder, just alter the following line:-

server.socket_host = "127.0.0.1"

to

server.socket_host = "0.0.0.0" or if you know it your private ip address.

[splnut]

Got it working but when I try to check a Genre in the Preferred Genre I get this:

Code: Select all

500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last):
  File "/mnt/disk/More/Downloader1/1/lib/sitepackages/cherrypy/_cprequest.py", line 645, in respond
    response.body = self.handler()
  File "/mnt/disk/More/Downloader1/1/lib/sitepackages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/mnt/disk/More/Downloader1/1/lib/sitepackages/cherrypy/_cpdispatch.py", line 29, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "MovieGrabber.py", line 3085, in save_config_imdb
    kwargs["preferred_rating2"] = decimal.Decimal(kwargs["good_rating2"]) - 0.1
TypeError: unsupported operand type(s) for -: 'Decimal' and 'float'

[binhex]

Got it working but when I try to check a Genre in the Preferred Genre I get this:

Code: Select all

500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last):
  File "/mnt/disk/More/Downloader1/1/lib/sitepackages/cherrypy/_cprequest.py", line 645, in respond
    response.body = self.handler()
  File "/mnt/disk/More/Downloader1/1/lib/sitepackages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/mnt/disk/More/Downloader1/1/lib/sitepackages/cherrypy/_cpdispatch.py", line 29, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "MovieGrabber.py", line 3085, in save_config_imdb
    kwargs["preferred_rating2"] = decimal.Decimal(kwargs["good_rating2"]) - 0.1
TypeError: unsupported operand type(s) for -: 'Decimal' and 'float'

hi splnut, glad your getting somewhere, now i would be intersted to hear how you resolved your issues if you have the time.

ok onto your posted traceback, ive tracked down the bug it was to do with comparing a decimal value with a float, i was assuming the float gets converted which it doesnt, for anybody who is having problems sleeping here is a great post on the subject :-) http://stackoverflow.com/questions/1062 ... comparison

ok short story is the fix will be included in my next release, for the time being you can get around this by making sure the preferred rating is always less than your good rating (which it should be in any case).

let me know how you get on.

binhex.