SABnzbd Smart Downloader

[minimad]

Hi All,

This oringally started as a modification of the script posted by popcornarsonist here:

http://forums.sabnzbd.org/index.php?topic=1335.0

I feel it's now taken on it's own life and now resides here:

http://my-svn.assembla.com/svn/SABnzbd_Smart_Downloader

The original thread has a lot of contributors and various versions of both the original tv script and movie script.  we wouldn't be where it is without those people.

I think to make it easier to help people keep track of development of this version of the script, it now deserves it's own thread.

If anyone disagree's then I'll keep posting to the original and close this one, but at 13 pages with multiple versions contained within, it can get tricky to work out who's script you are running.

EDIT:

'Trac' can be found here (error tickets etc)
http://my-trac.assembla.com/SABnzbd_Smart_Downloader

[rollingeyeball]

It does deserve it's own thread!
I now run this script and I'm very happy.

This has been a useless post to subscribe me to the thread and to attain the acclaimed first post of the thread.
Good day to you sirs.

[markus101]

Woohoo! I was going to suggest/make a new one myself, I'm gettign pretty lost in what is going on in that last thread, especially when its talking about two different scripts.

Maybe not as useless, but I'll steal #2 on this thread and subscribe to its goodness!

Thanks for all the awesome work Minimad!

-Markus

[rollingeyeball]

mmk, I need to reinstate my password for sabnzbd
What are the extra params I need to use for this?

[markus101]

Pretty sure you set you SABnzbd connection in the script to:

Code: Select all

mySAB="http://username:[email protected]:8080"

-Markus

[rollingeyeball]

I went OHH YEAH as I read that, then tested it and nogo.
Anyone else?

EDIT:
Fixd'

Code: Select all

# additional SAB parameters to e.g. specify the category, and/or script to run.  see http://sabnzbd.wikidot.com/automation-support
my_SAB_params="&cat=tv&ma_username=Admin&ma_password=Password"

Just for anyone that missed that glaring link in the comment like I did

[BrandonG777]

Working on a new version that maintains a history file. Will post tomorrow after I clean up the code a bit and a little more testing. 

[bnevets27]

I've been trying to get this script to run for a little while now. Using the latest as of writing, revision 12.  It runs and searches but keeps returning "FAILED PROCESSING" I'm on win xp using nzbs.org.


Also how do I get an output log? I can watch it run in the command prompt but I have a feeling I'm going to need to output a log. Bare with me this is the first time I've worked with scripts.

[rollingeyeball]

No clue about Failed processing, I havn't feen following this for long, so I don't get the debug AT ALL
However to get it to show you an output is pretty easy.

Run it from the commandline,
so just type in it's fullpath
for me that's
E:\Python26\python.exe  and it'll run there, and stay.

Or run it from with the IDLE editor.
If you've installed it (comes with python), right click the .py file 'edit in idle editor' then press f5

Oh, and HISTORY WOULD BE AWESOME.
So then it won't download if it matches something in history too?
making ignore folders is .. a slight inconvenience. but too much for me.
If i could let add a show, then just delete it from queue and res assured it won't download it again I'd be happy.

[markus101]

There isn't a log file unless you direct the output to a log file... simply by doing the following.

Code: Select all

python tv_rss.py > output.log

or the following to append to an already created log file

Code: Select all

python tv_rss.py >> output.log

The above examples will work on Linux or Windows.

I was having the Failed Processing message with Revision 7, haven't tried a newer version, I'll give that a shot a bit later tonight hopefully... Never seen that error before Revision 7 from the SVN, so it's something with his new code - might take a few days to work it out.

http://pastebin.com/f5bc3244f is the version I'm using with success, doesn't support different naming structures (Season 1  instead of Season 01), but it works quite well.

-Markus

PS History would be grrrreat, but I'd rather see a fix for () and ' first (which would make the History better due to the issue with regex's).
PPS - THANKS for the work guys.

[rollingeyeball]

It worked earlier..
Now I get this

MISSING: Show...
ENTERED isInQueueSmart
FAILED PROCESSING: SHOWNAME

Seems to work all good now, But when i got that failed error before, one show worked, others didn't. so..
Could you reword all the debugs perhaps?

[bnevets27]

yah!! I got version 0.88 to work! I was trying to use nzbs.org before but I just got a newzbin account (thanks to this form) and now it works. Not sure why that would matter but I'm currently happy I got it going. Now I need to change the formatting from Season 01 to Season 1 , I see that is in ver. 12 but that version isn't currently working. Any small change I can do to get that working in ver. 0.88?

Also I can't seem to get python to output a log file. I've tried "python tv_rss.py > output.log" in the command prompt and nothing happens, doesn't run the script, doesn't give me an error, nothing. Any ideas why I can't get this to work?
It is "python tv_rss.py*space*>*space*output.log" correct?

Thank you to all that have put all the work into making this, its really incredible how nicely this works and its still improving. Excited to seem my tv shows just appear on my comp. 

[minimad]

Hi bnevets27,

The only two confirmed feeds I know work are newzbin & tvbinz (although tvbinz seems to be activly blocking me now), I'm working on making sure that SSD (SABnzbd Smart Downloader) can handle gzip'd feeds (tvbinz is one, which is why you need feedburner for now).

Changing the season format from 01 to 1, not quite an easy change

These are version 0.88 changes:
Change line 232 to
config['with_ep_name'] = '%(seriesname)s - S%(seasno)dE%(epno)02d - %(epname)s'

Which changes the report name to 1 rather than 01, but I don't think that solves all of the issues, which is why there are major code changes in revision 12.

I'm also writing in a logging function, but due to work, it could be a week or so.

try python tv_rss.py > c:\output.log

@All

Please use the 'Trac' on the SVN site to log any errors etc, it should make it easier for me to know about them.


@rollingeyeball

Can you post me the entire output, I'm guessing it has something to do with the showname or report title, seeing the entire output will help.

@BrandonG777
Which version are you working against?

@markus101
Lucky you, your () and ' are the first ticket in 'Trac'

[BrandonG777]

This is my addition posted this in the other threat so I think most people missed it but here's my contribution.

Mine is for newzbin and the changes are to maintain a history file, verify that the movie hasn't been downloaded previously by checking the history file and also checking the dirs for existing movies. I also changed it to look for a watch file instead of dirs. It also supports a username and password for sabnzbd. This is my first time dealing with python so there are probably better ways to handle these things but it appears to be working.

Code: Select all

#!/usr/bin/python

import urllib
import xml.dom.minidom
import re
import time
import os
import sys

##########
# MOVIES #
##########

################################### change to your own values #####################################

# set this to the location of your downloads.  the first entry must be your live download area
movie_dirs=[ "/mnt/disk1/share/nzb/Movies", "/mnt/disk1/share/Movies" ]

# history file (must be create a blank file first)
history_file="/home/brandon/.movies.dat"

# a directory that will hold string to match against movies that you want no matter what score they get
watch_file="/mnt/disk1/share/nzb/watchfile.txt"

# your SABNZB address and credentials
my_SAB="http://hostname:8000"
username="brandon"
password="******"

# the RSS link to your search
url="newzbinurl"

################################### change to your own values #####################################

def removeIllegalChars(origstring):
	if sys.platform == "win32":
		newstring=re.sub("[\\?,]", ".", origstring)
		newstring=re.sub("[:]", ";", newstring)
	else:
		newstring=re.sub("[/?,]", ".", origstring)
#	print origstring+" becomes "+newstring
	return newstring
	

movie_list = []
for dir in movie_dirs:
	for movie in os.listdir(dir):
		movie_list.append(movie)
# Read History File
filein = open(history_file,"r")
for line in filein.readlines(): 
	movie_list.append(line)
filein.close()
print str(len(movie_list))+" movies already downloaded"

watch_list = []
watch_filein = open(watch_file,"r")
for line in watch_filein.readlines():
	watch_list.append(line)
watch_filein.close()
print "watching for "+str(len(watch_list))+" movies"

dom=xml.dom.minidom.parse(urllib.urlopen(url))
# go through each entry returned from the above newzbin search
for node in dom.getElementsByTagName("item"):
	
	report_title=node.getElementsByTagName("title")[0].childNodes[0].data
	print time.strftime("%Y-%m-%d %H:%M:%S")+": "+ report_title

	optimized_movie_title = report_title.split(" (")[0]
	if re.match(re.compile("[0-9][0-9][0-9][0-9]\)"), report_title):
		if re.match(re.compile("[0-9][0-9][0-9][0-9]\)"), report_title.split(" (")[1]):
			movie_year = report_title.split(" (")[1].split(")")[0]
	#optimized_movie_title = movie_title

	legal_movie_title = removeIllegalChars(report_title)
						
	more_info=node.getElementsByTagName("report:moreinfo")[0].childNodes[0].data
	msgid=node.getElementsByTagName("report:id")[0].childNodes[0].data
	
	# is there an imdb link?  if there is, follow it and check the score
	if re.compile("http://www\.imdb\.com.*").match(more_info):
		
		sock = urllib.urlopen(more_info)
		htmlSource = sock.read()                     
		sock.close()
		
		if re.compile("<b>[7-9]\.[0-9]/10</b>").search(htmlSource) and re.compile(">[0-9]*,[0-9][0-9][0-9] votes<").search(htmlSource):
			download = 1
		else:
			download = 0

# doesn't get a good imdb score, but am I watching for it?
		if download == 0:
			for watchfile in watch_list:
				if re.match(optimized_movie_title, watchfile):
					print optimized_movie_title+" is not very good but according to watch keyword \""+watchfile+"\" you want it - you're the boss!"
					download = 1
					break

			
		if download == 1:
			# it's worth a download but do we have it already?
			regex_movie_title_string = removeIllegalChars(optimized_movie_title)
			regex_movie_title = re.compile(regex_movie_title_string)
			regex_movie_year = re.compile(".*\([0-9][0-9][0-9][0-9]\).*")
			print "Checking to see if we have : \"" + regex_movie_title_string+"\"",
			for movie_file in movie_list:
				if re.match(regex_movie_title, movie_file) and re.match(regex_movie_year ,movie_file):
					print " - yes"
					download = 0
					break

		
			if download == 1:
				print " - no"

				# we want it, we don't have it, but is it in the queue already?
				queue=xml.dom.minidom.parse(urllib.urlopen(my_SAB+"/sabnzbd/api?mode=qstatus&output=xml&ma_username="+username+"&ma_password="+password))
				for job in queue.getElementsByTagName("job"):
					filename=job.getElementsByTagName("filename")[0].childNodes[0].data
					regex_filename_string = removeIllegalChars(optimized_movie_title)
					regex_filename = re.compile(regex_filename_string)
					if re.match(regex_filename, filename):
						download = 0

			if download == 1:
				print time.strftime("%Y-%m-%d %H:%M:%S")+": ++++++ added "+ optimized_movie_title+" ++++++"
				# this will prevent the same movie being downloaded twice in this session
				#os.makedirs(movie_dirs[0]+"\\"+legal_movie_title)
				# add this title to our view of what movies we have
				movie_list.append(legal_movie_title)
				
				# Append movie to history file
				fileout = open(history_file, "a")
				fileout.write( legal_movie_title+ "\n" )
				fileout.close()
				urllib.urlopen(my_SAB+"/sabnzbd/api?mode=addid&name="+msgid+"&ma_username="+username+"&ma_password="+password)
				

dom.unlink()

[minimad]

Oh,

And just so everyone knows.

tv_rss_088.py is the current working file and has been added to SVN.

I'll post when the newest revision is working, it's kind of not at the moment, its half way between the two code streams so it doesn't know if its coming or going so to speak.

I had to make some changes to the code for the new caching function which really doesn't like running inside PythonWin.

0.88 is what I'm using as 'Live'.