Autodownload script

[doubledrat]

Hi All,

I have been a usnet user for almost 8 years now (giganews and newsbin) and just discovered Sabnzbd and the ease of nzb files.  I just love all this post download management as it saves a TON of time. 

I am not a newzbin.com user as I just discovered the service (would greatly appreciate a sponsor to join) and my question is from an RSS standpoint how does this python script compare to using NZB-TV1.20?  http://two-sided.com/cs/  That program looks very slick from a standpoint of finding exactly what you are looking for and auto downloading the nzb files.

I am very eager to get more of my work flow automated.

Many thanks for the feedback.

Greg

well, I was using nzbtv and dropped it like a hot potato when I discoverd this script, but a lot of that is because I'm able to change the script to do what I want! 

NZBtv won't grab films based on their IMDB score.  You also can't be as creative with your search parameters.  NZBtv might be more appropriate for the non-technical though.  Try the scripts.  see what you think - they don't cost anything

PS there's a thread for newzbin invites

[pilGrim]

Thanks doubledrat, I like the auto dl for movies option along with a watch list and for TV shows I like how the script should reference my current collection to avoid dup downloads.  I am ready to give this a try.  I just installed SABnzbd a few days ago (already downloaded around 400 gig with it), and am a little fuzzy on a few script issues, would appreciate help from anyone in the group with the following questions:

1) My categories section under configure is different than what is listed in the online help.  I do not see a place to reference the path to my script for each category.  Under folders area, I see a place to reference where my scripts are located.  Do I put each script into a subfolder for each category?  Is there a naming convention that is important?  What about the extension of the file, I dled them as .txt files).  Basically can someone point me to some instructions on how to get the scripts to run via a scheduler as suggested in the first post? 

2) Doubledrat, can I use your TV script with MyTvNZB Feed? http://mytvnzb.foechoer.be/  I see the place in the script for the RSS feed, but wanted to make sure that other coding below was not specific to Newzbin.  I see xxhds has a movie version to work with non Newzbin sites, not sure if your TV script will have issues with another site.

3) My TV Show naming convention is as follows:  M:\TV Shows\30 Rock\30 Rock - Season 02\30 Rock - 02 - 08.avi   Can you guide me to the areas I would need to change in the script to accommodate this structure? I am worried about the renaming of the files after they are moved.  Or does the TV Folder management part of configuration still kick in?  I would like to know what to change so in the future I can update new versions that are posted.  Would be a great variable for the user to set at the top, maybe based on TV Folder sorting option settings.


4) with the TV Folder Folder management part of configuration all my files move and get named perfectly but I also get .nfo, .sfv, .nzb and sometimes a sample file as they must be inside the rar set.  Is there a way to make sure that only the main video file is moved over, or that the other files are deleted post clean up?

EDIT #4
I found the option under config/general to delete certain file extensions after processing.  I also have under switches the option for no sample files, but I still get sample fines from time to time.  Is this because they are inside the main RAR set or is there an issue with this function.  Having the sample files show up in my TV section really messes up my Meedio system after it imports the shows on my HTPC

5) anything else you think I need to know before giving this a real test?  My TV Show collection is VERY large, so I will test with a smaller collection to get everything worked out.

Sorry for so many questions and thank you in advance for guidance.  I am so excited to get this up and running as I spend way to much time hunting, dling, cleaning up nameing, and moving files!

Greg

[pilGrim]

Hey Doubledrat,

I program in VBA for excel so I am able to follow the basic logic of these scripts although I know nothing of python so would have a hard time making substantial changes.  Other than structural differences, it looks like the main difference from a logic standpoint between your movie script and xxhds's script is that he has a year check to only get newer movies if you wish and you have a watch list.  What I am not sure of if the parsing of the actual feed and how specific yours is to newzbin versus his to nzbs.org.  Is it safe to assume both of these scripts are rss feed specific to either newzbin (for yours) and nzbs.org for xxhds's? 

I like the year check for only newer movies as I already have a TON and really only want new, but I also like you watch list.  Any chance you could add the year check as a variable to turn off or on in the top part of your script?

Or xxhds, if you are still out there can you add the watch list functionality to your script? 

TIA for all the help and advice.

[doubledrat]

Thanks doubledrat, I like the auto dl for movies option along with a watch list and for TV shows I like how the script should reference my current collection to avoid dup downloads.  I am ready to give this a try.  I just installed SABnzbd a few days ago (already downloaded around 400 gig with it), and am a little fuzzy on a few script issues, would appreciate help from anyone in the group with the following questions:

1) My categories section under configure is different than what is listed in the online help.  I do not see a place to reference the path to my script for each category.  Under folders area, I see a place to reference where my scripts are located.  Do I put each script into a subfolder for each category?  Is there a naming convention that is important?  What about the extension of the file, I dled them as .txt files).  Basically can someone point me to some instructions on how to get the scripts to run via a scheduler as suggested in the first post? 

2) Doubledrat, can I use your TV script with MyTvNZB Feed? http://mytvnzb.foechoer.be/  I see the place in the script for the RSS feed, but wanted to make sure that other coding below was not specific to Newzbin.  I see xxhds has a movie version to work with non Newzbin sites, not sure if your TV script will have issues with another site.

3) My TV Show naming convention is as follows:  M:\TV Shows\30 Rock\30 Rock - Season 02\30 Rock - 02 - 08.avi   Can you guide me to the areas I would need to change in the script to accommodate this structure? I am worried about the renaming of the files after they are moved.  Or does the TV Folder management part of configuration still kick in?  I would like to know what to change so in the future I can update new versions that are posted.  Would be a great variable for the user to set at the top, maybe based on TV Folder sorting option settings.


4) with the TV Folder Folder management part of configuration all my files move and get named perfectly but I also get .nfo, .sfv, .nzb and sometimes a sample file as they must be inside the rar set.  Is there a way to make sure that only the main video file is moved over, or that the other files are deleted post clean up?

EDIT #4
I found the option under config/general to delete certain file extensions after processing.  I also have under switches the option for no sample files, but I still get sample fines from time to time.  Is this because they are inside the main RAR set or is there an issue with this function.  Having the sample files show up in my TV section really messes up my Meedio system after it imports the shows on my HTPC

5) anything else you think I need to know before giving this a real test?  My TV Show collection is VERY large, so I will test with a smaller collection to get everything worked out.

Sorry for so many questions and thank you in advance for guidance.  I am so excited to get this up and running as I spend way to much time hunting, dling, cleaning up nameing, and moving files!

[deep breath!]

ok, I'd never programmed in python before modding this script, so I'm sure you'll be fine.

1) this is in the post processing section, but is actually a standalone script, not a post processor.  I ended up with about 6 of them in the end, so I put them in a directory and scheduled this bat file to run in windows scheduler.  Obviously you need python installing too!

Code: Select all

for %%f in (d:\user\scripts\auto\*.py) do call "f:\Program Files\python\python.exe" "%%f"

or did I misunderstand the question?

2) Probably.  It's specific to the newzbin feed, but all it needs is the title and the nzb details to feed to sab, so should be relatively easy to mod.

3) anywhere that references dir basically, because it set to my layout of root/showname/season nn/epnum/season ep title

if you don't have an episode sub, then you'll have to be more creative with the existence code, but it's doable.  I prefer to keep them in a sub as then I know all the files associated with that download (I actually move the shows up one level as part of my post processing)

5)  I would definitely recommend you download to a new area while testing - as you said, you can tell the script about what already exists in another folder.


as for movies, the watch list code is pretty simple.  Might be easier to copy that into the other script?  Mine also has the multiple existing locations and I've just added logging to an HTML file of details of what is added to the queue.  If that sounds good, you might need to work on mine and change the RSS logic.  Let me know and I'll upload the latest version

[pilGrim]

Thanks Doubledrat,

My main confusion was around how to get the script to run.  I figured it may need windows scheduler, but figured I would wait for a reply before trying.  I was not sure if I need to install phython as maybe SABnzdb had done that, thanks for confirming.  There are very little good resources on the wiki to help you get started with scripts.

You also cleared up that this is not post processing which is why I thought the script was handling the moving and naming of the files after a download.  if I understand correctly the script is only really about getting the content into your queue and SABnzbd will handle all post download processing; which is working well for me right now.

I would appreciate a copy of your latest before I start tweaking things a bit.

I assume when the script runs there is a way to slow it down and watch it to ensure it is working properly?  I guess it is time for some reading up on python, been meaning to do this but never had a compelling reason till now.

I am in Hong Kong and about to head off to work as it is Monday morning here, so I will play with this when I get home tonight.

Thanks for the reply!

[pilGrim]

Hey Doubledrat,

EDIT: I realize now after looking at this that the error is becuase my RSS feed is probably a different format over the Newzbin feed.  Can you e-mail me or post a sample of a newzbin rss feed so I can see the differences and see if I can make this work?


Sorry to be such a pest.  I got python installed with a debugger and when I run your script I get the following error


2009-01-12 22:10:33: Ray.2004.DVDRiP.XviD-KJS (THIS WAS IN MY RSS FEED)
Traceback (most recent call last):
  File "I:\SABnbzd Working Dir\Scripts\Movies\Movies.py", line 67, in
    more_info=node.getElementsByTagName("report:moreinfo")[0].childNodes[0].data
IndexError: list index out of range

I have  a large movie collection and it seems to get fall outside an array? or is it a naming issue?


Any thoughts?

TIA

[minimad]

Hi pilGrim,

(first post, so be kind :-)  )

I have a modifed version of doubledrat's tv script which is able to process a different RSS feed to newzbin.

Your correct in that the format of the feed from newzbin is different from another one (i use tvbinz.net).  The newzbin feed is seperated by ' - ' (space hyphen space) and not a '.' (period).  so your example Ray.2004.DVDRIP.XviD-KJS would be Ray - 2004 - DVDRIP (i think, I haven't seen a proper newzbin entry just what i've gleaned from the code).

My modified scipt reformat's the tvbinz.net feed into the ' - ' format, basically to maintain a majority of code compatablity with the main script (makes life easier when Doubledrat or xxxhd update thier scripts)

I'll try to post it to pastebin when I've cleaned it up a little bit (still experimenting with the different season/episode formats so it doesn't work right now, it did until I tried processing a different format).

Expect a post with the link in about 30mins.

Regards

[minimad]

Hi All,

Here is the link for my version of the TV AutoDownloader script

http://pastebin.com/m161e1a6c

Important Changes:

New User config option 'NewzBin' : Selects if the script uses NewzBin SABnzbd options & feed.  Set to False if you use another RSS feed

Additional output detailing what it is doing (basically outputs the other information i.e. WANTED & NOT WANTED etc)

Removes '.' (period) and '_' (underscore) from series titles (should work for 95% of series, haven't worked out how to exclude ones that should have it in the title, or rather I haven't come across one yet)

Uses the Series Title ' - ' 'S'xx'E'xx format to look for episodes needed. comment line 116 and uncomment line 114 to revert back to DoubleDrat's format (he put this in the script originally).

Probale something else I forgot to mention

Let me know what you think

[doubledrat]

Hey Doubledrat,

2009-01-12 22:10:33: Ray.2004.DVDRiP.XviD-KJS (THIS WAS IN MY RSS FEED)
Traceback (most recent call last):
  File "I:\SABnbzd Working Dir\Scripts\Movies\Movies.py", line 67, in
    more_info=node.getElementsByTagName("report:moreinfo")[0].childNodes[0].data
IndexError: list index out of range

I have  a large movie collection and it seems to get fall outside an array? or is it a naming issue?


Any thoughts?

TIA

I think that error is because it's trying to decode what it expects to be the newzbin xml and your format is different.  The report:moreinfo entry is the imdb link.  the nb XML looks like this

Code: Select all

<item>
			<title>Turret Wars (Demo Version)</title>
			<guid isPermaLink="true">http://nburlremoved/</guid>
			<link>http://nburlremoved/</link>
			<comments>http://nburlremoved/#CommentsPH</comments>
			<description>
			<![CDATA[
			<ul>
			<li>
			ID: <a href="http://nburlremoved">3095588</a>
			(<a
			href="http://nburlremoved">Bookmark</a>)
			(<a href="http://nburlremoved/nzb/">NZB</a>)
			(<a href="http://nburlremoved/">NFO</a>)
			(<a href="http://games.softpedia.com/get/Shareware-Games/Turret-Wars.shtml">More Info</a>)
			(Size: 27.4MB)
			</li>
			<li>
				Attributes:
				Category - Games
				|  Platform - Windows |  Language - English |  Game Genre - Action
			</li>
			<li>Groups:  alt.binaries.pcgame</li>
			<li>Poster: [email protected] (AnoXL)</li>
			<li>PostDate: Thursday 26 Jun 2008, 02:52PM GMT</li>
			</ul>			]]>
			</description>
			<report:id>3095588</report:id>
			<report:category>Games</report:category>
			<report:attributes>
				<report:attribute type="Platform">Windows</report:attribute>
				<report:attribute type="Language">English</report:attribute>
				<report:attribute type="Game Genre">Action</report:attribute>
			</report:attributes>
			<report:groups>
				<report:group>alt.binaries.pcgame</report:group>
			</report:groups>
			<report:progress value="1">Report is complete</report:progress>
			<report:moreinfo>http://games.softpedia.com/get/Shareware-Games/Turret-Wars.shtml</report:moreinfo>
			<report:nfo>
				<report:fileid>188189611</report:fileid>
				<report:filename>Turret_Wars_1_75.nfo</report:filename>
				<report:link>http://nburlremoved/</report:link>
			</report:nfo>
			<report:nzb>http://nburlremoved/nzb/</report:nzb>
			<report:poster>[email protected] (AnoXL)</report:poster>
			<report:size type="bytes">28761370</report:size>
			<report:postdate>Thu, 26 Jun 2008 14:52:06 GMT</report:postdate>
			<report:stats>
				<report:views>217</report:views>
				<report:comments>1</report:comments>
			</report:stats>
			<pubDate>Thu, 26 Jun 2008 18:43:32 GMT</pubDate>
		</item>

latest tv version is here http://pastebin.com/f74ca2a4c

happy hacking!

[pilGrim]

Doubledrat and minimad, thanks for the posts.  Sorry to say I have been too busy to follow up on info you have both shared.  Hope to get to it today or tomorrow.  I will post back with progress or questions.

Thanks again!

[pilGrim]

Hey minimad / doubledrat,

any thoughts about altering the script to accept unique folder / episode naming structures?  Could the script pull the folder notaion from SABnzb d+, config, folders, folder sorting or could the script be altered to accept the same notation in the user setup at the top?  For example my TV Folder and naming convention is as follows:

%sn\%sn - Season %0s\%sn - %s - %0e.%ext

Being that I am a complete python newbie it is taking me a while to find all the spots to alter the script to accept my folder / naming convention and it occurs to me that the srcipt would be powerful for all if you could just set it at the top.  Far too many shows for me to rename as to match the code.  ;-)

TIA for any consideration or elbow grease on the idea,

Greg

[doubledrat]

Hey minimad / doubledrat,

any thoughts about altering the script to accept unique folder / episode naming structures?  Could the script pull the folder notaion from SABnzb d+, config, folders, folder sorting or could the script be altered to accept the same notation in the user setup at the top?  For example my TV Folder and naming convention is as follows:

%sn\%sn - Season %0s\%sn - %s - %0e.%ext

I had considered that approach, but to be honest was lacking motivation as it's working for me  as it is

certainly it's not inconceivable, as it would just be a case of doing a search and replace once each element was identified, but would require a logic rewrite to support, as the code scans directories at the moment.  Also, the ignore season x logic would be difficult to support in a variable world, as would the preventing duplicates which at the moment just creates the episode directory.  to prevent duplicates, the code would have to also have to keep track of which programmes had been added to the queue during this session, which is not the end of the world, but more than I want to be getting in to at the moment.


to tackle it yourself, I reckon you need to

• support 2 definition strings - 1 for the season and 1 for the actual episode - I can't think of a way of being able to determine that automatically while maintaining flexibility
• define a subroutine that will replace the %0s etc with the 3 supplied values that isMissing takes e.g. (series_title, season_number, episode_number).  your routine will have to support all the different variations that sab supports (maybe you can lift the routine they use by downloading the source)
• you'll have to change the isWanted, to use your routine to generate a string then just tack it onto the "dir" and do an if exists on it
• you'll have to change isMissing to do something similar
• then you can either accept the possibility of duplicates, or store the episode string as part of the "download" routine.  If you do this, then you will need to scan that list as part of the isInQueue routine

[minimad]

I think it's possible - maybe using code from SAB like doubledrat said or from tvnamer & tvdb_api.

I'm sure I've seen in one of the scripts I'm playing with, a list of different formats used by various people and has it coded into the script

I do have a version of this script that integrates with tvdb_api, which checks the show name against theTVDB.com and correctly formats it (resolves upper/lowercase issues) and puts the episodename in,but removes any other info like release group, format etc.  It does slow it down abit (a few extra seconds to query theTVDB.com) so it might not be too hard to do a midly customised version

I'll have a little think (at work :-)  ) about it.

I'm pretty sure that the tvdb_api handles your naming structure, so might not be too hard.

How are you using your directory structure, XBMC?

Minimad

[pilGrim]

How are you using your directory structure, XBMC?

Minimad

I came up with it on my own years ago.  I use Meedio which holds a database of your shows and there is a regex file importer so I just mask my directory structure.  I later realized I should have used something more consistent with what everyone else out there does, but I have a massive collection, so I have been loath to rename them all. 

the structure is:  M:\TV Shows\Series Name\Series Name - Season##\Series Name - Season## - Episode##.avi

I think giving the script the ability to mimic the SABnzbd TV Folder definition would make this script really helpful to many people out there.... but if that is not do able, and I found a way to auto rename what would be the best naming convention for the scripting tools out there? 

TIA

[minimad]

I think it's doable, just a little bit time-consuming, but I'm not doing it for any particular reason just for fun, I like a challenge.

Best Naming convention:  that's a toughie

I'm using Z:\'Series Name'\'Season ##'\'Series Name' - 'S##E##' - 'Episode Name'.avi  - ignore the ' those are just to show what makes an individual item.

BUT

I was using Z:\'Series Name'\'Season ##'\'Series Name'.'S##E##'.'Episode Name'.avi    replaced the . (period)  with ' - ' (space hyphen space)

The reason

One of the scripts i wanted to use used the new format (this was a couple of weeks ago before i started learning python & RegEx's).

I use XBMC which supports both formats and quite a few others so It all depends on what the program your using supports (sounds like anything).

How are you doing the renaming?  If you want to use a python script try tvnamer.py that uses the tvdb_api, with a slight mod (I've done it myself a few days ago but haven't got round to mailing the dev about it) it can rename & move an ENTIRE file structure based on theTVDB.com database.  I've also use tvnamer by natbur and tvrename by someone else.

anyway

try this -

http://pastebin.com/d56507bf1

Might not work (it's my TVDB intergated version) but i've changed it to your format.  Between this one and mine or doubledrat previous ones you should be able to get a working verison (or just get tvdb_api from here http://github.com/dbr/tvdb_api/tree/master  also holds tvnamer.py script).

Have Fun :-)