Media Rover

[kierse]

Hey funtek,

When SABNzbd 0.5.3 is setup to handle nzbmatrix categories* from RSS**, it causes the sorting script not to run on my windows box. It gives this error on the History page: ERROR: incorrect number of arguments!
My guess is that Mediarover sorting is assuming that there will be one non-deliminated string as the category, and when it gets my amended settings of "TV, tv.*", it's choking.

*(changing the Newzbin/Groups column on the categories view for the TV category to "TV, tv.*" instead of the "TV" that is default)
**(so SAB will put nzbs in the right queue category when you bookmark them on nzbmatrix and then have sab check your personal nzbmatrix rss feed)

Hmm interesting, I'll have to look into it.  My guess is that the comma is the culprit.  Whether it's causing MR to crap out or it's doing funny things to SABnzbd.  I'll have a look and get back to you

Kierse

[paulkoan]

2. When you update the code, are you unpacking the new version on top of the existing folder? I've run into problems when people unpack a new version on top of the existing one.  This is problematic as old code that's been removed from one update to another can stick around and cause problems.

Thanks for looking at this Kierse.  I am just doing a "git pull" to get the updates.  Perhaps I am getting the wrong branch?

[sabbato753]

Hi Kierse,

Thanks for the reply.  Quick question on smart titles - the files pulled from nzbs.org (for instance) don't have the ep title in the name of the NZB.  However, the final files usually contain the ep title.  When MR goes to do its thing, these titles don't end up in the final moved file.  Am I missing something?

Also, what arguments are REQUIRED when doing a manual episode sort?  It works fine via the automated script when SAB finishes, but running it manually has failed every time with some error regarding base 10. 

@paulkoan - please make sure to scrub episode titles and such from posted logs.    Nobody needs to see what you're downloading, especially any types of enforcement (moderators, law, etc.)  

[funtek]

Hi Kierse,

Thanks for the reply.  Quick question on smart titles - the files pulled from nzbs.org (for instance) don't have the ep title in the name of the NZB.  However, the final files usually contain the ep title.  When MR goes to do its thing, these titles don't end up in the final moved file.  Am I missing something?

Also, what arguments are REQUIRED when doing a manual episode sort?  It works fine via the automated script when SAB finishes, but running it manually has failed every time with some error regarding base 10. 

Mediarover generally goes by the FOLDER name instead of the video file name I think. This explains both of your problems: to manually sort a file, you need to sort the FOLDER it is in, not the file. This also explains the strange lack of episode names when the file has them originally, but the folder does not. I think the reason this is the behaviour, is that the folder name, is the name of the job in SABNZBD, and and name of the job is the name of the posting on your favored NZB hosting site.

Kierse: Please ignore my last bug report, there's something else going on, but I'm not sure if it's a SAB problem, a windows problem or a MediaRover problem. I've basically messed up sorting now so that it only works manually, but when SAB calls it, I get the error I reported previously. Updated to the newest files off of github. Meh, I'll figure it out.

[mike171562]

I'm trying to get a series called "MY SHOW", I have created the folder with the same name in my TV directory, and in my mediarover.conf I have tried serveral aliases, such as

alias = "MY SHOW -","MY SHOW - ","MY SHOW - / ","MY SHOW -* - */*"

but it doesnt seem to be grabbing the nzb as the title that appear in the searches are as you can see in the output below. Is there a wildcard I can use in the alias or a regex that will help it find these episodes? Thanks


2010-07-01 12:40 INFO - skipping 'MY SHOW - 1x05 - Car Brakes Fail/Physical Attack', not watching series - __init__.py:340
2010-07-01 12:40 INFO - skipping 'MY SHOW - 1x06 - Trapped in Freezer', not watching series - __init__.py:340
2010-07-01 12:40 INFO - skipping 'MY SHOW - 1x08 - Elevator Plunge/Blackout', not watching series - __init__.py:340
2010-07-01 12:40 INFO - skipping 'MY SHOW - 1x07 - Desert Breakdown/Tarantula', not watching series - __init__.py:340
2010-07-01 12:40 INFO - skipping 'MY SHOW - 1x01 - Burning Car/Boating Accident', not watching series - __init__.py:340
2010-07-01 12:40 INFO - skipping 'MY SHOW - 1x02 - Downed Powerline/Dog Attack', not watching series - __init__.py

[paulkoan]

2. When you update the code, are you unpacking the new version on top of the existing folder? I've run into problems when people unpack a new version on top of the existing one.  This is problematic as old code that's been removed from one update to another can stick around and cause problems.

Thanks for looking at this Kierse.  I am just doing a "git pull" to get the updates.  Perhaps I am getting the wrong branch?

Hmm.  Weird.  I made sure I was "git branch dev" and did "git pull", then downloaded the dev version tarball and extracted it (which I assumed was the same as git pull) and the two diffed differently.

Presumably they are not the same at all?

[kierse]

Hey paulkoan,

Hmm.  Weird.  I made sure I was "git branch dev" and did "git pull", then downloaded the dev version tarball and extracted it (which I assumed was the same as git pull) and the two diffed differently.

Ahh, that's your problem.  You aren't running off the remote dev branch.

Code: Select all

# git branch dev

creates a new branch called dev based on your current branch (which is probably master).  You want to create a new branch based on the remote dev branch.  To fix this, you can delete your repo and start again, or delete your incorrect dev branch with the following:

Code: Select all

# git branch -d dev
Deleted branch dev.

Ok, lets get a listing of your branches

Code: Select all

# git branch -a
* master
  origin/HEAD
  origin/dev
  origin/gh-pages
  origin/master

The * indicates the branch that's currently checked out (in this case its master).  As you can see, there are a couple remote branches (everything prefixed with origin/).  You want to create a local branch based on the remote dev branch (origin/dev):

Code: Select all

# git checkout -b dev origin/dev
Branch dev set up to track remote branch refs/remotes/origin/dev.
Switched to a new branch "dev"

This command creates a new local branch called dev based on the contents of the remote dev branch.  It also checks out the new dev branch so that you have access to its code.  List all your branches again:

Code: Select all

# git branch -a
* dev
  master
  origin/HEAD
  origin/dev
  origin/gh-pages
  origin/master

As you can see, you now have a new branch called dev and it has been checked out.

Code: Select all

# git reflog show dev
3dfcf1e... dev@{0}: branch: Created from origin/dev

As you can see, your new dev branch was created against the remote branch.  You are now running the latest code on the dev branch!

Hope that helps!

Kierse

[kierse]

Hi Kierse,

Thanks for the reply.  Quick question on smart titles - the files pulled from nzbs.org (for instance) don't have the ep title in the name of the NZB.  However, the final files usually contain the ep title.  When MR goes to do its thing, these titles don't end up in the final moved file.  Am I missing something?

Also, what arguments are REQUIRED when doing a manual episode sort?  It works fine via the automated script when SAB finishes, but running it manually has failed every time with some error regarding base 10. 

@paulkoan - please make sure to scrub episode titles and such from posted logs.    Nobody needs to see what you're downloading, especially any types of enforcement (moderators, law, etc.)  

As funtek pointed out, MR expects to be given a folder when sorting is called.  It then searches that folder for the episode file.  Here's an example of how you would manually have MR sort an episode:

Code: Select all

python mediarover.py episode-sort /path/to/some.download

See the episode-sort help for more details and additional examples:

Code: Select all

python mediarover.py episode-sort --help

As for sorting episodes with episode titles, that functionality is only (currently) supported when sorting episodes from Newzbin.  Newzbin has a very structured naming format for their nzbs (which includes the episode title).  None of the other indexing services follow any sort of a naming pattern (that I'm aware of) so there is no way for me to reliably scrape the episode titles.  In the future, I hope to pull this data from a third party service such as thetvdb.com or tvrage.com.  In the meantime, you'll have to do without 

Kierse

[kierse]

Please ignore my last bug report, there's something else going on, but I'm not sure if it's a SAB problem, a windows problem or a MediaRover problem. I've basically messed up sorting now so that it only works manually, but when SAB calls it, I get the error I reported previously. Updated to the newest files off of github. Meh, I'll figure it out.

After you reported the problem, I attempted to replicate it without success.  Here's what I did:

• added my nzbmatrix bookmark list as an RSS feed in SABnzbd (I didn't change any defaults)
• set movies category group field to "Movies, movies.*"
• set tv category group field to "TV, tv.*"

A couple of episodes were successfully downloaded and properly sorted by MR.  SAB called MR with the actual usenet group (expected behaviour) so I'm not sure what's going on with your setup.

Let me know if you figure it out or if you need any further help

Kierse

[kierse]

Hey mike171562,

I'm trying to get a series called "Worst Case Scenario", I have created the folder with the same name in my TV directory, and in my mediarover.conf I have tried serveral aliases, such as

alias = "The Worst Case Scenario -","Worst Case Scenario - ","Worst Case Scenario - / ","Worst Case Scenario -* - */*"

but it doesnt seem to be grabbing the nzb as the title that appear in the searches are as you can see in the output below. Is there a wildcard I can use in the alias or a regex that will help it find these episodes? Thanks

A couple of things:

1. I'm assuming you're running MR 0.4.1, is that right? I released 0.5.0 earlier today and it is considerably different than 0.4.1.  Update to the newest stable release and try again, I suspect that your problem has been fixed.

2. Unfortunately you can't use any kind of wild card / regular expression with aliases.  MR takes an alias as is and compares it to available nzb reports.

3. Sanitize your forum posts and remove the series name.  You don't want to get in trouble by the forum moderators 

Update your copy of MR and let me know how it goes.  There have been quite a few changes so go over the configuration file closely.

Kierse

[kierse]

Hey guys,

I'm happy to announce that Media Rover 0.5.0 is finally available!  For those interested, the full release announcement can be at http://mediarover.tv/2010/07/02/Media-Rover-0.5.0/. 

Here are the notable changes:

• quality management system
• ability to delay scheduling nzbs
• ability to order configured sources
• better support for manually sorting downloaded episodes
• ...and much, much, more!

Usage change:

Part of the 0.5.0 release includes a reorganisation of both shell scripts (mediarover.py and episodesort.py) and the elimination of one (episodesort.py).  All of Media Rovers functionality can now be accessed via the one script.  Here's a quick run down of the available command options:

Code: Select all

# python mediarover.py --help
Usage: mediarover.py [--version] [--help] COMMAND [ARGS]

Description: Media Rover is an automated TV download scheduler and catalogue
maintainer

Options:
  --version   show program's version number and exit
  -h, --help  show this help message and exit

Available commands are:
   schedule          Process configured sources and schedule nzb's for download
   episode-sort      Sort downloaded episode
   set-quality       Register quality of series episodes on disk
   write-configs     Generate default configuration and logging files
   migrate-metadata  Migrate metadata database from one version to another

See 'python mediarover.py COMMAND --help' for more information on a specific command.

• schedule - process configured sources and schedule new nzbs for download
• episode-sort - sort a downloaded episode
• set-quality - populate the metadata database with information regarding your existing series/episode collection
• write-configs - generate default configuration and logging files
• migrate-metadata - metadata database operations

For example, if you would like to process all configured sources and schedule missing episodes for download, you would run the following:

Code: Select all

# python mediarover.py schedule

If you would like to manually sort a downloaded episode, you would run the following:

Code: Select all

# python mediarover.py episode-sort /path/to/download/folder

If you would like to set the quality of your existing series episodes in order to make full use of Media Rovers new quality management system, you would run the following:

Code: Select all

# python mediarover.py set-quality --low mp4 --medium avi --high mkv --no-series-prompt

As per usual, please keep the bug reports coming and I'll do my best to fix them ASAP.  I've been running this code base for a while now and I'm pretty happy with its stability.  That being said, I'm sure there are a bunch of bugs yet to be discovered  . 

Enjoy!

Kierse

[wally007]

how exactly does scheduling work ?

[kierse]

Hey wally007,

how exactly does scheduling work ?

Here's a general idea of what's going on when you run the scheduler:

1. build a list of watched series by scanning each configured tv_root directory
2. retrieve delayed items from database and process them.  If any items are missing*, schedule them for download
3. scan each configured RSS source and process items.  If any items are missing*, schedule them for download
4. add new delayed items to database

* the most complicated job of the scheduler is determining if a given item should be downloaded.  This involves several steps:
1. check if the item belongs to a series that MR is watching for
2. make sure that the season / broadcast year of current item isn't being ignored by a series filter
3. if the item represents a multipart episode (ie s01e01-s01e02), check the config to see if the user preference is to accept it
4. make sure that exact item hasn't already been processed by the queue
5. check if episode is already on disk**
6. make sure episode isn't already in queue**

** determining if an episode is already on disk (note: same process applies when checking the queue):
1. compile list of single episodes found on disk for a given series.  List includes all parts of multiepisode files.  For example, two files s01e01-02, s01e03 translates into the following list of episodes: s01e01, s01e02, s01e03
2. check list to see if current episode is already on disk.  At this point if the potential download is missing, MR will stop processing and mark it desirable (meaning it should be downloaded).  If it's already on disk...
3. and quality management is on, compare the quality of potential download with that of current episode on disk.  If we don't know the quality of an episode on disk, MR defaults to:
  a) using the extension (if configured), or
  b) the default quality level for the series, or
  c) the default quality level for all tv

At this point, if MR hasn't found any reason to schedule the item for download it will be skipped.  If it is found to be desirable (missing, of more desirable quality, etc), it will be scheduled for download.

I hope this clears things up a bit.  Let me know if something is unclear

Kierse

[wally007]

whoops sorry i probably didnt phrase my question correctly .. i was more interested in this feature , which to my knowledge is new to .5 version

Here are the notable changes:
......
ability to delay scheduling nzbs
......

[kierse]

whoops sorry i probably didnt phrase my question correctly .. i was more interested in this feature , which to my knowledge is new to .5 version

Ahh, I see.  No that's nothing new.  Calling the 'schedule' command in 0.5.0 is the same as running mediarover.py in all previous versions.  It just processes all sources and schedules NZB's for download.  One of the things I did in this release was to  eliminate the separate shell scripts and combine them all under one script as separate callable commands. 

Does that make sense?

Kierse