Media Rover

[kierse]

I'm occasionally having an issue that sabnzbd downloads a duplicate episode (even when my desired quality is "high" and there is an episode with the .mkv extension already). It downloads the episode, goes to sort, and then identifies that the duplicate episode exists. At that point it then moves (renames) the file with quality and date / time info.

Appending the duplicate episode with quality and timestamp is normal behavior, downloading the duplicate is not however. It sounds like MR is properly identifying the existing episode (on disk) as being at the desired quality during sort but not when scheduling. Two things: what version of MR are you running? And can you send me a copy of your log files?

Shouldn't it either be deleting the old one or not downloading the show at all since the desired quality of the show already exists? What happens when my feed pulls down an avi (medium) and later finds a high quality version? I imagine that it would delete the .avi version and replace it with the .mkv. I'm just confused as to why it's replacing a .mkv with another .mkv.

Correct, MR will replace an existing episode with a more desirable version. However this behavior only occurs when moving from one quality level to another. So using the situation you described above, MR would replace the avi with the mkv. When sorting a true duplicate (two copies at the same quality level), the current approach is to keep both copies. The original thinking behind keeping both was that if/when the user manually scheduled a duplicate for download, they might want both copies for some reason.

After having typed all that, it occurred to me that the cause is most likely my use of two rss feeds from two sources. This apparently ends up queuing the download of one version from one RSS source and then another from a separate RSS source. In essence, it doesn't find the episode even though it's queued it for download already. Looking in my SABnzbd history, it shows the two versions of the show / episode in question being completed within 2 minutes of each other. What's the best way to prevent this from happening? I use them both in case one ends up having some shows / episodes that the other doesn't.

Hmm I don't think the problem lies with how you have MR configured. In my own configuration, I have 6 configured sources: 4 standard def and 2 HD feeds. I too have multiple configured to catch something that might fall through the cracks, and to test the sources supported by MR .

When scheduling items for download, MR makes every attempt to avoid scheduling duplicates. It checks the queue, nzb backup directory (if configured), and the filesystem before telling SABnzbd to start downloading something. Furthermore, this checking does attempt to work across sources. This means that any items identified for download from Newzbin will be checked against items that have already been identified for download from all other sources.

It does bring up the question of allowing the control of what the script does when it finds duplicate episodes of the same quality. Yes, the bandwidth has already been used but it would be nice to be able to tell it to just nuke the file since it's already there. Then again, there might be a reason why you would want to know that it downloaded duplicate episodes (like, say, to bring ones attention to the fact that they're being downloaded and prompt the user to come up with a fix).

As I sit here and respond to your post I've been thinking about how MR handles duplicates and I think I agree with you: MR should just replace the episode rather than keep a duplicate copy. If the user manually scheduled a duplicate for download they likely want to replace a defective copy already on disk. If MR is the one scheduling duplicates, there's a problem. However, the user shouldn't have to deal with duplicate files. A nice warning message in the logs should suffice.

---

It sounds to me like there is a problem with the quality management system in MR. I recently released 0.6.0 which included several bug fixes to quality management. It is possible the issue you are experiencing has been fixed. If you aren't running 0.6.0 try upgrading and seeing if it solves your issues. If you are running 0.6.0, send me your logs and I'll see if I can figure out what's going wrong.

I'm hard at work on 0.7.0 at the moment. I'll take a ook at how MR deals with duplicates and make it just replace the existing episode and generate a warning.

Kierse

[avatarr]

Thanks for taking the time to respond. I am not running the most recent version. I'll upgrade when I get a chance.

[Maverick519]

Upgrade my version of sabnzbd+ to the new 6.0 beta and I am getting this error now in the log of mediarover.

Code: Select all

2011-02-24 19:00 ERROR - SABnzbd 0.5.0 or greater required! - schedule.py:105
Traceback (most recent call last):
  File "/mediarover/mediarover/command/schedule.py", line 103, in schedule
    __schedule(broker, options)
  File "/mediarover/mediarover/command/schedule.py", line 223, in __schedule
    queue = init(params['root'], supported_categories, params)
  File "/mediarover/mediarover/queue/sabnzbd/__init__.py", line 276, in __init__
    self.__version_check()
  File "/mediarover/mediarover/queue/sabnzbd/__init__.py", line 265, in __version_check
    raise UnknownQueue("SABnzbd 0.5.0 or greater required!")
UnknownQueue: SABnzbd 0.5.0 or greater required!

Using Dev version of mediarover.

Maverick519

[kierse]

Hey Maverick519,

Upgrade my version of sabnzbd+ to the new 6.0 beta and I am getting this error now in the log of mediarover.

Code: Select all

2011-02-24 19:00 ERROR - SABnzbd 0.5.0 or greater required! - schedule.py:105
Traceback (most recent call last):
  File "/mediarover/mediarover/command/schedule.py", line 103, in schedule
    __schedule(broker, options)
  File "/mediarover/mediarover/command/schedule.py", line 223, in __schedule
    queue = init(params['root'], supported_categories, params)
  File "/mediarover/mediarover/queue/sabnzbd/__init__.py", line 276, in __init__
    self.__version_check()
  File "/mediarover/mediarover/queue/sabnzbd/__init__.py", line 265, in __version_check
    raise UnknownQueue("SABnzbd 0.5.0 or greater required!")
UnknownQueue: SABnzbd 0.5.0 or greater required!

Using Dev version of mediarover.

Maverick519

Ya, MR specifically looks for 0.5.x. You can disable this check by adding '__check_version__ = False'  to your config file under the sabnzbd queue (ie. [queue] [[sabnzbd]] __check_version__ = False), however I can't guarantee things will continue to function properly. I haven't looked at the 0.6.x branch at all yet.

Kierse

[Maverick519]

Will try that and report back how it works out for me.

Maverick519

[Maverick519]

Will try that and report back how it works out for me.

Maverick519

Well, I have been trying mediarover with the new 6.0 beta 2 branch and mediarover seems to be performing the same as before the version update.

Maverck519

[kierse]

Well, I have been trying mediarover with the new 6.0 beta 2 branch and mediarover seems to be performing the same as before the version update.

Well that's good, I guess that means there haven't been any major API changes then between 0.5.x and 0.6.x. Let me know if you run into any other problems/weird behaviour.

Kierse

[war59312]

You need to update http://kierse.github.com/mediarover/#usage .

[kierse]

You need to update http://kierse.github.com/mediarover/#usage .

Yes I do! Thanks for the heads up war59312

[avatarr]

I've never payed attention to this, but I recently upgraded my NAS's firmware, python version, sabnzbd version, and media rover version. I am pretty sure I have it all set up correctly, but I noticed that when I run "python mediarover.py schedule", the last line reads "Segmentation Fault". Is this normal? Here's a paste from the last two lines from my most recent execution of the scheduler:

Code: Select all

2011-03-06 19:48 INFO - no items to schedule for download - schedule.py:313
Segmentation fault

Also (and I realize this is a long shot), but when I check my crontab file, I have the correct entry for running the scheduler, but I noticed that SNL wasn't downloaded even though it showed up in my RSS feed. I manually executed the scheduler and it queued it for download. Can you think of any reason why the crontab wouldn't queue the download? Here's the line I have in my crontab:

Code: Select all

0-59/10 * * * * python ~/.mediarover/mediarover.py schedule > /dev/null

[kierse]

Hey avatarr,

I've never payed attention to this, but I recently upgraded my NAS's firmware, python version, sabnzbd version, and media rover version. I am pretty sure I have it all set up correctly, but I noticed that when I run "python mediarover.py schedule", the last line reads "Segmentation Fault". Is this normal? Here's a paste from the last two lines from my most recent execution of the scheduler:

Code: Select all

2011-03-06 19:48 INFO - no items to schedule for download - schedule.py:313
Segmentation fault

No, that's definitely not normal. What type of NAS are you running? Version of python? MR?

Also (and I realize this is a long shot), but when I check my crontab file, I have the correct entry for running the scheduler, but I noticed that SNL wasn't downloaded even though it showed up in my RSS feed. I manually executed the scheduler and it queued it for download. Can you think of any reason why the crontab wouldn't queue the download? Here's the line I have in my crontab:

Code: Select all

0-59/10 * * * * python ~/.mediarover/mediarover.py schedule > /dev/null

The problem you are describing was an issue with MR 0.7.0. Seeing as you recently updated I'm going to assume that's what you are running. Upgrading to 0.7.1 will fix this particular problem.

Kierse

[avatarr]

Hey avatarr,

I've never payed attention to this, but I recently upgraded my NAS's firmware, python version, sabnzbd version, and media rover version. I am pretty sure I have it all set up correctly, but I noticed that when I run "python mediarover.py schedule", the last line reads "Segmentation Fault". Is this normal? Here's a paste from the last two lines from my most recent execution of the scheduler:

Code: Select all

2011-03-06 19:48 INFO - no items to schedule for download - schedule.py:313
Segmentation fault

No, that's definitely not normal. What type of NAS are you running? Version of python? MR?

Also (and I realize this is a long shot), but when I check my crontab file, I have the correct entry for running the scheduler, but I noticed that SNL wasn't downloaded even though it showed up in my RSS feed. I manually executed the scheduler and it queued it for download. Can you think of any reason why the crontab wouldn't queue the download? Here's the line I have in my crontab:

Code: Select all

0-59/10 * * * * python ~/.mediarover/mediarover.py schedule > /dev/null

The problem you are describing was an issue with MR 0.7.0. Seeing as you recently updated I'm going to assume that's what you are running. Upgrading to 0.7.1 will fix this particular problem.

Kierse

I will upgrade to 0.7.1. This is probably a dumb question and has probably already been covered many times over, but what is the best / preferred method of "upgrading"? Is it just a matter of extracting and dumping everything in the archive into the already-existing mediarover folder? cp -R /path/to/071/* /path/to/070/ ? Also, is there somewhere I can add this to a wiki answer? I did some searching for the proper way to "upgrade" and the mediarover wiki is pretty sparse.

Here's another weird thing. The Segmentation Fault I reported last night for some reason has resolved itself. I ran the exact same command last night and again today, and now I get no segmentation faults - it executes correctly. No clue why. To answer your questions, however:
NAS: QNAP TS-439 Pro
Python: 2.7 (as provided by the QNAP .qpkg plugins)
MR: Was 0.7.0, will upgrade to 0.7.1

Thanks for your continued responses / support.

[kierse]

...but what is the best / preferred method of "upgrading"? Is it just a matter of extracting and dumping everything in the archive into the already-existing mediarover folder? cp -R /path/to/071/* /path/to/070/ ?

I suppose you could unpack a new release into an existing folder. If there are any files removed from release to release, they obviously won't be overwritten (though it shouldn't matter). Ideally you want to unpack a new release into its own folder. Seeing as you are running Unix/Linux, I would suggest creating a symlink:

Code: Select all

> cd /mr/parent/directory
> unzip /path/to/0.7.1.zip .
> ln -s 0.7.1 mediarover

This way you won't ever have to update your cronjob or anything pointing to the sorting script. Another option would be to use git:

Code: Select all

> cd /mr/parent/directory
> git clone git://github.com/kierse/mediarover.git

This will create a new directory called mediarover. Whenever there is a new release, you can get the latest code by running:

Code: Select all

> cd /path/to/mr
> git pull origin

Also, is there somewhere I can add this to a wiki answer? I did some searching for the proper way to "upgrade" and the mediarover wiki is pretty sparse.

No, I don't believe the current wiki has any of this information. As you noticed it is quite sparse  . I've been meaning to update the wiki to reflect some of the recent changes in MR; I'll be sure to add this information as well. In the meantime, I believe the wiki can be changed by anyone with a Github account. If you have an account and feel like making changes, go for it!

Here's another weird thing. The Segmentation Fault I reported last night for some reason has resolved itself. I ran the exact same command last night and again today, and now I get no segmentation faults - it executes correctly. No clue why. To answer your questions, however:
NAS: QNAP TS-439 Pro
Python: 2.7 (as provided by the QNAP .qpkg plugins)
MR: Was 0.7.0, will upgrade to 0.7.1

Thanks for your continued responses / support.

Hmm, strange indeed. I assume you copied the new release into the existing MR folder? I seem to recall hearing from someone else in the past that they had issues when doing that but its never been confirmed. Keep and eye on it and let me know if the problem reappears.

Kierse

[avatarr]

I'm still having an issue with the scheduling aspect of mediarover using a cron job. I have racked my brain and I can't for the life of me figure out why mediarover isn't scheduling. I don't know if it's script-related or if it has something to do with permissions in linux or what. The scope of my knowledge with respect to linux only goes so far and I don't really know how to troubleshoot beyond where I am.

I have tried a few different entries in my crontab. The first one was used before I upgraded from 0.6.x to 0.7.0, and with 0.6, it worked. What's difficult, however, is that around the time I upgraded MR, I also expanded my NAS to 9TB (4x3TB drives), which required a firmware upgrade. This, in turn, prompted the need to upgrade my version of sabnzbd which required me to also upgrade python. So the problem could be in any one of those areas and I have no idea how to track the issue down.

Here's what I'm working with now:
NAS: QNAP TS-439 Pro
Python: 2.7 (as provided by the QNAP .qpkg plugins)
MR: 0.7.1

This is the original entry I had in my crontab:

Code: Select all

0-59/10 * * * * python ~/.mediarover/mediarover.py schedule > /dev/null

After upgrading, this entry didn't seem to be working. I did some digging and thought it might have something to do with not using the explicit path, so I changed it to the following:

Code: Select all

0-59/10 * * * * python /root/.mediarover/mediarover.py schedule > /dev/null

Which still didn't work. After reading more about cron jobs, I changed the first part to remove the "0-59" (which I didn't think should have any effect) to:

Code: Select all

*/10 * * * * python /root/.mediarover/mediarover.py schedule > /dev/null

I also read some recommendations which said that in order to prevent emails getting sent out that you should replace the " > /dev/null" to " > dev/null 2>&1" which I also did to no avail.

Finally, I removed the dev/null from the command altogether just in case that was causing some input syntax redirect issue, so the entry as it stands now is:

Code: Select all

*/10 * * * * python /root/.mediarover/mediarover.py schedule

... and it still doesn't seem to be scheduling.

I know that the cronjob is running because I have another entry in there that touches a folder every minute and it does continue to update it when I run ls -la

Code: Select all

*/1 * * * * touch /root/temp/foo

The weird thing is that if I manually run the same command (i.e. "python /root/.mediarover/mediarover.py schedule"), it executes as it should.

[avatarr]

Edit: I think I figured out the issue here. I was using a script from 0.6 which had the many different parameters passed. I think updating to the new version of the script with modified syntax will take care of this issue, though I can't test it in this particular instance since I deleted it already.

Things are all kinda messed up. I got this error after manually running the scheduler:

Code: Select all

2011-03-12 15:37 INFO - --- STARTING --- - episode_sort.py:141
2011-03-12 15:37 INFO - begin processing tv directory: /share/Qmultimedia/TV/ - series.py:528
2011-03-12 15:37 INFO - watching 55 tv show(s) - episode_sort.py:243
2011-03-12 15:37 INFO - found download file at '/share/Qmultimedia/TV/Army_Wives_-_1x05_-_Independence_Day/Army.Wives.S01E05.german.sub.DVDRip.XviD-FoV.by.GhostUp10.avi' - episode_sort.py:272
2011-03-12 15:37 ERROR - unable to parse job title and create Episode object: missing episode season number - episode_sort.py:169
Traceback (most recent call last):
  File "/root/.mediarover/mediarover/command/episode_sort.py", line 163, in episode_sort
    __episode_sort(broker, options, **params)
  File "/root/.mediarover/mediarover/command/episode_sort.py", line 288, in __episode_sort
    raise InvalidJobTitle("unable to parse job title and create Episode object: %s" % e)
InvalidJobTitle: unable to parse job title and create Episode object: missing episode season number
FAILURE: unable to parse job title and create Episode object: missing episode season number!