Not extracting/Downloading par2's

[Freman]

Greetings.

I've found two problems with my install of SABnzbd+ on Gentoo.

1) It doesn't appear to be downloading the extra par2's when it finds issue with a file. I've had two failed downloads recently that failed verification but when I download the par2's manually and repair they unpack just fine.

2) Even when a download is completing successfully it's not unraring the files.

Both of these used to work on a 4.x version (before my HDD died, this is all a fresh install)

What have I broken?

Below is my config file

Code: Select all

__version__ = 18
[misc]
host = 192.168.0.2
port = 8080
enable_https = 0
https_port = 9090
https_key = server.key

log_dir = /var/log/sabnzbd
cache_dir = /var/lib/sabnzbd/data/cache
dirscan_dir = /var/lib/sabnzbd/data/dirscan
download_dir = /var/lib/sabnzbd/data/incomplete
complete_dir = /var/lib/sabnzbd/data/complete
nzb_backup_dir = /var/lib/sabnzbd/data/download
script_dir = /var/lib/sabnzbd/data/scripts
admin_dir = /var/lib/sabnzbd/data/admin
web_dir = Plush
web_dir2 = ""
email_dir = ""

api_key = 8c1f0fbb96e3f9139b1fedea1f816a04

folder_rename = 1
rss_rate = 60
cleanup_list = ,
cache_limit = ""
par_option = ""
web_color = ""
web_color2 = ""
replace_spaces = 0
dirscan_speed = 5
enable_filejoin = 1
create_group_folders = 0
no_penalties = 0
bandwidth_limit = 0
disable_api_key = 0
email_to = ""
password = ""
permissions = 775
auto_disconnect = 1
movie_extra_folder = 0
enable_date_sorting = 0
replace_dots = 0
movie_sort_extra = -cd%1
ionice = 10
enable_unzip = 1
download_free = ""
email_full = 0
tv_sort_countries = 1
date_categories = tv,
ssl_type = v23
date_sort_string = ""
schedlines = "1 0 * resume ", "0 11 * pause_all "
ignore_samples = 2
language = us-en
auto_browser = 0
pause_on_post_processing = 0
enable_unrar = 1
email_account = ""
email_server = ""
config_lock = 0
tv_sort_string = ""
dirscan_priority = 0
username = ""
enable_par_cleanup = 1
safe_postproc = 1
replace_illegal = 1
par2_multicore = 1
warned_old_queue = 0
enable_tsjoin = 1
check_new_rel = 1
no_dupes = 0
top_only = 1
quick_check = 1
movie_sort_string = ""
email_from = ""
dirscan_script = None
email_endjob = 0
unpack_check = 1
send_group = 0
refresh_rate = 0
movie_categories = movies,
email_pwd = ""
ignore_wrong_unrar = 0
nice = 10
enable_movie_sorting = 0
https_cert = server.cert
fail_on_crc = 0
auto_sort = 0
dirscan_opts = 3
enable_tv_sorting = 0
folder_max_length = 256
[logging]
max_log_size = 5242880
log_level = 1
log_backups = 5
enable_cherrypy_logging = 0
[newzbin]
username = ""
bookmark_rate = 60
bookmarks = 0
password = ""
unbookmark = 1
[nzbmatrix]
username = ""
apikey = ""
[categories]
[[misc]]
priority = -100
pp = ""
name = misc
script = Default
newzbin = Misc
dir = Misc
[[tv]]
priority = -100
pp = ""
name = tv
script = Default
newzbin = TV
dir = TV
[[unknown]]
priority = -100
pp = ""
name = unknown
script = Default
newzbin = Unknown
dir = Unknown
[[resources]]
priority = -100
pp = ""
name = resources
script = Default
newzbin = Resources
dir = Resources
[[apps]]
priority = -100
pp = ""
name = apps
script = Default
newzbin = Apps
dir = Apps
[[movies]]
priority = -100
pp = ""
name = movies
script = Default
newzbin = Movies
dir = Movies
[[consoles]]
priority = -100
pp = ""
name = consoles
script = Default
newzbin = Consoles
dir = Consoles
[[books]]
priority = -100
pp = ""
name = books
script = Default
newzbin = Books
dir = Books
[[games]]
priority = -100
pp = ""
name = games
script = Default
newzbin = Games
dir = Games
[[anime]]
priority = -100
pp = ""
name = anime
script = Default
newzbin = Anime
dir = Anime
[[music]]
priority = -100
pp = ""
name = music
script = Default
newzbin = Music
dir = Music
[[pda]]
priority = -100
pp = ""
name = pda
script = Default
newzbin = PDA
dir = PDA
[[emulation]]
priority = -100
pp = ""
name = emulation
script = Default
newzbin = Emulation
dir = Emulation
[servers]

Software versions:

Code: Select all

servah / $ unrar

UNRAR 3.80 freeware      Copyright (c) 1993-2008 Alexander Roshal

Code: Select all

servah / $ par2verify
You must specify a Recovery file.
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.

Code: Select all

servah / $ par2repair
You must specify a Recovery file.
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.

[shypike]

99% of the time this is caused by dubious NZB files.
So the only thing we need is the actual NZB file,
can you email it to bugs at abnzbd.org (include the UJRL of this thread ) ?

[robdeep]

I think I'm having the same issue with 0.5.0 running on XP. I have a job that looks complete and is in the "verifying" stage and says "starting repair", but it's not doing any repairs as far as I can tell.

I'm not sure how to delete this out of my queue now. It does not delete the job, when I attempt to.

[switch]

rob, try turning multicore par2 off in config>switches then restarting sabnzbd (click shutdown and relaunch)

[robdeep]

Hi Switch,

Thanks for replying. I did try turning off the multicore, but it did not help. After I restarted sabnzbd, I saw it try to retrieve some pars, but some were missing from the server. This NZB post was probably a bit to old. So I understand the missing articles will not allow this job to process, but it should have failed out the job. Now it's just stuck in my history with a status of "verifying..." even though it's not actually verifying. And I can't delete it

Any way to remove it manually? THanks

[robdeep]

did a "sabnzbd.exe --clean" that took care of my busted job in the queue.

[Freman]

Sorry it's taken me so long to get back - between work and rennovating, storms and flooding I have been a little busy.

Ok, I'm 99.999% positive it's not a NZB issue because I've used nzb's from many different sources since I made this post originally.

I've logged into the user I'm running sabnzbd under and tested that unrar, unzip, and all the par2 tools are accessable.

It doesn't have any errors in the log, it seems to think it's doing everything at least as far as I can tell, but it still doesn't do a par2repair (when necessary) or unzip/unrar (ever)

[shypike]

Are you using the +D option for all NZB's?

[Freman]

+D?

How do I know that?

I do have set - Default Post-Processing: +Delete - is that what you're talking about?

I had an earlier version all set up and hacked up for multi-user use but the hdd died, now I don't know what's wrong with this install

[shypike]

Indeed, +Delete.

Unless the NZB doesn't contain par2 files, SABnzbd will always attempt verify/repair.
What can go wrong is that par2-multicore that runs by default
isn't compatible with your system.
In that case you can say in Config->Switches that you want to use the "classic" version.
We know of no unrar problems (with Windows).

Other than that repair/unpack is very reliable,
otherwise we would get massive complaints.

Now again some bizarre NZB comes along that SABnzbd
cannot handle properly, but those are rare incidents.

[Freman]

I have par2_multicore = 0 in my config.

Honestly I don't think it's the nzb files because I've now downloaded over 30 things with nzb's from various sources and not one of them has par2repaired or unrared.

Believe me, I honestly don't think it a problem with SABnzbd, I've probably screwed up some part of my installation.

What I've done right now is start a download, I'll keep the nzb file aside and provide to whoever I need to, i'll also have the logs of the entire thing in case there's something useful in that.

In the mean time, must sleep.

[Freman]

Ok, so I couldn't sleep...

Attached is a log extract (didn't think you needed the whole download process just the post process)... and below is the output of unrar

And here is the information that I see you guys are now asking for in a sticky

----------
Version: 0.5
OS: Linux - 2.6.31-gentoo-r6
Install-type: SABnzbd-0.5.0-src.tar.gz
Skin: Plush
Firewall Software: None outgoing, iptables incoming, nat (depends on which server I connect to as to which gateway the traffic takes)
Are you using IPV6? I have a dual stack ipv6 network - but I connect to my usenet servers over ipv4
Is the issue reproducible? Happens for me every time
----------

Code: Select all

unrar x otv-ngshsotml-tvxvid.rar  /home/public/Video/Documentry/National\ Geographic

UNRAR 3.91 freeware      Copyright (c) 1993-2009 Alexander Roshal


Extracting from otv-ngshsotml-tvxvid.rar

Extracting  /home/public/Video/Documentry/National Geographic/National.Geographic.Secrets.History.Secrets.Of.The.Moon.Landing.PDTV.XviD-OTV.avi   3%

Extracting from otv-ngshsotml-tvxvid.r00

...         National.Geographic.Secrets.History.Secrets.Of.The.Moon.Landing.PDTV.XviD-OTV.avi   8%

~~~~~~~ Skip many rar files ~~~~~~~

Extracting from otv-ngshsotml-tvxvid.r22

...         National.Geographic.Secrets.History.Secrets.Of.The.Moon.Landing.PDTV.XviD-OTV.avi  96%

Extracting from otv-ngshsotml-tvxvid.r23

...         National.Geographic.Secrets.History.Secrets.Of.The.Moon.Landing.PDTV.XviD-OTV.avi  OK
All OK

[shypike]

Looking at the log, I can see that SABnzbd starts unrar.
Normally it would check the expected files after running unrar,
but that isn't done because we don't know how unrar will behave
on this file system (will it use latin-1 or utf-8?).
There's no exception, meaning that unrar ran fine.
However, it is logged that no new files were created.
The rar lists are all OK.
The exact data exchange between SABnzbd and unrar isn't logged,
so it hard to tell what is going on.

It also seems that you are using +Unpack and not the usual +Delete,
because at the end SABnzbd decides to move the rar file to
the final folder instead of deleting them.
I assume that's what you want? Benificial in this case, because unrar doesn't do its work.

Since you're on Linux, you are using Python sources.
I can email you a replacement module that contains additional logging
so that we can find out what's going on here.

[Freman]

I figured +DELETE was falling back to +UNPACK because it was seeing no files created (at least, that's the sort of sanity check I'd have written in given that in Australia we count all our bytes ).

I will gladly take a test module to help with more logging, can you get my email address from my profile (being a mod)?

I should be able to tell you if my system is working in utf-8, or latin1 (given I built it from the ground up) but I honestly have no idea (I'm going to use the caffeine excuse for now)

Edit: I'm going to have to tick the latin1 box for locale, I don't have any specifically configured but I'm fairly sure it falls back to the historical norm.

Code: Select all

priya ~ # locale
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

That's not to say I don't have access to all my locales, I can configure the environment before launching SABnzbd if you think that'll help?

[shypike]

The issue is only that Python doesn't think your system uses UTF-8.
That itself doesn't matter, it only means that SABnzbd will not check the resulting files.
However, SABnzbd doesn't see any files being unpacked at all,
which is true otherwise you would not complain.

I'll send you a patched newsunpack.pyto the registered email account,
when I have it ready.