Ubuntu 9.04 and SABnzbd 0.4.9 from jcfp : ERROR: CRC failed

[virtualshock]

I just refreshed my Ubuntu box from scratch with Jaunty Jackalope and the latest SABnzbd.  I have used SABnzbd almost from it's inception with various versions of Ubuntu.  I have never had any problems with it previously.
So far I have only found 1 NZB that has completed successfully since my upgrade.  I have tried 6 other NZBs which were 4+ GB video files which have all failed in the unrar stage. Here is what SABnzbd is displaying :

Stage Download
    [Time-Taken]: 2 hours 11 minutes 6 seconds
    [Avg-Speed]: 609kB/s

Stage Par2
    [PAR-INFO] the.blahblah.720p.bluray.x264-sinners: => Repairing : 100%

Stage Unrar
    [RAR-INFO] the.blahblah.720p.bluray.x264-sinners: => ERROR: CRC failed in "the.blahblah.720p.BluRay.x264-SiNNERS.mkv"

Could it be that I have picked files that were corrupted?  I have never had this many files fail on me before, I think something is wrong.  Does anyone have any suggestions for me to try?

Thanks

[sander]

Are you really using the jcfp version from PPA? That's not necessary anymore, as Jaunty has "sabnzbdplus" in the standard repositories. Have you tried that version?

And have you tried a manual unrar from the commandline?

[virtualshock]

I ran an unrar last night with Archive Manager using the files that were put into the "completed" directory by SABnzbd, it successfully created the ".mkv" that had failed with the CRC error by SABnzbd.  I ran "unrar x filename" on the command line and at the 99% point on the last rar file it said CRC failed.

You are correct about not using the jcfp version.  I followed the forum posting by jcfp not knowing that Jaunty had the package in it's repo.  The version that was installed is the Ubuntu repo version.

[sander]

I ran an unrar last night with Archive Manager using the files that were put into the "completed" directory by SABnzbd, it successfully created the ".mkv" that had failed with the CRC error by SABnzbd.  I ran "unrar x filename" on the command line and at the 99% point on the last rar file it said CRC failed.

Interesting: Archive Manager works OK, and unrar x not. Which version of Archive Manager resp unrar are you using?

I don't understand this: I thought Archive Manager (=file-roller) is just a GUI front-end for the underlying unrar. See http://packages.ubuntu.com/nl/jaunty/file-roller


And: can you mail me (a pointer to) the NZB you're using so that I can try myself?

[virtualshock]

FileRoller according to Synaptic is 2.26.1-0ubuntu1

I just ran the same test for a different binary, but this time both FileRoller and unrar had a CRC problem.  That is at least consistent, maybe that was just a bad binary.

This is frustrating because I just can count on 1 hand the number of CRC errors I have had over the last 3 years or so.  I just want to make sure my setup is OK.

I will PM you with the NZB that was not consistent.

[switch]

So which repository did you actually install from? The jaunty one or the one from ppa.launchpad.net/jcfp/?

Also if you know how to, try switching it to use python 2.6, you are not the first to respond with CRC errors with Jaunty.

[sander]

Also if you know how to, try switching it to use python 2.6, you are not the first to respond with CRC errors with Jaunty.

Do you really mean 2.6 (not 2.5)? Python 2.6 is already the default on Jaunty 9.04.

And apart from that: unrar itself is a binary and not Python dependant, so I guess your statement means the CRC problem is caused before unrar is invoked ... ?

[jcfp]

Having the standard ubuntu repository and ppa side by side in your apt sources can be savely done, even if both provide a sabnzbdplus package. The version of the packages on the ppa is designed such that in case both contain the same release -as is the case with 0.4.9 right now- the one from the ubuntu repositories takes precedence. Not that it matters much since they are essentially identical.

The crc error could have a number of causes, first and foremost bad uploads, but also a bug in unrar, python, packaging, or sab itself. Packaging could be ruled out by running sabnzbd+ directly from sources: shut down any running instances, download the -src.tar.gz release, extract, run the program with "python2.5 SABnzbd.py" and try downloading that very same nzb again.

Unfortunately I can't test the nzb right now even if I had it, me and my isp currently have this small disagreement about whether or not uploading 230 GB in 6 hours to a linux torrent fits the definitions of their "fair use policy"; until then my connection speed is capped a bit too low to enjoy usenet.

Do you really mean 2.6 (not 2.5)? Python 2.6 is already the default on Jaunty 9.04.

Default python on jaunty is indeed 2.6 but sabnzbdplus packages are hardcoded to use 2.5 only. I agree with your remark about unrar though, it certainly has potential to be the culprit here.

[virtualshock]

As jcfp stated, the Ubuntu repo version took precedence even though I thought I was installing jcfp's package.  Interestingly, the dependency in SABnzbd on python is for 2.5 not 2.6.  I have Ubuntu 9.04 installed and from the command line just typing "python" it is using python 2.6.2 .  Is this the problem?

[switch]

No, SABnzbd 0.4.x does not work properly with 2.6. However the person who also claimed a CRC error on IRC said it was fixed for him after switching it to 2.6.

[01:46] Hey duz, looks like the standard ubuntu release fixed the problem, but I also had to change the python version to 2.6
[01:47] Looks like the CRC errros have stopped

[Jaunty_Joke]

Hi, Its the guy from IRC, sorry but about an hour after I posted that reply I started seeing the same problems again.  I have since re-installed on a completely different system and I'm seeing the same errors.  I had thought that maybe it was a system/hardware issue so started from scratch on a completely different set of hardware.

Very simple procedure to reproduce from my end:

1; Install Ubuntu 9.04 from Alternate or Server CD
2: Apt-get update
3. apt-get dist-upgrade
4: apt-get install sabnzbplus
4 configure /etc/default/sabnzbplus

Code: Select all

# This file is sourced by /etc/init.d/sabnzbdplus
#
# When SABnzbd+ is started using the init script, the
# --daemon option is always used, and the program is
# started under the account of $USER, as set below.
#
# Each setting is marked either "required" or "optional";
# leaving any required setting unconfigured will cause
# the service to not start.

# [required] user or uid of account to run the program as:
USER=sysadmin

# [optional] full path to the configuration file of your choice;
#            otherwise, the default location (in $USER's home
#            directory) is used:
CONFIG=

# [optional] hostname/ip and port number to listen on:
HOST=
PORT=

# [optional] extra command line options, if any:
EXTRAOPTS=

5. run the service and configure the basics using the web interface:

Code: Select all

__version__ = 18
[misc]
log_dir = logs
auto_browser = 0
permissions = ""
api_key = 5956aa178d4074e443c04de53b729fa4
check_new_rel = 1
replace_spaces = 0
replace_illegal = 1
fail_on_crc = 0
enable_filejoin = 1
enable_unzip = 1
enable_unrar = 1
auto_disconnect = 1
enable_par_cleanup = 1
par_option = ""
no_dupes = 0
config_lock = 0
safe_postproc = 0
pause_on_post_processing = 0
cleanup_list = ,
ignore_samples = 0
send_group = 0
download_dir = downloads/incomplete
download_free = 0
complete_dir = /home/news
script_dir = ""
nzb_backup_dir = ""
cache_dir = cache
dirscan_dir = ""
dirscan_speed = 5
refresh_rate = 0
rss_rate = 60
bandwith_limit = 0
cache_limit = 120M
email_server = ""
email_to = ""
email_from = ""
email_account = ""
email_pwd = ""
email_endjob = 0
email_full = 0
schedlines = ,
dirscan_opts = 3
dirscan_script = None
top_only = 1
auto_sort = 0
enable_tv_sorting = 0
tv_sort_string = ""
ionice = ""
web_color = ""
web_color2 = ""
host = 0.0.0.0
port = 8080
web_dir = smpl
web_dir2 = ""
username = ""
password = ""
[logging]
enable_cherrypy_logging = 1
log_level = 1
max_log_size = 5M
log_backups = 5
[newzbin]
username = ""
password = ""
bookmarks = 0
unbookmark = 0
bookmark_rate = 60
[servers]
[[news.giganews.com:119]]
host = news.giganews.com
port = 119
username = 
password =
timeout = 120
connections = 10
fillserver = 0
ssl = 0
[rss]
[categories]
...

I tried again tonight ... here are the CRC errors I'm seeing

Code: Select all

2009-04-27 21:30:35,760 WARNING [decoder] CRC Error in <Article: [email protected], bytes=991040, partnum=13, art_id=None> (DE0C983E -> 2C00826E)
2009-04-27 21:30:38,868 WARNING [decoder] CRC Error in <Article: [email protected], bytes=991168, partnum=17, art_id=None> (1711D47B -> 237502E7)

some jobs will finish some have too many errors to be fixed and end up with RAR extraction errors.

Let me know if you need any more info

[sander]

FYI:

I downloaded without any problems the NZB/post that caused Virtualschock the CRC erros. I'm using Jaunty 9.04, SAB 0.4.10, python 2.6

One difference with virtualschock's output is that my download apparently didn't need a Par2-repair.
However, considering the python-dependance: the Par2-repair is done by 'par2' (right?), which is an executable too, and thus not python-dependant. Correct?

Code: Select all

Stage Download
    [Time-Taken]: 1 hour 7 minutes 35 seconds 
    [Avg-Speed]: 1165kB/s

	
		

Stage Par2
    [PAR-INFO] blablabla720p.BluRay.DTS.x264-CtrlHD: => Verified in 434.6s, all files correct
    [DEL-INFO] blablabla720p.BluRay.DTS.x264-CtrlHD: => Deleted 2 file(s)

	
		

Stage Unrar
    [RAR-INFO] blablabla720p.BluRay.DTS.x264-CtrlHD: => Unpacked 1 files/folders in 1281.3s
    [DEL-INFO] blablabla720p.BluRay.DTS.x264-CtrlHD: => Deleted 94 file(s) 

[sander]

And no problems either with the same NZB on Jaunty with the repository's sabnzbdplus 0.4.9 (and thus python 2.5).

So the only difference I now see is the missing files in virtualshock's download (see his par2 repair). Could the problem be caused by these missing files and subsequent SAB behaviour?

@virtualshock: you could check this hypthosis by activating IPv6 on your jaunty ("sudo apt-get install miredo", which should give you 2001: address within a minute or so), and then *only* using news.ipv6.eweka.nl as newsserver (6 threads): as eweka has the full post available and no missing articles, you could then verify my hypothesis.

[Jaunty_Joke]

@virtualshock: what hardware are you running on?  I'm curious to know if maybe thats the problem.

I'm setting this up on an old system  as a headless server:
AMD 3000 64 bit (only using the x86 version of ubuntu)
1Gb ddr
2xSeagate40 GB drives in a software RAID
1x320 drive mounted as /home

[switch]

Either of you using EXT4?

Edit: Also are you using SSL for usenet connections?