About CRC Errors

[guitarboy]

I've found that if I get a CRC error, it usually doesn't recur if I repeat the download.  So somewhere between the usenet host (I use newsdemon) and my PC something messes up, but it doesn't happen if I just download again.  It could be in the repair phase, in which case re-running PAR2 and unrar may be all I need to do, but that's too much work.  Much easier to just let sabnzbd take care of it for me.

[chokomon]

I have been getting A LOT of CRC errors recently; everything has worked fine for more than a year before this.  It's not #1, 2, or 3 from OP.  If I keep redownloading the same NZB, eventually it'll work without CRC errors.

I'm running Ubuntu 9.10, SAB 0.5.0.

Does this sound like a memory problem?  If so, can someone tell me how to verify?

EDIT:
It seems the PAR processing isn't working correctly.  When I get a CRC fail, the files appear in a folder in their rar files.  For one fail, I downloaded the PAR2 files on my Windows machines separately and ran them, and there was ONE file missing ONE block.  It fixed it and then I was able to unrar manually without error.  But the NZB had plenty of PARs to allow SAB to automatically fix the issue, but it didn't.  Memory problem?  Or something else?

[chokomon]

I've noticed that all my downloads with CRC errors passed the "Quick Check" - so SAB's par processing was never happening because of the "Enable Quick Check" switch.  I disabled that switch to force PAR checking on every download.  Haven't got a CRC error since making that change (but I haven't downloaded enough yet to know if this was really the problem).

If this is the problem, it means the "Quick Check" isn't working right, at least on my machine.

[chokomon]

UPDATE: It looks like unchecking "Enable Quick Check" (in Config / Switches) fixed my CRC error problem.  Since making the change, I've downloaded a bunch of stuff without a single CRC error (before, I was getting them almost every other download).

So, anyone who has this issue might want to try this.

[shypike]

UPDATE: It looks like unchecking "Enable Quick Check" (in Config / Switches) fixed my CRC error problem.  Since making the change, I've downloaded a bunch of stuff without a single CRC error (before, I was getting them almost every other download).
So, anyone who has this issue might want to try this.

You could tell us a bit more about this, like emailing NZB files that you have trouble with.
With this "fix" you are helping yourself, but no one else.
We're not going to be able to fix it with just reports like this.

[chokomon]

You could tell us a bit more about this, like emailing NZB files that you have trouble with.
With this "fix" you are helping yourself, but no one else.
We're not going to be able to fix it with just reports like this.

I reject the notion that my posts amount to "helping [my]self, but no one else."  To the contrary, I've told everyone how I solved the CRC error, possibly helping others with similar problems.  If I was really 'only helping myself', I wouldn't have posted anything after fixing my own problem.

I'm happy to provide more info - all you had to do was tell me what you needed (I'm not a programmer - I don't know what info you'd want).  I'll post the links to the NZBs that gave me CRC errors when I get home.

[shypike]

If I was really 'only helping myself', I wouldn't have posted anything after fixing my own problem.

You are right, of course.
But we are eager to fix any QuickCheck related problem, because it's highly efficient (provided it works).
I'll be happy to test your NZB files.
BTW: QuickCheck was extensively tested on Ubuntu 9.10 too.

[kitgerrits]

What I'm getting in my logs:
2010-03-29 02:44:53,959::ERROR::[rarfile:315] CRC mismatch! ofs =113
2010-04-17 23:01:51,187::ERROR::[rarfile:315] CRC mismatch! ofs =113
2010-04-17 23:21:29,949::ERROR::[rarfile:315] CRC mismatch! ofs =103
2010-04-18 12:14:20,933::ERROR::[rarfile:315] CRC mismatch! ofs =113
2010-04-18 23:37:26,337::ERROR::[rarfile:315] CRC mismatch! ofs =113

From my history:
[redacted]
(this is as far back as it will go in 0.5)

Oddly enough, the error has not occurred in any other downloads (maybe BiA is at fault?)

[josefismael]

full disclosure: I'll admit straight off the bat that I'm both a linux and newsgroup noob  apologies if this is answered elsewhere

After happily using your app for a few months, it's recently started failing CRC checks pretty frequently. However, when you look in the destination folder, its easy to see why -  the rar's are all different sizes  Where they should be a uniform size (example, 47MB), some are 5MB, some are 30MB, etc. I have easynews, so I know what the file sizes are supposed to be (via the par viewer).

Any thoughts on why sab is "giving up" on some of these rar's? here's some more details:

ubuntu 9.1
sab 0.4.12
seems to occur more frequently the larger the rar's are (20+ MB)

jm

[shypike]

Any thoughts on why sab is "giving up" on some of these rar's? here's some more details:

First of all: do you do a lot of unverified downloads?
In the sense that they are downloads without par2 files?
Do you have "Only post-process verified jobs" on in Config->Switches?

Two likely causes.
The NZB is incomplete in that it doesn't contain all articles composing a file.
You should be able to see that before downloading when the queue details show irregular expected RAR sizes.
The other cause is missing articles on the Usenet server.
You'll get lots of warnings about that.

A third one is a bug in SABnzbd of course, but it can really do the download basics
and it doesn't all of a sudden stop doing that.

[josefismael]

The NZB is incomplete in that it doesn't contain all articles composing a file.

The other cause is missing articles on the Usenet server.

appreciate your response.

what you said makes sense, but why would I be able to "manually" download the rar files to fix the job? My only thought is that maybe I'm hitting some sort of threshold or ceiling on the server side (however unlikely). I've dialed back my Connections from 150 to 50 and this seems to be doing the trick, without impacting my download speed too much.

I do have the "post process only verified jobs" checked, on the recommendation of someone else in this forum. I'm assuming from your statement that a "verified job" means "has par files"??

thanks again
jm

[shypike]

I've dialed back my Connections from 150 to 50 and this seems to be doing the trick, without impacting my download speed too much.

I do have the "post process only verified jobs" checked, on the recommendation of someone else in this forum. I'm assuming from your statement that a "verified job" means "has par files"??

1. The amount of connections is marketing hype. Usually a number below 20 is good enough. You may need to experiment to find the optimum.

2. Indeed.  Only downloads that include par2 files are considered verifiable.
This isn't strictly true, there are some other means to do verification.
SFV files, but SABnzbd ignores these and RAR integrity, but unrar will tell you when RAR files are corrupt.

[josefismael]

1. The amount of connections is marketing hype. Usually a number below 20 is good enough. You may need to experiment to find the optimum.

Word. Well so far this has helped, as 90+ MB rar's are now completing with no errors. Maybe that connection flooding upset something between me and Easynews? I may crank up the connections again just to be sure that was the resolution.

thanks again for the tips and info. Keep up the good work on your amazing product!!
jm

[colson22]

I thought I would share my recent CRC errors experience here. 

Test configuration

1 Server - Ubuntu 10.04 fully patched running VirtualBox

2 Virtual Hosts
  Windows Server 2003 Standard running SAB 0.5.2 - This is the only software installed on this machine - 256Mb Ram - 500G storage
  Ubuntu 10.04 Server - Full updated - SAB 0.5.2 installed from standard Ubuntu repositories (apt-get install method) - 512Mb Ram - 500G storage - Installed Python version Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)

2 NZB fetch methods tested

NZBMatrix RSS feed
NZBMatrix NZB URL pasted in to the Plush UI

All posts downloaded were larger then 500Mb

In each case the windows SAB downloaded, checked, extracted and deleted correctly while the Ubuntu client got CRC errors.

I tested both with and without quick verify, i also tried to pause downloads while post processing, same results in any configuration.

I will gladly provide any other relevant information to assist in troubleshooting this issue as Windows Server is not my preferred long term solution.

Thank you for you assistance,


Frank

[shypike]

I tested both with and without quick verify, i also tried to pause downloads while post processing, same results in any configuration.

When you do not use QuickCheck, all the work is done by par2 and unrar
and SABnzbd does nothing at all influencing the correctness of the files.
So the problem is in par2, unrar or your hardware/OS combination.
I wouldn't know what to test to help you further.
My experience with both virtual and physical Ubuntu is very different from yours.

When things fail, what happens if you run par2 and/or unrar manually?