Big NZBs make for slow/spikey downloads

[MonkeyFighter]

I regularly use individual nzb files for 20GB-50GB downloads.
I use sabnzbd on ubuntu 9.04 (64-bit) and 8.10 (32-bit).
I've tried both 0.4.8 and 0.4.11.

With these large nzb files sabnzdb likes to periodically chew up cpu and I've noticed that when it has the cpu pegged at 100% actual transfer rate drops to zero.  The reported download speed from within sabnzbd often shows full throttle instead of indicating the drop to zero.  It seems like the more articles downloaded from the nzb, the longer the periods of 100% cpu utilization and zero transfer rate get.  If I add a small nzb (say 2GB) to the head of the queue, it immediately goes back to low cpu utilization and maximum download speed until it completes at which point it goes back to the partially completed large nzb and slows down again.

The current system has 8GB of ram and sabnzbd is only using 1.5GB resident as reported in top.  There is no swapping occurring, most of the remaining 8GB is used for buffercache and there are 4 cores total, the other 3 are 99% idle.

If I strace the thread that is doing the writing to disk, I see it frequently sit on a futex for like half a second every few seconds in the middle of calling write() a bunch of times.

Has anyone seen and solved this problem?
Using smaller nzb files is not an option, these are individual nzbs for single posts.

[shypike]

Can you email me an example of a troublesome NZB file?
Preferably the newzbin report number or a zipped nzb to [email protected]

One reason could be that SAB has to look through larger lists to determine
what has to be downloaded next.

Last question: How is the option "Only get articles for top of the queue" in Config->Switches set?
Setting it "off" is more likely to cause such problems.

[MonkeyFighter]

Last question: How is the option "Only get articles for top of the queue" in Config->Switches set?
Setting it "off" is more likely to cause such problems.

I've tried it both ways, it does not seem to make a significant difference.

Can you email me an example of a troublesome NZB file?
Preferably the newzbin report number or a zipped nzb to [email protected]

I'm not a member of newzbin so I can't do a report number for you.
I can email an example nzb, but even compressed it is 2.4MB.
I am going to try sending you a URL.  Please post back here if you've received it and if it works for you.

I've verified that the example starts off slightly spikey (a cycle of about 7 seconds max throughput then 3 seconds of degraded throughput on a link that is roughly 50mbps) as the transfer progresses the ratio gets worse and worse.

[shypike]

On Windows I see little difference between this download and a smaller one.
It's likely to be spiky because the system cannot cope with the speed.
Messages  come in faster than they can be processed, so the downloading is temporarily throttled.
With my Core 2 duo and a fiber connection I get about 7500 KBytes/sec of the
theoretical max of 10.000 KBytes/sec.

Do you have the memory cache on (and set to al least 70M)?
Do you use SSL (costs more CPU time)?

I'll do some more testing (also on Linux) and keep you posted.
Maybe the behaviour is different on Linux.

[MonkeyFighter]

Memory cache is set to 140MB, I also tried 70MB and it didn't make a significant difference.
SSL is not in use.

FWIW, here is an example traffic graph - the first two-thirds is for a large 60GB nzb that has been running for 10-15 minutes.  The last third (where it is pretty much flat) is when I bumped 1.5GB nzb to the top of the queue.
http://i32.tinypic.com/x5s7y8.jpg

If it makes a difference, the underlying filesystem is XFS on a hardware raid capable of sustaining ~300MB/sec for streaming reads and writes.  At the time of testing in the graph there wasn't any other significant use of the filesystem.

[shypike]

The problem is likely in SABnzbd and the Python libraries.
Both are no speed kings.

Should you have full logging on (debug level) you will probably
see "delaying" and "undelaying" messages.
This means that SABnzbd cannot process the incoming articles fast enough
and it needs to stop incoming traffic.
As an experiment you can set logging level to errors/warnings only,
this should make some improvement.

Speedcapping will also smoothen the download speed, but it will not
increase the average speed.

We are looking at improving the speed, but the Python language we use
is not exactly the fastest thing on the planet.

Last question: do you have the yEnc package installed?
It's optional but it will increase the processing speed because it
replaces Python level yEnc decoding with C-level code.

So far I have not found a reason why big download jobs should be
slower than  smaller ones. But I'll keep looking.

[ardeck]

UP!

I have exactly the same problem with huge Nzb ( > 30 MB), for 100GB+ download.
Downloads are fast at first and then they slow down. It's not a network problem as it can always download up to 5 MB/s but I have the same kind of behavior. Between the download of 2 files there are gaps where nothing is downloaded.
Furthermore the memory consumption of the process is raising to  700MB, and the process is using a CPU core. I tried to restart the program, the browser of even the PC but there is no improvement.

I have Vista 64 SP2, core 2 duo, 6gB RAM and RAID 0 disks, 100Mb BW, sabnzbd 4.0.11. I don't think the problem is related to my configuration. I'm using the parameters you provided. With a list of nzb I have not noticed the problem.
I can divide the nzb in small parts but it's a pain.

Here's a sample of a log file :


2009-08-03 15:12:08,726::INFO::[downloader] Thread [email protected]:80: fetching part26of412.YkCZiJxNmJT1W&[email protected]
2009-08-03 15:12:08,730::INFO::[articlecache] Flushing to disk
2009-08-03 15:12:08,730::INFO::[sabnzbd] Saving data for SABnzbd_article_akhtqb in C:\Users\Lolo\AppData\Local\sabnzbd\cache\SABnzbd_article_akhtqb
2009-08-03 15:12:08,732::INFO::[sabnzbd] Saving data for SABnzbd_nzo_dsjtni in C:\Users\Lolo\AppData\Local\sabnzbd\cache\SABnzbd_nzo_dsjtni
2009-08-03 15:12:08,736::INFO::[downloader] Thread [email protected]:80: part3of412.YkCZiJxNmJT1W&[email protected] done
2009-08-03 15:13:23,697::INFO::[decoder] Decoding
2009-08-03 15:13:23,697::INFO::[downloader] Thread [email protected]:80: fetching part27of412.YkCZiJxNmJT1W&[email protected]
2009-08-03 15:13:23,697::INFO::[assembler] Decoding C:\Users\Lolo\Documents\downloads\incomplete\1249250292\BB095.part34.rar yenc
2009-08-03 15:13:23,706::INFO::[downloader] Thread [email protected]:80: part9of412.YkCZiJxNmJT1W&[email protected] done
2009-08-03 15:13:23,713::INFO::[downloader] Thread [email protected]:80: fetching part28of412.YkCZiJxNmJT1W&[email protected]


There is 1 minute between the downloading of 2 files.

[shypike]

Please sign up as  a Release tester in the Beta forum.
The pending 0.5.0 release has had huge changes. Maybe that will solve your problem.
BTW, your system specs should not a be a bottleneck.
I have a similar system and it easily saturates a 100 MBit fiber connection.

BTW: do the "slow" jobs consist of very large files or do they consist of very many standard sized files?

[lboregard]

i'm using the trunk version, and i'm having the same problems ... currently it's running but isn't downloading anything .... it's using aprox 90% of the memory (1gb), 1% cpu  ... but doesn't move ... i can't even connect to it through the browser 

[shypike]

I do see some slowdown with a very large NZB in the queue.
But on a very modest Atom 230 system, it downloading no slower than 2 MBytes/sec.
Replacing the big NZB with a  bunch of smaller jobs drives the speed up to about 4 MBytes/sec..
(Currently it's a Sunday evening over here, so prime-time. Often I can get upto 6 MByte/sec).

It's worth investigating this.

BTW: Did you set the option "Only Get Articles for Top of Queue" on Config->Switches on or off?
This can have quite some influence ("off" means more memory usage and possibly lower speeds).

[lboregard]

i enabled "only get articles ... " option (checked it) ... it seems to behave a little bit better so far, but it does stop downloading completely when i add any nzb file size ... i have the feeling it has to do with python 2.6, because i used to have large nzb running smoothly under 0.4.9, ubuntu 8.06 LTS, which i assume was running python 2.5 ... things went downhill after i upgraded to 0.4.11 and ubuntu 9.04 

[shypike]

We haven't properly tested with Python 2.6.
But the effect you describe does occur also with Python 2.5, but maybe a lot less.

I think it's possible to have Python 2.5 on your system too (in parallel with 2.6).

One problem we have is that nobody of the core team uses Linux for any serious downloading.
I just replaced my Ubuntu 8 server box out of frustration with Linux (unrelated to SABnzbd).

[lboregard]

thanks for the reply ... may i ask what are you guys using for the serious downloading ? freebsd ? opensolaris ?

[shypike]

I'm using an Acer Windows Home Server (Atom 230 CPU),
from which I get a surprising 6 MBytes/sec from the maximum 10 MByte/sec possible.

The only drawback is that unpacking is a bit slow (a 15 minute download,
usually takes between 5-15 minutes to unpack).
And I'd better not mention serious par2 repairs. But the latter is very very rare.

Mind you, the Ununtu server had a lowly Pentium III, but still managed to get upto 3.5 MBytes/sec.
But par2 and unrar were a lot worse than on the WHS box.

[lboregard]

a windows box ... whs is actually based on windows 2003 server sp2 ... the thing is running a windows server makes me think more in terms of newsleecher than sabnzbd (compiled delphi vs interpreted python).
i wouldnt have thought that sabnzbd development took place in a windows environment