Fluctuating Download Speeds

[djh82uk]

Hi All

I have Sab installed on my linux server which also acts as a media server & firewall/router

Everything was fine for a while, but now I get wildly fluctuating speeds. I have 100mb broadband. Sometimes I hit the full 11mb/s but then after a while it will drop to 7mb/s or 2mb/s and sometimes around 10k/s. Now and again it hits 0k/s and just sits there. If I download from my desktop using newsleecher I get 10-11mb/s all day long. And as that connection goes via the sab linux server as it act a router, I guess it shows my connection/internal network are ok (gigabit).

I increased the cache to 10Gb, and wiped what was on there and deleted any orphan jobs and no joy, I was on version 0.5 something from the official ubuntu repo so I uninstalled and re-installed 0.65 via the jcfp ppa.

I wiped my .ini file beforehand.

And I still get these fluctuating speeds, I tried a 8Gb mkv and ot started at 11mb/s, dropped to 2mb/s went upto 11mb/s again and is now sitting at 72kb/s. It's a core2duo machine with 2Gb ram, 80Gb system drive and a10Tb raid array which is where the files get extracted to.

System load during download goes from 0.05 to circa 1.1

Any thoughts?

Thanks

DJH

[shypike]

Always hard to tackle or even explain such issues.
You did all the right things.
BTW: how many connections do you use? SABnzbd doesn't handle a very large amount very well.
For the "standard" providers like Giganews and astraweb, more than 20 is not needed and may be counter-productive.
I get a consistent speed from all my boxes (Windows, Ubuntu, OSX, Win HomeServer).

The last very slow part might be explained by a non-responding server.

What happens if you "nice" the SABnzbd process, giving it less CPU time?

[djh82uk]

Hiya

I have number of connections set to 25 at the moment. It does the slowing right down thing all the time, while newsleecher does not.

I will try the nice command and see how that goes.

Thanks

Djh

[djh82uk]

Another thing I thought was memory usage, not much spare, this was taken with sabnzbd being idle with an empty Q:

28611 djh 1013696 kB /usr/bin/python /usr/bin/sabnzbdplus --daemon
9430 root 581868 kB java -Xmx100m -Dsubsonic.home=/var/subsonic -Dsubsonic.host=0.0.0.0 -Dsubsonic.p ...
9412 openldap 346484 kB /usr/sbin/slapd -h ldap:/// ldapi:/// ldaps:/// -g openldap -u openldap -F /etc/ ...
6688 root 325124 kB /usr/sbin/clamd
9827 ebox 321324 kB /usr/bin/perl /usr/share/perl5/EBox/EventDaemon.pm
9810 ebox 318980 kB /usr/bin/perl /usr/share/perl5/EBox/EventDaemon.pm
2962 mysql 280468 kB /usr/sbin/mysqld
9690 ebox 257504 kB /usr/bin/perl /usr/share/ebox/ebox-loggerd
7725 www-data 245712 kB /usr/sbin/apache2 -DNO_DETACH
7726 www-data 245712 kB /usr/sbin/apache2 -DNO_DETACH
7727 www-data 245712 kB /usr/sbin/apache2 -DNO_DETACH
7728 www-data 245712 kB /usr/sbin/apache2 -DNO_DETACH
7729 www-data 245712 kB /usr/sbin/apache2 -DNO_DETACH
21822 www-data 245712 kB /usr/sbin/apache2 -DNO_DETACH
7694 root 243404 kB /usr/sbin/apache2 -DNO_DETACH
7346 root 227440 kB /usr/sbin/collectd -f -C /etc/collectd/collectd.conf
9035 bind 221316 kB /usr/sbin/named -u bind
11336 ebox 219788 kB /usr/sbin/apache2 -k start -f /var/lib/ebox/conf/apache2.conf
8710 ebox-usercorner 216408 kB /usr/sbin/apache2 -f /var/lib/ebox/conf/user-apache2.conf -DNO_DETACH
9738 mumble-server 207292 kB /usr/sbin/murmurd -ini /etc/mumble-server.ini
2055 syslog 191628 kB rsyslogd -c4
8731 root 185676 kB /usr/sbin/apache2 -k start -f /var/lib/ebox/conf/apache2.conf
8595 root 182444 kB /usr/sbin/apache2 -f /var/lib/ebox/conf/user-apache2.conf -DNO_DETACH
9724 snort 165500 kB /usr/sbin/snort -m 027 -D -d -l /var/log/snort -u snort -g snort -A fast -c /etc ...
9704 snort 164956 kB /usr/sbin/snort -m 027 -D -d -l /var/log/snort -u snort -g snort -A fast -c /etc ...
6985 bind 142808 kB /usr/sbin/named -u bind -g
5269 root 122676 kB /usr/sbin/console-kit-daemon --no-daemon
11343 proxy 119028 kB /usr/sbin/squid -N
5493 djh 118652 kB sshd: djh@pts/1
2874 root 116476 kB sshd: djh [priv]
9699 postgres 104928 kB postgres: ebox eboxlogs [local] idle
3455 root 104724 kB smbd -F
5431 root 104652 kB smbd -F
9834 postgres 103704 kB postgres: ebox eboxlogs [local] idle
18386 root 103048 kB smbd -F
3512 root 102532 kB smbd -F
10914 root 101708 kB smbd -F
9814 postgres 101236 kB postgres: autovacuum launcher process
9812 postgres 101084 kB postgres: writer process
9777 postgres 100980 kB /usr/lib/postgresql/8.4/bin/postgres -D /var/lib/postgresql/8.4/main -c config_f ...
9813 postgres 100980 kB postgres: wal writer process

Sab is using Gb of ram, I only have 2

[beaston02]

I am new to sabnzbd (just installed yesterday)

I have a very similar issue, running on mac osx. Ive been playing with the Article Cache Limit settings, and as soon as it reaches its limit (currently set to 256M), my download speeds drop from between 2.5 and 3MB/s to <500k, and as low as 0k.

also, I let it run for about 30 minutes with the Article Cache Limit set to -1, and checked the memory usage of sabnzbd and it was up over 2GB. My macbook pro I am running it on has 6GB installed. I currently have the connections set to 2 (gradually dropped from 20 trying to find a magic number). Maybe im missing something, since I am new to all this, but I guess Im not the only one with this issue.

[shypike]

If memory keeps running up there are usually two explanations.
1. memory leak. possible but not likely, since it's very rarely reported for Windows and OSX.
2. SABnzbd cannot complete its files because articles are missing and you have a non-responding server

Which Usenet provider do you use?
Do you have multiple servers?

[djh82uk]

Hiya

I am using giganews, eventually the stuff downloads ok, and as I said I can run it via newsleecher and hit 10mb/s all day long. But sab is consistently fluctuating.

Thanks

Djh

[shypike]

We will be looking at this later.
Currently the focus is on getting SABnzbd fully working on OSX Lion.

I still have the question: do you have a single server or multiple?

[djh82uk]

Hi

I have a single server, connecting to the europe Giganews server.

DJH

[shypike]

I don't get it.
Even my VMWared Ubuntu gets an even speed (about 8MByte/sec).
Did you ever use the 0.5.x range?
I wonder if that gives you a better result.

[djh82uk]

Hi

Thats what I was using, and then moved to 0.65 to try and fix it.

DJH

[shypike]

Right, missed that part.
My memory isn't perfect and I don't re-read every thread every time.

When the memory usage builds up, do any files get saved in the job folder in "incomplete"?
I mean the actual files, not what's in the __admin__ folder.
Still trying to make a mental picture of what's may be going on.

There's been a thread about the memory growth as such.
It looks like the garbage collector for Python on Linux is worse than on OSX and Windows.
There are some work-arounds suggested too.
http://forums.sabnzbd.org/viewtopic.php?t=4713

[mollydog]

I know this is a few days old now but I found the router was the problem with similar symptoms. Disable any SPI firewall it has and try again.

[Mushin]

Hello I now have the same problem as described by djh82uk im also using 0.50 on ubuntu 10.04 and it worked for weeks without a problem sabnzbd (0.65) on my Windows 7 pc works without a problem. I use eweka and have the connections set to 4 on both Windows an Linux + memory usage is good only ~800/~4000 so that shoulnd be the problem now trying a newer version on Ubuntu if there are some results i will report back.

EDIT:

no luck with newest version still only around 45KiB/s

EDIT 2:

Disabled SPI firewall had no effect so not the problem here

[shypike]

Did you upgrade from an existing older version while keeping
settings and all admin data.
Can you try a clean install?