Very Slow download speed in 1.1.0 and later [VM or VPN]

[Sabpp]

When I updated to 1.1.0 I started experiencing very slow download speeds. This happens when I transition the code from 1.1.0 RC3 to RC4 and above. My download speed goes from wire speed (5.6MB/s) to 20-40KB/s.

I have tried setting server load balancing to "Prevent Load-balancing" but it doesn't fix anything.

RC3:
2016-10-26 20:53:30,173::DEBUG::[bpsmeter:270] bps: 5525935.61924

1.1.0:
2016-10-26 20:55:46,312::DEBUG::[bpsmeter:270] bps: 50744.2284521
2016-10-26 20:55:51,371::DEBUG::[bpsmeter:270] bps: 28236.6510364
2016-10-26 20:55:56,387::DEBUG::[bpsmeter:270] bps: 22702.6588401
2016-10-26 20:56:01,433::DEBUG::[bpsmeter:270] bps: 19748.5892482
2016-10-26 20:56:06,482::DEBUG::[bpsmeter:270] bps: 18040.8149414
2016-10-26 20:56:11,545::DEBUG::[bpsmeter:270] bps: 47290.5503227
2016-10-26 20:56:16,594::DEBUG::[bpsmeter:270] bps: 35147.691274
2016-10-26 20:56:21,650::DEBUG::[bpsmeter:270] bps: 25305.4613702
2016-10-26 20:56:26,651::DEBUG::[bpsmeter:270] bps: 20943.7889277
2016-10-26 20:56:31,667::DEBUG::[bpsmeter:270] bps: 19352.8161458

Any ideas?

[safihre]

How about 1.1.1RC2?

[Sabpp]

Thanks for the quick reply.

I am getting about 100 to 130 KB/s with 1.1.1RC2 .

2016-10-27 14:48:59,263::DEBUG::[bpsmeter:270] bps: 139257.672165
2016-10-27 14:49:04,266::DEBUG::[bpsmeter:270] bps: 105670.52802
2016-10-27 14:49:09,267::DEBUG::[bpsmeter:270] bps: 119876.724123
2016-10-27 14:49:14,282::DEBUG::[bpsmeter:270] bps: 135713.41407
2016-10-27 14:49:19,286::DEBUG::[bpsmeter:270] bps: 111404.663335

Let me know if you would like me to change any settings or test anything.

[safihre]

How many connections do you use to your Newsserver? Do you use a VPN or ssl?
There is 1 thing between versions 1.1.0RC 3 and 4 that could cause it.. But this is fixed in 1.1.1 for all other users. So I don't understand what's different about your setup..

[Sabpp]

OK here is some interesting Facts...

1. I am using a VPN (Hop-off in Netherlands)
2. I am using multiple Usenet providers
3. I am using SSL to the Usenet providers

As soon as I configure under 8 connects (7 / 0, 0 / 7, or 3 / 4) download speeds go back up to 3~4 MB/s.

What is it about under 8 connections?

[safihre]

Yes, the optimizations to reduce CPU usage are disabled when there's less than 8 connections.
Is it any better without VPN?

[Sabpp]

I appears there is no issue with the download speed when the VPN client is uninstalled and there is 8 or more connections. Very strange.

The crazy thing is I can hit my max download speed 5.6MB/s with 7 connections and the VPN is connected.

Is there a way to turn off those "optimizations" when you have more than 8 connections?

[safihre]

No unfortunately not. In 1.1.1 we already greatly reduced the optimizations, because other people were experiencing problems in 1.1.0 like yours.
Are you using the maximum number of connections for your Newsservers? How many connections do you have in total?

Basically if only 1 or 2 connections have new data, we slow down the checks for new data a tiny bit because the checks are more cpu expensive than the actual processing of the data.
So more number of connections is actually better in the new sabnzbd.

[Sabpp]

I have a total of 20 connections. I get the CPU optimization thing on a low power CPU like a NAS or first gen raspberry pi but I never really see the SABnzbd.exe use more that 3-5% of the CPU during download. This is on a Windows 7 Thin PC virtual machine with 2GB RAM and 3 virtual CPUs. The host CPU is an i5-2500K.

I think if you made those optimizations optional with a checkbox in the config I would be happy (as long as it fixed the download speed problem). If you want to try to figure out the root cause with the VPN I would be willing the help you out there as well.

[safihre]

It's main purpose is exactly for systems where there is enough cpu power, because there the check if connections are ready is performed way too often. Causing lots of unnecessary cpu usage.
There seems to be a problem with these special setups involving VM's and/or VPNs.
Adding the option wouldn't be usefull since it only seems to cause problems in these very specific setups..
Have to think how to best solve this.

[Sabpp]

Let me know if you want me to test any fix you come up with. Thanks for the help.

[tarlcabot]

I was getting the same thing. Very slow downloads with 1.1.1 with 10 connections and no VPN. Backed down to 1.0.3 and things sped up considerably. Also 1.1.1 was sluggish. Long delays between clicking on a tab and it coming up, the queue completely corrupted yesterday, and it didn't want to quit from the tray icon. I had to close it down several times by ending the process.

[safihre]

There's unrelated things going on I think.
Which tabs do you refer to? In Glitter or Plush?
We're only aware of some lasting speed issues on 1.1.1, working on that.
Could you see dthe logs to [email protected]? To see what is going wrong..

[Sabpp]

I did upgrade to 1.1.1 and I am seeing the same issues.

When the VPN is connected <8 connections = Fast, >=8 connection = Slow.
When the VPN is disconnected it is all fast.

Let me know if you want me to test anything.

[zeroday]

Finally found this topic and wanted to share my story

I have a VM with a debian instance which is already 1 or 2 years running fine, I've updated this instance all the time and it gives me perfect and normal speed as expected. (my max speed is 15MB/s and I see it sometimes, limited by my max 4 connection (and from today put towards 12 or 16, later on that more)
Now I wanted to move away from this instance as there is running too much on this server and I created an UBUNTU and after some initial tests an DEBIAN environment, also under VMware ESXI

I installed 1.1.1 under ubuntu and initially it seemed to work fine until I noticed a total drop of speed. Normally I only use 4 threads and it can fluctuate, but dropping from 10MB/s towards 500KB no
so I removed the VM and did the same under Debian as I thought UBUNTU was too heavily loaded with shit I do not want. So Debian it is. installed 1.1.1. and damn, same issues, so it was not the instance.

The difference I created was a temp folder towards my NFS share. so actually this was different from my good working instance. So I changed this, unfortunately without luck. no speed difference, starting high, after 600MB download it drops.
Than I checked every single setting in the GUI and modified it, so have 2 "identical" configurations, Now I noticed that the speed did actually improve a little (especially on 'checks like check on sfv, check on be able to complete) but still not stable.

Than I did following:
I copied the sabnzbd.ini from instance A (working ok) to instance B and restarted sabnzbd, als the installations with drives and shares are the same and even the location of sabnzbd is the same the swap of the ini file was painless.
now I noticed that the speed of my new instance was getting where I wanted it to be and it was stable.

Than I read this thread and saw that my usenetprovider of years had removed the max connections .. so i played a little with the connections and to my surprise I am maxing out my ISP connection towards 16MB/s when using 12 or 16 connections. Yeah !!

U dunno what is different in the ini files as there is a lot more settings than in the GUI,
Find here my ini file

http://pastebin.com/YA8tBKK8

I want to try to use the temp folder towards an NFS share again, due to the max size of the vmware and the big files I sometimes want to download.
but in short:

update: 24 hours later server instance has no job, 1 or 2 files in the queue load to 2.0 and very slow download, this means that there is somewhat of a memory leak, cause a reboot is solving the issue. I have also changed a VMWARE settings based on memory, and if that is not the solution, I will check not using thin provisioning. Now with a reboot the load is also towards 2.0 (download, unpack, unrar, control) but the speed is ok.

More to follow