Bug/Annoyance: missing articles on one server apparently slow down the download

Questions and bug reports for Beta releases should be posted here.
Forum rules
Help us help you:
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
User avatar
sander
Release Testers
Release Testers
Posts: 9061
Joined: January 22nd, 2008, 2:22 pm

Bug/Annoyance: missing articles on one server apparently slow down the download

Post by sander »

An Annoyance, and hopefully a bug in 0.4.0 Beta 2:

With two servers defined as 'normal' server (not backup), with one with a retention of 100 days and the other 14 days, a download of a post of 30 days old will slow down to around 600 kB/s. After making the 14-day-retention a backup server, the download goes to normal around 900 - 1000 kB/s.

In others words: missing articles on one server apparently slow down the download from the other server.  >:(
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
DeXeS
Release Testers
Release Testers
Posts: 206
Joined: January 28th, 2008, 1:04 pm
Location: The Netherlands

Re: Bug/Annoyance: missing articles on one server apparently slow down the downl

Post by DeXeS »

Probably because it first checks on the 14 day server and waits to get a error (because the file is older than 14 days) and after that goes on with the second server...

Don't know if it really is the answer but maybe putting the timeout from the 14 day server a little bit faster (standard is 120).

The second thing that might be the case is that 50% is done by the 100 day server at 1000kb/s and the 14 day server does the other 50% but at 0 kb/s so it ends up in a total of 500kb/s.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Bug/Annoyance: missing articles on one server apparently slow down the downl

Post by shypike »

Timeouts won't help you.
A server will tell you when it doesn't have the article, unfortunately some servers take a long time to decide that they don't have an article.
Such a server is not very suitable as a primary server.

Having said that, in your situation the 14-day server should be primary and the 100-day secundary. This is assuming that the 100-day server is more expensive than the 14-day one.
However, the behaviour of the 14-day server will inevitably slow down your downloads.
Making the 14-day server the backup-server (secondary) is not very useful (it doesn't harm either except financially).

I am looking at the possibility of using a retention rate value for the server, so that a server will not be used if a post is beyond its retention rate. However, I'm not too thrilled about the idea, because you can imagine that this will never work 100%. And so gives rise to endless complaints.
jack42
Newbie
Newbie
Posts: 7
Joined: May 11th, 2008, 10:12 am

Re: Bug/Annoyance: missing articles on one server apparently slow down the downl

Post by jack42 »

Being able to assign connections to servers based on a configured retention would be a great feature to have. I have 2 servers, one with a fast connection but a short retention, and one with a slow connection but a long retention. I would really like jobs to be passed to the slower one if they are too old for the fast one. Thing is, for this to really work well, another (younger) job would have to be passed to the fast server to keep that one busy while the slow one handled the older job. I can think of 2 ways this could be done. One way would be to have a separate queue for each server with jobs assigned according to the retention filter. The other way would be to have one queue but with an "active item" for each server. The server with the longer retention would just take the item from the top of the queue, but the shorter retention server would start by looking at the top first item, and if that was too old then it would go down to the next item on the queue and check that, until it found one that it could download.

Don't let fear of complaints put you off. Just put a use at your own risk warning for the feature in the docs.

Ideally, of course, you would want the jobs to be switched to the other server based on the articles actually being missing on the main one, but done intelligently when a certain number or proportion were missing rather than having to try every single article in the whole post, as it is now. This would be rather harder to get to work reliably, and would be a compromise between being sure that the job was really too old (and not just a bit damaged) and minimising the time that it takes to find out if it's too old. I think you're probably better off letting the user configure a fixed switchover point and then taking responsibility for deciding what that point should be. (I find that my allegedly 200 day server actually only has 190 days.) Assuming that the longer retention server is also used as a backup/fill server for the shorter one (ie if an article is missing then get it from the other one) then even if the value configured is too high the job will still get done in the current slower way. It'll get there eventually. If the value is too low then it just means jobs will get switched over unnecessarily, but that doesn't really matter for the few occasions when this happens. The overall user experience will be much better, even if it still isn't perfect.
Post Reply