Page 1 of 1

1.2.0: Propagation delay not working correctly

Posted: January 29th, 2017, 3:46 pm
by Starshadow
When I have a propagation delay time set, downloads that needed it have a tendency to never actually start downloading when the delay time runs out. The 'PROPAGATING' notice disappears but the downloads just hang out in the queue not doing anything. Sometimes they do start on their own, but I haven't figured out a rhyme or reason for why. In making the attached screenshot, I had 5 NZBs waiting for propagation with a delay of 45 minutes set. 1 download started on its own when the delay was up while the other 4 didn't do anything. I can start the hung downloads by manually pausing and resuming each one, and I've also seen them start after adding another NZB that did not require a propagation delay.

FYI, I've had this problem since the propagation delay feature was introduced, it's not new in 1.2.0.

Image

Re: 1.2.0: Propagation delay not working correctly

Posted: January 29th, 2017, 3:56 pm
by safihre
Interesting, we've never seen this before.
Do jou have "Check before download" turned on?
How many servers do you have?

Re: 1.2.0: Propagation delay not working correctly

Posted: January 29th, 2017, 4:37 pm
by Starshadow
I do not have "Check before download" turned on. I have 5 servers configured, 2 with priority 0, 2 with priority 1, and 1 with priority 2.

Re: 1.2.0: Propagation delay not working correctly

Posted: January 30th, 2017, 3:39 am
by safihre
Could you try if it works if you disable Check before download?
We have been having issues with it that we just can't reliably reproduce in order to fix it.

Re: 1.2.0: Propagation delay not working correctly

Posted: January 30th, 2017, 11:34 am
by safihre
Oops I see now that you do *not* have 'Check before download' enabled..

What OS are you on? We could give some code modifications that might help.

Re: 1.2.0: Propagation delay not working correctly

Posted: January 30th, 2017, 2:01 pm
by Starshadow
I'm on Linux (Gentoo, kernel 4.4.39, python 2.7.12). I'd be happy to try out some code changes. In the meantime, since I'm the only one with the issue, I'll try setting up a fresh default configuration and see if anything changes.

Re: 1.2.0: Propagation delay not working correctly

Posted: January 30th, 2017, 3:15 pm
by safihre
It's probably something to do with the number of servers and some timeouts. We have heard more problems of stalling (non starting) downloads, but just can't reproduce it..

Re: 1.2.0: Propagation delay not working correctly

Posted: January 30th, 2017, 5:15 pm
by Starshadow
Resetting my configuration and deleting my admin dir seems to have fixed it. I started from scratch and entered my own settings one by one. It's now identical to before but delayed downloads are no longer hanging. There were some interesting differences between my old and new ini, mostly retired options from previous versions of sabnzbd. I tried readding a few that seemed like they had a chance at being related (e.g. random_server_ip = 1) but I can't reproduce the hanging. My admin dir had been in use for least 4 years, so my bet is that something in there was the culprit. I'll keep an eye open for the next few days and see if I notice it again.

Re: 1.2.0: Propagation delay not working correctly

Posted: January 31st, 2017, 3:36 am
by safihre
Let us know indeed :)

Re: 1.2.0: Propagation delay not working correctly

Posted: January 31st, 2017, 10:31 pm
by Starshadow
I'm sad to report that I'm seeing downloads hang again. But I'm getting better at making it happen on cue. My python is rusty, but I dug around in the code and debug log and have some lead on what might be happening. Try this:

1. Have an empty queue.
2. Add some number of NZBs that will need propagation delay based on the current delay setting, and add one NZB that will reach the delay time and finish downloading before any others do.
3. Restart SABnzbd, and then don't touch it.
4. The first NZB will start and finish correctly.
5. After the first NZB is finished, all servers will have been appended to <NzbQueue>.__try_list for some reason.
6. As other NZBs reach the delay time, they will not start downloading, as all servers are currently in the try list and will not pass the has_articles_for check.
7. When some other event causes the try list to be reset, the hung downloads will start downloading as normal.

This happens even with only 1 server enabled.

Re: 1.2.0: Propagation delay not working correctly

Posted: February 1st, 2017, 3:37 am
by safihre
Thanks so much for helping us! That's some great debugging there.

We removed the TryList at the NZBQueue-level, could you see if this fixes it?
https://github.com/sabnzbd/sabnzbd/arch ... rylist.zip

Re: 1.2.0: Propagation delay not working correctly

Posted: February 1st, 2017, 10:36 am
by Starshadow
Thanks for the fix! It looks like it solved the issue. I'll keep running this version for a while and see if I notice any problems. Thanks again.