4.2.3: NZBs complete missing a ton of articles, retry does not attempt to grab blocks during the repair

[kyles]

Pretty straight forward and a bit surprising. It seems like retry doesn't actually try to grab articles again, and instead just gets stuck in a forever loop. Readding the exact same nzb allows for (all) the articles to get snatched, and a successful snatch + unpack. Haven't changed anything in Special, feels strongly like a bug based on req_completion_rate ( 100.2 ).

[safihre]

Your report is a bit hard to read, but if I understand correctly: when a download failed because of missing articles and you use Retry, it doesn't succeed. However, if you add that nzb again regularly, it does succeed even though the first time it failed?

Could you enable Debug logging in the Status window and then after you have such a job which failed using Retry buy succeeded when you added it again. Click Show Logging and send me the logs at [email protected]

[kyles]

Your report is a bit hard to read, but if I understand correctly: when a download failed because of missing articles and you use Retry, it doesn't succeed. However, if you add that nzb again regularly, it does succeed even though the first time it failed?

Yeah, I'm adding it before it's propagated out to all networks. Once it's distributed, the nzb completes (which was the manual addition of the same file, 15 minutes later). Apparently this is a very common problem, and is worked around / recommended with sab to add a delay.

Could you enable Debug logging in the Status window and then after you have such a job which failed using Retry buy succeeded when you added it again. Click Show Logging and send me the logs at [email protected]

No problem.

[kyles]

You've got mail, it's immediately reproducible. Hopefully I didn't mess up my client.

[safihre]

Thanks for the extensive testing.
Indeed you are right, the Retry in SABnzbd is not very smart. It doesn't really retry on article-level, only on file level.
Because it already finds partial files from the first try, it keeps on using those and tries to repair those.
We have a feature request for a while to fix this:
https://github.com/sabnzbd/sabnzbd/issues/2735

In any case I would suggest you use the Propagation Delay feature in SABnzbd, which uses the NZB age to dynamically wait for the post to be a certain age:
https://sabnzbd.org/wiki/configuration/4.2/switches

[kyles]

This feels pretty major to me. Imagine if a torrent just refused to complete forever because you ran out of pieces and weren't able to connect to the seeder, and then you had to manually re-add the torrent to get another maybe 60 pieces. I know the common case is aged articles that perhaps will never complete because they're gone, the arbitrary delay doesn't "technically" address the problem because it's still a guess that maybe it will complete now.

Anyway, is there anyway I can help? I know go and c, python is pretty sketchy for me but it sounds like it should never dump into the repair loop if articles are missing and it knows it's a new post (within a week, let's say). I'm pretty sure it would cover 99.98% of valid nzbs, and perhaps run a bit warm for invalid nzbs. The full fix would be to return a broken repair to the pipeline, but that may be beyond my capabilities.

[safihre]

You describe exactly why this is not a high priority: for older NZB's it's never going to be complete anyway, for newer NZB's we have the Propagation Delay.

The reality is a bit harder than it might seem at first:
Currently we just don't keep track of which articles failed.
Additionally, we don't have a way to write only specific parts of the file, so if Article 1 is present but Article 2 isn't, we don't have a way to only download Article 2 and write the resulting data after the data of Article 1.
So lot's of things to change before this can work.

The Retry has been limited like this for many years, but barely causes issues because we have Propagation Delay.