"Check before download"- methods?

Questions and bug reports for Beta releases should be posted here.
Forum rules
Help us help you:
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
semel
Jr. Member
Jr. Member
Posts: 58
Joined: January 25th, 2009, 3:45 am

"Check before download"- methods?

Post by semel »

Hi.

Thanx for a very handy and long awaited feature=)

However i have a few questions regarding it

1) What method does sabnzbd use? HEAD or STAT checks?(newsbin pro uses HEAD, nzb download checker uses STAT, nzb completion checker allows users to choose one of them)

STAT is much faster BUT on certain providers it doesnt work properly(readnews resellers-blocknews\usenetnnow etc

HEAD is slower but more accurate and all providers afaik support it


2) Does this check utilize backup servers?


3)Will we have a chance to set what method should be used manually? Blocknews is quite popular these days as a backup provider and if it is STAT method that is used..well..you get the picture.


Thanx
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: "Check before download"- methods?

Post by shypike »

STAT is used instead of HEADER.
The idea is to waste as little bandwidth as possible.
The pre-check is meant for people with download quotas or low bandwidths.
Using HEADER would mean more bytes to download.
Personally, I find STAT already slow.

Pre-check is a dry-run, so all servers are used in the normal pattern.

As the usefulness of pre-check is not certain, we don't go overboard offering options for it.
Your suggestion is noted.
You might suggest to blocknews to get their act together and conform to Usenet standards :)
User avatar
sander
Release Testers
Release Testers
Posts: 9113
Joined: January 22nd, 2008, 2:22 pm

Re: "Check before download"- methods?

Post by sander »

semel wrote: 3)Will we have a chance to set what method should be used manually? Blocknews is quite popular these days as a backup provider and if it is STAT method that is used..well..you get the picture.


Thanx
If you run SABnzbd from source, technically you could replace STAT by HEAD yourself: in sabnzbd/newswrapper.py do this:

Code: Select all

            # command = 'STAT <%s>\r\n' % (self.article.article)
            command = 'HEAD <%s>\r\n' % (self.article.article)

HTH
semel
Jr. Member
Jr. Member
Posts: 58
Joined: January 25th, 2009, 3:45 am

Re: "Check before download"- methods?

Post by semel »

The pre-check is meant for people with download quotas or low bandwidths.
to be honest i think it will help mostly users who deal with all those dmca takedowns (ggianews\supernews\newshosting etc) and AW's bad range(~200days affected) rather than those with quotas.(im talking here in numbers overall =))
Your suggestion is noted.
thanx
You might suggest to blocknews to get their act together and conform to Usenet standards
if it was that easy=)
If you run SABnzbd from source, technically you could replace STAT by HEAD yourself
i run it on win 7 64 and,alas, not familiar with compiling on it at all
User avatar
sander
Release Testers
Release Testers
Posts: 9113
Joined: January 22nd, 2008, 2:22 pm

Re: "Check before download"- methods?

Post by sander »

semel wrote:
STAT is much faster BUT on certain providers it doesnt work properly(readnews resellers-blocknews\usenetnnow etc

HEAD is slower but more accurate and all providers afaik support it
BTW: What is the result of a STAT on those non-compliant newsservers? A "500 What?", or something else?
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: "Check before download"- methods?

Post by shypike »

I'll see if something can be done for the next 0.7.0 Beta.
semel
Jr. Member
Jr. Member
Posts: 58
Joined: January 25th, 2009, 3:45 am

Re: "Check before download"- methods?

Post by semel »

sander wrote: BTW: What is the result of a STAT on those non-compliant newsservers? A "500 What?", or something else?
hmm it seems they've "fixed" this, sort of. i remember getting article not found via STAT check on blocknews when in reality they were available.

ive given it another try and here what i found

On new files STAT checks work fine on blocknews.

But on older files it is somewhat troubling

1) very slow(but blocknews is known for its slower speed on older stuff so we can do nothing about it)
2)some articles dont get processed with nzb download checker showing "501 newsgroup" message

i suspect it has something to do with time-outs during the checking process on blocknews 500 days file which is REALLY slow

head check is slow as well but it doesnt give this message and all articles are checked on the same nzb


In other words its not a good idea to check old files on any readnews reseller and its totally their fault

Anyways
I'll see if something can be done for the next 0.7.0 Beta.
i suppose it would still be nice to be able to change methods. Perhaps through INI if it is easier to do

Cheers

PS Btw STAT is not that slow as you say it is.

AW, 700 days old, 8.8Gb file - appr. 2 min
Though it depends on completion- if there are missing articles, a lot, it will be slow=)
Bren0man
Newbie
Newbie
Posts: 2
Joined: November 3rd, 2014, 5:43 am

Re: "Check before download"- methods?

Post by Bren0man »

Hey all,

I know I'm necroing a thread to the max but I just wanted to say a big thank you for implementing this feature and for creating this thread that I stumbled across via google. I've only recently resumed usenet activities after a hiatus of quite a few years. Since returning back I've had no end of incomplete downloads, frustration and wasted quota from troubleshooting.

After much googling and trial and error, I've come to realise that DMCAs are definitely a thing in 2014 (5 years ago I rarely encountered an incomplete file) and that the 'check before/abort incomplete download' options are my saviour (that and my complementary Astraweb service that my ISP provides is riddled with incomplete files which I assume are caused by DMCAs).

There's probably a good reason for this but would it not make sense for the pre-download check to be operational by default? It sure would have wonderful for this n00b lol.

Anywho, thanks again. If I had not come across this post, I'd still be searching for a solution, inching ever closer to snapping my keyboard over my own head and being forced to rely solely on torrents.

Cheers,
Bren0
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: "Check before download"- methods?

Post by shypike »

Bren0man wrote: There's probably a good reason for this but would it not make sense for the pre-download check to be operational by default? It sure would have wonderful for this n00b lol.
The check is slow with many Usenet providers. Also it isn't always reliable (also depending on provider).
We think the "Abort if" options are better.
The choice is yours, but we enable neither option by default.
Bren0man
Newbie
Newbie
Posts: 2
Joined: November 3rd, 2014, 5:43 am

Re: "Check before download"- methods?

Post by Bren0man »

Fair enough. Thanks again!
Post Reply