Downloader timed out

[msdv]

Hi all,

I have significant problems downloading from my ISP newsserver since a few days with latest 0.4.11 sabnzbd.
I could download previously without any problem. I have already asked the news-admin if any changes were introduced to the newsserver.
However I can download from this newsserver with other newsreaders (like newsbin).

In the log I see:

2009-07-06 11:38:34,706::WARNING::[downloader] Thread [email protected]:119: timed out
2009-07-06 11:38:39,713::WARNING::[downloader] Thread [email protected]:119: timed out

and more detailed:
2009-07-06 11:15:47,460::INFO::[downloader] [email protected]:119: Initiating connection
2009-07-06 11:15:47,479::DEBUG::[trylist] Appending binary-news.myprovider.de:119 to .__try_list
2009-07-06 11:15:47,480::INFO::[downloader] [email protected]:119: Initiating connection
2009-07-06 11:15:47,502::DEBUG::[downloader] [email protected]:119 last message -> 200  NNTPSwitch-0.12, 27795 groups available, posting allowed, slot 7, connections 3
2009-07-06 11:15:47,518::DEBUG::[downloader] [email protected]:119 last message -> 381 More Authentication Required
2009-07-06 11:15:47,527::DEBUG::[downloader] [email protected]:119 last message -> 200  NNTPSwitch-0.12, 27795 groups available, posting allowed, slot 12, connections 4
2009-07-06 11:15:47,543::DEBUG::[downloader] [email protected]:119 last message -> 381 More Authentication Required
2009-07-06 11:15:47,545::DEBUG::[downloader] [email protected]:119 last message -> 281 Authentication Accepted
2009-07-06 11:15:47,545::INFO::[downloader] Connecting [email protected]:119 finished
2009-07-06 11:15:47,546::INFO::[downloader] Thread [email protected]:119: fetching xxxx.89676.1@xxx
2009-07-06 11:15:47,571::DEBUG::[downloader] [email protected]:119 last message -> 281 Authentication Accepted
2009-07-06 11:15:47,572::INFO::[downloader] Connecting [email protected]:119 finished
2009-07-06 11:15:47,573::INFO::[downloader] Thread [email protected]:119: fetching xxxx
2009-07-06 11:15:52,579::DEBUG::[downloader] bps: 58.8441009874
2009-07-06 11:15:57,586::DEBUG::[downloader] bps: 23.6217509917
2009-07-06 11:16:02,593::DEBUG::[downloader] bps: 9.48205725675
2009-07-06 11:16:07,600::DEBUG::[downloader] bps: 3.80644549093
2009-07-06 11:16:12,613::DEBUG::[downloader] bps: 1.52629782215
2009-07-06 11:16:17,637::DEBUG::[downloader] bps: 0.61094485373
2009-07-06 11:16:22,644::DEBUG::[downloader] bps: 0.245256074634
2009-07-06 11:16:27,650::DEBUG::[downloader] bps: 0.0984576871823
2009-07-06 11:16:32,657::DEBUG::[downloader] bps: 0.0395226558114
2009-07-06 11:16:37,664::DEBUG::[downloader] bps: 0.0158651719744
2009-07-06 11:17:47,745::WARNING::[downloader] Thread [email protected]:119: timed out


Any idea what that means and how to better debug?
I can perfectly download from a second server.

Thx,
M.

[shypike]

In general it's impossible for us  to diagnose these kind of problems.
It looks like the login is accepted by the server, SABnzbd asks for an article,
but there is no reaction from the server.
What is the server timeout in the the Config->Server setup?
Which OS do you use?
After a couple of minutes, does downloading start anyway?
(I have seen the latter behavior on some Linux boxes in specific conditions).

The strange thing is that we didn't change any code having to do with
server communication since about 0.4.2.

You could sign-up for the "Release Test Board" (in the Beta board) and get a
Alpha copy of 0.5.0 (which has a redesigned communication module).

[msdv]

What is the server timeout in the the Config->Server setup?

120

Which OS do you use?

Ubuntu 9.04

After a couple of minutes, does downloading start anyway?
(I have seen the latter behavior on some Linux boxes in specific conditions).

Unfortunately not.

I suspect that the problem is either my connection (I changed my Internet Router) or the newsserver and not sabnzbd since it has already worked before.
Is there anything that I can to to increase logging or debugging?

Thx,
M.

[sander]

For hard-core debugging / logging, you could use ethereal a.k.a. wireshark to see which communication with 'binary-news.myprovider.de' is going on: you can see if the authentication really is successful, and what happens afterwards: does SAB send it's commands, is anything coming back, etc ...

And you could add another (free) newsserver to check whether that works...

[shypike]

I'll still advise you to use pre-release 0.5.0.
Shall I sign you up?

[msdv]

I'll still advise you to use pre-release 0.5.0.

yes.

[shypike]

BTW: you can check if the server is reachable.
Open a telnet session from a shell:

Code: Select all

 telnet binary-news.myprovider.de 119

You should get the server signon.
If not then it's definitely a problem of your server and./or router.

[msdv]

Server is reachable - and I can download with windows software (newsbin) on another box.
Only sabnzbd doesn't do anything.

Update:
To have absolutely the same environment, I have now taken the following approach:
I have installed on a virtual machine (running Vista) inside my Ubuntu box
a) newsbin latest version
b) sabnzbd lastest 0.5 alpha version

Result:
a) newsbin is downloading from my providers newsserver
b) sabnzbd is still throwing "timeout".

Taking this into account, is seems that is can't have anything to do with my router.
I also looked at lowlevel "wireshark" logging of network traffic. Definitely sabnzbd is logging into the newsserver successfully.
Except of a "RST (reset)" i don't see anything suspicious.

M.

[msdv]

Server is reachable - and I can download with windows software (newsbin) on another box.
Only sabnzbd doesn't do anything.

Update:
To have absolutely the same environment, I have now taken the following approach:
I have installed on a virtual machine (running Vista) inside my Ubuntu box
a) newsbin latest version
b) sabnzbd lastest 0.5 alpha version

Result:
a) newsbin is downloading from my providers newsserver
b) sabnzbd is still throwing "timeout".

Taking this into account, is seems that is can't have anything to do with my router.
I also looked at lowlevel "wireshark" logging of network traffic. Definitely sabnzbd is logging into the newsserver successfully.
Except of a "RST (reset)" i don't see anything suspicious.

As a result of my testing I now added a IPv6 newsserver as a second test - and also on this "eweka" server I receive the timeouts.
The only server that works flawlessly is a 3rd secure (SSl-based) server.
I am totally confused now.

M.

[sander]

Server is reachable - and I can download with windows software (newsbin) on another box.
Only sabnzbd doesn't do anything.

Update:
To have absolutely the same environment, I have now taken the following approach:
I have installed on a virtual machine (running Vista) inside my Ubuntu box
a) newsbin latest version
b) sabnzbd lastest 0.5 alpha version

Result:
a) newsbin is downloading from my providers newsserver
b) sabnzbd is still throwing "timeout".

Taking this into account, is seems that is can't have anything to do with my router.
I also looked at lowlevel "wireshark" logging of network traffic. Definitely sabnzbd is logging into the newsserver successfully.
Except of a "RST (reset)" i don't see anything suspicious.

M.

Maybe there is some confusion between SABnzbd and your newsserver. To verify that, you could add an extra server:
1) you could try the freely accessible test server stresstest.cheapernews.com. Hopefully it's still up; it was one week ago.
2) on Ubuntu: first "sudo apt-get install miredo", then fill out news.ipv6.eweka.nl as an extra server

If at least one of those extra servers work, it's indeed some confusion between SAB and your newwserver. You could then post the relevant wireshark logging to see who's waiting for who.

[msdv]

Further debugging:
I can successfully download from another (paid) pure NNTP server with sabnzbd 0.4.11!

I now have installed newsbin under wine and on the same machine download with my providers newsserver works (but only if I mark "send group command" - Otherwise also not much traffic). I think I can conclude from this that network is also OK.

Now to my observation: If I read sabnzbds logs within wireshark correctly - the "group" command is never send.
Although I have "send_group = 1" in my sabnzbd.ini

So question is: is the "send_group" switch in stable versions not working anymore?

[sander]

What happens if you (re)move sabnzbd.ini, start with an empty sabnzbd.ini, only fill out your newsserver with 1 connection, and then try to download again.

Do NOT copy info from the old sabnzbd.ini
Make sure no other newsgroup program is running or active.

Question: have you always used SAB with the send-group command? Or did it work without that option?

[msdv]

I started from scratch with the sabnzbd-0.5 version.
Here is log from wireshark:

Code: Select all

200  NNTPSwitch-0.12, 27795 groups available, posting allowed, slot 61, connections 3
authinfo xxxxx
381 More Authentication Required
authinfo pass xxxxx
281 Authentication Accepted
BODY <[email protected]>
QUIT

No send group command!

The newsserver of my ISP traditionally has needed the "send group" command. Recently they have send an announcement that it's not necessary anymore - doesn't seem to be the case.

M.

[shypike]

We have an option for that (also in 0.4.11).
It's in Config->Switches and is called Send Group.

This is the first time I heard of a server that actually requires the Group command.

[msdv]

I have that switch activated - but it doesn't do anything