Page 3 of 3
Re: 060b3: one faulty server spoiling the others?
Posted: March 19th, 2011, 7:52 am
by jcfp
shypike wrote:Can you try the attached replacement modules?
Greatly reduces the problem, only seen an error 9 file desciptor thing once, for a server that is supposed to work. Looks like the connection was established but then timed out? Log:
2011-03-19 13:16:33,918::INFO::[downloader:347]
[email protected]:563: Initiating connection
2011-03-19 13:18:33,961::INFO::[downloader:604] Thread
[email protected]:563: timed out
2011-03-19 13:18:33,975::DEBUG::[decoder:92] Decoding
2011-03-19 13:18:34,123::DEBUG::[decoder:92] Decoding
2011-03-19 13:18:34,225::DEBUG::[decoder:92] Decoding
2011-03-19 13:18:34,252::DEBUG::[bpsmeter:179] bps: 10355820.4082
2011-03-19 13:18:34,282::DEBUG::[decoder:92] Decoding
2011-03-19 13:18:34,561::INFO::[newswrapper:222] Failed to connect: (9, 'Bad file descriptor')
[email protected]:563
2011-03-19 13:20:33,963::INFO::[downloader:347]
[email protected]:563: Initiating connection
2011-03-19 13:20:34,371::INFO::[downloader:536] Connecting
[email protected]:563 finished
whereas the nonfunctional test server only results in this kind of error msg:
2011-03-19 13:22:34,953::INFO::[downloader:347]
[email protected]:119: Initiating connection
2011-03-19 13:23:35,020::INFO::[newswrapper:222] Failed to connect: (110, 'timed out')
[email protected]:119
(Neither of those causes the downloading to stop.)
Re: 060b3: one faulty server spoiling the others?
Posted: March 19th, 2011, 8:50 am
by shypike
sander wrote:
BTW / FWIW:
I thought SAB would throttle down it's connection setups to a non-responding server? In the current setup it keeps on connecting to
www.cisco.com each minute.
It will do quite a number of attempts before applying the 10 minute penalty.
Re: 060b3: one faulty server spoiling the others?
Posted: March 19th, 2011, 10:04 am
by shypike
jcfp wrote:
Greatly reduces the problem, only seen an error 9 file desciptor thing once, for a server that is supposed to work. Looks like the connection was established but then timed out?
Possibly these "error 9" events happen in the original code too,
but nobody noticed it because it didn't cause problems.
"error 9" normally means "operation on closed socket".
Which would mean that for some reason the socket was botched
and a simple new attempt would be sufficient.
Normally sockets are re-used even when a connection is dropped.
I can now reproduce the error 9 easily by setting an extremely short timeout (0.1 sec).
But this is a rather artificial situation.
Re: 060b3: one faulty server spoiling the others?
Posted: April 10th, 2011, 5:46 am
by sander
With 060 RC1 I get the same faulty behaviour: download hanging. SAB is working very hard on two bad servers (one unreachable, one out-of-retention), and ignoring the good server eweka.
Should I open a separate thread?
Code: Select all
2011-04-10 12:43:20,157::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:43:20,158::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:43:20,199::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:43:20,202::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:43:50,189::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:43:50,191::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:43:50,192::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:43:50,230::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:43:50,231::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:43:50,232::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:43:51,197::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:43:51,198::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:43:51,231::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:43:51,232::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:21,233::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:21,235::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:21,236::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:21,271::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:21,273::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:21,274::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:22,244::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:22,245::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:22,281::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:22,282::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:52,279::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:52,281::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:52,282::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:52,324::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:52,325::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:52,333::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:53,285::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:53,286::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:44:53,328::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:44:53,341::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:45:23,323::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:45:23,324::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:45:23,325::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:45:23,371::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:45:23,374::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:45:23,379::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:45:24,328::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:45:24,329::INFO::[downloader:351] [email protected]:119: Initiating connection
2011-04-10 12:45:24,371::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
2011-04-10 12:45:24,372::INFO::[newswrapper:223] Failed to connect: (110, '[Errno 111] Connection refused') [email protected]:119
Re: 060b3: one faulty server spoiling the others?
Posted: April 10th, 2011, 5:48 am
by sander
and CTRl-C and then restart SAB does not solve it.
After disabling the unreachable reader.ipv6.xsnews.nl, the download goes OK.
Re: 060b3: one faulty server spoiling the others?
Posted: April 10th, 2011, 6:22 am
by shypike
There is a difference with the previous situation.
The server isn't quite dead, it says "connection refused".
I'll have a look, assuming I can reproduce the same server behaviour.