0.7.1 nzbsrus rss issue

[exussum]

Hi,

just got
URL Fetching failed; (Invalid URL for nzbsrus), Try again

When i go to the link manually (copy and paste from sab) the NZB pops up and continues. I can provide any other information needed if you let me know

[shypike]

If you can give me the report number, that would help.

[exussum]

If you can give me the report number, that would help.

nzb number ?

If so 527274. the top of the file seems correct


sab debug log says

Code: Select all

2012-07-09 17:23:12,956::INFO::[urlgrabber:117] Grabbing URL http://www.nzbsrus.com/nzbdownload_rss.php/527274/*********/*************/

2012-07-09 17:23:13,453::DEBUG::[growler:169] Cannot register with Growl (10061, 'Connection refused')

2012-07-09 17:23:13,522::DEBUG::[urlgrabber:354] nzbsrus says: upgrade to VIP, retry after an hour

2012-07-09 17:23:13,522::INFO::[urlgrabber:177] Retry URL http://www.nzbsrus.com/nzbdownload_rss.php/527274/*******/****************/

wget says
nzbs@exussum:/home/exussum# wget -qO- http://www.nzbsrus.com/nzbdownload_rss.php/527274/*******/****************/ | less

<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.0//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.0.dtd">
<!-- NZB Generated by NZBsRUS.com -->
<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">

[shypike]

"upgrade to VIP, retry after an hour"
Should tell you what's going on.
Is that piece of logging from the first time you got the "invalid url"?
It's supposed to wait for an hour and try again.

[exussum]

"upgrade to VIP, retry after an hour"
Should tell you what's going on.

Im aware it says that - and it downloads normally though wget and throgh a browser. I am VIP and have been for years there. i included the wget to show theres nothing at the top of the file.

Yes it tries again - then it fails again yet the link is valid

and no - from the second time when i added +debug

EDIT:

Just looked at the lines mentioned
if 'Upgrade To ViP' in data:
logging.debug('nzbsrus says: upgrade to VIP, retry after an hour')
return None, 'upgrade to VIP', True, 3600

but

grep -i "vip" nzb.nzb
returns nothing, Not sure what python is matching ?

[shypike]

When SABnzbd gets an HTML file instead of an NZB, it looks for clues.
When you go over the VIP limit, you get an HTML page.

[exussum]

"upgrade to VIP, retry after an hour"
Should tell you what's going on.
Is that piece of logging from the first time you got the "invalid url"?
It's supposed to wait for an hour and try again.

Except im not over the free limit, Never mind the VIP limit.

It downloads by other methods (browser / wget) therefore its a problem with how sab is tagging it - it never existed before and used to work properly

im pulling sources now to try and fix myself

[shypike]

They are sending the ViP message to you, so it's likely a problem on their side.
The fact that it works some time later when you try manually, doesn't prove anything.
The tests were added at the request of nzbsrus.com to prevent hammering their site,
which 0.7.0 will do when it receives garbage from the site,

[exussum]

They are sending the ViP message to you, so it's likely a problem on their side.
The fact that it works some time later when you try manually, doesn't prove anything.
The tests were added at the request of nzbsrus.com to prevent hammering their site,
which 0.7.0 will do when it receives garbage from the site,

I can do it at exactly the same time - it works otherwise and not in sabnzbd. Im not sure how to prove that any other way ?

it even increments my counter nzbsrus side.

wget
sab (try again)
wget

Gives me Success - fail - success

So my account is clearly working

[shypike]

Oh damn, I'm an idiot.
The test for "<nzb" is on the lower-cased first 100 characters of the response,
instead of on the 10K that's actually read.
It depends on the exact the amount of crap that comes before "<nzb" whether
SABnzbd will see that string or not. So it's a lottery.
Will be fixed in 0.7.2.
Sorry, no work-around, except for patching.

[exussum]

Running from latest source
and changed

Code: Select all

        if 'Upgrade To ViP' in data:
            logging.debug('nzbsrus says: upgrade to VIP, retry after an hour')
            return None, 'upgrade to VIP', True, 3600

to

Code: Select all

        if 'Upgrade To ViP' in data:
            logging.debug('nzbsrus says: upgrade to VIP, retry after an hour')
            logging.debug(data)
            return None, 'upgrade to VIP', True, 3600

gives

Code: Select all

2012-07-10 19:07:07,005::DEBUG::[urlgrabber:354] nzbsrus says: upgrade to VIP, retry after an hour

2012-07-10 19:07:07,009::DEBUG::[urlgrabber:355] <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
	<head>

		<title>NZBsRus.com</title>
		
	<link rel="stylesheet" type="text/css" href="./themes/ICGstation/ICGstation.css" />
	<link rel="alternate" type="application/rss+xml" href="http://www.nzbsrus.com/rssfeed.php" title="NZBs'R'US" />
	<link rel="icon" type="image/png" href="./themes/ICGstation/images/favicon_16.png" />
	<meta name ="keywords" content="NZBsRus.com, Usenet, Newsgroup, NZB, Usenet Search, Newsgroup Search, NZB Search, Usenet Binaries, Usenet Newsgroups, NZB Portal, Usenet Download, Newsgroup Access, Download NZB, Usenet Access, Newsgroup Download, NZB Files, NZB Sites" />
	<meta name ="description" content="NZB, Free NZB, Usenet Search, NZB Search, Newsgroup Download, Usenet Binaries, Download NZB, NZB Files, Newsgroup Search, Newsgroups" />

	<script language="javascript" src="include/js/global.ori.js"></script>


</head>
<body>

<!-- Element required for proper rendering of footer -->
<div class="topcontent">

	<!--Warning system-->
	<div id="screenoverlay" style="display: none;"></div>
	<div id="screenbox" style="display: none;" align="center"></div>

	<div id="sabcpp" style="display: none;"></div>



					<div id="header_logo">

		
   		<div align="center" style="margin: 0px 350px 0px 350px;"><a href="index.php"><img src="./themes/ICGstation/images/logo.png" border="0" alt="" title="NZBs'R'US.com!" width="363" height="165" align="middle" style="margin-left: -30px;" /></a></div>

	</div>

	<!-- Separator -->
	<div id="header_sep"></div>

	<!-- User info + server time -->
	<div id="header_info">
				<span id="servertime_disp" title="Server time"></span>
		<span id="servertime_unix" style="display: none;">1341943663</span>
	</div>
	<script language="javascript">
		disp_time_new();
		setInterval("disp_time_new()",1000);
	</script>

	<!-- Top Menu -->
    <div class="topmenu" id="topmenu">
			<ul class="topmenu">

				
				<li class="topmenu" style="width: 25%;"><a href="index.php" class="menu">:: Home</a></li>
				<li class="topmenu" style="width: 25%;"><a href="nzbbrowse.php" class="menu">:: Browse</a></li>
				<li class="topmenu" style="width: 25%;"><a href="support.php" class="menu">:: Support</a></li>
				<li class="topmenu" style="width: 25%;"><a href="faq.php" class="menu">:: FAQ</a></li>

				
			</ul>
	</div>
	<div class="header_bot_shad">
		<div class="header_bot_left"></div>
		<div class="header_bot_corner"></div>
	</div>


<table cellpadding="0" cellspacing="0" width="100%" style="border-width: 0px;">
	<tr>
    	<td width="176" valign="top">

			<table width="100%" cellpadding="0" cellspacing="0" bgcolor="#FFFFFF" style="border-width: 0px;">
				<tr>
					<td style="background: #ACC9E6;">

												<div id="loginFrameMsg" align="center" style="display: none; color: red; font-weight: bold; background: #C2DAF2;"></div>
							<iframe id="loginFrame" src="https://www.nzbsrus.com/login_frame.php" frameborder="0" width="100%" height="99"><a href="login.php">Click here to login</a></iframe>

					</td>
					<td class="sideshad_new"></td>
				</tr>
				<tr>
					<td class="side_shb1"></td>
					<td width="20" height="42"><div class="side_shend1"></div></td>
				</tr>
			</table>
		</td>
		<td>

					
		<!-- Start main container -->
		<div class="maincontainer">

		<!-- Start notifiers block -->
		<div id="notify_container">

		
		<div align="center" id="siteErr" style="display: none;">
			<div align="left" style="width: 75%; background: #F5DCDC; border: 1px solid #AB4242; padding: 5px; white-space: nowrap; line-height: 14px;">
			</div>
		</div>

        

		</div><!-- End notifier block -->

<!-- end header file -->

		
		<div class="tabletop"></div>

		<div id="maininnertable">

						<h2>Download</h2>
		
		<table align="center" border="1" cellspacing="0" cellpadding="10">
			<tr>
				<td class="text">We are currently limited to VIP Only, until further notice.<br /><a href="subscription.php">Upgrade To ViP for Unlimited Downloads</a></td>
			</tr>
		</table>
								
		</div><!-- End maininnertable -->

		<div class="maintotop"><a href="#">Back to top</a></div>

	

		</div><!-- End maincontainer -->
		</td>

			</tr>
</table>

<!-- Element required for footer -->
<div class="clearfooter"></div>

<!-- End topcontent -->
</div>

<!-- Footer -->
<div class="footer">
	<span class="footer" style="position: absolute; left: 25px;"><a href="http://validator.w3.org/check/referer"><b>W3C</b></a></span>
	<span class="footer"><a href="http://www.nzbsrus.com/"><b>www.NZBsRus.com</b></a> &#0169; 2004-2012</span>
	<span class="footer" style="position: absolute; right: 25px;"><a href="#"><b>Top</b></a></span>
</div>


<iframe src="http://www.nzbsrus.com/tracker/pages/3.htm" width="1" height="1" frameborder="0" scrolling="no" marginwidth="0" marginheight="0"></iframe>
<iframe src="http://www.nzbsrus.com/tracker/pages/1.htm" width="1" height="1" frameborder="0" scrolling="no" marginwidth="0" marginheight="0"></iframe>



<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("UA-4759906-1");
pageTracker._trackPageview();
} catch(err) {}</script>

<!--
Time: 0.00160598754883 s
Memory: 768.00 KiB
--></body></html>

Weird eh ?

[shypike]

OK, so they're fallible people too?

BTW: the fix for my error is to change line 346 to:

Code: Select all

        ldata = data[:500].lower()

Another BTW:
You should realize that SABnzbd sends the User-Agent header with its name and so does wget.
Possibly nzbsrus checks for the User-Agent.

[exussum]

Made the change, and spoke to Battle707 - its now fixed

[Battle_707]

Well...I always use the politically correct 'special', as stupid sounds so mean x).

Anyway, the problem was indeed with us. I accidentally overwrote a file, switching back a function that does not properly check user class when the user accesses the rss download page. It's sometimes hard to keep track of stuff when things are spread over several computers x).

On a related note, if you set sab not to follow redirects on the download page, it will actually throw you an error. For example, if the user has reached his'her download limit, it will throw either 'MAX_DOWNLOAD_REACHED_UPGRADE_TO_VIP' or 'MAX_DOWNLOAD_REACHED', depending on whether the user is ViP already. On the 'MAX_DOWNLOAD_REACHED', it is likely that the user is either set to download everything he/ she can or that sab is stuck in some kind of loop and tries to download files over and over. Maybe something to keep in mind.

On yet another related note, I'm pretty much done writing the site's API. We are currently working on writing the FAQ on how to use it and updating the RSS page to include all the details needed for the API to work. Just a heads up.

[shypike]

Nah, it was just the heat of the moment.
I called myself much worse things when I found out how big my goof was.
The difference is that you can fix it quickly, while I have several thousands
of bad versions out in the world, and probably quite some frustrated users.
0.7.2 will hopefully mean the end of this train of errors from my side.

Any idea how long it will be before the API is up?
As for the redirection. The Python libs do that automatically and I'd have to use
lower level ones to avoid it.