Recently been getting loads of 'hanging' downloads

[AutomaticCoding]

Example:-


If I restart sabnzbd then they all kick in and finish, but, it's kind of annoying. Also, the log file hasn't just abandoned them, the very last load of lines are about them:-

Code: Select all

013-04-20 00:47:10,826::INFO::[nzbqueue:211] Saving queue
2013-04-20 00:47:10,878::INFO::[__init__:904] Saving data for queue9.sab in /mnt/user/Appdata/sabnzbd/admin/queue9.sab
2013-04-20 00:47:11,299::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:47:30,529::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:47:46,775::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:47:47,845::INFO::[assembler:87] Decoding /mnt/user/Appdata/sabnzbd/Downloads/incomplete/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile.part04.rar yenc
2013-04-20 00:48:02,646::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:48:17,821::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:48:34,041::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:48:38,993::INFO::[assembler:87] Decoding /mnt/user/Appdata/sabnzbd/Downloads/incomplete/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile.part05.rar yenc
2013-04-20 00:48:51,067::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:49:06,431::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:49:21,355::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:49:34,301::INFO::[assembler:87] Decoding /mnt/user/Appdata/sabnzbd/Downloads/incomplete/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile.part06.rar yenc
2013-04-20 00:49:37,857::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:49:57,274::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:50:13,133::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:50:19,465::INFO::[decoder:158] Badly formed yEnc article in [email protected]
2013-04-20 00:50:19,485::INFO::[downloader:378] [email protected]:563: Initiating connection
2013-04-20 00:50:20,531::INFO::[downloader:574] Connecting [email protected]:563 finished
2013-04-20 00:50:26,140::INFO::[assembler:87] Decoding /mnt/user/Appdata/sabnzbd/Downloads/incomplete/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile.part07.rar yenc
2013-04-20 00:50:27,001::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:50:44,799::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:51:00,514::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:51:09,709::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:51:10,174::INFO::[assembler:87] Decoding /mnt/user/Appdata/sabnzbd/Downloads/incomplete/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile.part08.rar yenc
2013-04-20 00:51:19,365::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:51:30,639::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:51:40,936::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:51:49,845::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:51:51,915::INFO::[assembler:87] Decoding /mnt/user/Appdata/sabnzbd/Downloads/incomplete/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile.nfo yenc
2013-04-20 00:51:51,945::INFO::[assembler:87] Decoding /mnt/user/Appdata/sabnzbd/Downloads/incomplete/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile/thisIsntProbablyTheDownloadFileNameButHey,StopReadingMyLogFile.part09.rar yenc
2013-04-20 00:52:01,890::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab
2013-04-20 00:52:12,950::INFO::[__init__:904] Saving data for totals9.sab in /mnt/user/Appdata/sabnzbd/admin/totals9.sab

I CTRL+H'd the top most download in the image's name, so, as you can see, it's still "decoding" said file, it's just taking figuratively forrrevvvvvverrrrrr; in fact, we can calculate how long it's taking. 3 total downloads, however, one would only start processing after it's done, so, in the time it took to download 2 more it would have of had to process, so, considering my download speed is 3MB/s:-
http://www.wolframalpha.com/input/?i=%2 ... in+minutes

So, four minutes it took to 'decode' said rar set before dropping down into processing, I have two computers, identical hardware/OS/applications running and the other one is instant & takes a lot less time to unrar. Any tips on what to do? It does eventually finish, however, it does scale based on the size. I've had a 13GB download that took ~ 6 hours to finish downloading (After it's downloadED) and ~ 4 hours to unrar.

EDIT:- Re-reading this thread it sounds like I'm pissed off at you guys, which, I assure you I'm not, it's obviously my fault (Because not everyone is having this issue), I'm simply asking for any tips.

EDIT:- It got worse, apparently sabnzbd felt like giving up on listening on port 8081 (It stopped listening, it didn't get swapped & decline TCP connections, it stopped listening), so, I said fuck it and deleted my configuration & deleted my binaries. Let's hope this works.

[shypike]

On what kind of hardware is this running and how much memory do you have?

[AutomaticCoding]

On what kind of hardware is this running and how much memory do you have?

This build in fact has more memory than the memory of the other build, which, is the exact same hardware and runs a lot faster.

Here's the output of "free":-

Code: Select all

             total       used       free     shared    buffers     cached
Mem:      16605268   13502960    3102308          0      90072   11594072
-/+ buffers/cache:    1818816   14786452
Swap:            0          0          0

And, in fact, right after this post I'm going to make a swap-space partition; 26.58% of the CPU in use (Although, multicore utilization isn't perfect I have to assume, so, due to that the real total is probably skewed). As for the hardware itself:-
Drive sabnzbd is running off (And downloading to):- http://www.newegg.com/Product/Product.a ... 6820147185
PSU (irrelevant, but I have the newegg order open anyway so):- http://www.newegg.com/Product/Product.a ... 6817703027
MOBO (Same as PSU, but, it's a great MOBO. IPMI = 10/10):- http://www.newegg.com/Product/Product.a ... 6813182253
CPU:- http://www.newegg.com/Product/Product.a ... 6819117286
RAM (Somewhat important, I think):- http://www.newegg.com/Product/Product.a ... 6820239370

[shypike]

I was after the CPU, amount of memory and the OS.
SABnzbd is both memory and CPU hungry.
It's also single core.

[AutomaticCoding]

I was after the CPU, amount of memory and the OS.
SABnzbd is both memory and CPU hungry.
It's also single core.

I gave you the CPU (Model & usage), the RAM (Model & usage) and the OS is unraid (Linux) running directly from the sabnzbd github page. As for the CPU core(s), really? Oh well, mistaken computer CPU then, I'll take your word for it, not in the mood to google around.

[shypike]

I'm sorry about the confusion.
I wanted to know the amount of memory, I cannot determine it from what you wrote before.
There's something in the system log, but I don't know the units.
SABnzbd is known to be troublesome when using anything less than 256M.
Having a large queue and especially having very large items makes things worse.
The CPU is probably not a factor.

I don't support UnRaid, primarily because it refuses to be virtualized under VMWare.
I cannot have a physical box for each and every OS.

[xeddog]

This sounds exactly like a couple of problem downloads I have had recently. I am running Ubuntu linux 12.04 on a Intel I7 with 6GB ram. When I look at my system monitor resources, all 8 of my cpu's are almost flatlined, and memory usage is extremely low at less than 20%. Here is what I see -

In the Queue list I see TWO entries for the same "hung" download (at one point there was even three of them). Each shows a full progress bar, time left is 0:00:00, age is 7 hrs, and size is 793MB

In the History section I see (showname) - "Repair: starting repair" in green, size 87MB, and status is Verifying. Interestingly, the red "X" to terminate the process is not there so there isn't really a good way to terminate this download.

The graph at the bottom of the screen that shows the data downloaded is completely flat at zero.

Hope this helps,

X

[shypike]

Sounds like the repair process (done by the par2 program) is really slow (can happen) or is hung.
We have no control over the quality of the par2 program.
You can kill par2 with a task manager and subsequently SABnzbd will list the job as failed.
There is a brief moment when a job appears in both the queue and the history,
you may be seeing a display anomaly.

[xeddog]

After two hours nothing had changed. I looked at my system monitor for all processes and there was nothing there for par2 processing. I shutdown SABnzbd and restarted, but all entries in both Queue and History came back just as before. Then I tried just going to my system monitor and killed sabnzbdplus. A restart was only different in that there was only 1 entry in the Queue listeng, but other than that everything else was identical. So I tried a shutdown and reboot of my computer and when it finally came back up I was back where I started with two entries in the Queue section, and the same one entry in the history section.

I may have ruined any chance of diagnosing further, but I shutdown sabnzbd, opened a terminal, and ran sabnzbdplus -c. that got rid of it. I will try to download that same nzb again and see what happens.

X

Edit: I should add that I am running two copies of sabnzbdplus on two different machines. Both copies had a similar problem but my "production" machine eventually completed after a few hours. Here are the differences:

Machine 1 I call my production machine and is pretty much dedicated to running CouchPotatoServer, SickBeard, and SABnzbdplus. Ubuntu 32-bit linux 12.04 using xfce desktop. The processor is an Intel I7 and the machine only has 4GB ram. The version of SABnzbdplus is shown as 0.7.0Beta3. However, on this machine I am using a professional indexer that I would rather not name.

Machine 2 is my "main" machine because I do everything else on it. This one is running Ubuntu 64-bit linux 12.04 using the gnome classic (gnome-session-fallback) for desktop. This is also an I7 processor (but faster), and has 6GB ram. SABnzbdplus shows version 0.6.15. But on this machine I have installed Newznab+ to use for my indexer.


Edit2: I just tried the same nzb again and this time it just failed saying "1861 articles were missing". I was able to remove it by clicking on the red "X" on the right side of the history entry.

[jcfp]

Start by updating sab. http://forums.sabnzbd.org/viewtopic.php?f=16&t=387

[AutomaticCoding]

First of all, sorry for not replying in so long.

I'm sorry about the confusion.
I wanted to know the amount of memory, I cannot determine it from what you wrote before.
There's something in the system log, but I don't know the units.
SABnzbd is known to be troublesome when using anything less than 256M.
Having a large queue and especially having very large items makes things worse.
The CPU is probably not a factor.

I don't support UnRaid, primarily because it refuses to be virtualized under VMWare.
I cannot have a physical box for each and every OS.

16GB total, 13.5GB used, 3GB available. Those numbers are off by a bit (Due to the difference between KiB and KB) but, yeah. As for the queue, it's only around 15 items long, each ~ 2-8GB in size.

This sounds exactly like a couple of problem downloads I have had recently. I am running Ubuntu linux 12.04 on a Intel I7 with 6GB ram. When I look at my system monitor resources, all 8 of my cpu's are almost flatlined, and memory usage is extremely low at less than 20%. Here is what I see -

In the Queue list I see TWO entries for the same "hung" download (at one point there was even three of them). Each shows a full progress bar, time left is 0:00:00, age is 7 hrs, and size is 793MB

In the History section I see (showname) - "Repair: starting repair" in green, size 87MB, and status is Verifying. Interestingly, the red "X" to terminate the process is not there so there isn't really a good way to terminate this download.

The graph at the bottom of the screen that shows the data downloaded is completely flat at zero.

Hope this helps,

X

I don't see anything in history until it finally feels like pushing over to the history, then everything works as normal.

Sounds like the repair process (done by the par2 program) is really slow (can happen) or is hung.
We have no control over the quality of the par2 program.
You can kill par2 with a task manager and subsequently SABnzbd will list the job as failed.
There is a brief moment when a job appears in both the queue and the history,
you may be seeing a display anomaly.

Refer to the above, par2 works fine & repairs at a reasonable rate.

Start by updating sab. http://forums.sabnzbd.org/viewtopic.php?f=16&t=387

Although this wasn't directed at me, I'm not running on Ubuntu & I'm running directly from the github source code.

[xeddog]

I'm not sure if I spoke too soon or not. On my "production" machine it said the download was complete, but when I expanded the entry I am not so sure. This is what I see:

Code: Select all

- 	(name of show).720p.hdtv.x264-2hd 		875 MB 	Completed      X	

Completed at 08:47:37 on Monday 22 April 2013 

Download:

Downloaded in 5 minutes 31 seconds at an average of 2.6 MB/s

Source:

(name of show).720p.hdtv.x264-2hd.nzb

Unpack:

[136197] Unpacked 1 files/folders in 16 seconds

Repair:

[136197] Repair failed, not enough repair blocks (455 short)
[sample-136197] Repair failed, not enough repair blocks (40 short)
Trying SFV verification
Verified successfully using SFV files

Category:

*

Path:

/home/twoblues/Videos/SABnzbd+/complete/(name of show).720p.hdtv.x264-2hd

So it says the download completed (that's good), it seems to have unpacked ok (good), but also repair failed because of not enough repair blocks (bad), SFV verification succeeded (that's good), and the post processing script sabtoSickBeard.py was not executed so the episode was not moved to it's final resting place nor properly renamed (bad). It is, however, in the sab completed folder (good??). I am also wondering why the Category said "*" instead of "tv" like all of the others that come from SickBeard (bad???).

I guess I will have to manually move it and and then see if it is good by watching it tonight or tomorrow night.

X