Decoder failure: Out of memory

[MiG]

I keep receiving this message on two similar 720p jobs. Strange thing is that if I download a larger 1080p, also similar job, it easily goes past the number of articles and MBs where the 720p ones would get stuck, and then finishes with flying colours.

Cache limit was set to 256MB, I've tried increasing it to 450MB and lowering to 128MB, both to no avail.
This coincides with me switching indexers. I've tried pulling the same jobs from different indexers, but the decoder error keeps popping up, so it's either the jobs somehow, or I found a bug.

[sander]

On what platform is SAbnzbd running?

Can you post the logging & screenshot?

Can you share the offending NZB in some way?

[safihre]

Out of memory error's are really low level, indicating that your system can't give us the memory we want to decode stuff.
In this case it would only help to reduce the download speed so less memory is needed, because we can't control the particular memory consumption of the decoding. The settings for Cache limit only apply to the steps after the decoding, not before or during. That is dependend on your CPU speed.

How is the memory usage of the device? Could also be corrupt memory.

[MiG]

On what platform is SAbnzbd running?

Win7 x64.

Can you post the logging & screenshot?

Code: Select all

2019-04-17 18:09:56,378::INFO::[downloader:704] Connecting [email protected] finished
2019-04-17 18:09:56,400::INFO::[downloader:704] Connecting [email protected] finished
2019-04-17 18:09:56,405::INFO::[downloader:704] Connecting [email protected] finished
2019-04-17 18:09:56,444::WARNING::[decoder:139] Decoder failure: Out of memory
2019-04-17 18:09:56,446::INFO::[decoder:141] Decoder-Queue: 0, Cache: 0, 0, 134217728
2019-04-17 18:09:56,446::INFO::[decoder:142] Traceback: 
Traceback (most recent call last):
  File "sabnzbd\decoder.pyo", line 124, in run
  File "sabnzbd\decoder.pyo", line 221, in decode
MemoryError
2019-04-17 18:09:56,446::INFO::[downloader:279] Pausing
2019-04-17 18:09:56,451::INFO::[downloader:783] Thread [email protected]: forcing disconnect
2019-04-17 18:09:56,559::INFO::[downloader:783] Thread [email protected]: forcing disconnect
2019-04-17 18:09:56,660::INFO::[downloader:783] Thread [email protected]: forcing disconnect

What kind of screenshot do you need?

Can you share the offending NZB in some way?

Sure.

[sander]

Win7 ... how much RAM?

[MiG]

Out of memory error's are really low level, indicating that your system can't give us the memory we want to decode stuff.
In this case it would only help to reduce the download speed so less memory is needed, because we can't control the particular memory consumption of the decoding. The settings for Cache limit only apply to the steps after the decoding, not before or during. That is dependend on your CPU speed.

How is the memory usage of the device? Could also be corrupt memory.

Yeah, I read the wiki article
First thing I did was check memory usage, which seemed quite normal (at least 8GB of physical memory available, 16GB total).
I've also been pulling in the maximum my provider gives me for a long time now with this machine (22-26MByte/s), so it's a bit puzzling why it would be an issue now. Lowering speed, at least with the existing job that produces said error, still does so. it's at 87% (391MB out of 445MB) and does not progress.

I'll also try to do a memtest soon, to rule out hardware issues.

[MiG]

Win7 ... how much RAM?

16GB, ~8GB physical memory in use right now.

[sander]

Thanks for NZB via PM.

On my system (Ubuntu Linux), no problem at all. So I can't reproduce.

Do you have another system on which you can run SABnzbd and download the NZB, and see if you can reproduce the OoM ... ?

[safihre]

This problem can't be related to the NZB, it's specific to the system.
I think you should run a memory checking tool!

[MiG]

Memtest86 checks out, full pass without errors. Tried the built in BIOS mem check as well for completeness' sake, same result.

Tried NZBget on the same system with the same NZB, and there it works... I'm puzzled.

It now appears to result from any nzb I try, so something is really wrong here. Is there any more detailed debugging that I can do to see what exactly is causing it?

[safihre]

Really weird..

[MiG]

Think I've found the problem. The NZB backup dir contains 22k files, about 1.2GB worth of old NZBs which I assume hits the memory limit per process (prob 2GB for Win7 home premium x64?). I moved those away, and now it appears to work again. I assume these are all loaded or indexed?

Still not sure why the error message only appears during download, rather than preventing SAB from succesfully launching.

[sander]

Think I've found the problem. The NZB backup dir contains 22k files, about 1.2GB worth of old NZBs which I assume hits the memory limit per process (prob 2GB for Win7 home premium x64?). I moved those away, and now it appears to work again. I assume these are all loaded or indexed?

If you move those 22k files back into the NZB backup dir, does the problem occur again?

[MiG]

This is where it gets even more puzzling... Just downloaded a set of files (~22GB) successfully, then stopped SAB, moved the dir back and started it again. There was a nearly identical set of duplicates that I used as a test case, and that downloaded with flying colours. No error messages, no speed issues, nothing.
Before I removed the backup dir, SAB's downloads had just ground to a complete halt, despite settings changes, reboots etc.

I'll keep monitoring it. If the issue returns, do you have any suggestions on what to put a magnifying glass on?

[safihre]

They are scanned when adding new jobs, if you have Duplicate Check enabled. Do you?