Memory leak?
Forum rules
Help us help you:
Help us help you:
- Are you using the latest stable version of SABnzbd? Downloads page.
- Tell us what system you run SABnzbd on.
- Adhere to the forum rules.
- Do you experience problems during downloading?
Check your connection in Status and Interface settings window.
Use Test Server in Config > Servers.
We will probably ask you to do a test using only basic settings. - Do you experience problems during repair or unpacking?
Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Memory leak?
When adding many large jobs to the queue, SAB seems to encounter some kind of memory leak. It ends up using about 50% of my system memory (1GB) and never releases that again, even after all downloads are finished. Also, downloading and post-processing gets slower over time.
I've paused all my downloads, waited for SAB to settle and then took a bunch of screenshots. IMHO the memory consumption in this scenario should go back to almost zero, because:
- nothing in Cache
- nothing to do (no de-yencing, no repairing, no unraring, no downloading, no anything)
as you can see, SAB is sitting around on its ass lazily (which is ok)
Webinterface tells the same story - nothing to do. But you can see that the time between finished jobs suddenly jumps at around 4am in the history queue. before, it took SAB ~20 minutes per job, then it's sometimes well over the one hour mark.
Memory usage is 47% of 2GB, even though there is nothing to do.
I've paused all my downloads, waited for SAB to settle and then took a bunch of screenshots. IMHO the memory consumption in this scenario should go back to almost zero, because:
- nothing in Cache
- nothing to do (no de-yencing, no repairing, no unraring, no downloading, no anything)
as you can see, SAB is sitting around on its ass lazily (which is ok)
Webinterface tells the same story - nothing to do. But you can see that the time between finished jobs suddenly jumps at around 4am in the history queue. before, it took SAB ~20 minutes per job, then it's sometimes well over the one hour mark.
Memory usage is 47% of 2GB, even though there is nothing to do.
Re: Memory leak?
Some Linux ports of Python refuse to return memory to the OS
or are at least very reluctant to do so.
A proper OS will swap out unused memory anyway, so it shouldn't hurt much.
or are at least very reluctant to do so.
A proper OS will swap out unused memory anyway, so it shouldn't hurt much.
Re: Memory leak?
I only have 200MB Swap because my OS is on a 2GB USB Stick the OS is debian. Are you sure this couldn't be handled via code?
Re: Memory leak?
First of all:
SABnzbd loves memory, and loves using it. (If you want a NZB-downloader with a small footprint, consider nzbget. See http://ipv6-or-no-ipv6.blogspot.com/201 ... -1110.html for an instruction).
SABnzbd uses more memory with large (> 10GB) downloads
In general, it's my experience on Linux, that the OS will fill up all memory. The reclaiming of memory will only happen when needed by other applications.
Now about the memory leak: IMHO (and confirmed by http://en.wikipedia.org/wiki/Memory_leak), something is only a memory leak if a program uses a lot of memory when not needed AND it cannot be reclaimed by the OS / other programs. So, what happens when SABnzbd is idle, has nothing in the queue, and you start up other memory-loving programs? Does SABnzbd's memory usage go down then?
SABnzbd loves memory, and loves using it. (If you want a NZB-downloader with a small footprint, consider nzbget. See http://ipv6-or-no-ipv6.blogspot.com/201 ... -1110.html for an instruction).
SABnzbd uses more memory with large (> 10GB) downloads
In general, it's my experience on Linux, that the OS will fill up all memory. The reclaiming of memory will only happen when needed by other applications.
Now about the memory leak: IMHO (and confirmed by http://en.wikipedia.org/wiki/Memory_leak), something is only a memory leak if a program uses a lot of memory when not needed AND it cannot be reclaimed by the OS / other programs. So, what happens when SABnzbd is idle, has nothing in the queue, and you start up other memory-loving programs? Does SABnzbd's memory usage go down then?
Re: Memory leak?
Usually that number is the amount requested by the processes, Which isnt equal to the amount used. It can also show weirdly for shared libraries as you have sickbeard running too that may be a cause.
You also have paused jobs, so the memory for downloading them is still "active" though 1GB is still a large number
You also have paused jobs, so the memory for downloading them is still "active" though 1GB is still a large number
Re: Memory leak?
I don't have an issue with SAB using 1GB of memory while actually doing stuff, but "stockpiling" that memory seems like a bad approach, especially since there can be absolutely nothing meaningful in there once the queue is paused/empty and all yencs are decoded - or am I wrong?
I will check whether that memory can be reclaimed by the OS/other processes and report back here.
I will check whether that memory can be reclaimed by the OS/other processes and report back here.
Re: Memory leak?
Okay, so I've thrown in a bunch of large downloads yesterday night and went to bed. This morning, things looked like this:
It's currently roughly 9am, so SAB hasn't been doing anything for six hours.
Console shows the following:
Not quite as bad as yesterday, but still SAB uses a solid 768MB or RAM for doing nothing.
So, how do I efficiently use up all the memory in my system to see whether SAB gives it back to me? My idea was tmpfs (or Ramdisk for Windows users). I create a ramdisk with size 2GB and start filling it, then monitor what the memory consumption of SAB does. Note that Linux does not allocate the memory upon creation of the tmpfs (opposed to Windows' Ramdisk if I'm not mistaken). The memory only gets used once you start filling the tmpfs. So I can easily create a 2GB tmpfs, even if that would consume all my system's memory if I fill it up to the last bit.
After 400M, pretty much all the free memory should have been used up. Of course, much of the used memory was "cached" by the kernel and other system thingies, which will release the memory if things get a bit uncomfortable. I'll skip the 500M screen and go right to the 1000M screen, as things look pretty much the same. Even after 500M, the system found some other ram to give up. SAB didn't give in a bit of it's memory so far.
So, "let's fry the fucker" I figured and made a 1000M file. This would get the system seriously into trouble. As you can see, my three python daemons claim >40% of memory, and I'm sure the kernel would like a couple of MBs too. So at 1000M, things will get tight.
Note the speed. I'm copying from /dev/zero (a virtual device creating only zeros) to a ramdisk, and the speed is 20mb/s !!! why? Because the system has to swap out everything to the slow USB drive. Sab gave me 1% of RAM back, still consuming shitloads.
And here I encountered a surprise. When I came back to my consoles from making my screenshots, SAB was gone.
Now, thinking that this is a coincidence would be a bit much, right? SAB has never crashed for me before. So I assume it crashed because it was out of memory. This means that although SAB allocates 40% of the memory of my system, it can't even use it itsself anymore. Now this seems like a proper memory leak to me.
Just to finish the story, here's how things look after unmounting the tmpfs.
Lots of free memory, and an overall happy system.
Cheers
It's currently roughly 9am, so SAB hasn't been doing anything for six hours.
Console shows the following:
Not quite as bad as yesterday, but still SAB uses a solid 768MB or RAM for doing nothing.
So, how do I efficiently use up all the memory in my system to see whether SAB gives it back to me? My idea was tmpfs (or Ramdisk for Windows users). I create a ramdisk with size 2GB and start filling it, then monitor what the memory consumption of SAB does. Note that Linux does not allocate the memory upon creation of the tmpfs (opposed to Windows' Ramdisk if I'm not mistaken). The memory only gets used once you start filling the tmpfs. So I can easily create a 2GB tmpfs, even if that would consume all my system's memory if I fill it up to the last bit.
After 400M, pretty much all the free memory should have been used up. Of course, much of the used memory was "cached" by the kernel and other system thingies, which will release the memory if things get a bit uncomfortable. I'll skip the 500M screen and go right to the 1000M screen, as things look pretty much the same. Even after 500M, the system found some other ram to give up. SAB didn't give in a bit of it's memory so far.
So, "let's fry the fucker" I figured and made a 1000M file. This would get the system seriously into trouble. As you can see, my three python daemons claim >40% of memory, and I'm sure the kernel would like a couple of MBs too. So at 1000M, things will get tight.
Note the speed. I'm copying from /dev/zero (a virtual device creating only zeros) to a ramdisk, and the speed is 20mb/s !!! why? Because the system has to swap out everything to the slow USB drive. Sab gave me 1% of RAM back, still consuming shitloads.
And here I encountered a surprise. When I came back to my consoles from making my screenshots, SAB was gone.
Now, thinking that this is a coincidence would be a bit much, right? SAB has never crashed for me before. So I assume it crashed because it was out of memory. This means that although SAB allocates 40% of the memory of my system, it can't even use it itsself anymore. Now this seems like a proper memory leak to me.
Just to finish the story, here's how things look after unmounting the tmpfs.
Lots of free memory, and an overall happy system.
Cheers
Re: Memory leak?
Program crash or the Linux oom-killer in action? Check system logs.Now, thinking that this is a coincidence would be a bit much, right? SAB has never crashed for me before. So I assume it crashed because it was out of memory.
Re: Memory leak?
Yup, you're right. It was the oom-killer. The fact stands that SAB rather gets killed than giving up some of its allocated memory.
Re: Memory leak?
Hmm currently im downloading a file with sab and its sitting around 112mb of memory with a Article Cache Limit of 128M. Total memory on the box is 192Mb (VPS) though mine has not ran as long as yours, Constant start up and shut downs to free memory for other applications on there.
Ill try leaveing it open see if it starts consuming more. What OS and what version of sab ? (And python) ?
Ill try leaveing it open see if it starts consuming more. What OS and what version of sab ? (And python) ?
Re: Memory leak?
Debian wheezy amd64
Python 2.7.2-9 (debian wheezy package)
Sab 0.6.10
Python 2.7.2-9 (debian wheezy package)
Sab 0.6.10
Re: Memory leak?
Is the tool 'pdb' useful for finding a memory leak? I tried to following the instructions on http://www.lshift.net/blog/2008/11/14/t ... mory-leaks , but after typing 'r' to run SABnzbd.py, it ran, but a CTRL-C stopped it all completely. So I was not sent back to the pdb shell ... :-(
And if a quick workaround is needed, an hourly call to API-restart (http://wiki.sabnzbd.org/api#toc21) could help.
FWIW: One ore two years ago I also researched memory usage by SAB, and how to limit it. It must be somewhere in the forum.
And if a quick workaround is needed, an hourly call to API-restart (http://wiki.sabnzbd.org/api#toc21) could help.
FWIW: One ore two years ago I also researched memory usage by SAB, and how to limit it. It must be somewhere in the forum.
Re: Memory leak?
Ah, found it:
http://forums.sabnzbd.org/viewtopic.php?t=4713
and "ulimit -d 500111 -m 500111 -v 500111" to limit SAB's mem usage to 500 MB. I don't know what happens if you do a 50GB download with those settings.
http://forums.sabnzbd.org/viewtopic.php?t=4713
and "ulimit -d 500111 -m 500111 -v 500111" to limit SAB's mem usage to 500 MB. I don't know what happens if you do a 50GB download with those settings.
Re: Memory leak?
hmm im using
Debain lenny 32 bit and thats the only difference
Wonder if theres a problem with 64 bit python ?
Debain lenny 32 bit and thats the only difference
Wonder if theres a problem with 64 bit python ?
Re: Memory leak?
Except on OSX we don't support 64bit Python.
And even on OSX there might be some issues, which we are looking at.
64bit may work, it's just that we don't test with it.
And even on OSX there might be some issues, which we are looking at.
64bit may work, it's just that we don't test with it.