3.2.0RC2 constantly crashes after postprocessor (docker hotio)

[sander]

Also, in that hotio docker:

Code: Select all

bash-5.1# find / -name libc.so.6 -print
bash-5.1#

No libc.so.6?!

An no other library that offers malloc_trim:

Code: Select all

bash-5.1# grep -irn malloc_trim /usr/lib/
bash-5.1#

What kind of stripped down OS is this?

Maybe @jcfp can shine his light on this?

[Puzzled]

I get lots of hits for Symbol not found: malloc_trim and Docker so it's probably a common thing. There should probably be a check before calling it, or do it in a try.

[sander]

I get lots of hits for Symbol not found: malloc_trim and Docker so it's probably a common thing.

Oh what OS / docker image?

[Puzzled]

Oh what OS / docker image?

Usually Alpine Linux but several images.

[sander]

Indeed, Alpine seems to have no libc.so.6 & thus reproducible:

Code: Select all

docker run -it --rm alpine /bin/ash

apk add python3

 # python3
Python 3.8.7 (default, Dec 26 2020, 08:45:55)
[GCC 10.2.1 20201203] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> ctypes.CDLL("libc.so.6").malloc_trim(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/ctypes/__init__.py", line 386, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.8/ctypes/__init__.py", line 391, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: Symbol not found: malloc_trim
>>>

See ... easy to detect.

After
LIBC = ctypes.CDLL("libc.so.6")

SAB should check if malloc_trim() works ok. If not, set LIBC to None to avoid it calling

[sander]

sabnzbd/__init__.py:38:KERNEL32 = LIBC = None
sabnzbd/__init__.py:61: LIBC = ctypes.CDLL("libc.so.6")

Set LIBC to None if a malloc_trim() results in a Traceback.

Then the "if" should avoid calling it:

sabnzbd/postproc.py : 1044: if sabnzbd.LIBC:
sabnzbd/postproc.py : 1045: sabnzbd.LIBC.malloc_trim(0)

I tried, but so far it didn't work. Probably PEBKAC

[Puzzled]

What did you try?

How about

Code: Select all

try:
    LIBC.malloc_trim()
except:
    LIBC=None

in __init__.py?

[sander]

Yes, I did that. But within the hotio docker, with a SAB restart ... so not sure I did the right things / the changes survived the sab/docker restart.


EDIT:

Ah, got it working;

Code: Select all

bash-5.1# ./SABnzbd.py -s 0.0.0.0:8081
SJ: going in
SJ: no libc no malloc
2021-02-18 14:55:04,796::INFO::[SABnzbd:1149] --------------------------------
2021-02-18 14:55:04,797::INFO::[SABnzbd:1150] SABnzbd.py-3.2.0RC2

[Puzzled]

Do you use docker commit after doing the changes?

[jcfp]

What kind of stripped down OS is this?

Probably using some alternative c standard lib without memory management stuff (dietlibc or similar, as typically found in embedded systems). Checking at startup like puzzled proposed should be an easy fix.

[sander]

I'll send a PR

[sander]

https://github.com/sabnzbd/sabnzbd/pull/1800

[sander]

What kind of stripped down OS is this?

Probably using some alternative c standard lib without memory management stuff (dietlibc or similar, as typically found in embedded systems). Checking at startup like puzzled proposed should be an easy fix.

Ah: https://pkgs.alpinelinux.org/contents?b ... &repo=main

After installing that, there is a /lib/libc.so.6 , but it's just a link to /lib/libc.musl-x86_64.so.1, and so no malloc_trim() in it.

Anyway: Alpine uses musl (not glibc), which has no malloc_trim().

[fringe09]

Issue resolved! Pulled the latest nightly image and it no longer crashes. Thank you for fixing it quickly.

[sander]

Issue resolved! Pulled the latest nightly image and it no longer crashes. Thank you for fixing it quickly.

Ah, that's nice: hotio nightly does "Every commit to develop branch".

Cool.

Thanks for reporting. TIL ... not each Linux system has libc.