sabctools: "ELF load command alignment not page-aligned" when importing?

[OneCD]

Hi guys, recently had a new error reported. It relates to the 'sabctools' module and might be another manylinux issue.

This is on an ARMv7 QNAP NAS running QTS (modified Ubuntu).

I can confirm this error doesn't occur on ARMv5 or AARCH64.

So, we are running SABnzbd from 'master' branch, in a venv, and are now reinstalling 'sabctools' (as it looks like manylinux2014 builds cannot be seen by Python/SABnzbd, and rebuilding a wheel for it locally appears to fix this).

Code: Select all

[~] # . /share/CACHEDEV1_DATA/.qpkg/SABnzbd/venv/bin/activate
(venv) [~] # pip install --upgrade --force-reinstall sabctools --no-binary sabctools --no-cache-dir
Collecting sabctools
  Downloading sabctools-7.1.0.tar.gz (234 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 234.0/234.0 kB 5.0 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: sabctools
  Building wheel for sabctools (pyproject.toml) ... done
  Created wheel for sabctools: filename=sabctools-7.1.0-cp311-cp311-linux_armv7l.whl size=157294 sha256=53c09062182b73ba8786885f35dad232581b5aac1b4162578d6ccd48e14a405c
  Stored in directory: /share/CACHEDEV1_DATA/.qpkg/Entware/tmp/pip-ephem-wheel-cache-ffcwsg0w/wheels/23/df/ee/6d1470a58e799dd336431ec37a3112fab028b783186f231c33
Successfully built sabctools
Installing collected packages: sabctools
  Attempting uninstall: sabctools
    Found existing installation: sabctools 7.1.0
    Uninstalling sabctools-7.1.0:
      Successfully uninstalled sabctools-7.1.0
Successfully installed sabctools-7.1.0

Looks good.

Now, let's try manually importing it:

Code: Select all

(venv) [~] # python -c "import sabctools; print("sabctools.__version__")"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/venv/lib/python3.11/site-packages/sabctools/__init__.py", line 2, in <module>
    from sabctools.sabctools import *
ImportError: /share/CACHEDEV1_DATA/.qpkg/SABnzbd/venv/lib/python3.11/site-packages/sabctools/sabctools.cpython-311.so: ELF load command alignment not page-aligned

I'm quite clueless on how to solve this, although I did find mention of this error here: https://github.com/linuxdeploy/linuxdeploy/issues/204

Any advice please?

[jcfp]

What's the page size and the load alignment on that system?

Try these two commands (modify the filename and path of the sabctools .so file so it points to the one you built):

Code: Select all

$ getconf PAGESIZE
4096

$ readelf -l /usr/lib/python3/dist-packages/sabctools/sabctools.cpython-310-x86_64-linux-gnu.so

Elf file type is DYN (Shared object file)
Entry point 0x0
There are 11 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000003a10 0x0000000000003a10  R      0x1000
  LOAD           0x0000000000004000 0x0000000000004000 0x0000000000004000
                 0x000000000001f865 0x000000000001f865  R E    0x1000
  LOAD           0x0000000000024000 0x0000000000024000 0x0000000000024000
                 0x0000000000003380 0x0000000000003380  R      0x1000
  LOAD           0x0000000000027990 0x0000000000028990 0x0000000000028990
                 0x0000000000000878 0x0000000000000918  RW     0x1000
[...]

The alignment of the LOAD lines (0x1000 = 4096 in the example output) should be an exact multiple of the pagesize.

[OneCD]

Thank you @jcfp, I'll translate those commands for QTS so the affected user can execute them.

QTS doesn't contain 'getconf', so I'll use this instead to display the page size if that's OK? I'm not sure how reliable this method is:

Code: Select all

grep -ir pagesize /proc/1/smaps

[OneCD]

Here's the response from the affected user:

Hi OneCD.
Back Online and here ist the Result from the 2 shell commands.

Code: Select all

[~] # grep -ir pagesize /proc/1/smaps
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
KernelPageSize:       32 kB
MMUPageSize:          32 kB
[~] #

Code: Select all

[~] # readelf -l /share/CACHEDEV1_DATA/.qpkg/SABnzbd/venv/lib/python3.11/site-packages/sabctools/sabctools.cpython-311.so

Elf file type is DYN (Shared object file)
Entry point 0x0
There are 6 program headers, starting at offset 52

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x00b130 0x0000b130 0x0000b130 0x00010 0x00010 R   0x4
  LOAD           0x000000 0x00000000 0x00000000 0x0b144 0x0b144 R E 0x1000
  LOAD           0x00be28 0x0000ce28 0x0000ce28 0x00438 0x00460 RW  0x1000
  DYNAMIC        0x00bef0 0x0000cef0 0x0000cef0 0x00110 0x00110 RW  0x4
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x10
  GNU_RELRO      0x00be28 0x0000ce28 0x0000ce28 0x001d8 0x001d8 R   0x1

 Section to Segment mapping:
  Segment Sections...
   00     .ARM.exidx
   01     .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .rel.plt .init .plt .text .fini .rodata .ARM.exidx .eh_frame
   02     .init_array .fini_array .data.rel.ro .dynamic .got .data .bss
   03     .dynamic
   04
   05     .init_array .fini_array .data.rel.ro .dynamic
[~] #

It looks like the page sizes don't match.

Does this suggest a config issue with 'gcc'?

[jcfp]

It looks like the page sizes don't match.

Does this suggest a config issue with 'gcc'?

Indeed, pagesize 64kB and alignment at 4kB won't work.

Something on the toolchain causing this, not necessarily gcc though; could be the linker or something like https://github.com/pypa/manylinux/issues/735

[OneCD]

Apologies, I’m still trying to understand the process.

I was working on the basis that if an existing arch-compatible wheel could not be downloaded for ‘sabctools’, then a new one is compiled from source. Is this correct?

If a ‘sabctools’ wheel is built locally from the source distribution, is manylinux still involved?

[safihre]

No, locally there's no manylinux. It will just compile a new local wheel.

[jcfp]

I didn't mean to point out manylinux as the cause of the issue, but rather the breakage in patchelf it was affected by.

Does qnap have a public bug tracker for their ubuntu derivative?

[OneCD]

Does qnap have a public bug tracker for their ubuntu derivative?

No chance. They are quite private (comes with the Taiwanese culture, I think).

Is there a way I (or the affected user) can test 'patchelf'?

BTW: I just purchased an old QNAP NAS online with the same ARMv7 arch, so I'll be able to test this myself when it arrives.

[jcfp]

Trial and error to narrow down the cause. You could try compiling without creating a wheel, and see if the alignment of the .so is sane at that point.

[OneCD]

I'm no compiling guru, so I'm kinda in the dark here. Any help on-offer is much appreciated.

[jcfp]

You probably want to use a venv for this to avoid polluting the system's python install.

Grab and extract the sabctools source from github, install build and/or test dependencies ("python3 -m pip install -U -r requirements.txt -r tests/requirements.txt"), then run "python3 setup.py build" in the base dir of the extracted source code. If you grab the (~20MB) github "source code" tarball rather than the 200 kB release tarball, you can also run tests after the build step ("python3 -m pytest tests").

See https://buildd.debian.org/status/fetch. ... 8589&raw=0 for the kind of output to expect (search the log for "I: pybuild base:240" to see the commands used during build).

[OneCD]

Thank you @jcfp, I’ll give that a go.

Cheers!