Page 1 of 1
sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 15th, 2023, 4:30 pm
by OneCD
Hi guys, recently had a new error
reported. It relates to the 'sabctools' module and might be another manylinux issue.
This is on an ARMv7 QNAP NAS running QTS (modified Ubuntu).
I can confirm this error doesn't occur on ARMv5 or AARCH64.
So, we are running SABnzbd from 'master' branch, in a venv, and are now reinstalling 'sabctools' (as it looks like manylinux2014 builds cannot be seen by Python/SABnzbd, and rebuilding a wheel for it locally appears to fix this).
Code: Select all
[~] # . /share/CACHEDEV1_DATA/.qpkg/SABnzbd/venv/bin/activate
(venv) [~] # pip install --upgrade --force-reinstall sabctools --no-binary sabctools --no-cache-dir
Collecting sabctools
Downloading sabctools-7.1.0.tar.gz (234 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 234.0/234.0 kB 5.0 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: sabctools
Building wheel for sabctools (pyproject.toml) ... done
Created wheel for sabctools: filename=sabctools-7.1.0-cp311-cp311-linux_armv7l.whl size=157294 sha256=53c09062182b73ba8786885f35dad232581b5aac1b4162578d6ccd48e14a405c
Stored in directory: /share/CACHEDEV1_DATA/.qpkg/Entware/tmp/pip-ephem-wheel-cache-ffcwsg0w/wheels/23/df/ee/6d1470a58e799dd336431ec37a3112fab028b783186f231c33
Successfully built sabctools
Installing collected packages: sabctools
Attempting uninstall: sabctools
Found existing installation: sabctools 7.1.0
Uninstalling sabctools-7.1.0:
Successfully uninstalled sabctools-7.1.0
Successfully installed sabctools-7.1.0
Looks good.
Now, let's try manually importing it:
Code: Select all
(venv) [~] # python -c "import sabctools; print("sabctools.__version__")"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/share/CACHEDEV1_DATA/.qpkg/SABnzbd/venv/lib/python3.11/site-packages/sabctools/__init__.py", line 2, in <module>
from sabctools.sabctools import *
ImportError: /share/CACHEDEV1_DATA/.qpkg/SABnzbd/venv/lib/python3.11/site-packages/sabctools/sabctools.cpython-311.so: ELF load command alignment not page-aligned
I'm quite clueless on how to solve this, although I did find mention of this error here:
https://github.com/linuxdeploy/linuxdeploy/issues/204
Any advice please?
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 16th, 2023, 4:58 am
by jcfp
What's the page size and the load alignment on that system?
Try these two commands (modify the filename and path of the sabctools .so file so it points to the one you built):
Code: Select all
$ getconf PAGESIZE
4096
$ readelf -l /usr/lib/python3/dist-packages/sabctools/sabctools.cpython-310-x86_64-linux-gnu.so
Elf file type is DYN (Shared object file)
Entry point 0x0
There are 11 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000003a10 0x0000000000003a10 R 0x1000
LOAD 0x0000000000004000 0x0000000000004000 0x0000000000004000
0x000000000001f865 0x000000000001f865 R E 0x1000
LOAD 0x0000000000024000 0x0000000000024000 0x0000000000024000
0x0000000000003380 0x0000000000003380 R 0x1000
LOAD 0x0000000000027990 0x0000000000028990 0x0000000000028990
0x0000000000000878 0x0000000000000918 RW 0x1000
[...]
The alignment of the LOAD lines (0x1000 = 4096 in the example output) should be an exact multiple of the pagesize.
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 16th, 2023, 5:57 pm
by OneCD
Thank you @jcfp, I'll translate those commands for QTS so the affected user can execute them.
QTS doesn't contain 'getconf', so I'll use this instead to display the page size if that's OK? I'm not sure how reliable this method is:
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 20th, 2023, 2:54 pm
by OneCD
Here's the response from the affected user:
Hi OneCD.
Back Online and here ist the Result from the 2 shell commands.
Code: Select all
[~] # grep -ir pagesize /proc/1/smaps
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
KernelPageSize: 32 kB
MMUPageSize: 32 kB
[~] #
Code: Select all
[~] # readelf -l /share/CACHEDEV1_DATA/.qpkg/SABnzbd/venv/lib/python3.11/site-packages/sabctools/sabctools.cpython-311.so
Elf file type is DYN (Shared object file)
Entry point 0x0
There are 6 program headers, starting at offset 52
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
EXIDX 0x00b130 0x0000b130 0x0000b130 0x00010 0x00010 R 0x4
LOAD 0x000000 0x00000000 0x00000000 0x0b144 0x0b144 R E 0x1000
LOAD 0x00be28 0x0000ce28 0x0000ce28 0x00438 0x00460 RW 0x1000
DYNAMIC 0x00bef0 0x0000cef0 0x0000cef0 0x00110 0x00110 RW 0x4
GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x10
GNU_RELRO 0x00be28 0x0000ce28 0x0000ce28 0x001d8 0x001d8 R 0x1
Section to Segment mapping:
Segment Sections...
00 .ARM.exidx
01 .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .rel.plt .init .plt .text .fini .rodata .ARM.exidx .eh_frame
02 .init_array .fini_array .data.rel.ro .dynamic .got .data .bss
03 .dynamic
04
05 .init_array .fini_array .data.rel.ro .dynamic
[~] #
It looks like the page sizes don't match.
Does this suggest a config issue with 'gcc'?
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 21st, 2023, 4:00 am
by jcfp
OneCD wrote: ↑June 20th, 2023, 2:54 pmIt looks like the page sizes don't match.
Does this suggest a config issue with 'gcc'?
Indeed, pagesize 64kB and alignment at 4kB won't work.
Something on the toolchain causing this, not necessarily gcc though; could be the linker or something like
https://github.com/pypa/manylinux/issues/735
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 21st, 2023, 8:55 am
by OneCD
Apologies, I’m still trying to understand the process.
I was working on the basis that if an existing arch-compatible wheel could not be downloaded for ‘sabctools’, then a new one is compiled from source. Is this correct?
If a ‘sabctools’ wheel is built locally from the source distribution, is manylinux still involved?
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 21st, 2023, 11:09 am
by safihre
No, locally there's no manylinux. It will just compile a new local wheel.
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 21st, 2023, 3:04 pm
by jcfp
I didn't mean to point out manylinux as the cause of the issue, but rather the breakage in patchelf it was affected by.
Does qnap have a public bug tracker for their ubuntu derivative?
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 21st, 2023, 3:40 pm
by OneCD
jcfp wrote: ↑June 21st, 2023, 3:04 pm
Does qnap have a public bug tracker for their ubuntu derivative?
No chance. They are quite private (comes with the Taiwanese culture, I think).
Is there a way I (or the affected user) can test 'patchelf'?
BTW: I just purchased an old QNAP NAS online with the same ARMv7 arch, so I'll be able to test this myself when it arrives.
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 22nd, 2023, 6:39 am
by jcfp
Trial and error to narrow down the cause. You could try compiling without creating a wheel, and see if the alignment of the .so is sane at that point.
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 22nd, 2023, 4:06 pm
by OneCD
I'm no compiling guru, so I'm kinda in the dark here. Any help on-offer is much appreciated.
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 23rd, 2023, 2:17 am
by jcfp
You probably want to use a venv for this to avoid polluting the system's python install.
Grab and extract the sabctools source from github, install build and/or test dependencies ("python3 -m pip install -U -r requirements.txt -r tests/requirements.txt"), then run "python3 setup.py build" in the base dir of the extracted source code. If you grab the (~20MB) github "source code" tarball rather than the 200 kB release tarball, you can also run tests after the build step ("python3 -m pytest tests").
See
https://buildd.debian.org/status/fetch. ... 8589&raw=0 for the kind of output to expect (search the log for "I: pybuild base:240" to see the commands used during build).
Re: sabctools: "ELF load command alignment not page-aligned" when importing?
Posted: June 23rd, 2023, 2:26 am
by OneCD
Thank you @jcfp, I’ll give that a go.
Cheers!