par2 verify fails on 3.2.1 RC2

[mesa57]

Same download goes well with 2.3.9 but verify fails on 3.2.1 RC2

Logfile 2.3.9 : https://trello-attachments.s3.amazonaws ... D2.3.9.log

Logfile 3.2.1 RC2 : https://trello-attachments.s3.amazonaws ... .1.RC2.log

Edit: correction, we tested with 3.2.1 RC2

[safihre]

Could you send me the NZB? [email protected]
It looks like a really specific edge case having to do with CRC error's and how they are handled.

[safihre]

Actually, it looks like this is indeed a regression.
Currently in 3.x we discard the data if we get a CRC error, while before even though the data was bad, we would still save it.

[safihre]

You can download the patched 3.2.1 release here (does require a Github account):
https://github.com/sabnzbd/sabnzbd/acti ... /701684577

[mesa57]

Ok, thanks, we will test.
Btw. we also tested with some lower 3.x.x releases, same problem.

[mesa57]

Ok, seems not to be resolved with that commit.
Log : https://trello-attachments.s3.amazonaws ... github.log

Nzb is easy to find with binsearch/nzbindex : http://binsearch.info/?q=ffrekjkrjtkejkejktkekekk

[sander]

I can't download with SAB3.

I tried sabnzbd-2.3.9, and also that version also gives me "Repair failed, not enough repair blocks (4087 short)"

If you set SABnzbd's logging to +Debug, you will see this:

Code: Select all

2021-03-30 21:01:02,750::INFO::[assembler:319] Error during inspection of RAR-file /root/Downloads/incomplete/ffrekjkrjtkejkejktkekekk.1/ffrekjkrjtkejkejktkekekk.part52.rar
2021-03-30 21:01:02,750::DEBUG::[assembler:320] Traceback: 
Traceback (most recent call last):
  File "/sabnzbd-2.3.9/sabnzbd/assembler.py", line 247, in check_encrypted_and_unwanted_files
    zf = rarfile.RarFile(filepath, all_names=True)
  File "/sabnzbd-2.3.9/sabnzbd/utils/rarfile.py", line 667, in __init__
    self._parse()
  File "/sabnzbd-2.3.9/sabnzbd/utils/rarfile.py", line 867, in _parse
    self._file_parser.parse()
  File "/sabnzbd-2.3.9/sabnzbd/utils/rarfile.py", line 978, in parse
    self._parse_real()
  File "/sabnzbd-2.3.9/sabnzbd/utils/rarfile.py", line 1002, in _parse_real
    h = self._parse_header(fd)
  File "/sabnzbd-2.3.9/sabnzbd/utils/rarfile.py", line 1084, in _parse_header
    return self._parse_block_header(fd)
  File "/sabnzbd-2.3.9/sabnzbd/utils/rarfile.py", line 1619, in _parse_block_header
    header_crc, pos = load_le32(start_bytes, 0)
  File "/sabnzbd-2.3.9/sabnzbd/utils/rarfile.py", line 2576, in load_le32
    raise BadRarFile('cannot load le32')
BadRarFile: cannot load le32

If you Google that, you find https://github.com/sabnzbd/sabyenc/issues/14 ... bad rar files.

If you inspect the first rar file, it's not a rar file at all:

Code: Select all

root@7256da37145d:~/Downloads/incomplete/ffrekjkrjtkejkejktkekekk.1# hd *01.rar | head -1
00000000  05 5f 24 11 28 d4 aa 21  a9 8b 14 da 6f a7 cb 9f  |._$.(..!....o...|

When I do a full download of rar and par2 files, and do a par2repair, the result is:

Code: Select all

Repair is required.
54 file(s) exist but are damaged.
You have 10626 out of 14713 data blocks available.
You have 342 recovery blocks available.
Repair is not possible.
You need 3745 more recovery blocks to be able to repair.

So ... bad post in a few ways?

[safihre]

I have the same findings as Sander. I tried to download this file on 2.3.9 and also there it fails for me.
Just to be sure I also bought a subscription to Eweka and tried again with 2.3.9, and it still fails. Interestingly enough it fails with 4032 blocks short (both on 2.3.9 and 3.2.1), so more blocks short than Sander's result from last night. Maybe this post is slowly disappearing from the servers?

I also tried to load it in NZBGet, but it just aborts the download after a few seconds due to how bad the NZB is.

So to summarize: there was a bug in 3.x where we discarded data when there were CRC errors, that will be fixed in 3.2.1.
Now 3.2.1 behaves the exact same way as 2.3.9 and it still fails, so not sure what more I can do. It's just a bad post...

[mesa57]

You can see in the 2.3.9 log that unrar is succesfull.
But we will continue testing the problem.

[safihre]

I can no longer get it to succeed on 2.3.9. Can you try it again on 2.3.9?
So that's why I think it's also failing now on the new 3.2.1: gone from the servers.

[mesa57]

We tried again and it worked with 2.3.9. But I must apologize, we did not use the nzb from binsearch, but the nzb uploaded by the user via spotnet.
Apparently those are different.
As i cannot upload attachments here, the original nzb is here :
https://trello-attachments.s3.amazonaws ... s_etc_.zip

I hope you can access it, it is on our Trello board.

[mesa57]

More news: found why the nzb is not working in 3.2.1 (fixed version).
There is a space in the nzb name, just before the .nzb
If we remove that the fixed version of 3.2.1 works well
So the fix for the CRC errors works ok !
About the nzb name : this is something we can fix in spotnet ourselfs.

Log successfull download with 3.2.1 : https://trello-attachments.s3.amazonaws ... spatie.log

[mesa57]

Btw. our tester reports that 2.3.9 has no problems with a space in the nzb name.

[sander]

More news: found why the nzb is not working in 3.2.1 (fixed version).
There is a space in the nzb name, just before the .nzb
If we remove that the fixed version of 3.2.1 works well

Can you search your sabnzbd.log for a line containing "Error renaming" and post that line here?

[mesa57]

There is no error renaming, but another error :

Code: Select all

2021-03-31 07:54:58,592::INFO::[newsunpack:1133] Scanning "\\?\C:\<USERNAME>s\<USERNAME>\Downloads\incomplete\Military Magazines etc \ffrekjkrjtkejkejktkekekk.par2"
2021-03-31 07:54:58,594::INFO::[misc:1050] [N/A] Running external command: ['D:\\Program files\\SABnzbd-3.2.1RC2\\win\\multipar\\par2j64.exe', 'r', '-uo', '-vs2', '-vd\\\\?\\C:\\<USERNAME>s\\<USERNAME>\\Downloads\\incomplete\\Military Magazines etc ', '\\\\?\\C:\\<USERNAME>s\\<USERNAME>\\Downloads\\incomplete\\Military Magazines etc \\ffrekjkrjtkejkejktkekekk.par2', '\\\\?\\C:\\<USERNAME>s\\<USERNAME>\\Downloads\\incomplete\\Military Magazines etc \\*']
2021-03-31 07:54:58,817::INFO::[newsunpack:1154] Par verify failed on \\?\C:\<USERNAME>s\<USERNAME>\Downloads\incomplete\Military Magazines etc \ffrekjkrjtkejkejktkekekk.par2!
2021-03-31 07:54:58,818::INFO::[postproc:807] Verification and repair finished for Military Magazines etc 
2021-03-31 07:54:58,824::INFO::[nzbstuff:1782] [N/A] Purging data for job Military Magazines etc  (delete_all_data=False)
2021-03-31 07:54:58,831::INFO::[notifier:122] Sending notification: Download mislukt - Military Magazines etc  (type=failed, job_cat=*)
2021-03-31 07:54:58,854::INFO::[database:293] Added job Military Magazines etc  to history

Full log at https://trello-attachments.s3.amazonaws ... a-punt.log