Not trying to repair with PAR anymore

[jms667]

Hello,

Probably since I've upgraded to 1.2.3 (running on a synology box), corrupted files are not repaired with PAR anymore.
After the download has completed, only a RAR-based verification is done, and if corrupted files are detected, the post processing is stopped (never trying to repair with downloaded PAR files like before). I can manually repair the files with quickpar (@windows) and extract them without downloading more files.

I have noticed the same behavior for my last 4/5 last downloads. Never succeed in repairing since I have upgraded.

Thanks for your help.

[safihre]

Can you switch logging to +Debug in the status window?
And then check what it is doing during post-processing. What does it show for PAR2 output?

[jms667]

Thanks for the reply.
I set the output log into debug mode.
A download is running, I will post back the result in a couple of hours.

[jms667]

Here is the output. It seems that it's a library issue. Any idea of how to fix it ?

2017-04-07 18:59:15,606::DEBUG::[newsunpack:1194] Starting par2: ['/usr/local/<USERNAME>/bin/nice', '-n15', '/usr/local/<USERNAME>/bin/par2', 'r', '', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.par2', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.01', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.02', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.03', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.04', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.08', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.09', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.10', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.11', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.12', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.13', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.14', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.15', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.17', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.18', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.20', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.21', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.25', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.27', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.28', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.29', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.30', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.35', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.36', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.38', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.40', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.48', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.49', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.51', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.52', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.53', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.55', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.57', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.59', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.63', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.64', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.67', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.68', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.70', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.71', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.73', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.74', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.81', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.84', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.vol000+02.par2', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.vol002+04.par2', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.vol062+64.par2', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.vol126+70.par2', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.75', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.32', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.91', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.78', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.37', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.72', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.60', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.42', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.50', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.80', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.79', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.47', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.45', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.19', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.33', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.46', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.65', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.89', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.56', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.77', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.05', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.54', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.85', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.82', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.58', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.34', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.par2', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.66', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.06', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.23', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.39', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.24', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.26', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.31', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.16', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.76', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.62', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.44', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.69', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.41', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.83', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.90', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.61', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.43', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.07', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.22', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.vol006+08.par2', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.vol014+16.par2', '/volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.vol030+32.par2']
2017-04-07 18:59:15,650::DEBUG::[newsunpack:1510] PAR2 output was
/usr/local/<USERNAME>/bin/par2: error while loading shared libraries: libgomp.so.1: cannot open shared object file: No such file or directory
2017-04-07 18:59:15,650::INFO::[newsunpack:1034] Par verify failed on /volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/<HASH>a004de6.par2!
2017-04-07 18:59:15,658::INFO::[downloader:772] Thread [email protected]: forcing disconnect
2017-04-07 18:59:16,543::DEBUG::[newsunpack:1698] build_filelists(): joinables: []
2017-04-07 18:59:16,544::DEBUG::[newsunpack:1699] build_filelists(): zips: []
2017-04-07 18:59:16,544::DEBUG::[newsunpack:1700] build_filelists(): rars: []
2017-04-07 18:59:16,544::DEBUG::[newsunpack:1701] build_filelists(): 7zips: []
2017-04-07 18:59:16,544::DEBUG::[newsunpack:1702] build_filelists(): ts: []
2017-04-07 18:59:16,544::DEBUG::[__init__:896] Saving data for __verified__ in /volume1/data/Download-Temp/incomplete/<HASH>a004de6.01/__ADMIN__
2017-04-07 18:59:16,545::INFO::[postproc:722] Verification and repair finished for

[safihre]

How did you install this?
Docker or SynoCommunity?
If you use SynoCommunity, can you try the sabnzbd-testing package? It has a different par2 module.

[jms667]

SynoCommunity package yes.
I will try with sabnzbd-testing, thanks.

[sander]

How did you install this?
It has a different par2 module.

Module, or executable?

Code: Select all

/bin/par2: error while loading shared libraries: libgomp.so.1: cannot open shared object file: No such file or directory

It looks llike the OP's par2 is multi-threading, which is using GOMP (GNU Offloading and Multi Processing), but the OP's hasn't the needed library installed.


On ubuntu, so probably the same on Synology:

Plain par2: no threads, no libgomp:

Code: Select all

root@1c6015fbbce8:/# strings $(which par2) | grep  -e gomp -e threads
root@1c6015fbbce8:/#

The threaded par2 has/needs libgomp:

Code: Select all

sander@Streamer13:~$ strings $(which par2) | grep  -e gomp -e threads
libgomp.so.1
omp_get_num_threads
omp_get_max_threads
omp_set_num_threads
  -t<n>    : Number of threads to use (

[safihre]

Strange, the regular sabnzbd package should be single core.
Is the testing one working?
Otherwise we need to report to SynoCommunity.
What NAS model and DSM software version do you use?

[desperado591]

I am bringing up this post again, as my problem seems quite the same as the thread openers problem:

After upgrading to version 2.0.0 SAB will not try to repair any files that are incomplete, regardless if that happens on my Synology NAS or on my Windows PC. As long as the files are complete, everything is running fast and fine, but as soon as there is something missing, it is saying something like "trying to verify the rar files" (I am from Germany, so my English is not the best, sorry) and then it just stops.

Manually downloading the par files and repairing it with quickpar or multipar on my PC is no problem, so the necessary programs are installed on my PC at least.

As I am the second one here with the same problem, it seems that there could be an issue with SAB 2.0 don´t you think?

[sander]

The OP did not come back after our help & questions. Not so nice of him/her; now we can't find the cause and solution on his Synology.

Anyway ... now about you: post the sabnzbd.log of your Windows sabnzbd not repairing here or on pastebin. Deutsche logging is kein Problem ... dass kónnen wir auch lesen.

[desperado591]

OK, I have changed the loglevel to debug and downloaded a file which I knew was incomplete.
But the logfile is really huge and ranging a long time back... is there a way to show just the log of the last download that caused the problem? Or must I clear the log before downloading?

[sander]

Copy the 30 lines around your "trying to repair"

[jms667]

The OP did not come back after our help & questions. Not so nice of him/her; now we can't find the cause and solution on his Synology.

Anyway ... now about you: post the sabnzbd.log of your Windows sabnzbd not repairing here or on pastebin. Deutsche logging is kein Problem ... dass kónnen wir auch lesen.

Hi,

That's not fair to say that. I answered each question and was systematically giving feed back but the last posts are missing because of the ransomware you faced off.
The last status (last post) was that I had posted 15 days ago on the synocommunity forum. Only feedbak I got was that par2 command are working fine with sabnzbd 2.0 and DSM 6, but I'am running DSM 5.2. Please have a look there : https://github.com/SynoCommunity/spksrc/issues/2735

A couple of days a go, I figured out that I had a second xpenology test vm running on my esx (running same DSM version as my 'production' one). I started it again and noticed that sabnzbd was running version 1.0.3-16. I did a copy of the par2 binary from testing DSM to production DSM. Now, execution of the par2 command is again working fine (no libgomp.so.1 error anymore) :

Code: Select all

NANOBOOT> /volume1/@appstore/sabnzbd/bin/par2
Not enough command line arguments.
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.

par2cmdline comes with ABSOLUTELY NO WARRANTY.

This is free software, and you are welcome to redistribute it and/or modify
it under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version. See COPYING for details.


Usage:

  par2 c(reate) [options] <par2 file> [files] : Create PAR2 files
  par2 v(erify) [options] <par2 file> [files] : Verify files using PAR2 file
  par2 r(epair) [options] <par2 file> [files] : Repair files using PAR2 files

You may also leave out the "c", "v", and "r" commands by using "parcreate",
"par2verify", or "par2repair" instead.

Options:

  -b<n>  : Set the Block-Count
  -s<n>  : Set the Block-Size (Don't use both -b and -s)
  -r<n>  : Level of Redundancy (%%)
  -c<n>  : Recovery block count (Don't use both -r and -c)
  -f<n>  : First Recovery-Block-Number
  -u     : Uniform recovery file sizes
  -l     : Limit size of recovery files (Don't use both -u and -l)
  -n<n>  : Number of recovery files (Don't use both -n and -l)
  -m<n>  : Memory (in MB) to use
  -v [-v]: Be more verbose
  -q [-q]: Be more quiet (-q -q gives silence)
  --     : Treat all remaining CommandLine as filenames

If you wish to create par2 files for a single source file, you may leave
out the name of the par2 file from the command line.

Yesterday I downloaded a corrupted package/file which has been repaired successfully with PAR. I'm using now this temporary workaround but there is definitively a issue with par binary embedded in synocomunity package with DSM 5.2.

If it could help I can post working an non-working par binary somewhere.

[sander]

the last posts are missing because of the ransomware you faced off.

Ah, yes!

Hopefully the github question will give you a solution.

[desperado591]

Copy the 30 lines around your "trying to repair"

öhm... I can´t find anything like "trying to repair" in the logfile. There are dozens of pages dealing with the relevant dowload. So I copied the end of that passage and startet at the first quick check that failed:

2017-04-26 19:34:24,717::INFO::[newsunpack:1808] Quick-check of file Prince - Rome 20Ten.part06.rar failed!
2017-04-26 19:34:24,717::INFO::[newsunpack:1042] Scanning "\\?\F:\Usenet\Downloads\incomplete\Prince-Rome-20Ten\Prince - Rome 20Ten.part.par2"
2017-04-26 19:34:24,717::DEBUG::[newsunpack:1769] build_filelists(): joinables: []
2017-04-26 19:34:24,717::DEBUG::[newsunpack:1770] build_filelists(): zips: []
2017-04-26 19:34:24,717::DEBUG::[newsunpack:1771] build_filelists(): rars: [u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part01.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part02.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part03.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part04.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part05.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part06.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part07.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part08.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part09.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part10.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part11.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part12.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part13.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part14.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part15.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part16.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part17.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part18.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part19.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part20.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part21.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part22.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part23.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part24.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part25.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part26.rar', u'\\\\?\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part27.rar']
2017-04-26 19:34:24,717::DEBUG::[newsunpack:1772] build_filelists(): 7zips: []
2017-04-26 19:34:24,717::DEBUG::[newsunpack:1773] build_filelists(): ts: []
2017-04-26 19:34:24,717::DEBUG::[newsunpack:1177] Par2-classic/cmdline = False
2017-04-26 19:34:24,717::DEBUG::[newsunpack:1220] Starting par2: ['F:\\Programme\\SABnzbd\\win\\par2\\par2.exe', 'r', '', '\\\\.\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\Prince - Rome 20Ten.part.par2', '\\\\.\\F:\\Usenet\\Downloads\\incomplete\\Prince-Rome-20Ten\\*']
2017-04-26 19:34:24,765::ERROR::[postproc:551] Nachbearbeitung von Prince-Rome-20Ten fehlgeschlagen ()
2017-04-26 19:34:24,765::INFO::[postproc:553] Traceback:
Traceback (most recent call last):
File "sabnzbd\postproc.pyo", line 322, in process_job
File "sabnzbd\postproc.pyo", line 650, in parring
File "sabnzbd\newsunpack.pyo", line 1068, in par2_repair


I hope that was the relevant part of the logfile.

PS: I tried to set an email notification on this thread so that I can reply faster, but it seems not to work. And of course I have checked my spam folder...
PPS: Just in case it is necessary - on my Synology I am running also DSM 5.2 as does the thread opener