Page 1 of 2
par2 skipped in 0.7.14
Posted: August 2nd, 2013, 3:09 am
by droopy
Hi,
It seems sabnzbd fail to process the par2 file in the 0.7.14 release (using it on a Sunologic NAS).
According to the log it add correctly the par2 (and the volumes par2 file) to the queue
Code: Select all
2013-08-02 00:15:04,948::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [00/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.nzb" yEnc (1/1) added to queue
2013-08-02 00:15:04,955::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [01/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.nfo" yEnc (1/1) added to queue
2013-08-02 00:15:04,962::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [02/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.par2" yEnc (1/1) added to queue
2013-08-02 00:15:04,997::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [03/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.part01.rar" yEnc (1/78) added to queue
...
2013-08-02 00:15:06,054::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [34/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol000+01.PAR2" yEnc (1/2) added to queue
2013-08-02 00:15:06,061::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [35/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol001+02.PAR2" yEnc (1/3) added to queue
2013-08-02 00:15:06,069::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [36/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol003+04.PAR2" yEnc (1/5) added to queue
2013-08-02 00:15:06,078::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [37/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol007+06.PAR2" yEnc (1/7) added to queue
2013-08-02 00:15:06,088::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [38/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol013+10.PAR2" yEnc (1/11) added to queue
2013-08-02 00:15:06,102::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [39/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol023+20.PAR2" yEnc (1/21) added to queue
2013-08-02 00:15:06,125::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [40/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol043+39.PAR2" yEnc (1/40) added to queue
2013-08-02 00:15:06,161::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [41/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol082+78.PAR2" yEnc (1/79) added to queue
2013-08-02 00:15:06,197::INFO::[nzbstuff:443] File Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS - [42/42] - "Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol160+78.PAR2" yEnc (1/79) added to queue
but later it fail to process them. the par2. It result it doesn't download thr volume par2 file (at least they are not in the download folder)
Code: Select all
2013-08-02 01:03:16,986::INFO::[newsunpack:1064] Extra pars = [<NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol000+01.PAR2, type=None>, <NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol001+02.PAR2, type=None>, <NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol003+04.PAR2, type=None>, <NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol007+06.PAR2, type=None>, <NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol013+10.PAR2, type=None>, <NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol023+20.PAR2, type=None>, <NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol043+39.PAR2, type=None>, <NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol082+78.PAR2, type=None>, <NzbFile: filename=Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol160+78.PAR2, type=None>]
2013-08-02 01:03:16,988::INFO::[assembler:154] <Article: [email protected], bytes=786998, partnum=43, art_id=None> manquants
2013-08-02 01:03:16,991::INFO::[newsunpack:1075] Found new par2file Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.vol000+01.PAR2
2013-08-02 01:03:16,998::INFO::[assembler:154] <Article: [email protected], bytes=786878, partnum=44, art_id=None> manquants
2013-08-02 01:03:17,008::INFO::[assembler:154] <Article: [email protected], bytes=786901, partnum=45, art_id=None> manquants
2013-08-02 01:03:17,018::INFO::[assembler:154] <Article: [email protected], bytes=787099, partnum=46, art_id=None> manquants
2013-08-02 01:03:17,038::INFO::[assembler:154] <Article: [email protected], bytes=787055, partnum=47, art_id=None> manquants
2013-08-02 01:03:17,058::INFO::[assembler:154] <Article: [email protected], bytes=786907, partnum=48, art_id=None> manquants
2013-08-02 01:03:17,070::INFO::[assembler:154] <Article: [email protected], bytes=786810, partnum=49, art_id=None> manquants
2013-08-02 01:03:17,078::INFO::[assembler:154] <Article: [email protected], bytes=786734, partnum=50, art_id=None> manquants
2013-08-02 01:03:17,088::INFO::[assembler:154] <Article: [email protected], bytes=786949, partnum=51, art_id=None> manquants
2013-08-02 01:03:17,098::INFO::[assembler:154] <Article: [email protected], bytes=786874, partnum=52, art_id=None> manquants
2013-08-02 01:03:17,108::INFO::[assembler:154] <Article: [email protected], bytes=787154, partnum=53, art_id=None> manquants
2013-08-02 01:03:17,118::INFO::[assembler:154] <Article: [email protected], bytes=786909, partnum=54, art_id=None> manquants
2013-08-02 01:03:17,128::INFO::[assembler:154] <Article: [email protected], bytes=786910, partnum=55, art_id=None> manquants
2013-08-02 01:03:17,138::INFO::[assembler:154] <Article: [email protected], bytes=786738, partnum=56, art_id=None> manquants
2013-08-02 01:03:17,141::INFO::[newsunpack:856] Par verify failed on /volume1/Download/incomplete/Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS/Game.of.Thrones.S03E05.PROPER.720p.HDTV.x264-KILLERS.par2!
If i use another client to download the files and use them with QuickPar everything works fine so the par2 files aren't corrupt.
Any clue ?
Regards,
Re: par2 skipped in 0.7.14
Posted: August 2nd, 2013, 1:28 pm
by shypike
There are often problems with installed par2cmdline versions on embedded systems.
Does any download ever complete successfully?
Re: par2 skipped in 0.7.14
Posted: August 2nd, 2013, 1:43 pm
by droopy
very few.
Is it possible to update the par2cmdline tools installed or configure another one on sabnzbd ?
Re: par2 skipped in 0.7.14
Posted: August 2nd, 2013, 2:09 pm
by shypike
I don't know. Tools like par2cmdline, unzip and unrar are platform-specific and
not delivered by the SABnzbd team.
One thing you can try is open an ssh session (assuming you know hat that is) and
try to run par2cmdline manually.
Find the "incomplete" folder where the "bad" show is and run this command:
par2 r bla.par2 *
where "bla.par2" is one of the downloaded par2 files.
Post a copy of the output of par2cmdline here.
Re: par2 skipped in 0.7.14
Posted: August 2nd, 2013, 2:53 pm
by droopy
par2 is not found on my system (Synology DS213).
I though it was preinstalled, should I install it manually ?
Re: par2 skipped in 0.7.14
Posted: August 2nd, 2013, 4:48 pm
by zaadstra
I have the same issue on a DS213+. Just got an update to 0.7.14-2 (Synocommunity) but no change. Seems the verification part is broken for a while.
In the past par2's were checked and if blocks missing, extra par2 blocks were downloaded. I assume the par2 always was included with SABnzbd+.
Now it looks it just does not attempt to grab extra blocks, and I see in binsearch that there are enough blocks.
The completed emails says nothing about the failed download and the history overview doesn't warn either.
I updated from 0.7.9 to 0.7.14 yesterday, today another update to 0.7.14-2. In 0.7.9 the behaviour of the repair system and the reporting wasn't perfect at all but it looks it got worse.
This is a part of the log:
2013-08-02 23:20:15,102::INFO::[newsunpack:838] Scanning "/volume1/download/sabnzbd/incomplete/VA-blah_MP3/VA-blah_MP3.par2"
2013-08-02 23:20:15,235::INFO::[newsunpack:856] Par verify failed on /volume1/download/sabnzbd/incomplete/VA-blah_MP3/VA-blah_MP3.par2!
2013-08-02 23:20:15,238::INFO::[postproc:596] Par2 check finished on VA-blah_MP3
2013-08-02 23:20:15,240::INFO::[misc:783] Creating directories: /volume1/download/sabnzbd/complete/VA-blah_MP3
2013-08-02 23:20:15,541::INFO::[emailer:123] Notification e-mail succesfully sent
2013-08-02 23:20:15,561::INFO::[postproc:521] Cleaning up VA-blah_MP3 (keep_basic=False)
The automatic system has worked in an excellent way in the past (long time ago). I wish that there is an option to grab all par2's so at least I can fix the download myself. But of course better, a working repair system with correct listing in the history :-)
Re: par2 skipped in 0.7.14
Posted: August 2nd, 2013, 5:22 pm
by zaadstra
I did a debug run, it looks like par2 does not like some of the parameters:
2013-08-03 00:07:17,777::DEBUG::[newsunpack:975] Starting par2: ['/usr/local/sabnzbd/bin/nice', '-n15', '/usr/local/sabnzbd/bin/ionice', '-c2', '-n4', '/usr/local/sabnzbd/bin/par2', 'r', '/volume1/download/sabnzbd/incomplete/VA-blah_MP3-testlog/VA-blah_MP3.par2', '/volume1/download/sabnzbd/incomplete/VA-blah_MP3-testlog/*']
2013-08-03 00:07:18,071::DEBUG::[newsunpack:1267] PAR2 output was
The source file does not exist: /volume1/download/sabnzbd/incomplete/VA-blah_MP3-testlog/__ADMIN__
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.
par2cmdline comes with ABSOLUTELY NO WARRANTY.
.....bla bla helptext....
out the name of the par2 file from the command line.
2013-08-03 00:07:18,072::INFO::[newsunpack:856] Par verify failed on /volume1/download/sabnzbd/incomplete/VA-blah_mP3-testlog/VA-blaHMP3.par2!
Interesting line is:
The source file does not exist: /volume1/download/sabnzbd/incomplete/VA-blah_MP3-testlog/__ADMIN__
What is it looking for?
Re: par2 skipped in 0.7.14
Posted: August 2nd, 2013, 5:33 pm
by zaadstra
shypike wrote:I don't know. Tools like par2cmdline, unzip and unrar are platform-specific and
not delivered by the SABnzbd team.
One thing you can try is open an ssh session (assuming you know hat that is) and
try to run par2cmdline manually.
Find the "incomplete" folder where the "bad" show is and run this command:
par2 r bla.par2 *
where "bla.par2" is one of the downloaded par2 files.
Post a copy of the output of par2cmdline here.
Here is the output:
nas> /usr/local/sabnzbd/bin/par2 r /volume1/download/sabnzbd/complete/VA-Blah_MP3-testlog/VA-Blah_MP3.par2 *
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.
par2cmdline comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it and/or modify
it under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version. See COPYING for details.
Loading "VA-Blah_MP3.par2".
Loaded 90 new packets
There are 44 recoverable files and 0 other files.
The block size used was 129416 bytes.
There are a total of 2000 data blocks.
The total size of the data files is 256134937 bytes.
Verifying source files:
Target: "000-va-b-2cd-2012.jpg" - found.
Target: "000-va-b-2012.m3u" - found.
Target: "000-va-d-2012.nfo" - found.
Target: "000-va-2cd-2012.sfv" - found.
Target: "101-psy.mp3" - missing.
Target: "102-justin.mp3" - found.7%
Target: "103-.mp3" - found.
.........
Target: "220-e.mp3" - found.
Scanning extra files:
File: "busybox" - no data found.
File: "ionice" - no data found.
File: "nice" - no data found.
File: "par2" - no data found.
File: "par2create" - no data found.
File: "par2repair" - no data found.
File: "par2verify" - no data found.
File: "renice" - no data found.
File: "unrar" - no data found.
Repair is required.
1 file(s) are missing.
43 file(s) are ok.
You have 1949 out of 2000 data blocks available.
Repair is not possible.
You need 51 more recovery blocks to be able to repair.
nas>
Re: par2 skipped in 0.7.14
Posted: August 2nd, 2013, 6:13 pm
by zaadstra
Re: par2 skipped in 0.7.14
Posted: August 21st, 2013, 1:29 pm
by zaadstra
Any ideas? I'm stuck on this point.
Re: par2 skipped in 0.7.14
Posted: August 23rd, 2013, 5:49 am
by shypike
I'm just as stuck as you are.
The reality is that embedded systems (like NAS) have all sorts of behaviour that's
different from mainstream Linux-based systems.
We simply cannot test all of these.
Are those "extra" files listed by par2cmdline present in the job folder?
Re: par2 skipped in 0.7.14
Posted: August 23rd, 2013, 2:52 pm
by id2test2
I use Nas4Free, and the following change makes automatic repair using Par files work again:
Code: Select all
diff newsunpack.py.orig newsunpack.py
971c971
< wildcard = os.path.join(os.path.split(parfile)[0], '*')
---
> wildcard = os.path.join(os.path.split(parfile)[0], '\*')
Re: par2 skipped in 0.7.14
Posted: August 24th, 2013, 3:14 am
by shypike
Very odd, this shouldn't make any difference or make things worse.
What a strange port of par2cmdline.
Anyway, release 0.7.15 will fix this in another way.
Re: par2 skipped in 0.7.14
Posted: August 24th, 2013, 4:37 pm
by zaadstra
shypike wrote:Very odd, this shouldn't make any difference or make things worse.
What a strange port of par2cmdline.
Anyway, release 0.7.15 will fix this in another way.
Nice
Can this be accompanied by an option or switch to have all extra Par2 block2 downloaded, instead of 0 (in v7.14) or only needed.
Re: par2 skipped in 0.7.14
Posted: August 25th, 2013, 1:46 pm
by shypike
zaadstra wrote:
Can this be accompanied by an option or switch to have all extra Par2 block2 downloaded, instead of 0 (in v7.14) or only needed.
That will be in 0.8.0