Develop Pars randomly failing

[lordjynx]

I'm currently running from github sources on develop branch (last commit e6094141a089b849880f997ef2ded3f2ab6be84e - dated May 24th, 2014). I'm having an issue with seemingly random downloads being failed as having "Invalid par2 files, cannot verify or repair" error. If I run multipar and manually check, they repair. The included par2.exe also states the files can be repaired successfully.

Any ideas? I can submit a sample nzb if needed.

TIA!

LJ

[lordjynx]

More info from the logs:

2014-06-02 20:24:31,573::INFO::[__init__:532] Backing up C:\Python27\sabnzbd-settings\nzbs\SHOW.nzb.gz
2014-06-02 20:24:31,635::INFO::[__init__:532] Backing up \\?\C:\Temp\SHOW\__ADMIN__\SHOW.nzb.gz
2014-06-02 20:24:31,730::INFO::[__init__:836] \\?\C:\Temp\SHOW\__ADMIN__\__renames__ missing
2014-06-02 20:24:31,792::INFO::[postproc:83] Saving postproc queue
2014-06-02 20:24:31,792::INFO::[__init__:881] Saving data for postproc2.sab in C:\Python27\sabnzbd-settings\admin\postproc2.sab
2014-06-02 20:24:31,808::INFO::[postproc:278] Starting PostProcessing on SHOW => Repair:True, Unpack:True, Delete:True, Script:deletecrapfiles.cmd, Cat:tv show
2014-06-02 20:24:31,822::INFO::[postproc:560] Par2 check starting on SHOW
2014-06-02 20:24:31,838::INFO::[postproc:581] Running repair on set SHOW
2014-06-02 20:24:31,854::INFO::[newsunpack:1034] Scanning "\\?\C:\Temp\SHOW\SHOW.vol00+4.par2"
2014-06-02 20:24:34,959::INFO::[newsunpack:1265] Main packet not found...
2014-06-02 20:24:34,959::INFO::[newsunpack:1269] Extra pars = [<NzbFile: filename=SHOW.vol04+8.par2, type=None>, <NzbFile: filename=SHOW.vol12+4.par2, type=None>, <NzbFile: filename=SHOW.par2, type=None>]
2014-06-02 20:24:34,990::INFO::[newsunpack:1052] Par verify failed on \\?\C:\Temp\SHOW\SHOW.vol00+4.par2!
2014-06-02 20:24:35,006::INFO::[postproc:608] Par2 check finished on SHOW

[sander]

The line

Code: Select all

Scanning "\\?\C:\Temp\SHOW\SHOW.vol00+4.par2"

looks strange: What is the "\\?\" at the beginning?

[lordjynx]

The line

Code: Select all

Scanning "\\?\C:\Temp\SHOW\SHOW.vol00+4.par2"

looks strange: What is the "\\?\" at the beginning?

Yeah, I'm not sure either. I also have more in the debug logs ex:

2014-06-02 23:14:37,017::DEBUG::[newsunpack:1146] Par2-classic = True
2014-06-02 23:14:39,638::DEBUG::[newsunpack:1180] Starting par2: ['C:\\Python27\\sabnzbd\\win\\par2\\par2-classic.exe', 'r', '\\\\?\\C:\\Temp\\SHOW\\SHOW.vol00+4.par2', '\\\\?\\C:\\Temp\\SHOW\\*']
2014-06-02 23:14:39,670::INFO::[newsunpack:1265] Main packet not found...

LJ

[sander]

I think you should check your folders settings in SABnzbd (sabnzbd.ini) and check for strange characters.

[shypike]

"\\?\C:\Temp\SHOW\SHOW.vol00+4.par2" is the notation required on Windows to handle paths longer than 255 characters.
This is new in the coming 0.8.0 release.
"Main packet not found" means that the par2 file is damaged that it isn't recognized any more.
I doubt that there is a relation between the two.

Please enable debug logging, so that you get more diagnostic information.
What's especially important is the output of the par2 tool, which is logged too.

[lordjynx]

Shypike,

I'll PM you a pastebin.

Thanks for looking into this!

LJ

[lordjynx]

I went into Queue Repair and hit the ADD button to get it to spin out the debugged errors again.

I should note in the meantime - I extracted the original .nzb from the .gz archive, added it manually to sab, then started another download of it... worked fine... I can't figure out why it's working sometimes, then failing others.

LJ

[lordjynx]

Skypike,

I have a FULL debug log now from another one that failed. Here's the PAR2 snipit.. I can dump the whole log to paste bin if you want it?

2014-06-03 10:43:59,428::INFO::[nzbqueue:232] Saving queue
2014-06-03 10:43:59,428::INFO::[__init__:881] Saving data for queue10.sab in C:\Python27\sabnzbd-settings\admin\queue10.sab
2014-06-03 10:43:59,444::INFO::[postproc:83] Saving postproc queue
2014-06-03 10:43:59,444::INFO::[postproc:278] Starting PostProcessing on MOVIE => Repair:True, Unpack:True, Delete:True, Script:deletecrapfiles.cmd, Cat:movies
2014-06-03 10:43:59,444::INFO::[__init__:881] Saving data for postproc2.sab in C:\Python27\sabnzbd-settings\admin\postproc2.sab
2014-06-03 10:43:59,460::INFO::[postproc:560] Par2 check starting on MOVIE
2014-06-03 10:43:59,663::INFO::[__init__:836] \\?\C:\Temp\MOVIE\__ADMIN__\__verified__ missing
2014-06-03 10:43:59,663::INFO::[postproc:581] Running repair on set 141878-2
2014-06-03 10:43:59,678::DEBUG::[bpsmeter:273] bps: 2590099.03612
2014-06-03 10:43:59,678::INFO::[downloader:674] Thread [email protected]: forcing disconnect
2014-06-03 10:43:59,710::INFO::[newsunpack:1641] Cannot Quick-check missing file MOVIE.jpg!
2014-06-03 10:43:59,710::INFO::[newsunpack:1034] Scanning "\\?\C:\Temp\MOVIE\141878-2.par2"
2014-06-03 10:43:59,740::DEBUG::[newsunpack:1606] build_filelists(): joinables: []
2014-06-03 10:43:59,740::DEBUG::[newsunpack:1607] build_filelists(): zips: []
2014-06-03 10:43:59,740::DEBUG::[newsunpack:1608] build_filelists(): rars: []
2014-06-03 10:43:59,756::DEBUG::[newsunpack:1609] build_filelists(): 7zips: []
2014-06-03 10:43:59,756::DEBUG::[newsunpack:1610] build_filelists(): ts: []
2014-06-03 10:43:59,756::DEBUG::[newsunpack:1146] Par2-classic = True
2014-06-03 10:43:59,772::DEBUG::[newsunpack:1180] Starting par2: ['C:\\Python27\\sabnzbd\\win\\par2\\par2-classic.exe', 'r', '\\\\?\\C:\\Temp\\MOVIE\\141878-2.par2', '\\\\?\\C:\\Temp\\MOVIE\\141878-2*']
2014-06-03 10:43:59,802::INFO::[downloader:674] Thread [email protected]: forcing disconnect
2014-06-03 10:43:59,802::INFO::[newsunpack:1265] Main packet not found...
2014-06-03 10:43:59,802::INFO::[newsunpack:1269] Extra pars = [<NzbFile: filename=141878-2.vol0+1.par2, type=None>]
2014-06-03 10:43:59,819::INFO::[newsunpack:1280] Found new par2file 141878-2.vol0+1.par2
2014-06-03 10:43:59,835::DEBUG::[newsunpack:1473] PAR2 output was
The recovery file does not exist: \\?
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.
par2cmdline comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it and/or modify
it under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version. See COPYING for details.
Usage:
par2 c(reate) [options] <par2 file> [files] : Create PAR2 files
par2 v(erify) [options] <par2 file> [files] : Verify files using PAR2 file
par2 r(epair) [options] <par2 file> [files] : Repair files using PAR2 files
You may also leave out the "c", "v", and "r" commands by using "parcreate",
"par2verify", or "par2repair" instead.
Options:
-b<n> : Set the Block-Count
-s<n> : Set the Block-Size (Don't use both -b and -s)
-r<n> : Level of Redundancy (%%)
-c<n> : Recovery block count (Don't use both -r and -c)
-f<n> : First Recovery-Block-Number
-u : Uniform recovery file sizes
-l : Limit size of recovery files (Don't use both -u and -l)
-n<n> : Number of recovery files (Don't use both -n and -l)
-m<n> : Memory (in MB) to use
-v [-v]: Be more verbose
-q [-q]: Be more quiet (-q -q gives silence)
-- : Treat all remaining CommandLine as filenames
If you wish to create par2 files for a single source file, you may leave
out the name of the par2 file from the command line.


The The recovery file does not exist: \\? section above appears to be good information, I hope!

Thanks again!

LJ

[sander]

My guess: par2 does not understand the "\\?\"

Checking:

Code: Select all

C:\tempie\par2cmdline-0.4-x86-win32>par2 r \\?\C:\tempie\stuff\onzin.par \\?\C:\tempie\stuff\onzin.*
The recovery file does not exist: \\?
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.

Yep.

Without the leading "\\?\" par2 does understand it:

Code: Select all

C:\tempie\par2cmdline-0.4-x86-win32>par2 r C:\tempie\stuff\onzin.par C:\tempie\stuff\onzin.*
par2cmdline version 0.4, Copyright (C) 2003 Peter Brian Clements.

par2cmdline comes with ABSOLUTELY NO WARRANTY.

FWIW: I found this information interesting:

Windows NT has a maximum path size defined as MAX_PATH which is 255 characters. It is possible to use more characters by calling the wide (W) version of CreateFile and prefixing "\\?\" to the path. The "\\?\" tells the function to turn off path parsing. This lets you use paths that are nearly 32,000 Unicode characters long. You must use fully-qualified paths with this technique. This also works with UNC names.

The "\\?\" is ignored as part of the path.

For example, "\\?\D:\documents\faq.txt" is seen as "D:\documents\faq.txt".

Programs expecting to find legal file lengths may fall over attempting to open a file with a long path. If, for example the buffer they are putting the path into expects a legal file length.

[shypike]

Looks like a bug, will be investigated.

[sander]

Looks like a bug, will be investigated.

... a bug in par2, I would say, right?

[lordjynx]

Sander - I mentioned to skypike that it started after the unrar 5.01 upgrade off github... so, it's very likely, it's a par2 issue... or at least a combination of the two working together.. lol

LJ

[shypike]

It's not a par2 issue. par2 doesn't handle "long-paths".
SABnzbd should not send those to par2.
Likely something went wrong when I synced "develop" with "0.7.x".

[lordjynx]

Ah... well, at any rate.. once again, thanks shypike.. greatly appreciate your assistance!

LJ