Multi-core PAR2 for Linux

[beecher]

Hi,

Would it be possible to add multi-core par2 on the the next sabnzbd linux release. It is available on linux too: http://chuchusoft.com/par2_tbb/download.html

Thanks.

[shypike]

We only distribute the sources for Linux.
This is used by package creators to create binaries for various Linux distributions.
This means we cannot add unrar or par2 binaries (it would not be useful).

We only make binary distributions for Windows and OSX.
There are just too many Unix/Linux/xBSD distributions.

[bav]

just upgrade your local par2 installation with the binaries provided there:

Download: [316kB] Source code (GPLv2).

Download: [180kB] FreeBSD 32-bit i386/ia32 binary (command line only).

Download: [180kB] FreeBSD 64-bit x86_64 binary (command line only).

Download: [188kB] GNU/Linux 32-bit i386/ia32 binary for kernel 2.6 / GCC 4 (command line only).

Download: [188kB] GNU/Linux 64-bit x86_64 binary for kernel 2.6 / GCC 4 (command line only).

Download: [664kB] Mac OS X 10.4 32-bit Intel/PowerPC and Mac OS X 10.5 64-bit Intel/PowerPC "fat"-universal binary (command line only). Note: the 64-bit PowerPC version is UNTESTED (because of lack of access to a G5 Mac).

Download: [200kB] 32-bit binary for Windows XP/Vista (command line only). Note: this version is untested under Windows 2000 - it may work, it may not.

Download: [228kB] 64-bit binary for 64-bit Windows XP or 64-bit Windows Vista (command line only). Warning: a 64-bit capable CPU is not sufficient to use this binary. You must also have a 64-bit version of Windows installed.

...needed intel libraries are included in the package and sabnzbd will use them, no problem at all. PAR2 or RAR/UNRAR within the sabnzbd distribution isn't needed when using it under linux.

Successfully tested with the latest Debian x64 and Ubuntu x64 - both working like a charm.

The only problem I see it's working so good and fast, that my server is getting unresponsive because of high load values (load average ~ 16+ with 8 cores) when SABNzbd is processing the next download at the same time the PAR2verify process is still running. But it's nice to see the par2 verify/repair fly!

btw. the program also got a good review from Intel itself, check http://software.intel.com/en-us/blogs/2 ... e-utility/

edit: OK, I found a little cosmetic problem: sabnzbd doesn't recognize when the new PAR2 ends

Code: Select all

Stage Par2
    [PAR-INFO] xxx ;) => Repairing : 100% 

--> looks like the optimized PAR2 has a slighty different console ouput when it finishes its work.

[shypike]

edit: OK, I found a little cosmetic problem: sabnzbd doesn't recognize when the new PAR2 ends

Code: Select all

Stage Par2
    [PAR-INFO] xxx ;) => Repairing : 100% 

--> looks like the optimized PAR2 has a slighty different console ouput when it finishes its work.

Very strange, because we use the same for the Windows distribution.
There it behaves the same as the classic par2.
I'll see if I can test this in the coming weeks.

[shypike]

I cannot confirm your problem.
I tested on Linux Mint with the chuchusoft binary for Linux.
The history says "[item] Repaired in XX seconds" for both the classic par2 and the tbb par2.

[adaemox]

Incase anyone else wonders (under Ubuntu 10.04), after downloading I believe you replace the par2 file located in /usr/bin with the par2 (and supporting files) from the site listed above. I've not yet tested in SABnzbd+ but from the command line it is loading with the par2 command.

[war59312]

Is there logs anywhere to see what sabnzbd is doing when using par2?

The logs in ~/.sabnzbd/logs are useless really (for my needs) as they only log errors and when sabnzbd starts and exits.

I'd like to see what sabnzbd is really up to.

Thanks!

[shypike]

All output from par2 is processed by SABnzbd.
If you don't trust SABnzbd, run par2 from the command line on the downloaded files.

[war59312]

It's not that I'd just like to confirm that it is working correctly. I "trust it", but want to be sure I have not broken something.

I changed out a few files hoping to speed things up a bit. And added some command line options in switches page and simply want to confirm they are working as expected, a bit hard to tell without real logs.

Also need to confirm that IONice is working.

Update: OK I Changed line 29 in /etc/init.d/sabnzbdplus:

DEFOPTS="--daemon"

To:

DEFOPTS="--daemon --logging 2 --log-all"

Now I can see a good amount of information as to what is happening.

Sadly it does NOT show the par2 command line. Bummer!

Good news is that I can confirm that ionice is working. Whoooooooo!!!

2011-10-20 22:57:32,366::INFO::[sabnzbdplus:472] ionice binary... found (/usr/bin/ionice)

2011-10-20 23:01:36,856::DEBUG::[newsunpack:531] Running unrar ['/usr/bin/ionice', '-c2', '-n4', '/usr/bin/unrar', 'x', '-idp', '-o-', '-or', '-p-', 'fileName.rar', 'dirLocation/']

Thanks again!

[shypike]

--log-all is not recommended.
It only adds a ton of extra logging related to article manipulation.
I'll extend the logging for the par2 command a bit.