multi format unpackaging: .zip, .rar, .iso, .7z, .gzip, ...

[chest069]

Would love to see this ability added too. Great idea.

[neilt0]

Can you retain the option of using unzip and unrar on non X86 Linux machines (e.g. PPC and ARM9 NAS boxes) in case there's no easy way to get a binary of the p7zip program?

[real_per]

Is this feature planned in any way ?
Or can someone point me to a post processing script, that would do the trick ?

[elfurbe]

I suggested this a while ago when we were still on the SourceForge forums and was told that 7z had some issues with some features of the RAR format so it wasn't tenable as a full replacement.  I have, so far, never come across a RAR set that 7z can't extract, but I'm not calling anyone a liar.

[shypike]

Probably in 0.5.0, definitely not in 0.4.0

[Haskell Noob]

I know this is an old post, but the basic premise of supporting 7zip at some point hasn't changed, or has it?

[shypike]

It's  just that it's an awful lot of work with only limited benefits.

[trog]

It's  just that it's an awful lot of work with only limited benefits.

Just chiming in late on this one - I've been meaning to ask about this for a while, because I think the benefits would be a bigger than merely opening up a new range of formats for unzipping.

I upload large volumes of binaries to Usenet (primarily gaming related, but right now I'm uploading the new Debian v6.0.0 release) and I always find it vexing to have to use RAR because it's still technically payware and a proprietary system.

If it was possible, I'd much rather shift to an open format like 7z for my archiving needs - it would pretty much be a good drop-in replacement for me uploading.

I know I could switch over to 7zip now and just let people extract their archives manually, but I'm a very happy sabnzbd user and would love to see 7z supported natively in the hope that it would encourage more submissions to Usenet to be compressed with 7z as opposed to RAR.

I am sure it's a lot of work to make the switch, but just wanted to chime in and say that it would certainly be greatly appreciated by me!

[sander]

It's  just that it's an awful lot of work with only limited benefits.

Can it be put in a custom post-process script by a user her/himself? (Maybe a stupid question; I don't know anything about the scripts ...)

[shypike]

Yes, that's possible, but not very efficient.
The 7z files will have been copied to "complete" before the script runs.

For the programmers among you.
What is needed for 7zip is an equivalent of the function rar_extract_core()
in the module sabnzbd/newsunpack.py
This function starts unrar with the right parameters and analyzes the
console output generated by it.
It's a fairly isolated piece of code with little interaction with other parts of SABnzbd.

[sander]

shypike, some questions on post-processing:

I've created a script cd-ing to $1 and then doing a 7z-unzip in that directory and some file writing in that directory. I've filled out the post-processing-script directory in sabnzbd/ However, I don't see any results in that $1 directory.

So:
- should sab 060 run the post process? Can you see it being called in the stdout output below?
- has the post-process-script the same input parameters as the pre-process? So like http://wiki.sabnzbd.org/user-scripts ?

Code: Select all

2011-02-21 11:45:24,248::INFO::[downloader:607] Thread [email protected]:119: forcing disconnect
2011-02-21 11:45:24,373::INFO::[postproc:230] Starting PostProcessing on 0394857.7z => Repair:True, Unpack:True, Delete:True, Script:None
2011-02-21 11:45:24,375::INFO::[postproc:461] Par2 check starting on 0394857.7z
2011-02-21 11:45:24,383::INFO::[postproc:476] Running repair on set 0394857.7z
2011-02-21 11:45:24,401::INFO::[postproc:83] Saving postproc queue
2011-02-21 11:45:24,403::INFO::[__init__:823] Saving data for postproc1.sab in /home/sander/.sabnzbd/admin/postproc1.sab
2011-02-21 11:45:24,416::INFO::[newsunpack:731] Quick-check for 0394857.7z is OK, skipping repair
2011-02-21 11:45:24,418::INFO::[newsunpack:788] Deleting /home/sander/Downloads/incomplete/0394857.7z/0394857.7z.par2
2011-02-21 11:45:24,422::INFO::[newsunpack:802] Deleting /home/sander/Downloads/incomplete/0394857.7z/0394857.7z.vol00+1.par2
2011-02-21 11:45:24,424::INFO::[postproc:491] Par2 check finished on 0394857.7z
2011-02-21 11:45:24,426::INFO::[misc:976] Creating directories: /home/sander/Downloads/complete/0394857.7z
2011-02-21 11:45:24,454::INFO::[postproc:279] Running unpack_magic on 0394857.7z
2011-02-21 11:45:24,458::INFO::[newsunpack:178] Filejoin starting on /home/sander/Downloads/incomplete/0394857.7z
2011-02-21 11:45:25,710::INFO::[newsunpack:182] Filejoin finished on /home/sander/Downloads/incomplete/0394857.7z
2011-02-21 11:45:25,713::INFO::[postproc:281] unpack_magic finished on 0394857.7z
2011-02-21 11:45:26,018::INFO::[postproc:434] Cleaning up 0394857.7z (keep_basic=False)
2011-02-21 11:45:26,028::INFO::[postproc:83] Saving postproc queue
2011-02-21 11:45:26,029::INFO::[__init__:823] Saving data for postproc1.sab in /home/sander/.sabnzbd/admin/postproc1.sab
2011-02-21 11:45:26,031::INFO::[downloader:225] Post-processing finished, resuming download

[shypike]

The pre-process script is not related to post-processing.
It has it own rules: http://wiki.sabnzbd.org/user-pre-queue-script

There is no general post-processing script.
You either set one per job or you associate one with a category.
Check whether the queue entry for the NZB has a script set.

Your logging says: Repair:True, Unpack:True, Delete:True, Script:None

[sander]

Aha: the post-process-script is *not* run on all downloads. You have to select a/the scrip manually per download (or category) on the right hand side of the download (most right dropdown).

Clear. Thanks!

After doing that, my 7z-unzip script works on the first test downloads.

To do:
- handle spaces in the directory name (my current sh-script cannot handle that)
- handle password protected 7z files (meaning: skip them)

[sander]

screenshot attached ...

[sander]

OK, done: I've implemented 7z-unzipping as a post-process script and it works on all the 7z-downloads I've tested.

Developed and tested on Ubuntu. It will need some tweaking for Windows; I can assist


@shypike: some catch-all post-proces-script would be a nice feature. The script could for the existence of *.7z and do it's thing.

Final remark: I see little posts with 7z, so let's see if anybody uses my script. If it is used a lot, the 7z-unzipping could get into SAB's mainstream