[SOLVED] Deobfuscate.py fails with Found multiple larger files, aborting when using +UNPACK and rars are over 40 MB
Posted: October 18th, 2019, 8:16 pm
Error:
If you have Processing set to +UNPACK and your rars are over 40MB the script deobfuscate.py doesn't rename the extracted file to the job-name because it detects the rars as additional 'large files' and aborts with error text
Fix 1:
Change
OR
Fix 2:
Change
FEEDBACK
Of the two Fix 1 is the more appropriate remedy as you don't see many old style rar names any more.
I don't use +Delete as I prefer to hold onto the rar files for a little bit until I've checked the extracted files.
Background info
As someone who just updated to version 2.3.9 from a much earlier version because I was finally fed up manually figuring out which was the par2 file from all the random names and other obfuscation and because it's now running as a service on a server I figured I should belatedly start utilising +UNPACK and figure out how to automatically rename the extracted file to the original name.
Problem resolving thought processes waffle.
Attempt 1 Deobfuscate.py
Script Message
Attempt 2 Deobfuscate.py
Script Message
Attempt 3 Sorting %dn.%ext
Why are all the files in the root? - search forum - oh it should be %dn/%dn.%ext
Why are all the rar files in the folder renamed to %dn.%ext followed by .1 .2 .3 etc?
Search forum again, apparently that's how sorting works, it's either all files or none. There's no way to separate rars from mkv for renaming. - Time for another look at deobfuscate.py
Full error message from deobfuscate.py
Search for "No par2 files or large files found"
Figure out that phrase is just the final else statement for something went wrong
Search for "Found multiple larger files, aborting."
Ah now we're getting somewhere, that error message is because there is more than one file (I like comments in code) and it doesn't know which one to rename, and it's handily listed them for me... why is a rar file being listed?
Oh look, there's a line above that has the following in it.
Now I just need to modify the code so the limit is above 75MB, lets make it 100MB just for a bit of room.
Reads notes section.
Reads other variable
I think this is the better solution so change to this 'fix'.
Reads notes section a bit more
Always read any Notes first.
If you have Processing set to +UNPACK and your rars are over 40MB the script deobfuscate.py doesn't rename the extracted file to the job-name because it detects the rars as additional 'large files' and aborts with error text
Solution is to duplicate Deobfuscate.py and give it another name (e.g. DeobExclRars.py or Deobfuscate100MB.py), place it in a new script folder out of the install directory (remembering to change the config to point to new script folder) then modify the script in one of two ways.Found multiple larger files, aborting
No par2 files or large files found
Fix 1:
Change
add the file extension rarEXCLUDED_FILE_EXTS = ('.vob', '.bin')
Limitation is it wont work for old style r00, r02, etc names.EXCLUDED_FILE_EXTS = ('.vob', '.bin', '.rar')
OR
Fix 2:
Change
to a larger value e.g. 100MBMIN_FILE_SIZE = 40*1024*1024
Limitation is it won't work for rar files above 100MB.MIN_FILE_SIZE = 100*1024*1024
FEEDBACK
Of the two Fix 1 is the more appropriate remedy as you don't see many old style rar names any more.
I don't use +Delete as I prefer to hold onto the rar files for a little bit until I've checked the extracted files.
Background info
As someone who just updated to version 2.3.9 from a much earlier version because I was finally fed up manually figuring out which was the par2 file from all the random names and other obfuscation and because it's now running as a service on a server I figured I should belatedly start utilising +UNPACK and figure out how to automatically rename the extracted file to the original name.
Problem resolving thought processes waffle.
Attempt 1 Deobfuscate.py
Script Message
Apparently I need to install python 2.7.16 - DONE'"python"' is not recognized as an internal or external command, operable program or batch file.
Attempt 2 Deobfuscate.py
Script Message
Looks like deobfuscate.py doesn't do that - Time to try SortingNo par2 files or large files found
Attempt 3 Sorting %dn.%ext
Why are all the files in the root? - search forum - oh it should be %dn/%dn.%ext
Why are all the rar files in the folder renamed to %dn.%ext followed by .1 .2 .3 etc?
Search forum again, apparently that's how sorting works, it's either all files or none. There's no way to separate rars from mkv for renaming. - Time for another look at deobfuscate.py
Full error message from deobfuscate.py
Remote into server and copy deobfuscate.py to local machine and open in text editor.Found multiple larger files, aborting.
No par2 files or large files found
Search for "No par2 files or large files found"
Figure out that phrase is just the final else statement for something went wrong
Search for "Found multiple larger files, aborting."
Ah now we're getting somewhere, that error message is because there is more than one file (I like comments in code) and it doesn't know which one to rename, and it's handily listed them for me... why is a rar file being listed?
Oh look, there's a line above that has the following in it.
Lets find where the variable MIN_FILE_SIZE is definedif file_size > MIN_FILE_SIZE
That's 40MB, so anything over that is inspected. How big are my rars? 75MB! that'll be the problem then.MIN_FILE_SIZE = 40*1024*1024
Now I just need to modify the code so the limit is above 75MB, lets make it 100MB just for a bit of room.
then install it in the scripts folder.MIN_FILE_SIZE = 100*1024*1024
Reads notes section.
And I'd better rename it to deobfuscate100MB.py and create a new script folder elsewhere to prevent it being overwritten.NOTES:
4) If you want to modify this script, make sure to copy it out
of this directory, or it will be overwritten when SABnzbd is updated.
Reads other variable
Why aren't rar files included in this excluded file extension list?EXCLUDED_FILE_EXTS = ('.vob', '.bin')
This should work for all new version *.rar but not old style .r00EXCLUDED_FILE_EXTS = ('.vob', '.bin', '.rar')
I think this is the better solution so change to this 'fix'.
Reads notes section a bit more
3) If there are multiple larger (>40MB) files, then the script will not
rename anything, since it could be a multi-pack.
Always read any Notes first.