Media Processing Script

[auskento]

Hi Everyone,

Been a little while since i posted an update. The script has undergone many changes in the last two weeks.

Additions:

MUSIC category processing - assumes MP3 / album downloads. Will process the folder as ARTIST/ALBUM/Track - Songtitle.mp3 (This is how i like to store music)
If it finds a graphic file with the word front or cover, it will be copied to the destination folder as album art.

Implementation of a config file: Its much easier to manage settings from a .CONF file.
Just execute the script to have a default config file created - edit to suit

Expanded adult content handling to include network sites (ie Bangbros, Reality Kings and any others) see the postproc.net file
This file is not required - it was a request from the original guy who asked me to develop a script in the first place.

Version history. Execute script with --version for full list of recent changes 

Changes:

Added various functions to the script to better handle logging and script exit.
File logging is now configurable in the config file.

Removed the mv command from all but a renaming function in the ISO handling part of the script. It caused all sorts of problems if the script encountered an error, or the script was set to not remove things.

Fixed the TV processing to handle any format - if it isnt detected as episode, season or DVD, it will be processed anyway without showname information.

Lots of other little handling changes

Let me know if you have any issues

you can get the files from the following location

http://postproc.incinerator.com.au


NOTE: The postproc.sh.x file is a 'compiled' script - make sure you set it to +x, and you can rename it to whatever (i recommend just leaving as postproc.sh since the conf files and net files are named the same.
The text version of the script is available upon request - i just want to see who is using / changing it


[attachment deleted by admin]

[inpheaux]

Just a heads-up, I didn't delete your stuff because it was expressedly against the rules, but because I forgot to lock-down the attachment script before we went live. In the future, for sharing scripts - for security reasons - I suggest either using your own hosting (as you do already) or using something like pastebin.

[auskento]

No problem, i will just leave it hosted at my little hosting site.  It gets updated there more often anyway, since my editing script is setup to copy it (as long as im happy with the functionality)

[auskento]

Ran into a very big logistics error in the script after adding MUSIC / MP3.  Since Music Video also comes in via the MUSIC category, the script would barf when the script attempted to process it as a MP3.
I am in the midst of a rewrite of the script to process things more based on their file extension, and using the category information only to set destination folder settings.
I have got it working quite well atm, but needs a bit more testing - I also functionalised the bulk of the script to tidy up excess repetitiveness, and allow easier addition of new file types.

I ran into the following issues last night which im going to try and resolve.

1) FLAC audio - probably will be treated as a general file copy since there is no tag information
2) WMV Audio - again probably treated as a general file copy since the tag information isnt easily retrievable
  Will probably be able to handle this by removing the SAMPLESIZE check IF it comes through in the MUSIC category.
3) RATDVD - compressed DVD image, general file copy

If anyone is interested in testing the updated script, let me know and i will post you a copy.

With a long weekend here in aus this weekend, i dont expect to get a publicly usable update out until middle of next week.

[quarx]

Hi, i am a big newbie and i did not manage to run your script on my nas QNAP-109 with sabnzbd 0.3.0 (probably because i am a big newbie)

I get the following error (chmod +x is ok):
bash: ./postproc.sh: cannot execute binary file

How do you run your "compiled" script ? is it in perl, python, c ?

And last question, your script looks really interesting,
would it be possible to provide me with your source code ?

Thanks

[auskento]

New update posted

Code: Select all

Jan 28th 2008 - Script has had a major rewrite due to a handling issue discovered once the Music handling was put in place
                  Since the MUSIC category can include Music DVD's, converts, HDTV etc, the script has been modified to handle content
                  based on its file extension more so than the category it comes in through from Newzbin.
                  The script has also been changed so the file handling is done as a function - cleaning up the detection and processing
                  Many functions were added to the script to reduce repetitive code - this has allowed the ability of turning file logging on and off
                         - better handling of file copying, directory creation, and errors that may result
                         - and most importantly leaving content behind if the script does fail.
                 Script no longer being compiled

[auskento]

Update posted, been away a lot the last month, so nothing posted until now.
Please let me know of anything else I can try and incorporate, or fix.

  Feb 25th 2008 - Made some adjustments to VOB processing
                        There is now an option called VOBISO.  Set it to yes or no, to enable conversion of a VIDEO_TS structure to an ISO
                        If VOBISO is set to NO, then the files forming the DVD, and any graphic files, will be copied to the destination folder
                        Also adjusted VOB file processing to handle singular VOB files - such as Music video's to be copied rather than attempt ISO conversion

  Feb 11th 2008 - Added file permission and ownership settings
                        If you want access to the files through a web interface (ie HTTPS or AutoIndex) the files need to be visible to the web user (http) - www-data is default in Ubuntu
                        Set the required file permissions - read required and the user to give ownership to
                        Use full ownership:  ie root:www-data

  Jan 29th 2008 - Added extra category handling
                        Category can be determined by the folder name as follows:  "MUSIC - ", "TV - " , "MUSIC - ", "XXX - "
                        (It is important that there is a SPACE then a DASH then another SPACE to set category in folder name)
                        Implemented a DEFAULT_CATEGORY directive in conf file (DEFAULT=MOVIES)
                        Script will attempt to handle any content that can't be determined as the DEFAULT category
                        (Category is required to determine final file location and the type of handling)

[th3joker]

pair of dimes pointed me here for the latest version, I'm still getting problems

I keep getting the error:

find: -printf: unknown option
/postproc.conf is invalid - please modify or delete to create new file

[auskento]

The postproc.conf error means there are settings missing from your conf file.

The find error i need more information about.

What distro are you using?
if you type find from a terminal prompt, what does it respond with?

[th3joker]

Using Mac OS X Leopard

find
usage: find [-H | -L | -P] [-EXdsx] [-f file] [file ...] [expression]

postproc.conf:
# Dont use any spaces in the base folder pathnames
# Set to off to disable logging to file
LOGGING=ON
LOG_PATH=/Volumes/Storage/sabnzbd/postproc.log
MP3_FOLDER=/Volumes/Leopard/Users/adrian/Music
TVDEST=/Volumes/TV
MOVIEDEST=/Volumes/Movies
XXXDEST=
TEMPFOLDER=/Downloads/Working
SAMPLESIZE=25M
# Set Remove = YES to remove the FOLDER_PATH upon successful post processing
REMOVE=NO
DEFAULT=MOVIES
# The following items ensure that files are visible via web interfaces to files - set to YES and set permissions as required
CHANGEOWNER=NO
FILEACCESS=655
FILEOWNER=root:www-data
# Ensure the MKISOFS is installed to create ISO files from DVD structures.  Set to know to leave VIDEO_TS folders alone
VOBISO=YES

[auskento]

Put something in the XXX section, even if you dont use it, just set it somewhere.

I developed this on my Ubuntu server, so am not 100% sure of its x-platform compatibility.

Do you get a line number from the error at all?

[th3joker]

No line number just this:
find: -printf: unknown option
/postproc.conf is invalid - please modify or delete to create new file

Updated postproc.conf:
# Dont use any spaces in the base folder pathnames
# Set to off to disable logging to file
LOGGING=ON
LOG_PATH=/Volumes/Storage/sabnzbd/postproc.log
MP3_FOLDER=/Volumes/Leopard/Users/adrian/Music
TVDEST=/Volumes/TV
MOVIEDEST=/Volumes/Movies
XXXDEST=/Volumes/Movies/Adult
TEMPFOLDER=/Volumes/Storage/sabnzbd/Downloads/Working/Cache
SAMPLESIZE=25M
# Set Remove = YES to remove the FOLDER_PATH upon successful post processing
REMOVE=NO
DEFAULT=MOVIES
# The following items ensure that files are visible via web interfaces to files - set to YES and set permissions as required
CHANGEOWNER=NO
FILEACCESS=655
FILEOWNER=root:www-data
# Ensure the MKISOFS is installed to create ISO files from DVD structures.  Set to know to leave VIDEO_TS folders alone
VOBISO=YES

[auskento]

Try running the following command at console

Change to the folder where the script is running from

find postproc.conf -printf %h

It should return a single dot
.

If you get the error, thats what is causing the problem, need to change the command to suit mac

I expect there may be a limitation in the find command.

[auskento]

The Mac implementation of find doesnt have printf support

http://developer.apple.com/documentatio ... ind.1.html

I will look into a way of getting the same information without using that command.

[auskento]

Updated to remove usage of the find with -printf command

Test it out and let me know your results.