OS X + HandBrakeCLI + Python = convert, iTunes and e-mail

[jetshred]

I'm getting this error for all the files I have where the Category is TV.  Any advice?

Code: Select all

File "/Users/media/Scripts/nachos_to_handbrake.py", line 173, in <module>
    tv_details = re.match( TV_FORMAT, filename ).groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'

If the file does not have TV as the category then it gets added to iTunes as a Movie with the file name as the title.  In other word, this script works properly for me except when the file is a TV show.  Also, I do not have a Newzbin account (but wish I did).

[randyharris]

I'm getting this error for all the files I have where the Category is TV.  Any advice?

Code: Select all

File "/Users/media/Scripts/nachos_to_handbrake.py", line 173, in <module>
    tv_details = re.match( TV_FORMAT, filename ).groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'

If the file does not have TV as the category then it gets added to iTunes as a Movie with the file name as the title.  In other word, this script works properly for me except when the file is a TV show.  Also, I do not have a Newzbin account (but wish I did).

I solved this problem by using a Sorting for TV option that jived with what is coded in the script.

[johnnytk36]

I love your script, i took out the OSX stuff and it works great on linux.

I had to also disable the email as it wasn't working, though i didn't try to mess with it as sabnzbd emails me anyway.

My question is, can you please add a option to ad custom handbrake CLI commands instead of a preset? I do not like any of the built in presets. I have my own custom preset for the GUI but it wont work in the CLI, so i want to pass all its settings to the CLI manually.

I don't know python, so i don't know how hard this would be.

I would appreciate it a lot.

JKutz

[ddorato]

Solved.... Or Not...  See below...

[ddorato]

Ok... Maybe I do have a question...

I'm trying to encode the video files in AppleTV format...  The script converts the file but can't import into iTunes...  Any ideas?

[imthenachoman]

Nice script all-in-one script.

I use a different solution since my Windows PC has more processing power than my Macbook Pro.
I download via RSS Feeds with my Windows PC with TV sorting enabled. Than I use this nifty program to rename TV episode.
http://neverfear.org/blog/view/21/Autom ... V_episodes

Dexter.S04E12.nzb results into Dexter  -  4x12 - The Getaway.mkv, yes it fetches information from the internet.

Than I invoke TVconvert.exe http://code.google.com/p/sabscripts/ which will convert it to iPhone format and tags it with AtomicParsely. It deletes the original and puts the converted mp4 on my macbook pro via a netshare. On the macbook it has action folders enabled which adds it to my iTunes.

However, I like your python script, but I am not sure whether you tag the videos as "TV Shows" with the appropiate EpisodeNo and SeasonNo.

Yeah it strips show information from the file name. I modified the script to be able to run a 2nd script (to do whatever) after conversion but not sure if other people would find use of it.

[imthenachoman]

I'm getting this error for all the files I have where the Category is TV.  Any advice?

Code: Select all

File "/Users/media/Scripts/nachos_to_handbrake.py", line 173, in <module>
    tv_details = re.match( TV_FORMAT, filename ).groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'

If the file does not have TV as the category then it gets added to iTunes as a Movie with the file name as the title.  In other word, this script works properly for me except when the file is a TV show.  Also, I do not have a Newzbin account (but wish I did).

What is the file name being generated by SabNZBD for the TV show? And what are your config settings?

[imthenachoman]

I love your script, i took out the OSX stuff and it works great on linux.

I had to also disable the email as it wasn't working, though i didn't try to mess with it as sabnzbd emails me anyway.

My question is, can you please add a option to ad custom handbrake CLI commands instead of a preset? I do not like any of the built in presets. I have my own custom preset for the GUI but it wont work in the CLI, so i want to pass all its settings to the CLI manually.

I don't know python, so i don't know how hard this would be.

I would appreciate it a lot.

JKutz

Great idea. Will work on that sir/madam.

[imthenachoman]

Ok... Maybe I do have a question...

I'm trying to encode the video files in AppleTV format...  The script converts the file but can't import into iTunes...  Any ideas?

Are you the one I spoke to via e-mail?

Either way, for anyone else too, if you convert a video to AppleTV the file extension has to match a valid AppleTV format. Handbrake will create a .txt file if you tell it but iTunes will complain when you try to import it. So make sure whatever format the video is converted to, the extension ought to match.

[randyharris]

Ok... Maybe I do have a question...

I'm trying to encode the video files in AppleTV format...  The script converts the file but can't import into iTunes...  Any ideas?

My guess is that your file output from SAB is not in the same format as imthenachman's script expects it to be, I had this issue.

[imthenachoman]

I'm getting this error for all the files I have where the Category is TV.  Any advice?

Code: Select all

File "/Users/media/Scripts/nachos_to_handbrake.py", line 173, in <module>
    tv_details = re.match( TV_FORMAT, filename ).groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'

If the file does not have TV as the category then it gets added to iTunes as a Movie with the file name as the title.  In other word, this script works properly for me except when the file is a TV show.  Also, I do not have a Newzbin account (but wish I did).

I'm working on a modified version of the script that wont error if the file name does not match. It will print an error but it will still import.

[johnnytk36]

I thought yall might want to see my improved version. I took out all the mac OSX stuff as im running this on linux, its not to hard to put back in. I have not tested the email stuff on linux, i have it disabled, but might use it one day soon.

I've added the ability to easily add your own custom HandBrake commands. You can switch between the Custom Commands and Presets really easy.

i've also added the ability to change the file permissions and group ID after the converted file is made. SABnzb's permisions are not applied to the converted file as scripts are run after that part in their code. I run a special script after every download to set permissions to the way i want anyway, as i want every file in a certain group for reasons i wont get into to.

Code: Select all

#!/usr/bin/python

###############################################################################
# PLEASE DO NOT REMOVE
#
# Author: Anchal 'Nacho' Nigam (and lots of help from #python on irc.freenode.net)
# E-Mail: [email protected]
# Date:   February 1, 2010
###############################################################################

###############################################################################
# CONFIG START - read bottom of page for explanation
###############################################################################

DATE_TIME_FORMAT = "[ %Y-%m-%d @ %H:%M:%S ]  "
SEPERATOR = " : "

FILE_TYPES = [ "*.avi", "*.mkv" ]

TV_FORMAT = "^(?P<show>.*) - (?P<season_number>\d+)x(?P<episode_number>\d+) - (?P<episode_ID>.*)$"

OUTPUT_PREFIX = ""
OUTPUT_POSTFIX = ""
OUTPUT_EXTENSION = "m4v"
#OUTPUT_DIRECTORY = "/"

DELETE_ORIGINAL = True

HANDBRAKE_PATH = "/usr/bin/HandBrakeCLI"
HANDBRAKE_PRESET = "High Profile"
HANDBRAKE_COMMAND_OUTPUT = False
HANDBRAKE_COMMAND_OUTPUT_APPEND = True
Use_Handbrake_Custom_Commands_or_Preset = "Custom"

HANDBRAKE_Custom_Commands = [ '-e', 'x264',  '-q', '20.0', '-a', '1,1', '-E', 'faac,ac3', '-B', '160,160', '-6', 'stereo,auto', '-R', 'Auto,Auto', '-D', '0.0,0.0', '-f', 'mp4', '--decomb', '--loose-anamorphic', '-m', '-x', 'level=30:cabac=0:ref=3:mixed-refs=1:analyse=all:me=umh:no-fast-pskip=1' ]


Change_Output_File_Permissions_and_Group_id = True
File_Permisions = '0775'
Group_id = 1001


EMAIL = False
EMAIL_SMTP = 'smtp.gmail.com'
EMAIL_AUTH = True
EMAIL_LOGIN = ''
EMAIL_PASSWORD = ''
EMAIL_TO = [ '', ]
EMAIL_FROM = ''

###############################################################################
# CONFIG END
###############################################################################

###############################################################################
# MAIN PROGRAM
###############################################################################

import sys
import glob, os, re, subprocess, smtplib, email.utils
from email.mime.text import MIMEText
from email.utils import COMMASPACE
from time import strftime
from encodings import ascii



def date_time():
    return strftime( DATE_TIME_FORMAT )

# 1        The final directory of the job (full path)
# 2        The name of the NZB file
# 3        Clean version of the job name (no path info and ".nzb" removed)
# 4        Newzbin report number (may be empty
# 5        Newzbin or user-defined category
# 6        Group that the NZB was posted in e.g. alt.binaries.x

FINAL_DIRECTORY = len( sys.argv ) >= 2 and sys.argv[1] or ""
NZB_FILE_NAME = sys.argv[2] if len( sys.argv ) >= 3 else ""
JOB_NAME = sys.argv[3] if len( sys.argv ) >= 4 else ""
NEWZBIN_NZB_NUMBER = sys.argv[4] if len( sys.argv ) >= 5 else ""
NEWZBIN_CATEGORY = sys.argv[5] if len( sys.argv ) >= 6 else ""
NZB_USENET_CATEGORY = sys.argv[6] if len( sys.argv ) >= 7 else ""

print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Final Directory", FINAL_DIRECTORY )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "NZB Filename", NZB_FILE_NAME )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Job Name", JOB_NAME )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "NewzBin NZB ID", NEWZBIN_NZB_NUMBER )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "NewzBin Category", NEWZBIN_CATEGORY )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Usenet Category", NZB_USENET_CATEGORY )

# set output directory to final directory if not set
try:
    OUTPUT_DIRECTORY
except NameError:
    OUTPUT_DIRECTORY = FINAL_DIRECTORY

email_message = "Results:\n"
success = 0
fail = 0

print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "File Types", FILE_TYPES )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "TV Format", TV_FORMAT )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Output Directory", OUTPUT_DIRECTORY )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Output Prefix", OUTPUT_PREFIX )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Output Postfix", OUTPUT_POSTFIX )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Output Extension", OUTPUT_EXTENSION )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Handbrake Path", HANDBRAKE_PATH )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Handbrake Preset", HANDBRAKE_PRESET )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Handbrake Custom Commands", HANDBRAKE_Custom_Commands )
print date_time() + ( "%-30s" + SEPERATOR + "%s" ) % ( "Handbrake Verbose", HANDBRAKE_COMMAND_OUTPUT )

# store the output of each handbrake iteration
handbrake_output = []

#Determine if to use handbrake preset or Custom Commands
if ( Use_Handbrake_Custom_Commands_or_Preset == "Custom" ):
   HANDBRAKE_parameters = HANDBRAKE_Custom_Commands
elif ( Use_Handbrake_Custom_Commands_or_Preset == "Preset" ):
   HANDBRAKE_parameters = [ '--preset=' + HANDBRAKE_PRESET ]

#set the file permisions
pfiles = int(File_Permisions, 8)


# iterate through all the file types
for file_type in FILE_TYPES:
    print date_time() + "Processsing file type " + file_type
    
    # for each file type find all files
    for input_file in glob.glob( FINAL_DIRECTORY + '/' + file_type ):
        
        # get directory and file name with extension for the input file and remove extension
        ( directory, filename ) = os.path.split( input_file )
        filename = re.match( '^(.*)[.]' + file_type + '$', filename ).group( 1 )

        output_file = os.path.join( OUTPUT_DIRECTORY, "%s%s%s.%s" % ( OUTPUT_PREFIX, filename, OUTPUT_POSTFIX, OUTPUT_EXTENSION ) )
        # check if the output file exists
        if( os.path.exists( output_file ) ) :
            print date_time() + "\t" + input_file + SEPERATOR + 'skipping' + SEPERATOR + output_file + ' exists'
            #email_message += "\n" + 'SKIPPED' + SEPERATOR + input_file + SEPERATOR + 'already exists'
            continue
        
        output_file_processing = os.path.join( OUTPUT_DIRECTORY, "%s%s%s_processing.%s" % ( OUTPUT_PREFIX, filename, OUTPUT_POSTFIX, OUTPUT_EXTENSION ) )
        print date_time() + "\t" + input_file
        
        print date_time() + "\t\t" + 'encoding' + SEPERATOR + output_file_processing
       
        # set the HandBrake subprocess string
        Handbrake_process_starting_cmd = [ HANDBRAKE_PATH, '-i', input_file, '-o', output_file_processing ]
        print Handbrake_process_starting_cmd + HANDBRAKE_parameters
        
        p = subprocess.Popen( Handbrake_process_starting_cmd + HANDBRAKE_parameters , stdout = ( None if HANDBRAKE_COMMAND_OUTPUT else subprocess.PIPE ), stderr = ( None if HANDBRAKE_COMMAND_OUTPUT else subprocess.STDOUT ) )
        ( stdout, stderr ) = p.communicate()
        handbrake_output.append( [ input_file, stdout ] )
        
        # make sure HandBrake exited successfully
        if( p.wait() != 0 ):
            print date_time() + "\t\t" + "FAILED"
            email_message += "\n" + 'FAILED' + SEPERATOR + input_file + SEPERATOR + 'error encoding'
            fail += 1
            continue
        
        # rename file
        os.rename( output_file_processing, output_file )
        print date_time() + "\t\t" + 'done encoding' + SEPERATOR + output_file      
       
        # Change owner and set permissions
        if ( Change_Output_File_Permissions_and_Group_id ):
             os.chmod(output_file,pfiles)
             os.chown(output_file,-1,Group_id)

        

        # delete the original if requested
        if( DELETE_ORIGINAL ):
            sys.stdout.write( date_time() + "\t\t" + "deleting" )
            sys.stdout.flush()
            os.unlink( input_file )
            print SEPERATOR + 'done'
            
        email_message += "\n" + 'succeeded' + SEPERATOR + input_file
        success += 1

# send e-mail if requested
if( EMAIL ):
    sys.stdout.write( date_time() + "Sending e-mail" )
    sys.stdout.flush()
    if( email_message == "Results:\n" ):
        print SEPERATOR + 'skipping' + SEPERATOR + 'nothing to send'
    else:
        msg = MIMEText( email_message )
        msg['To'] = COMMASPACE.join( EMAIL_TO )
        msg['Subject'] = 'Post processing results: %s success / %s fail' % ( success, fail )
    
        server = smtplib.SMTP( EMAIL_SMTP )
        try:
            # authenticate if required
            if( EMAIL_AUTH ):
                server.ehlo()
                # start TLS if required
                if( server.has_extn( 'STARTTLS' ) ):
                    server.starttls()
                    server.ehlo()
                server.login( EMAIL_LOGIN, EMAIL_PASSWORD )
            server.sendmail( EMAIL_FROM, EMAIL_TO, msg.as_string() )
        except Exception, e:
            print SEPERATOR + 'ERROR'
        else:
            print SEPERATOR + 'done'
        finally:
            # quit/close connection
            server.quit()

# if we want to see handbrake output
if( HANDBRAKE_COMMAND_OUTPUT_APPEND ):
    for i in handbrake_output:
        print "\n\n"
        print i[0]
        print i[1]

print date_time() + 'DONE'
sys.exit( 0 )


"""
###############################################################################
# CONFIG HELP
###############################################################################

DATE_TIME_FORMAT = "[ %Y-%m-%d @ %H:%M:%S ]  "
    Use the table at http://docs.python.org/library/time.html#time.strftime to build the format for the time in the output
    Example:    "[ %Y-%m-%d @ %H:%M:%S ]  "
    Output:     ""[ 2010-02-01 @ 16:07:36 ]  "

SEPERATOR = " : "
     The string to use between sections in a line in the output
    Example:    " : "

FILE_TYPES = [ "*.avi", "*.mkv" ]
     A list of file types used to find files to encode
    Example:    [ "*.avi", "*.mkv" ]

TV_FORMAT = "^(?P<show>.*) - (?P<season_number>\d+)x(?P<episode_number>\d+) - (?P<episode_ID>.*)$"
     A named regex pattern that will match the output of the file created by SabNZBD
    The regex names must match those of applescript options (use _ for space).
    Example:    "^(?P<show>.*) - (?P<season_number>\d+)x(?P<episode_number>\d+) - (?P<episode_ID>.*)$"
    
OUTPUT_PREFIX = ""
    The string to add to the beginning of each output filename

OUTPUT_POSTFIX = " - iPhone"
    The string to add to the end of each output filename (before the extension)

OUTPUT_EXTENSION = "mp4"
    The extension to use for the output file

#OUTPUT_DIRECTORY = "/"
    The directory to save the output files in
    If this line is commented out (with a #) this script will save the file in the same directory as the input file

DELETE_ORIGINAL = False
    Set to True if you want to delete the input file
    Set to False if you do not

HANDBRAKE_PATH = "/usr/bin/HandBrakeCLI"
    The path to handbrakecli

HANDBRAKE_PRESET = "iPhone & iPod Touch"
    The preset name to use for this script
    Look at http://trac.handbrake.fr/wiki/CLIGuide#presets for a list of presets
    At some point I might modify this script to accept other HandBrake paramaters

HANDBRAKE_COMMAND_OUTPUT = False
    Set to True if you want to see the output of HandBrakeCLI as it is encoding
    Set to False if you do not

Use_Handbrake_Custom_Commands_or_Preset = "Custom"
	Set to "Custom" if you want to use the Custom Handbrake parameters.
	Set to "Preset" if you want to use the Preset that you set above


HANDBRAKE_COMMAND_OUTPUT_APPEND = False
    Set to True if you want to see the output of HandBrakeCLI at the end, after it has finished encoding all files
    Set to False if you do not

EMAIL = True
    Send e-mail upon completion of this script

EMAIL_SMTP = 'smtp.gmail.com'
    smtp address of your e-mail

EMAIL_AUTH = True
    True if your smtp provider requires authentication
    False if not

EMAIL_LOGIN = '[email protected]'
    Login for authentication if required

EMAIL_PASSWORD = 'H0ku5p0ku5'
    Password for authentication if required
    
EMAIL_TO = [ '[email protected]', ]
    A list of e-mail addressees to send the e-mail to

EMAIL_FROM = '[email protected]'
    Whow the e-mail will be freom
    Some smtp providers might require you to use your e-mail address

"""

###############################################################################
# PLEASE DO NOT REMOVE
#
# Author:    Anchal 'Nacho' Nigam (and lots of help from #python on irc.freenode.net)
# E-Mail:    [email protected]
# Date:        February 1, 2010
###############################################################################

[macfloripa]

Nice work on the script. I use it with linux.

I wonder if you can provide an option to set the process priority so that handbrake doesn't chew up all of my CPU.

I suppose that someone who knows python would know how to use nice(), or setrlimit()?

Thanks

[johnnytk36]

Here is what i do. I have SABnzbd call a script that calls this script. What this allows is fro me to do some file permission stuff and set the priority (nice) value for the conversion script. Since the priority value for the conversion script is set, any process it spawns will have the same priority value. I have seen a HUGE difference in the stability of my linux server since doing this. SOmetimes i couldnt do anything on my server if handbrake was converting anything.

I plan on adding a way to have the handbrake process run in the background separately from SABnzbd one day, so that SABnzbd can still post process other files in my queue.

Here is a copy of the script i use to call the conversion script listed here.

Code: Select all

#!/bin/sh

cd "$1"
cd ..

chmod_workingdir="$PWD"

echo
echo Started as $0
echo
echo "CHMOD Working Directory =" "$chmod_workingdir"
echo
echo

chown -hR :file_access "$1"
chmod -R 775 "$1"
chmod -R g+s "$1"
# Set the TV Show's Root Directory Permissions- Only if it was modified in the last 15 Minutes
cd "$chmod_workingdir"
find . -type d -mmin -15 -maxdepth 0 -exec chmod 775 '{}' ';'


nice -n 3 /home/johnnytk36/.sabnzbd/userscripts/nachos_to_handbrake.py "$1" "$2" "$3" "$4" "$5" "$6" "$7" &

[macfloripa]

Perfect!

My life just got better.