Page 5 of 9
Re: [Synology NAS] Fixed foreign accents (2012/09/16)
Posted: January 23rd, 2013, 2:38 am
by EftyEdge
@LapinFou
First - i would to thank you. Your script seems to be exactly what i'm looking for. But i have two questions about it. Sorry but i found no answer in this thread.
- 1. I installed SABnzbd from GIT and not from a package. Did i have to change the first line and how?
2. When i finally use your script - how do i setup the postprocessing scripts from Sickbeard, Couchpotato and Headphones?
Thanx in advance.
Efty
Re: [Synology NAS] Fixed foreign accents (2012/09/16)
Posted: January 23rd, 2013, 5:09 am
by LapinFou
Hello!
First question, why not using the packages available here:
http://www.synocommunity.com/? Synology Python package will NOT conflict with SynoCommunity one. Both can be installed at the same time. It is also much easier to install and update.
To answer your questions:
1- The 1st line of the script must be updated according to where your python is installed. It must be done with vi through a telnet session.
2- At the moment you must select my script OR the Sickbeard script when the d/l is started under SABnzbd. If necessary, I can have a look to see if it is possible to launch Sickbeard post-processing from my script. But it is not planned yet.
Not sure if it is the expected answers.... keep me in touch!
Version 1.8 Update (2013/02/19)
Posted: February 16th, 2013, 10:00 am
by LapinFou
Code: Select all
#!/usr/local/python/bin/python -OO
#-*- coding: iso-8859-15 -*-
#
# If a file has been archieved under an ISO-8859 environment and unarchived
# under an UTF8 environment, then you will get an encoding format problem.
# The file will not be readable through SAMBA.
#
# Renaming script for SABnzbd runnning under Synology NAS.
# By default the NZB software is running under UTF-8 encoding format
# in order to correctly handle the french accents (éèàç...) a SABnzbd
# post-script must be run in order to convert the file encoding.
#
# To fix this problem, you must convert the encoding format
# to the UTF8 (default Synology encoding)
# The script is trying to detect if the original file/directory are coming
# from a RAR archive. In this case the unrar command on your Syno system will
# unpack in CP850 format (DOS).
# NB: in all cases, files will be readable through samba, even if the detection
# failed. But converted characters will not be good, ex: Î? instead é
#
# Remark: I guess it should work for any other encoding style. Just replace
# ISO-8859-15 (Western Europe) by the one coresponding to your country:
# http://en.wikipedia.org/wiki/Character_encoding#Common_character_encodings
#
# Done by LapinFou
# date | version | comment
#--------------------------------------
# 12-04-22 | 1.0 | Initial version
# 12-04-22 | 1.1 | Change encoding to ISO-8859-15
# | Added CP437 special characters (0x80-0x9A)
# 12-04-24 | 1.2 | Mixed encoding is now supported
# | UTF-8 encoding format detected
# 12-04-24 | 1.3 | Fixed typo line 57 (test must be 0xA0, not 0xA1)
# 12-05-24 | 1.4 | Added an exception for "Â" character
# | Added 7z unpack
# | Added move option
# | Added Syno index option
# 13-02-15 | 1.5 | Added an option to activate Sickbear post-processing
# | More evoluate move option (merge is now managed)
# | Argv1 folder is not renamed anymore (already managed by SABnzbd)
# 13-02-18 | 1.6 | Argv1 folder is now renamed (not managed by SABnzbd)
# 13-02-19 | 1.7 | Changed CP437 detection range
# 13-02-19 | 1.8 | Changed CP437 DOS encoding style with CP850
#
# get library modules
import sys
import os
import subprocess
import shutil
# If empty, then no move
# Format must be synology full path (case sensitive). For ex: /volume1/video/News
MoveToThisFolder = ''
# If MoveMergeSubFolder = False, then equivalent to unix command:
# mv -rf srcFolder destFolder
# In case of conflict between an already existing sub-folder in the destination folder:
# the destination sub-folder will be replaced with source sub-folder
#
# If MoveMergeSubFolder = True, then equivalent to unix command:
# cp -rf srcFolder/* destFolder/
# rm -rf srcFolder
# In case of conflict between an already existing sub-folder in the destination folder:
# the destination sub-folder will be merged with source sub-folder (kind of incremental)
MoveMergeSubFolder = True
# /!\ IndexInSynoDLNA and SickBeardPostProcessing are exclusive
# =============================================================
# If "True", then the folder will be indexed into Synology DLNA
# By default it is "False"
IndexInSynoDLNA = False
# If "True", the folder will be send to SickBeard for Post-Processing
# By default it is "False"
SickBeardPostProcessing = False
# If "True", all .7z files will be unpacked then source .7z file will be deleted
# By default it is "False"
Unpack7z = True
########################
# ----- Functions ---- #
########################
# Special character hex range:
# CP850: 0x80-0xA5 (fortunately not used in ISO-8859-15)
# UTF-8: 1st hex code 0xC2-0xC3 followed by a 2nd hex code 0xA1-0xFF
# ISO-8859-15: 0xA6-0xFF
# The function will detect if fileDirName contains a special character
# If there is special character, detects if it is a UTF-8, CP850 or ISO-8859-15 encoding
def renameFunc(fullPath, fileDirName):
encodingDetected = False
# parsing all files/directories in odrer to detect if CP850 is used
for Idx in range(len(fileDirName)):
# /!\ detection is done 2char by 2char for UTF-8 special character
if (len(fileDirName) != 1) & (Idx < (len(fileDirName) - 1)):
# Detect UTF-8
if ((fileDirName[Idx] == '\xC2') | (fileDirName[Idx] == '\xC3')) & ((fileDirName[Idx+1] >= '\xA0') & (fileDirName[Idx+1] <= '\xFF')):
print os.path.join(fullPath, fileDirName) + " -> UTF-8 detected: Nothing to be done"
encodingDetected = True
break;
# Detect CP850
elif ((fileDirName[Idx] >= '\x80') & (fileDirName[Idx] <= '\xA5')):
utf8Name = fileDirName.decode('cp850')
utf8Name = utf8Name.encode('utf-8')
os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
print os.path.join(fullPath, utf8Name) + " -> CP850 detected: Renamed"
encodingDetected = True
break;
# Detect ISO-8859-15
elif (fileDirName[Idx] >= '\xA6') & (fileDirName[Idx] <= '\xFF'):
utf8Name = fileDirName.decode('iso-8859-15')
utf8Name = utf8Name.encode('utf-8')
os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
print os.path.join(fullPath, utf8Name) + " -> ISO-8859-15 detected: Renamed"
encodingDetected = True
break;
else:
# Detect CP850
if ((fileDirName[Idx] >= '\x80') & (fileDirName[Idx] <= '\xA5')):
utf8Name = fileDirName.decode('cp850')
utf8Name = utf8Name.encode('utf-8')
os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
print os.path.join(fullPath, utf8Name) + " -> CP850 detected: Renamed"
encodingDetected = True
break;
# Detect ISO-8859-15
elif (fileDirName[Idx] >= '\xA6') & (fileDirName[Idx] <= '\xFF'):
utf8Name = fileDirName.decode('iso-8859-15')
utf8Name = utf8Name.encode('utf-8')
os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
print os.path.join(fullPath, utf8Name) + " -> ISO-8859-15 detected: Renamed"
encodingDetected = True
break;
if (encodingDetected == False):
print os.path.join(fullPath, fileDirName) + " -> No special characters detected: Nothing to be done"
return
# scan .7z files and unpack them
def unpack7zFunc(DirName):
print "Scanning for .7z file(s), then unpack them"
print "Scanning files..."
DetectedFiles = False
for dirname, dirnames, filenames in os.walk(DirName):
for filename in filenames:
if (filename[-3:] == ".7z"):
print "Unpack %s..." %(filename)
DetectedFiles = True
try:
filepath = os.path.join(dirname, filename)
syno7z_cmd = ['/usr/syno/bin/7z', 'x', '-y', filepath]
p = subprocess.Popen(syno7z_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if (str(out) != ''):
print("7z result: " + filepath + " successfully unpacked")
os.remove(filepath)
print(filepath + " has been deleted")
if (str(err) != ''):
print("7z failed: " + str(err))
except OSError, e:
print("Unable to run 7z: "+str(e))
if DetectedFiles:
print "Scanning for .7z files Done !"
else:
print "No .7z file Detected !"
return
# add folder in the Syno index database (DLNA server)
def addToSynoIndex(DirName):
print "Adding folder in the DLNA server"
synoindex_cmd = ['/usr/syno/bin/synoindex', '-A', DirName]
try:
p = subprocess.Popen(synoindex_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if (str(out) == ''):
print("synoindex result: " + DirName + " successfully added to Synology database")
else:
print("synoindex result: " + str(out))
if (str(err) != ''):
print("synoindex failed: " + str(err))
except OSError, e:
print("Unable to run synoindex: "+str(e))
return
########################
# --- Main Program --- #
########################
print "Launching CharTranslator Python script..."
print ""
# Get scripts directory of the SABnzbd from its config.ini file
if (SickBeardPostProcessing == True):
print 100*'-'
print "SickBeardPostProcessing option is ON"
print "Locating SABnzbd config.ini file..."
# Get SABnzbd rundir folder
currentFolder = os.getcwd()
# SABnzbd config.ini location
SabScriptsFolder = ''
confFile = '../../var/config.ini'
# Check that file does exit
if os.path.isfile(confFile):
SabConfigFile = open('../../var/config.ini', 'r')
# Parse each lines in order to get scripts folder path
for line in SabConfigFile.readlines():
if line[:len('script_dir')] == 'script_dir':
# Get script_dir result
SabScriptsFolder = line.split('=')[1]
# Remove 1st space + \n
if (SabScriptsFolder[0] == ' '):
SabScriptsFolder = SabScriptsFolder[1:]
SabScriptsFolder = SabScriptsFolder.replace('\n', '')
break
SabConfigFile.close
# Check that SABnzbd script folder has been found
if (SabScriptsFolder == ''):
print 100*'#'
print "SABnzbd script_dir parameter not found!"
print 100*'#'
sys.exit(1)
else:
print "SABnzbd script_dir parameter is: '%s'" %SabScriptsFolder
# Load SickBeard module
SickBeardScript = os.path.join(SabScriptsFolder, 'autoProcessTV.py')
# Check that SickBeard post-processing is present into SABnzbd scripts folder
if os.path.isfile(SickBeardScript):
sys.path.append(SabScriptsFolder)
print "Loading SickBeard 'autoProcessTV' module"
import autoProcessTV
print 100*'-'
print ""
else:
print 100*'#'
print "Unable to find SickBeard autoProcessTV.py script in folder:"
print SickBeardScript
print 100*'#'
sys.exit(1)
# Exit if the file doesn't exist
else:
print 100*'#'
print "Unable to find SABnzbd config.ini file in this folder:"
print os.path.join(currentFolder, confFile)
print 100*'#'
sys.exit(1)
# Change current directory to SABnzbd argument 1
os.chdir(sys.argv[1])
# display directory of the SABnzbd job
currentFolder = os.getcwd()
print "Current folder is " + currentFolder
# rename SABnzbd job directory (coming from SABnzbd: never in CP850 format)
print "Renaming destination folder to UTF-8 format..."
renameFunc('', currentFolder)
currentFolder = os.getcwd()
print "Destination folder renamed !"
print ""
# Unpack 7z file(s)
if (Unpack7z == True):
print 100*'-'
unpack7zFunc(currentFolder)
print 100*'-'
print ""
# process each sub-folders starting from the deepest level
print 100*'-'
print "Renaming folders to UTF-8 format..."
for dirname, dirnames, filenames in os.walk('.', topdown=False):
for subdirname in dirnames:
renameFunc(dirname, subdirname)
print "Folder renaming Done !"
print 100*'-'
print ""
# process each file recursively
print 100*'-'
print "Renaming files to UTF-8 format..."
for dirname, dirnames, filenames in os.walk('.'):
for filename in filenames:
renameFunc(dirname, filename)
print "Files renaming Done !"
print 100*'-'
print ""
# Move current folder to an another destination if the option has been configured
if (MoveToThisFolder != ''):
print 100*'-'
print "Moving folder:"
print os.getcwd()
print "to:"
print MoveToThisFolder
# Check if destination folder does exist and can be written
# If destination doesn't exist, it will be created
if (os.access(MoveToThisFolder, os.F_OK) == False):
os.makedirs(MoveToThisFolder)
os.chmod(MoveToThisFolder, 0777)
# Check write access
if (os.access(MoveToThisFolder, os.W_OK) == False):
print 100*'#'
print "File(s)/Folder(s) can not be move in %s" %(MoveToThisFolder)
print "Please, check Unix permissions"
print 100*'#'
sys.exit(1)
# If MoveMergeSubFolder is True, then move all file(s)/folder(s) to destination
# then remove source folder
destFolder = os.path.join(MoveToThisFolder, os.path.split(os.getcwd())[-1])
if (MoveMergeSubFolder):
print " Info: Merge option is ON (incremental copy)"
try:
synoCopy_cmd = ['/bin/cp', '-Rpf', os.path.abspath(currentFolder), MoveToThisFolder]
p = subprocess.Popen(synoCopy_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if (str(err) != ''):
print("Copy failed: " + str(err))
sys.exit(1)
except OSError, e:
print("Unable to run cp: " + str(e))
sys.exit(1)
os.chdir(MoveToThisFolder)
shutil.rmtree(currentFolder)
# If MoveMergeSubFolder is False, remove folder with same (if exists)
# then move all file(s)/folder(s) to destination and remove source folder
else:
print " Info: Merge option is OFF (existing data will be deleted and replaced)"
# Remove if destination already exist
if os.path.exists(destFolder):
shutil.rmtree(destFolder)
shutil.move(currentFolder, MoveToThisFolder)
os.chdir(MoveToThisFolder)
# Update currentFolder variable
os.chdir(destFolder)
currentFolder = os.getcwd()
print 100*'-'
print ""
# Add multimedia files in the Syno DLNA if the option has been enabled
if (IndexInSynoDLNA == True) & (SickBeardPostProcessing == False):
print 100*'-'
addToSynoIndex(currentFolder)
print ""
print 100*'-'
# Send to SickBeard for post-processing
elif (IndexInSynoDLNA == False) & (SickBeardPostProcessing == True):
print 100*'-'
print "Launching SickBeard post-processing..."
autoProcessTV.processEpisode(currentFolder, sys.argv[2])
print "SickBeard post-processing done!"
print ""
print 100*'-'
# Display error message + advise if both options are enabled
elif (IndexInSynoDLNA == True) & (SickBeardPostProcessing == True):
print 100*'#'
print "IndexInSynoDLNA and SickBeardPostProcessing options are exclusive"
print "Please check your configuration"
print ""
print "If you want to have both options enables at the same time, please processed as follow:"
print " 1- Enable only SickBeardPostProcessing option"
print " 2- In SickBeard GUI -> Config -> Notifications -> Enable 'Synology Indexer'"
print 100*'#'
sys.exit(1)
print ""
print "Character encoding translation done!"
Tutorial update (2013/02/16)
Posted: February 16th, 2013, 11:15 am
by LapinFou
The tutorial has been updated. I added more explanations about how options are working.
If you got any troubles, please post a message!!
Re: [Synology NAS] Fixed foreign accents (2013/02/16)
Posted: February 18th, 2013, 12:02 pm
by StreameR
Today I've made a fresh install for SABnzbd and dowloaded some musicfiles. The script worked fine and after downloading and unpacking I saw the message: Character encoding translation done!
The only thing the script didn't change, were the files which contained the letter "ö", for example:
./0200 - Bl›f - Liefs Uit Londen.flac -> No special characters detected: Nothing to be done
The original name of the Artist is Blöf and al the files containing that specific character, weren't changed.
Other files worked like a charm, for example:
./0848 - André Hazes - 'n Vriend.flac -> CP437 detected: Renamed
Did I do something wrong here, or is the script unable to change the specific letter "ö"?
Your work is appreciated, hope you can help me out here, or do I have to change all those files still manually?
The problem is I can't move change or delete the files and still have to edit them via a TelNet session....
Re: [Synology NAS] Fixed foreign accents (2013/02/16)
Posted: February 18th, 2013, 1:13 pm
by LapinFou
Hello StreameR,
First could you try again with the updated version (1.6) below?
I realized I've removed something which I should not, between version 1.4 and 1.5.
Maybe this is the root cause of your problem.
Then; if you still got the problem, could you contact me by PM in order to send me the NZB file which doesn't work well.
Re: [Synology NAS] Fixed foreign accents (2013/02/16)
Posted: February 18th, 2013, 2:13 pm
by StreameR
LapinFou wrote:Hello StreameR,
First could you try again with the updated version (1.6) below?
I realized I've removed something which I should not, between version 1.4 and 1.5.
Maybe this is the root cause of your problem.
Then; if you still got the problem, could you contact me by PM in order to send me the NZB file which doesn't work well.
Thanks for the quick respond, and excuse me for being a noob.... I don't know how to update the script to the 1.6 version, is there some kind of tutorial available that guides me through this session?
[edit]
maybe I found the way to update the script, could you tell me if I am in the right direction?
I've downloaded notepad++, and copy/past the script (v1.6) above this post. After pasting the text into notepad++ I did this: Edit -> EOL Conversion -> Unix
After that I saved the file as CharTranslator.py
Now I have to copy that created file to the location /usr/local/sabnzbd/var/scripts and replace the "original" CharTranslator.py file?
If the above routing is correct, could you please confirm this to me? Now I have to find out how to copy the created file to the right dir on the DSM.....
[/edit]
Re: [Synology NAS] Fixed foreign accents (2013/02/18)
Posted: February 18th, 2013, 4:56 pm
by LapinFou
You are complety right. Now if you copy the file in your video share folder from windows, your file should be visible in:
/volume1/video/CharTranslator.py
Then from a telnet session use the command:
cp -rf /volume1/video/CharTranslator.py /usr/local/sabnzbd/var/scripts/
Re: [Synology NAS] Fixed foreign accents (2013/02/18)
Posted: February 18th, 2013, 5:39 pm
by StreameR
Okay, thanks for the info!
Someone at the Dutch Synology forum also helped me, and recommended me to use the package Config File Editor by MertyMade. This worked for me, and atm I am downloading the files again to see if the problem is solved in Version 1.6
I will let you know what the results are, thanks a lot so far!
[UPDATE]
Strange, when I take
THIS file everything seems to work fine and I get the following message:
Launching CharTranslator Python script...
Current folder is /volume1/downloads/complete/50f7e7ae69cb932c11c8e2e7
Renaming destination folder to UTF-8 format...
/volume1/downloads/complete/50f7e7ae69cb932c11c8e2e7 -> No special characters detected: Nothing to be done
Destination folder renamed !
----------------------------------------------------------------------------------------------------
Scanning for .7z file(s), then unpack them
Scanning files...
No .7z file Detected !
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
Renaming folders to UTF-8 format...
Folder renaming Done !
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
Renaming files to UTF-8 format...
./1998_150[CD] Bløf - Aan De Kust (Live).mp3 -> ISO-8859-15 detected: Renamed
Files renaming Done !
----------------------------------------------------------------------------------------------------
Character encoding translation done!
When I download
THIS file (a part of a bigger collection songs) than the script doesn't change it..
./0155 - Bl›f - Aan De Kust.flac -> No special characters detected: Nothing to be done
I'm using v 1.6 of the script and I'm curious what causes this, and if I have to change the files manually?
[/UPDATE]
Re: [Synology NAS] Fixed foreign accents (2013/02/18)
Posted: February 19th, 2013, 4:54 am
by LapinFou
Could you contact me by PM? I need the NZB file which is failing in order to do some testing at home and to fix the problem.
Unfortunately, so far, you must fix the file through a telnet session or wait for the version 1.7 which will surely fix the problem!!
Re: [Synology NAS] Fixed foreign accents (2013/02/18)
Posted: February 19th, 2013, 9:44 am
by LapinFou
OK.
Version 1.7 is out.
The problem below is coming from the usage of
Scandinavian code page + a hole in my detection function.
The hole was punched. But unfortunately, I can not detect the difference between code 437 and 865 (Danish, Norwegian).
So, with the fix below, the character will be badly renamed. However, this time, the file will be accessible from Windows/File Station.
I hope Scandinavian people can leave with this restriction.
Depending of where your NZB source is coming from, you may replace the .decode('cp437') Python code, with the one matching your country:
Python Standard Encodings
Re: [Synology NAS] Fixed foreign accents (2013/02/19)
Posted: February 19th, 2013, 11:12 am
by LapinFou
I hope I'm done for today!!
I'm waiting StreameR feedback.
The version 1.8 is out.
In fact the problem was, I should use cp850 decoding style instead cp437.
Re: [Synology NAS] Fixed foreign accents (2013/02/19)
Posted: February 19th, 2013, 11:26 am
by StreameR
LapinFou wrote:I hope I'm done for today!!
I'm waiting StreameR feedback.
The version 1.8 is out.
In fact the problem was, I should use cp850 decoding style instead cp437.
lol, me too! Good to see the problem is solved, and glad that I could help you out there!
Keep up the good work m8!
Re: [Synology NAS] Fixed non-ASCII chars and more...(2013/02
Posted: September 1st, 2013, 3:48 am
by LapinFou
Latest version of CharTanslator script is now included into the latest SABnzbd (SynoCommunity version for Synology NAS).
Version 1.9 Update (2013/10/02)
Posted: October 2nd, 2013, 8:33 am
by LapinFou
Code: Select all
#!/usr/local/python/bin/python -OO
#-*- coding: iso-8859-15 -*-
#
# If a file has been archieved under an ISO-8859 environment and unarchived
# under an UTF8 environment, then you will get an encoding format problem.
# The file will not be readable through SAMBA.
#
# Renaming script for SABnzbd runnning under Synology NAS.
# By default the NZB software is running under UTF-8 encoding format
# in order to correctly handle the french accents (éèàç...) a SABnzbd
# post-script must be run in order to convert the file encoding.
#
# To fix this problem, you must convert the encoding format
# to the UTF8 (default Synology encoding)
# The script is trying to detect if the original file/directory are coming
# from a RAR archive. In this case the unrar command on your Syno system will
# unpack in CP850 format (DOS).
# NB: in all cases, files will be readable through samba, even if the detection
# failed. But converted characters will not be good, ex: Î? instead é
#
# Remark: I guess it should work for any other encoding style. Just replace
# ISO-8859-15 (Western Europe) by the one coresponding to your country:
# http://en.wikipedia.org/wiki/Character_encoding#Common_character_encodings
#
# Done by LapinFou
# date | version | comment
#--------------------------------------
# 12-04-22 | 1.0 | Initial version
# 12-04-22 | 1.1 | Change encoding to ISO-8859-15
# | Added CP437 special characters (0x80-0x9A)
# 12-04-24 | 1.2 | Mixed encoding is now supported
# | UTF-8 encoding format detected
# 12-04-24 | 1.3 | Fixed typo line 57 (test must be 0xA0, not 0xA1)
# 12-05-24 | 1.4 | Added an exception for "Â" character
# | Added 7z unpack
# | Added move option
# | Added Syno index option
# 13-02-15 | 1.5 | Added an option to activate Sickbear post-processing
# | More evoluate move option (merge is now managed)
# | Argv1 folder is not renamed anymore (already managed by SABnzbd)
# 13-02-18 | 1.6 | Argv1 folder is now renamed (not managed by SABnzbd)
# 13-02-19 | 1.7 | Changed CP437 detection range
# 13-02-19 | 1.8 | Changed CP437 DOS encoding style with CP850
# 13-10-02 | 1.9 | Fixed an issue with some NZB and Sickbeard option
# | In order to simplify the support, the script version is now displayed
#
# get library modules
import sys
import os
import subprocess
import shutil
scriptVersionIs = 1.9
# If empty, then no move
# Format must be synology full path (case sensitive). For ex: /volume1/video/News
MoveToThisFolder = ''
# If MoveMergeSubFolder = False, then equivalent to unix command:
# mv -rf srcFolder destFolder
# In case of conflict between an already existing sub-folder in the destination folder:
# the destination sub-folder will be replaced with source sub-folder
#
# If MoveMergeSubFolder = True, then equivalent to unix command:
# cp -rf srcFolder/* destFolder/
# rm -rf srcFolder
# In case of conflict between an already existing sub-folder in the destination folder:
# the destination sub-folder will be merged with source sub-folder (kind of incremental)
MoveMergeSubFolder = True
# /!\ IndexInSynoDLNA and SickBeardPostProcessing are exclusive
# =============================================================
# If "True", then the folder will be indexed into Synology DLNA
# By default it is "False"
IndexInSynoDLNA = False
# If "True", the folder will be send to SickBeard for Post-Processing
# By default it is "False"
SickBeardPostProcessing = False
# If "True", all .7z files will be unpacked then source .7z file will be deleted
# By default it is "False"
Unpack7z = True
########################
# ----- Functions ---- #
########################
# Special character hex range:
# CP850: 0x80-0xA5 (fortunately not used in ISO-8859-15)
# UTF-8: 1st hex code 0xC2-0xC3 followed by a 2nd hex code 0xA1-0xFF
# ISO-8859-15: 0xA6-0xFF
# The function will detect if fileDirName contains a special character
# If there is special character, detects if it is a UTF-8, CP850 or ISO-8859-15 encoding
def renameFunc(fullPath, fileDirName):
encodingDetected = False
# parsing all files/directories in odrer to detect if CP850 is used
for Idx in range(len(fileDirName)):
# /!\ detection is done 2char by 2char for UTF-8 special character
if (len(fileDirName) != 1) & (Idx < (len(fileDirName) - 1)):
# Detect UTF-8
if ((fileDirName[Idx] == '\xC2') | (fileDirName[Idx] == '\xC3')) & ((fileDirName[Idx+1] >= '\xA0') & (fileDirName[Idx+1] <= '\xFF')):
print os.path.join(fullPath, fileDirName) + " -> UTF-8 detected: Nothing to be done"
encodingDetected = True
break;
# Detect CP850
elif ((fileDirName[Idx] >= '\x80') & (fileDirName[Idx] <= '\xA5')):
utf8Name = fileDirName.decode('cp850')
utf8Name = utf8Name.encode('utf-8')
os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
print os.path.join(fullPath, utf8Name) + " -> CP850 detected: Renamed"
encodingDetected = True
break;
# Detect ISO-8859-15
elif (fileDirName[Idx] >= '\xA6') & (fileDirName[Idx] <= '\xFF'):
utf8Name = fileDirName.decode('iso-8859-15')
utf8Name = utf8Name.encode('utf-8')
os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
print os.path.join(fullPath, utf8Name) + " -> ISO-8859-15 detected: Renamed"
encodingDetected = True
break;
else:
# Detect CP850
if ((fileDirName[Idx] >= '\x80') & (fileDirName[Idx] <= '\xA5')):
utf8Name = fileDirName.decode('cp850')
utf8Name = utf8Name.encode('utf-8')
os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
print os.path.join(fullPath, utf8Name) + " -> CP850 detected: Renamed"
encodingDetected = True
break;
# Detect ISO-8859-15
elif (fileDirName[Idx] >= '\xA6') & (fileDirName[Idx] <= '\xFF'):
utf8Name = fileDirName.decode('iso-8859-15')
utf8Name = utf8Name.encode('utf-8')
os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
print os.path.join(fullPath, utf8Name) + " -> ISO-8859-15 detected: Renamed"
encodingDetected = True
break;
if (encodingDetected == False):
print os.path.join(fullPath, fileDirName) + " -> No special characters detected: Nothing to be done"
return
# scan .7z files and unpack them
def unpack7zFunc(DirName):
print "Scanning for .7z file(s), then unpack them"
print "Scanning files..."
DetectedFiles = False
for dirname, dirnames, filenames in os.walk(DirName):
for filename in filenames:
if (filename[-3:] == ".7z"):
print "Unpack %s..." %(filename)
DetectedFiles = True
try:
filepath = os.path.join(dirname, filename)
syno7z_cmd = ['/usr/syno/bin/7z', 'x', '-y', filepath]
p = subprocess.Popen(syno7z_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if (str(out) != ''):
print("7z result: " + filepath + " successfully unpacked")
os.remove(filepath)
print(filepath + " has been deleted")
if (str(err) != ''):
print("7z failed: " + str(err))
except OSError, e:
print("Unable to run 7z: "+str(e))
if DetectedFiles:
print "Scanning for .7z files Done !"
else:
print "No .7z file Detected !"
return
# add folder in the Syno index database (DLNA server)
def addToSynoIndex(DirName):
print "Adding folder in the DLNA server"
synoindex_cmd = ['/usr/syno/bin/synoindex', '-A', DirName]
try:
p = subprocess.Popen(synoindex_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if (str(out) == ''):
print("synoindex result: " + DirName + " successfully added to Synology database")
else:
print("synoindex result: " + str(out))
if (str(err) != ''):
print("synoindex failed: " + str(err))
except OSError, e:
print("Unable to run synoindex: "+str(e))
return
########################
# --- Main Program --- #
########################
print "Launching CharTranslator Python script (v%s) ..." %scriptVersionIs
print ""
# Get scripts directory of the SABnzbd from its config.ini file
if (SickBeardPostProcessing == True):
print 100*'-'
print "SickBeardPostProcessing option is ON"
print "Locating SABnzbd config.ini file..."
# Get SABnzbd rundir folder
currentFolder = os.getcwd()
# SABnzbd config.ini location
SabScriptsFolder = ''
confFile = '../../var/config.ini'
# Check that file does exit
if os.path.isfile(confFile):
SabConfigFile = open('../../var/config.ini', 'r')
# Parse each lines in order to get scripts folder path
for line in SabConfigFile.readlines():
if line[:len('script_dir')] == 'script_dir':
# Get script_dir result
SabScriptsFolder = line.split('=')[1]
# Remove 1st space + \n
if (SabScriptsFolder[0] == ' '):
SabScriptsFolder = SabScriptsFolder[1:]
SabScriptsFolder = SabScriptsFolder.replace('\n', '')
break
SabConfigFile.close
# Check that SABnzbd script folder has been found
if (SabScriptsFolder == ''):
print 100*'#'
print "SABnzbd script_dir parameter not found!"
print 100*'#'
sys.exit(1)
else:
print "SABnzbd script_dir parameter is: '%s'" %SabScriptsFolder
# Load SickBeard module
SickBeardScript = os.path.join(SabScriptsFolder, 'autoProcessTV.py')
# Check that SickBeard post-processing is present into SABnzbd scripts folder
if os.path.isfile(SickBeardScript):
sys.path.append(SabScriptsFolder)
print "Loading SickBeard 'autoProcessTV' module"
import autoProcessTV
print 100*'-'
print ""
else:
print 100*'#'
print "Unable to find SickBeard autoProcessTV.py script in folder:"
print SickBeardScript
print 100*'#'
sys.exit(1)
# Exit if the file doesn't exist
else:
print 100*'#'
print "Unable to find SABnzbd config.ini file in this folder:"
print os.path.join(currentFolder, confFile)
print 100*'#'
sys.exit(1)
# Change current directory to SABnzbd argument 1
os.chdir(sys.argv[1])
# display directory of the SABnzbd job
currentFolder = os.getcwd()
print "Current folder is " + currentFolder
# rename SABnzbd job directory (coming from SABnzbd: never in CP850 format)
print "Renaming destination folder to UTF-8 format..."
renameFunc('', currentFolder)
currentFolder = os.getcwd()
print "Destination folder renamed !"
print ""
# Unpack 7z file(s)
if (Unpack7z == True):
print 100*'-'
unpack7zFunc(currentFolder)
print 100*'-'
print ""
# process each sub-folders starting from the deepest level
print 100*'-'
print "Renaming folders to UTF-8 format..."
for dirname, dirnames, filenames in os.walk('.', topdown=False):
for subdirname in dirnames:
renameFunc(dirname, subdirname)
print "Folder renaming Done !"
print 100*'-'
print ""
# process each file recursively
print 100*'-'
print "Renaming files to UTF-8 format..."
for dirname, dirnames, filenames in os.walk('.'):
for filename in filenames:
renameFunc(dirname, filename)
print "Files renaming Done !"
print 100*'-'
print ""
# Move current folder to an another destination if the option has been configured
if (MoveToThisFolder != ''):
print 100*'-'
print "Moving folder:"
print os.getcwd()
print "to:"
print MoveToThisFolder
# Check if destination folder does exist and can be written
# If destination doesn't exist, it will be created
if (os.access(MoveToThisFolder, os.F_OK) == False):
os.makedirs(MoveToThisFolder)
os.chmod(MoveToThisFolder, 0777)
# Check write access
if (os.access(MoveToThisFolder, os.W_OK) == False):
print 100*'#'
print "File(s)/Folder(s) can not be move in %s" %(MoveToThisFolder)
print "Please, check Unix permissions"
print 100*'#'
sys.exit(1)
# If MoveMergeSubFolder is True, then move all file(s)/folder(s) to destination
# then remove source folder
destFolder = os.path.join(MoveToThisFolder, os.path.split(os.getcwd())[-1])
if (MoveMergeSubFolder):
print " Info: Merge option is ON (incremental copy)"
try:
synoCopy_cmd = ['/bin/cp', '-Rpf', os.path.abspath(currentFolder), MoveToThisFolder]
p = subprocess.Popen(synoCopy_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if (str(err) != ''):
print("Copy failed: " + str(err))
sys.exit(1)
except OSError, e:
print("Unable to run cp: " + str(e))
sys.exit(1)
os.chdir(MoveToThisFolder)
shutil.rmtree(currentFolder)
# If MoveMergeSubFolder is False, remove folder with same (if exists)
# then move all file(s)/folder(s) to destination and remove source folder
else:
print " Info: Merge option is OFF (existing data will be deleted and replaced)"
# Remove if destination already exist
if os.path.exists(destFolder):
shutil.rmtree(destFolder)
shutil.move(currentFolder, MoveToThisFolder)
os.chdir(MoveToThisFolder)
# Update currentFolder variable
os.chdir(destFolder)
currentFolder = os.getcwd()
print 100*'-'
print ""
# Add multimedia files in the Syno DLNA if the option has been enabled
if (IndexInSynoDLNA == True) & (SickBeardPostProcessing == False):
print 100*'-'
addToSynoIndex(currentFolder)
print ""
print 100*'-'
# Send to SickBeard for post-processing
elif (IndexInSynoDLNA == False) & (SickBeardPostProcessing == True):
print 100*'-'
print "Launching SickBeard post-processing..."
autoProcessTV.processEpisode(currentFolder)
print "SickBeard post-processing done!"
print ""
print 100*'-'
# Display error message + advise if both options are enabled
elif (IndexInSynoDLNA == True) & (SickBeardPostProcessing == True):
print 100*'#'
print "IndexInSynoDLNA and SickBeardPostProcessing options are exclusive"
print "Please check your configuration"
print ""
print "If you want to have both options enables at the same time, please processed as follow:"
print " 1- Enable only SickBeardPostProcessing option"
print " 2- In SickBeard GUI -> Config -> Notifications -> Enable 'Synology Indexer'"
print 100*'#'
sys.exit(1)
print ""
print "Character encoding translation done!"