Script to automate fixing failed downloads
Posted: July 24th, 2021, 2:07 am
I hate downloading something only to have it finish with 10 or so missing blocks. I've come to learn that if you download the same thing that was posted earlier or, strangely enough, the exact same thing from a different indexer you can combine the two (or more) into one working download.
I wrote a script to automate this. In short, when you download the same thing over and over it will rename the folders to thing.1 thing.2 and you can use these to repair your download.
Here is an example run in "normal" mode. "Normal" mode rescans all the folders and tries to repair them. Once in a blue moon sabnzbd says something failed to repair when in fact it's repairable.
Here's an example of something I fixed today (name changed). It failed with 5 blocks above, downloaded the exact same thing from a different indexer (same date etc), it failed with ~100 blocks missing but them together had enough to repair.
Here is the script if anyone wants to play with it. It's written in bash so run it with "bash whateveryounamedit.sh" or chmod it. It's possible there are some bugs but it works for metm.
Also after it successfully repairs just "retry" the first download again, it will detect as done, unrar, cleanup etc.
I wrote a script to automate this. In short, when you download the same thing over and over it will rename the folders to thing.1 thing.2 and you can use these to repair your download.
Here is an example run in "normal" mode. "Normal" mode rescans all the folders and tries to repair them. Once in a blue moon sabnzbd says something failed to repair when in fact it's repairable.
Code: Select all
attempting to repair usenet incomplete downloads... normal mode.
repairing folder './Human.Toilet.2021.iNTERNAL.720p.BluRay.x264-PooP'
You need 5 more recovery blocks to be able to repair.
done.Code: Select all
attempting to repair usenet incomplete downloads...
append 'hc' for hardcore mode, 'normal' for normal mode...
repairing folder './Human.Toilet.2021.iNTERNAL.720p.BluRay.x264-PooP'
- found './Human.Toilet.2021.iNTERNAL.720p.BluRay.x264-PooP.1'
Repair complete.Code: Select all
#!/bin/bash
# to use:
# download same release exactly as the failed one.
# make sure the nzb is named _exactly_ the same.
# if it fails too you can keep running this and downloading other nzbs until it repairs.
# do not stop this once it starts running, it may leave folders in an unwanted state.
# how it works:
# looks for directories similiar to /name /name.1 /name.2
# gets into /name, symlinks the others inside and uses all pars/rars for repair.
# does _not_ unrar on success, do that inside sab so it names/cleans up properly.
# hardcore mode tries the par2 inside the other folders in case they are different.
# normal mode only scans folders without a .1 since sab sometimes fails for no reason.
# sabnzbd incomplete folder to operate on.
INCOMPLETE="/250g/usenet/incomplete"
# nonroot user to drop permissions to when ran as root, preferably same as sabnzbd runs under.
NONROOT="poop"
[ ! -d "${INCOMPLETE}" ] && echo "'${INCOMPLETE}' not found." && exit
[ "$(whoami)" == 'root' ] && echo 'dropping root...' && su "${NONROOT}" -c "$0" "$1" && exit
[ "$1" == 'hc' ] && HARDCORE="1"
[ "$1" == 'normal' ] && NORMAL="1"
echo -n 'attempting to repair usenet incomplete downloads... '
[ -z "${HARDCORE}${NORMAL}" ] && echo '' && echo "append 'hc' for hardcore mode, 'normal' for normal mode..."
[ -n "${HARDCORE}" ] && echo 'HARDCORE MODE!'
[ -n "${NORMAL}" ] && echo 'normal mode.'
echo 'starting in 3 sec...'
sleep 3
cd "${INCOMPLETE}" || exit
find . -maxdepth 1 -type d -print0 | sort -z | while read -d '' -r DIR; do
[ "${DIR}" == '.' ] && continue
if [ -d "${INCOMPLETE}/${DIR}.1" ] && [ -z "${NORMAL}" ]; then
echo ''
echo "repairing folder '${DIR}'"
cd "${INCOMPLETE}/${DIR}" || exit
MAINPAR="$(find . -maxdepth 1 -type f -name "*.par2" | grep -vE '\.vol[0-9]{1,3}\+[0-9]{1,3}\.par2')"
[ ! -f "${MAINPAR}" ] && MAINPAR="$(find . -maxdepth 1 -type f -name "*.par2" | sort | head -n1)"
[ ! -f "${MAINPAR}" ] && echo 'setting MAINPAR failed...' && continue
MD5SUM="$(md5sum "${MAINPAR}" | cut -c1-32)"
for j in $(seq 1 9); do
if [ -d "../${DIR}.${j}" ]; then
echo "- found '${DIR}.${j}'"
ln -s "../${DIR}.${j}" "x${j}"
if [ $(find "x${j}/" -maxdepth 1 -type f -name "*.par2" | wc -l) -ge 1 ]; then
# rename potentally dupe par2 files in /x# to please par2repair
rename ".par2" ".x${j}.par2" "x${j}/"*.par2
[ -n "${HARDCORE}" ] && mv "x${j}/"*".x${j}.par2" .
fi
fi
done
if [ -n "${HARDCORE}" ]; then
# try each diff par2
for j in $(seq 1 9); do
if [ -L "x${j}" ]; then
echo "-- trying .${j} par2"
ALTPAR="$(find . -maxdepth 1 -type f -name "*.x${j}.par2" | grep -vE '\.vol[0-9]{1,3}\+[0-9]{1,3}\.x[1-9]\.par2')"
[ ! -f "${ALTPAR}" ] && ALTPAR="$(find . -maxdepth 1 -type f -name "*.x${j}.par2" | sort | head -n1)"
if [ -f "${ALTPAR}" ]; then
MD5ALT="$(md5sum "${ALTPAR}" | cut -c1-32)"
if echo "${MD5SUM}" | grep -q "${MD5ALT}" ; then
echo ' par2 is a dupe, skipping.'
else
MD5SUM="${MD5SUM} ${MD5ALT}"
par2repair "${ALTPAR}" ./*.vol*.par2 {,x[1-9]/}*.rar 2>&1 | grep -E 'more recovery blocks|Repair complete'
fi
else
echo ' setting ALTPAR failed.'
fi
fi
done
else
par2repair "${MAINPAR}" {,x[1-9]/}*.{par2,rar} 2>&1 \
| grep -E 'more recovery blocks|Repair complete'
fi
# done. move things back and remove symlinks.
for j in $(seq 1 9); do
if [ -L "x${j}" ]; then
if [ $(find -maxdepth 2 -type f -name "*.x${j}.par2" | wc -l) -ge 1 ]; then
[ -n "${HARDCORE}" ] && mv ./*".x${j}.par2" "x${j}/"
rename ".x${j}.par2" ".par2" "x${j}/"*.par2
fi
rm "x${j}"
fi
done
elif [ ! -d "${INCOMPLETE}/${DIR}.1" ] && [ $(find "${INCOMPLETE}/${DIR}" -maxdepth 1 -type f -name "*.par2" | wc -l) -ge 1 ] && [ -n "${NORMAL}" ]; then
echo ''
echo "repairing folder '${DIR}'"
cd "${INCOMPLETE}/${DIR}" || exit
par2repair ./*.par2 ./*.rar 2>&1 | grep -E 'more recovery blocks|Repair complete'
fi
done
echo ''
echo 'done.'