Automatically testing video's for content

[ojilles]

Having had some trouble lately with posts that contain invalid video files (either with Codec virusses or without) I was thinking that one possible solution would be hooking up ffmpeg/ffprobe in a post processing script. If that can't determine the container/codecs in use, ditch the download.

Has anyone tried this?

(Sorry, a forum search didn't turn up anything relevant -- mostly just conversion scripts to mp4, etc.)

[sander]

Having had some trouble lately with posts that contain invalid video files (either with Codec virusses or without) I was thinking that one possible solution would be hooking up ffmpeg/ffprobe in a post processing script. If that can't determine the container/codecs in use, ditch the download.

Has anyone tried this?

(Sorry, a forum search didn't turn up anything relevant -- mostly just conversion scripts to mp4, etc.)

The upcoming SAB 0.7.17 will contain a feature to scan for unwanted content based on the file extension (so: .exe). See http://forums.sabnzbd.org/viewtopic.php ... t=unwanted

A postprocessing script can indeed use ffprobe to determine things, but a postprocessing script can't "ditch" a download; it can however tell the result (for example "avi is not a valid format")
Is that OK for you?

[ojilles]

The problem is, extensions don't say much. I've had a bunch of downloads that were avi's as extension, but trying to play them with f.ex. VLC didn't work. Inspecting them with ffprobe told me it wasn't much of a video at all. Not sure about the intent of the uploader, but I'd like to weed these out somehow.

[sander]

Not sure about the intent of the uploader, <snip>

Spreading their malware.

[sander]

Here's a post-processing script for your needs.

Code: Select all

#!/usr/bin/env python

# A SABnzbd post-processing script to check if files with video format extensions
# indeed contain video format.
# The tool 'ffprobe' is needed. On Ubuntu it's part of ffmpeg

import sys
import os
from os import walk

if len(sys.argv) < 2:
    sys.exit('Usage: %s directory-name' % sys.argv[0])

dirname = sys.argv[1]

if not os.path.exists(dirname):
    sys.exit('ERROR: Directory not found' % sys.argv[1])

if os.popen('ffprobe -version 2>&1 ').readline().find('ffprobe version') < 0 :
	sys.exit('ffprobe not found. Install it first.')


# Extensions to be scanned (note the dot at the beginning):
extlist = [ '.avi', '.mpg', '.mpeg', '.divx', '.xvid' ]

for path, subdirs, files in os.walk(dirname):
    for file in files:
        extension = os.path.splitext(file)[1].lower()
        if extension in extlist:
                fullfilename = os.path.join(path, file)
		cmd = 'ffprobe ' + fullfilename + ' 2>&1 '
		for thisline in os.popen(cmd).readlines():
			# Invalid data found when processing input ... is not good
			# Metadata ... is good
			if thisline.find('Invalid data') >= 0:
				exitmessage = "No valid video format in " + file
				sys.exit(exitmessage)

# All well
sys.exit(0)

In case of no video format found, SABnzbd will say "Exit(1) No valid video format in <file name>".

IMHO the "unwanted exentsions" feature in the upcomping SAB 0.7.18 will help you more; the goal of fake-uploaders is to spread their viruses in .EXE files, which they can't hide. On the other side they can always put in a real video format (although fake content) in their .AVI.
Anyway, maybe the script will fulfill your need for now.

[ojilles]

Thanks, I'll give that a shot!

[sander]

SABnzbd 0.7.18RC1 has been released, with "Support for detecting unwanted extensions inside RAR files". See viewtopic.php?f=8&t=17705

It's under Config -> Switches ... search for 'unwanted' and fill out the action (Pause) and EXE