pre-check on contents of post for unwanted content

[sander]

wait for sab 0.7.18

[sander]

SABnzbd 0.7.18RC1 has been released, with "Support for detecting unwanted extensions inside RAR files". See http://forums.sabnzbd.org/viewtopic.php?f=8&t=17705

It's under Config -> Switches ... search for 'unwanted' and fill out the action (Pause) and EXE

[gpharos]

Thank you for a job very well done! It has been driving me crazy not to mention the BW i've been using. Last 6 files downloaded had the setup.exe for add'l codec support in them. AVG would remove the threat but I was sitting here with 20+ GB of drive space used. Thank you once again for a great piece of work.

[sander]

Thank you for a job very well done! It has been driving me crazy not to mention the BW i've been using. Last 6 files downloaded had the setup.exe for add'l codec support in them. AVG would remove the threat but I was sitting here with 20+ GB of drive space used. Thank you once again for a great piece of work.

Good to hear it works for you.

We have to build up experience and verify the following hypothesis: "the unwanted extension can 99% sure be detected in the first or the last rar file".
If so, in a future release, SAB will need to be instructed in to check that first and last rar first to achieve detection as early as possible.

Maybe you can help: can you set the SAB logging to +Debug (Status -> Latest Warnings -> Logging), and post back the sabnzbd.log lines that contain "is in rar file" like this:

Code: Select all

$ grep -i "is in rar file" ~/.sabnzbd/logs/sabnzbd.log*

/home/sander/.sabnzbd/logs/sabnzbd.log.2:2014-06-30 20:57:10,973::DEBUG::[assembler:128] Unwanted extension is in rar file /home/sander/Downloads/incomplete/...LiMiTED.DVDRip.XviD-Ecf.part01.rar

/home/sander/.sabnzbd/logs/sabnzbd.log.1:2014-06-30 23:00:35,220::DEBUG::[assembler:128] Unwanted extension is in rar file /home/sander/Downloads/incomplete/...1080P.MP4.part15.rar

In the first post the EXE is in part01.rar, in the second post the EXE is in part15.rar (which is the last rar of that specific post), so confirming the hypothesis for this N=2 case.

Thanks.

[LiltTrublMakr]

The last 4 or 5 I have had with the exe extension have been in either in the first or last rar. Only one of them was in the last.

[TwoZeroOne]

This function is working perfectly. It filters out those irritating double packed movies for which you need a password. Actually it's working a bit too good. When downloading a movie with rarred subs included, it also fails.

Is it possible to either force a download manually and skip this test for one download only? Or what would be better to add a size limit. So basically if an included archive is bigger then 10MB, mark as failed, else if smaller then download it.

[shypike]

When downloading a movie with rarred subs included, it also fails.

SABnzbd already contained checks for this kind of download.
Can you email an example NZB to [email protected]?
Please add the URL of this message.

[nawo69]

The latest batch of fake R6 films all had a exe file in the last rar file.

[shypike]

My argument was that you should not check for the ".rar" extention.
You should check for ".exe" though.

[sander]

... a exe file in the last rar file.

Thank you for sharing.

[JoeMBlair]

I'm wondering if you could add like a category whitelist so it wouldn't abort/pause a download if it came from one of those categories.

[wolf5]

Maybe add a way to get the first and last RAR file to check for those EXE files before getting everything in between?

Also, could you change the feature from Extension filtering to filename filtering using regular expressions? That would be way more powerful. "*.exe|*.com" and even more advanced filters.

[sander]

FWIW:

In my personal github repository https://github.com/sanderjo/sabnzbd I've pushed the code for "Put the last rar immediately the first rar, so that unwanted extensions will get detected earlier.".

If you want to test this feature from my personal github, here's a post that has an unwanted exe in the last rar: http://www.binsearch.info/index.php?q=Chef+x264-tGc
In this 4.61 GB post, it only takes a 152 MB download (and 1 minute 11 seconds) before the unwanted extension gets detected.

I've sent a pull request to Shypike / SABnzbd: https://github.com/sanderjo/sabnzbd/com ... 1b2ffc9ca6

Logging from sabnzbd.log:

Code: Select all

2014-09-21 08:49:52,885::INFO::[__init__:626] Adding Chef.tGc (1).nzb

2014-09-21 08:51:43,314::DEBUG::[nzbstuff:839] Unwanted Extension: putting last rar after first rar
2014-09-21 08:51:43,314::DEBUG::[nzbstuff:853] Unwanted Extension: First rar at 2, Last rar at 43
2014-09-21 08:51:43,314::DEBUG::[nzbstuff:854] Unwanted Extension: Last rar is <NzbFile: filename=Chef.tGc.part42.rar, type=None>
2014-09-21 08:52:03,035::DEBUG::[assembler:367] Unwanted file Chef.tGc/Codec/Setup.exe
2014-09-21 08:52:03,037::WARNING::[assembler:128] WARNING: In "Chef x264-tGc" unwanted extension in RAR file. Unwanted file is Chef.tGc/Codec/Setup.exe 
2014-09-21 08:52:03,037::DEBUG::[assembler:129] Unwanted extension is in rar file /home/sander/Downloads/incomplete/Chef x264-tGc/CheftGc.part42.rar
2014-09-21 08:52:03,037::DEBUG::[assembler:131] Unwanted extension ... pausing

[sander]

To keep this thread uptodate, especially my last message:

In SAB 0.7.19 (released Nov 1, see http://forums.sabnzbd.org/viewtopic.php?f=8&t=18258), the feature "Earlier detection of Unwanted Extension if in last rar file" has been implemented. To achieve this, the last rar is downloaded immediately after the first rar. If there is an Unwanted Extension in a post with a large file (movie) plus some smaller files, it will get detected very early.