Page 3 of 3
Re: pre-check on contents of post for unwanted content
Posted: June 22nd, 2014, 3:14 am
by sander
wait for sab 0.7.18
Re: pre-check on contents of post for unwanted content
Posted: June 30th, 2014, 2:02 pm
by sander
SABnzbd 0.7.18RC1 has been released, with "Support for detecting unwanted extensions inside RAR files". See
http://forums.sabnzbd.org/viewtopic.php?f=8&t=17705
It's under Config -> Switches ... search for 'unwanted' and fill out the action (Pause) and EXE
Re: pre-check on contents of post for unwanted content
Posted: July 1st, 2014, 3:27 pm
by gpharos
Thank you for a job very well done! It has been driving me crazy not to mention the BW i've been using. Last 6 files downloaded had the setup.exe for add'l codec support in them. AVG would remove the threat but I was sitting here with 20+ GB of drive space used. Thank you once again for a great piece of work.
Re: pre-check on contents of post for unwanted content
Posted: July 1st, 2014, 3:54 pm
by sander
gpharos wrote:Thank you for a job very well done! It has been driving me crazy not to mention the BW i've been using. Last 6 files downloaded had the setup.exe for add'l codec support in them. AVG would remove the threat but I was sitting here with 20+ GB of drive space used. Thank you once again for a great piece of work.
Good to hear it works for you.
We have to build up experience and verify the following hypothesis: "the unwanted extension can 99% sure be detected in the first or the last rar file".
If so, in a future release, SAB will need to be instructed in to check that first and last rar first to achieve detection as early as possible.
Maybe you can help: can you set the SAB logging to +Debug (Status -> Latest Warnings -> Logging), and post back the sabnzbd.log lines that contain "is in rar file" like this:
Code: Select all
$ grep -i "is in rar file" ~/.sabnzbd/logs/sabnzbd.log*
/home/sander/.sabnzbd/logs/sabnzbd.log.2:2014-06-30 20:57:10,973::DEBUG::[assembler:128] Unwanted extension is in rar file /home/sander/Downloads/incomplete/...LiMiTED.DVDRip.XviD-Ecf.part01.rar
/home/sander/.sabnzbd/logs/sabnzbd.log.1:2014-06-30 23:00:35,220::DEBUG::[assembler:128] Unwanted extension is in rar file /home/sander/Downloads/incomplete/...1080P.MP4.part15.rar
In the first post the EXE is in part01.rar, in the second post the EXE is in part15.rar (which is the last rar of that specific post), so confirming the hypothesis for this N=2 case.
Thanks.
Re: pre-check on contents of post for unwanted content
Posted: July 6th, 2014, 6:18 am
by LiltTrublMakr
The last 4 or 5 I have had with the exe extension have been in either in the first or last rar. Only one of them was in the last.
Re: pre-check on contents of post for unwanted content
Posted: July 12th, 2014, 4:18 am
by TwoZeroOne
This function is working perfectly. It filters out those irritating double packed movies for which you need a password. Actually it's working a bit too good. When downloading a movie with rarred subs included, it also fails.
Is it possible to either force a download manually and skip this test for one download only? Or what would be better to add a size limit. So basically if an included archive is bigger then 10MB, mark as failed, else if smaller then download it.
Re: pre-check on contents of post for unwanted content
Posted: July 12th, 2014, 5:04 am
by shypike
TwoZeroOne wrote:When downloading a movie with rarred subs included, it also fails.
SABnzbd already contained checks for this kind of download.
Can you email an example NZB to
[email protected]?
Please add the URL of this message.
Re: pre-check on contents of post for unwanted content
Posted: July 20th, 2014, 4:27 am
by nawo69
The latest batch of fake R6 films all had a exe file in the last rar file.
Re: pre-check on contents of post for unwanted content
Posted: July 20th, 2014, 11:20 pm
by shypike
My argument was that you should not check for the ".rar" extention.
You should check for ".exe" though.
Re: pre-check on contents of post for unwanted content
Posted: July 21st, 2014, 12:13 am
by sander
nawo69 wrote:... a exe file in the last rar file.
Thank you for sharing.
Re: pre-check on contents of post for unwanted content
Posted: July 22nd, 2014, 3:36 pm
by JoeMBlair
I'm wondering if you could add like a category whitelist so it wouldn't abort/pause a download if it came from one of those categories.
Re: pre-check on contents of post for unwanted content
Posted: July 26th, 2014, 12:57 pm
by wolf5
Maybe add a way to get the first and last RAR file to check for those EXE files before getting everything in between?
Also, could you change the feature from Extension filtering to filename filtering using regular expressions? That would be way more powerful. "*.exe|*.com" and even more advanced filters.
Re: pre-check on contents of post for unwanted content
Posted: September 21st, 2014, 1:32 am
by sander
FWIW:
In my personal github repository
https://github.com/sanderjo/sabnzbd I've pushed the code for "Put the last rar immediately the first rar, so that unwanted extensions will get detected earlier.".
If you want to test this feature from my personal github, here's a post that has an unwanted exe in the last rar:
http://www.binsearch.info/index.php?q=Chef+x264-tGc
In this 4.61 GB post, it only takes a 152 MB download (and 1 minute 11 seconds) before the unwanted extension gets detected.
I've sent a pull request to Shypike / SABnzbd:
https://github.com/sanderjo/sabnzbd/com ... 1b2ffc9ca6
Logging from sabnzbd.log:
Code: Select all
2014-09-21 08:49:52,885::INFO::[__init__:626] Adding Chef.tGc (1).nzb
2014-09-21 08:51:43,314::DEBUG::[nzbstuff:839] Unwanted Extension: putting last rar after first rar
2014-09-21 08:51:43,314::DEBUG::[nzbstuff:853] Unwanted Extension: First rar at 2, Last rar at 43
2014-09-21 08:51:43,314::DEBUG::[nzbstuff:854] Unwanted Extension: Last rar is <NzbFile: filename=Chef.tGc.part42.rar, type=None>
2014-09-21 08:52:03,035::DEBUG::[assembler:367] Unwanted file Chef.tGc/Codec/Setup.exe
2014-09-21 08:52:03,037::WARNING::[assembler:128] WARNING: In "Chef x264-tGc" unwanted extension in RAR file. Unwanted file is Chef.tGc/Codec/Setup.exe
2014-09-21 08:52:03,037::DEBUG::[assembler:129] Unwanted extension is in rar file /home/sander/Downloads/incomplete/Chef x264-tGc/CheftGc.part42.rar
2014-09-21 08:52:03,037::DEBUG::[assembler:131] Unwanted extension ... pausing
Re: pre-check on contents of post for unwanted content
Posted: November 15th, 2014, 4:49 am
by sander
To keep this thread uptodate, especially my last message:
In SAB 0.7.19 (released Nov 1, see
http://forums.sabnzbd.org/viewtopic.php?f=8&t=18258), the feature "Earlier detection of Unwanted Extension if in last rar file" has been implemented. To achieve this, the last rar is downloaded immediately after the first rar. If there is an Unwanted Extension in a post with a large file (movie) plus some smaller files, it will get detected very early.