PRE-processing script

[MfHYoJEcYf50]

With a preprocessing script, I could weed out both duplicate msgids from for example newzbin's RSS-feed for saved searches, and also previously downloaded msgids.

To do this, I would need a database-backend, which I understand you will implement in 0.5.0 (sqlite3).

I would use this primarily for downloading new music.

I have some database experience and am familiar with regular expressions, and can write queries for a pre processing python-script myself.

Such queries would, for my purpose of downloading new music:
* Filter out duplicates in the RSS
* Filter out previously downloaded msgids

I could also extend your database model, and add functionality that could:
* Filter out previously downloaded albums based on regex-matching
* Connect to a playlist, for example for Amarok's SQLEmbedded database, and filter out albums-that-I-have
* Connect to a playlist, for example for Amarok's SQLEmbedded database, and return a prioritization for each msgid based on the rate (1-5) that I have given an artist

I would also hope to be able to extend this to consider last.fm's recommended artists list, for further prioritization.

Ideally, I would in addition like to see SABnzbd implement prioritization, with top-prioritization per newzbin-category / newsgroups, and sub-level prioritization within each category.
That way, I could say that movies are to be downloaded with toppriority 1 and subpriority undefined, and music were to be downloaded with toppriority 2 and subpriority undefined / defined by a pre-processing script.
In which case, if an album for an artist that I have given a score of 5 shows up, I can give that msgid toppriority 1 and subpriority 1, so that I grab that immediately.

[MfHYoJEcYf50]

Ok, so no reply so far.

Maybe I threw too many ideas in there.

Anyhow, looking at the 5.0 alpha 2, you've got:
CREATE TABLE "history"
(
    "id" INTEGER PRIMARY KEY,
    "completed" INTEGER NOT NULL,
    "name" TEXT NOT NULL,
    "nzb_name" TEXT NOT NULL,
    "category" TEXT,
    "pp" TEXT,
    "script" TEXT,
    "report" TEXT,
    "url" TEXT,
    "status" TEXT,
    "nzo_id" TEXT,
    "storage" TEXT,
    "path" TEXT,
    "script_log" BLOB,
    "script_line" TEXT,
    "download_time" INTEGER,
    "postproc_time" INTEGER,
    "stage_log" TEXT,
    "downloaded" INTEGER,
    "completeness" INTEGER,
    "fail_message" TEXT,
    "url_info" TEXT,
    "bytes" INTEGER,
    "meta" TEXT
)

, which seems okay, though not normalized.

A simple python-script can go through a directory and just insert the dir-names into "name", and then with your existing option which doesn't download duplicates, that should be sufficient for my purposes.

I still think you should add a pre-processing hook though.

I would appreciate it if you could comment on this.

[shypike]

We have our minds on other stuff currently.
Like getting 0.5.0 in shape to release it.
I appreciate your concern, but I don't think we'll follow up this in the short term.
I think your ideas are very useful for a later date.

We are also looking at a plug-in system for later, but there's not much progress on that.

A pre-processing hook looks feasible.
It would be a single hook and not one per category.

[Mills]

Second on preprocessing script please!

Thanks for all your hard work,

Mills