Compressed Headers support

[TheBiGW]

Astraweb has just implemented compressed headers. Are there any plans to implement this into SABnzbd+? This is from their website.

We have recently introduced two new commands at the server level which require news reader authors to implement in order to take advantage of the faster header downloads.
By extending the NNTP protocol and incorporating header compression directly into the server with support from news readers, you will not need to run any additional, complicated proxy style applications (accelerators). You won't need to worry about configuring proxies/accelerators to bypass firewalls.

Some advantages you will notice by using compressed headers:
- You download less data as it is all compressed. This is a big bonus for those with ISPs that have monthly data caps.
- Your downloads finish faster, giving a perceived header download speed increase between 5-15 times faster.

We are proposing two new commands as an extension to the NNTP protocol: XZVER and XZHDR as counterparts to XOVER and XHDR respectively.
These two new optional commands will appear in the list returned from the "HELP" command.


Request: XZVER [range]
Response:

224 compressed data follows
=ybegin line=128 size=-1

=yend crc=FFFFFFFF
.


Request: XZHDR header [range]
Response

221 compressed data follows
=ybegin line=128 size=-1

=yend crc=FFFFFFFF
.


Where [range] is in the standard article number range as per XOVER.
Notes:
- The multi-line responses conform to standard NNTP protocol with a single period "." on a line by itself.
- Line terminals are \r\n
- The actual XOVER / XHDR response is compressed with zlib, then yEnc encoded.
- The compression conforms with zlib's deflate() method and there is NO gzip done on it. ie. Just pure gzip'ed data with no extra padding info to make it gzip-file compliant.
- The yEnc spec requires that the "size" parameter in "=ybegin line=123 size=456" be present, but for efficiency and performance sake, the total size is not known ahead of time so it has been set to some arbitrary value that should be ignored. The yEnc CRC32 value should be sufficient to validate the payload. On top of that, zlib has builtin adler32 which is an alternate form of CRC32.


It is assumed that the implementation for XZVER be duplicated from XOVER, with two additional steps at the end, that being: yEnc decode the data, then zlib inflate the data.
============================================================

[switch]

The only use for compressed headers is when retrieving listings of what articles are in the newsgroup.

SABnzbd doesn't work in this way, it get's what articles are needed from the nzb file, so we are never retrieving the headers.

[huleboeren]

Speaking of NNTP changes
Will this affect sabnzbd?

Usenet Growth Reveals Need for 64-bit Based Article Numbering
http://www.giganews.com/news/article/64-bit-Usenet.html

[switch]

Nope, we don't use article numbers, but article-id's
http://forums.sabnzbd.org/index.php?topic=1139.0

[sab12397]

Does anyone know of a freeware Windows application that supports compressed headers?  TIA