[Monotone-devel] Re: results of mercurial user survey

[Graydon Hoare]

Richard Levitte - VMS Whacker wrote:
> lapo> IMHO the data stream should contain uncompressed data and be
> lapo> compressed at the stream level.
>
> Exactly.

I agree that you'll probably win by doing full stream compression. I'll 
offer a couple points in my defense to explain why I did it this way.

  - A reasonable chunk of the protocol stream contains incompressible 
material: high-entropy cryptographic stuff or merkle trie nodes.

  - Mixing a streaming interface with pipelined and multiplexed i/o 
might not be quite as easy as you like. In particular: you might find 
that you have to add some logic for flushing gzip's state down at the 
bottom of the probe loop if the sending side has gone dead (and this 
might possibly introduce bubbles into the pipeline).

Those points aside, I certainly wouldn't *stop* anyone from trying it. I 
just didn't want to deal with such issues when I first wrote it, so I 
didn't bother.

> lapo> BTW: what about bzip2?
>
> Does it come with a nice API similar to zlib?

You don't want to use bzip2 for this. It'll chew too much CPU and 
intermediate storage to make it worth any compression gain.

-graydon