Re: [Monotone-devel] Re: results of mercurial user survey

[Richard Levitte - VMS Whacker]

In message <address@hidden> on Sun, 30 Apr 2006 19:58:14 -0700, Graydon Hoare 
<address@hidden> said:

graydon> Richard Levitte - VMS Whacker wrote:
graydon> > lapo> IMHO the data stream should contain uncompressed data and be
graydon> > lapo> compressed at the stream level.
graydon> > 
graydon> > Exactly.
graydon> 
graydon> I agree that you'll probably win by doing full stream
graydon> compression. I'll offer a couple points in my defense to
graydon> explain why I did it this way.
graydon> 
graydon>    - A reasonable chunk of the protocol stream contains
graydon> incompressible material: high-entropy cryptographic stuff or
graydon> merkle trie nodes.

True, but you don't lose much by having them in the stream.

graydon>    - Mixing a streaming interface with pipelined and
graydon> multiplexed i/o might not be quite as easy as you like. In
graydon> particular: you might find that you have to add some logic
graydon> for flushing gzip's state down at the bottom of the probe
graydon> loop if the sending side has gone dead (and this might
graydon> possibly introduce bubbles into the pipeline).

Uhmm, not sure where you're going there.  I imagine that each session
would have its own individual zlib stream to work with.  It's possible
that a bit of extra flushing would be desirable, and in that case,
just introduce it!

I see another huge benefit with this move: it would most probably
avoid creating large blobs of memory to contain the gzipped data.
That in itself might prove to be a win.

graydon> Those points aside, I certainly wouldn't *stop* anyone from
graydon> trying it. I just didn't want to deal with such issues when I
graydon> first wrote it, so I didn't bother.

I'll see if I can give it some time this week.

Cheers,
Richard

-----
Please consider sponsoring my work on free software.
See http://www.free.lp.se/sponsoring.html for details.

-- 
Richard Levitte                         address@hidden
                                        http://richard.levitte.org/

"When I became a man I put away childish things, including
 the fear of childishness and the desire to be very grown up."
                                                -- C.S. Lewis