Re: [Monotone-devel] Netsync performance improvement patch

[Matt Johnston]

On Fri, Aug 12, 2005 at 02:52:13PM -0700, Eric Anderson wrote:
> Eric Anderson writes:
>  > Summary: The attached patch changes the recieve buffer from a string
>  > to a string_queue.  This changes an O(n^2) algorithm to an O(n)
>  > algorithm.  The practical effect on a smallish database is a 3.48x
>  > CPU usage reduction on the pull side.  On a somewhat extreme case, it
>  > resulted in a 24.7x CPU reduction on the pull side.
> 
> Any comments from people on this fix?

The general approach is good I think - I'm not seeing such
significant improvements (119sec versus 177sec for the
a pull of a monotone tree with the check_sane_history() call
removed), but still worthwhile. I guess it depends on the
repository's ancestry structure - repositories with longer
history will be more limited by delta reconstruction etc.

Trying switching the outbuf in netsync.cc to a string_queue,
I see a slowdown on both the client and the server side. I
haven't profiled that yet, so not sure why that's happening.

I guess one possible objection to the patch is that
string_queue.hh is doing raw pointer operations, something
that most of the code in monotone avoids. It does appear to
have a fair amount of sanity checking through it - anyone
got thoughts on the matter?  Personally I think it might be
justified for a data structure like that.

I assume that a boost::circular_buffer isn't suitable since
that doesn't have contiguous storage?

Matt