Re: Performance issue of libtool-2.4.4

[Gary V. Vaughan]

Hi Richard,

On Feb 10, 2015, at 10:35 AM, Richard Purdie <address@hidden> wrote:
> On Mon, 2015-02-09 at 23:36 +0000, Richard Purdie wrote:
>> On Mon, 2015-02-09 at 13:05 +0000, Richard Purdie wrote:
>>> In an effort to get to the bottom of this I made a git bisection, timing
>>> the performance of building xz with make -j1 using each different
>>> libtool.
>>> 
>>> The issues come down to this commit:
>>> 
>>> http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
>>> 
>>> libtool: rewritten over funclib.sh instead of general.m4sh.
>>> 
>>> Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
>>> in the fix in master mentioned above, I get 27s.
>>> 
>>> So whilst things are better (thanks!), the above change is still causing
>>> a regression in the performance somewhere else. Any ideas what else in
>>> that rather large change may be causing this?
>> 
>> To further narrow this down, of the changes in the above commit, the
>> problem appears to be in the changes to the option parsing code. I've
>> included the diff below which if I apply on top of the above, I get the
>> speed back. I've left the func_split_short_opt/func_split_long_opt code
>> in there but that is worth a tiny part of the speed, the issues are
>> around the addition of the func_options call.
>> 
>> As yet I don't know enough about the code in question to know why this
>> is an issue but traces of libtool show a lot more looping in code to do
>> with argument parsing and quoting.
> 
> To be more specific, if I take my "good" libtool and add:
> 
> func_options_prep ${1+"$@"}
> 
> it slows the build down by 0.5s on a 21s build. If I look at
> func_options_prep and comment out the line:
> 
>    func_run_hooks func_options_prep ${1+"$@"}
> 
> I get the 0.5s back.
> 
> In func_run_hooks, if I comment:
> 
>    func_quote_for_eval ${1+"$@"}
>    func_run_hooks_result=$func_quote_for_eval_result
> 
> I get the 0.5s back. The issue is all the quoting of the various return
> values through all this looping. It doesn't appear to be hitting the
> printf/sed in func_quote_for_eval which would be an obvious slow path,
> its just the shear number of loops run through with the commandline
> arguments. The change adds a number of calls to func_run_hooks, not just
> the single test case I have above and all combined, it slows things down
> significantly.
> 
> So is there a way we can change things so its not calling
> func_quote_for_eval all the time with all the looping that entails?

One possibility would be to add a post-processing script that rewrites
the hook functions used by func_options into a a single top-level blob of
sequential shell code.  I imagine that adding some carefully chosen comment
strings would make extracting the right parts in the right order relatively
straight forward... it might even be that inlining func_options_prep and
any hook functions it calls would be enough?

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)