Re: [PATCH 0/5] speed up gnulib-tool some more

[Bruno Haible]

Hello Ralf,

Ralf Wildenhues wrote:
> This patch series is a repost and update of the unapplied parts of
> <http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/16147>
> plus a couple of new patches; prompted by
> <http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/20351/focus=20370>.

I'm glad that you addressed some of the issues that I mentioned in
<http://lists.gnu.org/archive/html/bug-gnulib/2009-01/msg00008.html>.

> The first two patches, taken together, have a net effect of 10-20%
> speedup on normal gnulib-tool usage.  The third patch is a minor
> peephole optimization.

Here are the times of a full "gnulib-tool --test --with-tests" run
that I measured on a Linux/glibc platform:

current git:
real    1020.054s
user     514.568s
sys      506.424s

patch 1:
real    1499.414s
user     805.882s
sys      728.198s

patches 1 + 2:
real     847.445s
user     513.072s
sys      408.298s

patches 1 + 2 + 3:
real     844.850s
user     509.708s
sys      405.389s

patches 1 + 2 + 3 + 4:
real     517.247s
user     362.823s
sys      155.662s

So, all together, it's a 2x speedup again! (after some of your speedups
were already integrated a year ago.)

I'll look into these in detail.

In particular, patch 1 and 2 each removed many blank lines from the output.
One needs to verify whether this is harmless.

Regarding part 1, I would like to put in a command-line option --no-cache
that preserves the old, slow but simpler code. It is good practice, for
every nontrivial optimization, to have a way to disable the optimization.
This helps in two situations:
  - When a bug crops up, and you want to know whether the cache introduced it.
  - When in the future, the optimization turns out to be a blocker for new
    developments that were not known at this time.

Regarding part 1 also, I would like to investigate the possible speedup that
use of the new associative arrays of bash 4 can bring. See
  <http://www.gnu.org/software/bash/manual/html_node/Arrays.html>
  <http://tldp.org/LDP/abs/html/bashver4.html>

> It exposes a segfault with (unmaintained) pdksh on GNU/Linux.  I don't
> think we need to cater to this.

Yes, agreed.

> IRIX 6.5 sed has too many problems for 
> gnulib-tool both without and with this patch series, but using GNU sed
> there seems to work fine.

Can you put this info into the doc somewhere?

> I'm willing to rework the patches for suggested/required changes

Me too. But I've run out of time for today.

Bruno