speed up gnulib-tool a bit

[Ralf Wildenhues]

Hello Bruno, all,

For some invocations, gnulib-tool is rather slow[1], which impacts
productivity a bit.  The following tries to improve things without
sacrificing portability, while hopefully it doesn't compromise
readability.  ;-)

First, `sort -u' is already used in gnulib-tool, so it would seem only
consistent to drop `uniq' where possible, even as it does not lead to a
noticeable speedup.

Then, the transitive closure algorithm need not walk any module names in
the dependency graph more than once.  (Looking up and checking a module
is rather expensive due to the file operations and fork&exec involved.)

Also, the rewriting of file names may be done in batch.  Note that this
may cause the loop that calls `func_add_or_update' to be executed in a
subshell, but this shouldn't matter: the function does not change any
state.  (Should I provide a similar change for func_create_testdir?)

And the include directives need only be read once.

What do you think?

Cheers,
Ralf

[1] Times taken on a modern system with:
  time gnulib-tool --import acl alloca argmatch assert backupfile \
    base64 c-strtold calloc canon-host canonicalize chown config-h \
    closeout cycle-check d-ino d-type diacrit dirfd dirname error \
    euidaccess exclude exitfail fcntl fcntl-safer fdl fileblocks  \
    filemode filenamecat fnmatch-gnu fprintftime fsusage ftruncate \
    fts getdate getgroups getline getloadavg getndelim2 getopt \
    getpagesize gettext gettime gettimeofday getugroups getusershell \
    group-member hard-locale hash hash-pjw host-os human idcache \
    inttostr inttypes lchmod lchown lib-ignore

before:
43.07user 25.70system 1:00.87elapsed 112%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+5660467minor)pagefaults 0swaps

after:
18.70user 9.34system 0:26.49elapsed 105%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1995848minor)pagefaults 0swaps

PS: For the bootstrapping of gettext, it saves roughly 40s.

PPS: should the number of files used by a project exceed a few hundred,
it may make sense to exploit bash3's `+=' with a func_append, in order
to avoid the quadratic scaling associated with string enlargement:
http://lists.gnu.org/archive/html/libtool-patches/2006-05/msg00016.html

sort-u.diff
Description: Text document

transitive.diff
Description: Text document

files-rewrite.diff
Description: Text document

includes.diff
Description: Text document