Re: libtool optimization

[Kyle Sallee]

I recently tested dash for other purposes than libtool.
A statically linked dash shell can be started repeatedly
to execute a tiny scripts that contain only shell syntax
at the amazing rate of 6 times what can be accomplished
with a dynamically linked bash.
However, when executing a script of small size
that runs for a moderate duration and is almost entirely sh syntax
then dash was not even twice as fast as bash.
What others receive as test results will vary, naturally.
However, I expect the conclusion would be that
dash is a good startup performer,
but is nearly as slow in execution as bash.

The reason that the old stable branch of libtool was slow
was not so much an effect of the interpreter.
There were some very unfortunate use of case statements
which made the shell run very slow.
But the largest impact was because libtool would
invoke sed and other helper program 300 some times
if necessary to create an object file.
Or was it 3000 times?
It invoked sed plenty during compilation of koffice.
Testing determined that installing a statically linked sed
expedited old libtool, merely because statically linked
sed starts faster than dynamically linked sed
and was invoked frequently by libtool.

I have no speed tests for the new stable libtool.
But as far as the old stable libtool is concerned
using a faster shell is like crying tears in the rain.
No one would notice.

However, the lack of certain substitution functions in dash
such as not supporting ${VARIABLE:0:15}
in some cases might require that the missing
functionality be supplied by external programs.
However, even a single invocation of a helper program such
as sed could cost a great deal of performance.

Consequently, the optimal use of a shell like dash is probably
to invoke tiny scripts that can accomplish all the tasks by using
only the functionality provided by dash syntax
instead of invoking other programs such as sed, cat, cut, tr.
Consequently, re-hacking a program as large and complex as libtool
to the syntax of dash could lead to less performance than using
a more feature rich shell such as bash.

A libtool designed for a faster shell performing slower may be
counter-intuitive?
Consequently, that realization might only be
discovered after hacking the code considerably.
Therefore, I wanted to add the above information to save
someone time and effort if anyone was about to try it.

Rehacking libtool to support a faster shell is a good idea.
But that should be done last after all the libtool shell syntax
is optimized and invocation of external programs is minimized.
Once libtool is properly optimized
then the impact of shell interpreter startup and execution speed
will have a significant impact upon libtool speed.

Lastly, I want to put greater emphasis on what Mr. Michel Briand stated.

Michel Briand has a good point about dash not supporting:
> VAR+="val"

Dash also does not support array variables which might be even
faster for storing and accessing data instead of creating super long strings
using VAR+=" val"

Using case statements on long strings was a big performance hit with
old libtool.
If the string was long enough
then even an invocation to grep could have been faster.
In some situations long strings may need
to be grepped or sedited to have some
portion extracted or replaced.
Having variables stored in array makes finding and modifying data faster.

bash also support [[ =~ ]] syntax for locating a string or substring
in both strings and arrays
It may be bashism, but it is very useful.

Also dash syntax does not support ${VARIABLE//old_val/new_val} substitutions.

Without the ${VARIABLE:x:y} and ${VARIABLE//old/new}
and VAR+=" val" and arrays
handling data in dash tends to be slower and more complex than bash
and solutions may tempt programmers to invoke sed, cut, tr for even the
most trivial tasks that can be handled easily internally with a shell like bash.

The faster shell might not create the faster libtool program.
The eventual optimizing of libtool
to a feature rich shell such as bash while using bashism and arrays might
produce a faster libtool than a libtool optimized for dash.


On Sat, Oct 25, 2008 at 3:56 AM, Michel Briand <address@hidden> wrote:
>
> Bob Friesenhahn <address@hidden> - Wed, 22 Oct 2008
> 09:35:26 -0500 (CDT)
>
>>On Wed, 22 Oct 2008, Michel Briand wrote:
>>> You don't see my point. In autogen I want to have a way to select a
>>> different shell (for configure & libtool). I'm not saying that I'll
>>> choose a shell that fails to run the tools or that I'll not ask the
>>> user for the best shell available...
>>
>>As Ralph detailed earlier, it is possible for the end user to specify
>>the shell to use.  It is actually pretty challenging to know which
>>shell on a system works since they are often riddled with bugs or
>>obscure behavior.  Sometimes it seems like things works but wrong
>>results are discovered later.  Even though versions of bash also have
>>bugs, it is easier to understand the bugs in one implementation so
>>autoconf currently checks for bash first and then sticks with it if it
>>is available.
>>
>>The shell selection functionality is part of Autoconf and some of the
>>world's most experienced developers have been involved in designing
>>the selection logic.
>>
>
> I agree with that.
>
> The good way is to give the choice for a quicker shell, given that it
> will not break the scripts. Adding an option to choose the shell that
> would be easily understandable by average user.
>
>>> Furthermore you seems to indicate that Dash would not run configure ?
>>
>>In my testing, dash is a real speed daemon and does usually work to
>>perform normal package configuration.  However, I learned the hard way
>>that wrong things could happen if dash was used by the maintainer of a
>>package.
>>
>
> Having tested with /bin/dash, I've noted that I do not want to use
> this kind of construct in configure.ac:
>
> VAR+="val"
>
> It doesn't work with dash...
>
> Cheers,
> Michel
>
>
>
> _______________________________________________
> http://lists.gnu.org/mailman/listinfo/libtool
>