Re: Failing to use gnulib bootstrap in libtool

[Eric Blake]

On 09/02/2010 10:36 AM, Gary V. Vaughan wrote:
> Is gnulib bootstrap designed for reuse in other projects?

Yes; I know that it is already shared among coreutils, grep, libvirt, 
and several other projects.  However, improvements are certainly welcome.

I'll tackle the questions I know an answer to, but since I don't (yet) 
use bootstrap on m4, I'm not the best person to answer.

> 2. source_base, m4_base etc
> ===========================
>
> Why hardcode another copy of these locations into bootstrap, where they can 
> fall out of sync.  Far better to either extract them from or inject them into 
> `gnulib-cache.m4'.  Extracting them with `autom4te --language=Autoconf 
> --trace=gl_SOURCE_BASE --trace=gl_M4_BASE ... configure.ac' seems like the most 
> robust way of tackling this problem to me.

Seems reasonable to me, as well.  Do you have patches in mind?

> 4. bootstrap.conf
> =================
>
> Why is bootstrap.conf called below the definition of find_tool(), but the other 
> functions
> are redefined much later on.  It would be far more flexible to encapsulate most 
> of the
> functionality of `bootstrap' into shell functions (preferably well commented so 
> that they
> can be easily understood!) before sourcing `bootstrap.conf' so that those 
> functions could
> be overridden safely before they are called by the remaining code.

Indeed, that does sound like a more flexible approach for allowing 
package-specific overrides.

> I don't like the way gnulib `bootstrap' forces a `gnulib-tool --import' 
> invocation on every
> run, when a `gnulib-tool --update' is quite sufficient most of the time.  If 
> the calls
> to gnulib-tool were in a function set before sourcing bootstrap.conf, I could 
> redefine the
> function for Libtool without upsetting other users of the script by fighting 
> for a change
> of semantics.

This is also my biggest pet peeve with bootstrap; and why I don't yet 
use it for m4; I find the gnulib-tool --update to be much faster 
(especially on Cygwin).


> 5. option parsing
> =================
>
> The option parse loop is not as robust as it could be: For example, it doesn't 
> support
> `./bootstrap -fc --gnulib-srcdir ../gnulib' for a couple of reasons.
>
> I wonder whether you would accept a generated script to check in to the gnulib 
> repo,
> along with a few lines of shell and an extra m4sh file to regenerate that file 
> whenever
> the source is edited?

Having bootstrap be generated seems reasonable (we maintain a couple 
other generated files in gnulib, such as INSTALL being generated from 
install.texi as copied from autoconf) - as long as the Makefile rules 
are in place, then it seems okay to do commits that touch a master file 
and update bootstrap to match; all other projects will just be copying 
bootstrap (and not any upstream file).

> 7. buildreqs
> ============
>
> Why do we have to keep 2 copies of this information in sync?  `bootstrap' can
> easily extract the prequisite versions of Autoconf, Automake, Libtool and
> Gettext from configure.ac for us.  I didn't have this in my bootstrap script
> so I don't have code to paste in, but it seems like it would just be a few more
> lines in `extract_package_values'.

For the autotools, yes, bootstrap can extract prereq versions.  But for 
other tools (think bison, help2man, etc.), that's something that 
bootstrap can't extract.  So, as long as buildreqs have to support 
minimum versions (or - if it is expected that any version will do), no 
one has bothered to automate minimum version detection for the subset of 
tools where configure.ac already contains that information.

> 8. gnulib submodule
> ===================
>
> > git_modules_config () {
> >    test -f .gitmodules&&  git config --file .gitmodules "$@"
> > }
> >
> > gnulib_path=`git_modules_config submodule.gnulib.path`
> > : ${gnulib_path=gnulib}
>
> So when I don't have .gitmodules, gnulib_path is set to the empty string, which
> prevents the follow line from falling back to a default value.

There's already outstanding issues that gnulib submodule handling needs 
to be made smarter for people who do NOT have git installed, but DO have 
a snapshot of the correct gnulib version accessible.  This whole area 
needs some loving care.

> Actually, I can't follow the logic of the case statement that follows, although
> I think it is not supposed to do nothing at all when I have no `.gitmodules' 
> file,
> and pass `--gnulib-srcdir=../gnulib' to try and get a reference submodule 
> installed.
>
> The gnulib submodule code in GNU M4 is, by comparison, easy to follow... and it
> works.  Aside from adding enough comments to make the logic understandable, and
> fixing it to work in the common case of `--gnulib-srcdir=../gnulib' with no 
> `.gitmodules'
> file, I think the whole thing should be put inside a function before 
> `bootstrap.conf',
> so that I can override it with my preferred semantics.

To some degree, recent gnulib-srcdir changes in bootstrap are due to my 
desire to copy m4's bootstrap logic.

> And why does GNULIB_SRCDIR contain the location of the referenced gnulib tree
> for the first half of the script, and then get reused to hold the location of 
> the
> submodule itself for the rest of the script?  `$gnulib_path' seems to hold the
> right value already, so there's no need to change the meaning of `GNULIB_SRCDIR'
> partway down.

Yeah, stuff like that is what makes it hard to maintain in the first place.

> 11. gnulib_tool_options
> =======================
>
> This is broken quite badly:
>
>    `--import': Most of the time I'd rather use `--update', but there's no way
>         to override it without using a gnulib-local patch (and I couldn't even
>         get that to work for `bootstrap' since it's not even in a module).
>    `--lib $gnulib_name' (and others): where `gnulib_name=lib$package' by 
> default.
>         Surely most people want to call it `libgnu' (not `liblibtool'!)?  And in
>         any case, these are already stored in gnulib-cache.m4, so we shouldn't
>         have to keep two references in sync.

That's one of the problems I see with bootstrap - currently, none of the 
projects using gnulib's bootstrap are storing gnulib-cache.m4 in VCS. 
gnulib-tool --update only works if gnulib-cache.m4 is stored in VCS (as 
it is with M4).  If others don't mind the paradigm shift towards storing 
_both_ bootstrap and gnulib-cache.m4 in their project, then we could 
make bootstrap use the --update option.

> 12. AUTOHEADER and lack of AUTORECONF
> =====================================
>
> Autoreconf does a better job here too.  I suggest replacing this entire block
> with:
>
>      ${AUTORECONF-autoreconf} $autoreconf_flags

The biggest issue here is that gettextize and gnulib-tool do not (yet) 
play well together; but there are also some libtool interplay issues. 
M4's bootstrap script gets around this by running gettextize first, then 
gnulib-tool, and finally AUTOPOINT=true autoreconf.  gnulib bootstrap 
gets around this by running the tools separately into two different 
directories (tmp1 for gnulib-tool, tmp2 for gettext), then merging the 
results while favoring gnulib-tool.  I don't know if one approach is 
inherently better than the other, but do know that M4's bootstrap is faster.

--
Eric Blake   address@hidden    +1-801-349-2682
Libvirt virtualization library http://libvirt.org