Re: more problems with --create-testdir and --add-import

[Bruno Haible]

Hello Werner,

The differences between --create-testdir and --add-import are
explained by their different use cases. The --help text gives
a brief summary already:

      --import              import the given modules into the current package
      --add-import          augment the list of imports from gnulib into the
                            current package, by adding the given modules;
                            if no modules are specified, update the current
                            package from the current gnulib

      --create-testdir      create a scratch package with the given modules

For --import and --add-import, the situation is that the developer is working
on his package, and gnulib-tool has to look at the existing infrastructure
and command-line options like --source-base, so that the result fits the
taste and wishes of the developer.

For --create-testdir, the situation is that a gnulib maintainer, or sometimes
a developer, want to check the code in gnulib in a way that detects as much
portability problems as possible in advance. It creates a complete package,
meant to be transported to a build machine and built there.

Now regarding the differences:
> Doing
> 
>   gnulib-tool --create-testdir --dir=src/libs/gl wcwidth
> 
> creates the following directory structure:
> 
>   src/libs/gl/build-aux
>   src/libs/gl/gllib
>   src/libs/gl/glm4
> 
> However, doing
> 
>   gnulib-tool --add-import --dir=src/libs/gl wcwidth
> 
> afterwards, I have two new directories:
> 
>   src/libs/gl/lib
>   src/libs/gl/m4
> 
> Obviously, the `gllib' and `glm4' directories created during the
> `--create-testdir' incantation have the wrong `gl' prefix.

In --import / --add-import modes, default settings for --source-base
and --m4-base have been provided that match what the majority of
GNU packages use: lib/ for the C source code from gnulib, and m4/ for
the autoconf macros.

In --create-testdir mode, a different prefix is used, so as to verify
that no module nor gnulib-tool accidentally hardcodes 'lib' or 'm4'.
In the early days of gnulib, this bug existed in at least one module.

> Comparing the contents of the gnulib directory created by
> 
>   gnulib-tool --create-testdir \
>               --dir=src/libs/gl \
>               wcwidth
> 
> and
> 
>   gnulib-tool --add-import \
>               --dir=src/libs/gl \
>               --source-base=gllib \
>               --m4-base=glm4 \
>               wcwidth
> 
> I see four new files in `m4':
> 
>   gnulib-cache.m4
>   gnulib-comp.m4
>   gnulib-tool.m4
>   onceonly.m4
> 
> Why?

The role of gnulib-cache.m4 is to remember the --source-base, --lgpl,
and many other command-line options that were used, so that the
developer can run --add-import the next time without specifying the
same long list of options once again. For --create-testdir this is
not needed, because there is no need to run --add-import on a
directory created by --create-testdir. Instead, you just throw away
the created testdir and create a new one.

The role of gnulib-comp.m4 is to allow the resulting files from
--import to be omitted from a version control system, and your
collaborators can nevertheless retrieve them through
"gnulib-tool --update". For --create-testdir this is not needed,
because such testdirs are so short-lived that they are never
stored in a VCS.

onceonly.m4 is needed for Autoconf 2.59, but not for newer versions.
In --import / --add-import, this file is being added unless you
specify the minimum autoconf version to a value >= 2.60 through an
AC_PREREQ invocation in configure.ac. For --create-testdir this
file is always being added; I don't know why you don't see it.

gnulib-tool.m4 is currently useless. We could omit it from --import /
--add-import, but it doesn't harm.

> Similarly, there are unexpected differences in `lib' and `build-aux':
> 
>   . `--create-testdir' uses (L)GPL2, while `--add-import' uses
>     (L)GPL3

Rather, --add-import modifies the headers to match the --lgpl option
that you passed, because the expectation is that the result could
be distributed as a tarball and generate confusion when people analyze
the copyright headers.

Whereas tarballs created after --create-testdir are not widely
distributed. For this use-case it is more important that the testdir
and the gnulib checkout can share hardlinks, so that modifications
in the testdir can flow back directly in the gnulib checkout and be
committed there.

>   . `--add-import' adds a `Reproduce by:' line to Makefile.am (this is
>     a good thing), but the `-DGNULIB_STRICT_CHECKING=1' string is
>     missing from `AM_CPPFLAGS'.  Again, this is something which looks
>     strange.

The -DGNULIB_STRICT_CHECKING=1 line generates failures in situations
where the current GCC version would compile the code without problems
but it would have problems with other GCC versions. This is something
you want to have turned on in testdirs but not in production code.

There's also another major difference: When --with-tests is specified,
with --import / --add-import we reuse the main configure.ac, whereas
--create-testdir creates a second configure.ac in the gltests/ directory.
This doubles the time needed to run './configure', but it helps detect
some missing module dependencies.

All in all, the differences are explained by the different use-cases.

Thanks for the questions. I think I should add comments about this
in gnulib-tool.

Bruno