Re: Instructions on new bootstrap?

[Bruno Haible]

Paul Smith wrote:
> how all this is supposed to work in the modern era?

I'm updating the documentation:


2022-07-30  Bruno Haible  <bruno@clisp.org>

        doc: Update regarding bootstrap split.
        Reported by Paul Smith <psmith@gnu.org>.
        * doc/gnulib-tool.texi (gettextize and autopoint): Mention autogen.sh
        instead of bootstrap.
        (VCS Issues): Describe three alternative approaches.
        * doc/gnulib.texi (Developer tools): Mention also autopull.sh and
        autogen.sh.
        * top/bootstrap (usage): Fix copy&paste mistake.

diff --git a/doc/gnulib-tool.texi b/doc/gnulib-tool.texi
index f99f214caf..ba1598fd9a 100644
--- a/doc/gnulib-tool.texi
+++ b/doc/gnulib-tool.texi
@@ -797,7 +797,7 @@ at the time the Gnulib snapshot was taken.
 After running @code{gettextize}, invoke @code{gnulib-tool} and import
 the @code{gettext} module.  Also, copy the latest version of gnulib's
 @file{build-aux/po/Makefile.in.in} to your @file{po/} directory (this
-is done for you if you use gnulib's @file{bootstrap} script).
+is done for you if you use gnulib's @file{autogen.sh} script).
 
 @item
 If you get an error message like
@@ -828,7 +828,7 @@ Invoke @code{gnulib-tool}, and import the @code{gettext-h} 
module.
 
 Regardless of which approach you used to get the infrastructure in
 place, the following steps must then be used to preserve that
-infrastructure (gnulib's @file{bootstrap} script follows these rules):
+infrastructure (gnulib's @file{autogen.sh} script follows these rules):
 
 @enumerate
 @item
@@ -982,26 +982,32 @@ add because they were missing.
 
 @end itemize
 
-Gnulib includes the file @file{build-aux/bootstrap} to aid a developer
-in using this setup.  You would copy this file and add it to your
-project's VCS.  It can be customized using @file{bootstrap.conf} which
-is also put under VCS in your project.  Read the script for further
-information, and look at how other projects are using it (see links in
-@url{https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=users.txt;hb=HEAD}).
+Most packages nowadays use the first among these two approaches.  Over
+time, three ways of handling version control have evolved.
 
-In projects that use git for version control, it is possible to use a
-git submodule containing the precise commit of the gnulib repository, so
-that each developer running @file{bootstrap} will get the same version
-of all gnulib-provided files.  The location of the submodule can be
-chosen to fit the package's needs; here's how to initially create the
-submodule in the directory @file{gnulib}:
+In the cases (A) and (B), a ``git submodule'' is used to reference
+the precise commit of the gnulib repository, so that each developer
+running @file{autopull.sh} will get the same version of all gnulib-provided
+files.
+
+The alternative is to always follow the newest Gnulib automatically.
+Note that this can cause breakages at unexpected moments, namely
+when a broken commit is pushed in Gnulib.  It does not happen often,
+but it does happen.
+
+@itemize
+@item (A)
+In this approach, the developers use a git submodule manually.
+
+The location of the submodule can be chosen to fit the package's needs;
+here's how to initially create the submodule in the directory @file{gnulib}:
 
 @smallexample
 $ git submodule add -- https://git.savannah.gnu.org/git/gnulib.git gnulib
 @end smallexample
 
 @noindent
-Thereafter, @file{bootstrap} will run this command to update the
+Thereafter, the developer will run this command to update the
 submodule to the recorded checkout level:
 
 @smallexample
@@ -1024,6 +1030,37 @@ $ git config alias.syncsub "submodule foreach git pull 
origin master"
 $ git syncsub
 @end smallexample
 
+@item (B)
+In this approach, the @code{autopull.sh} program (see @ref{Developer tools})
+is used to aid a developer in using this setup.  You copy this program and
+its companion files into your package and place them under version control.
+It can be customized using @file{bootstrap.conf} which you also put under
+version control.
+
+@item (C)
+In this approach, you write the @code{autopull.sh} and @code{autogen.sh}
+files by hand.
+
+@code{autopull.sh} is most easily written as a script that
+invokes
+@smallexample
+./gitsub.sh pull || exit 1
+@end smallexample
+@noindent
+where @code{gitsub.sh} is described in @ref{Developer tools}.
+
+@code{autogen.sh} typically contains an explicit @code{gnulib-tool}
+invocation, followed by
+@smallexample
+aclocal -I m4 \
+  && autoconf \
+  && autoheader && touch config.h.in \
+  && automake --add-missing --copy \
+  && rm -rf autom4te.cache \
+  || exit $?
+@end smallexample
+@end itemize
+
 @item
 Some projects take a ``middle road'': they do commit Gnulib source
 files as in the first approach, but they do not commit other derived
diff --git a/doc/gnulib.texi b/doc/gnulib.texi
index 1386c44ed0..8d24699216 100644
--- a/doc/gnulib.texi
+++ b/doc/gnulib.texi
@@ -7026,16 +7026,42 @@ tools right away.
 @item top/gitsub.sh
 This program manages the subdirectories of a Git checkout that come from
 other packages, including Gnulib.
-@item build-aux/bootstrap
-This program manages the Git submodules, including Gnulib, and is also a
-wrapper around @code{gnulib-tool} and @code{automake}, that generates
-files from other files.@*
+
+@item top/bootstrap
+@itemx top/autopull.sh
+@itemx top/autogen.sh
+@itemx top/bootstrap-funclib.sh
+This is a set of three programs and a function library, that manage
+the source directory of a package, preparing for the state where
+@samp{./configure} can be used.
+
+@code{autopull.sh} is a program for fetching dependencies that may
+require network accesses.  It manages the Git submodules, including
+Gnulib --- assuming that Gnulib is a Git submodule. It also can fetch
+the PO files for internationalized packages.
+
+@code{autogen.sh} is a program that is meant to be run after
+@code{autopull.sh}.  It generates all autogeneratable files that are
+omitted from version control.  Usually this means that it invokes
+@code{gnulib-tool} and @code{automake}, that generate files from other
+files.
+
+@code{bootstrap} is a wrapper around both: It invokes @code{autopull.sh}
+and then immediately @code{autogen.sh}.@*
 Note: Because this program mixes version control management and
 generation of files in non-obvious ways, it has a number of usability
 issues for the advanced developer.
+
+@code{bootstrap-funclib.sh} is a function library for these three
+programs.  It is not meant to be used directly.
+
+All three programs make use of a configuration file, called
+@code{bootstrap.conf}.
+
 @item build-aux/bootstrap.conf
-This is the template configuration file.  After copying it into your
-package, you need to customize it.
+This is the template configuration file.  After copying it into the
+top-level directory of your package, you need to customize it.
+
 @item build-aux/po/Makefile.in.in
 @itemx build-aux/po/remove-potcdate.sin
 These are auxiliary files used by @code{bootstrap}.  You don't have to
diff --git a/top/bootstrap b/top/bootstrap
index e9d8ad82aa..cf8d007f64 100755
--- a/top/bootstrap
+++ b/top/bootstrap
@@ -26,7 +26,7 @@
 
 # Please report bugs or propose patches to bug-gnulib@gnu.org.
 
-scriptversion=2022-07-24.15; # UTC
+scriptversion=2022-07-29.23; # UTC
 
 me="$0"
 medir=`dirname "$me"`
@@ -59,10 +59,6 @@ Options:
                            and history on your machine, and do not want
                            to waste your bandwidth downloading them again.
                            Defaults to \$GNULIB_REFDIR
-  --bootstrap-sync         if this bootstrap script is not identical to
-                           the version in the local gnulib sources,
-                           update this script, and then restart it with
-
   --bootstrap-sync         if this bootstrap script is not identical to
                            the version in the local gnulib sources,
                            update this script, and then restart it with