bug-gnulib
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bootstrap: Obey another environment variable GNULIB_REFDIR


From: Bruno Haible
Subject: bootstrap: Obey another environment variable GNULIB_REFDIR
Date: Sun, 24 Jul 2022 14:21:08 +0200

The handling of the GNULIB_SRCDIR environment variable in 'bootstrap'
is hard to understand, despite the long documentation in `bootstrap --help`.

"
Gnulib sources can be fetched in various ways:

 * If this package is in a git repository with a 'gnulib' submodule
   configured, then that submodule is initialized and updated and sources
   are fetched from there.  If $GNULIB_SRCDIR is set (directly or via
   --gnulib-srcdir) and is a git repository, then it is used as a reference.

 * Otherwise, if $GNULIB_SRCDIR is set (directly or via --gnulib-srcdir),
   then sources are fetched from that local directory.  If it is a git
   repository and $GNULIB_REVISION is set, then that revision is checked
   out.

 * ...
"

What this documentation does not say, is that the interpretation of
$GNULIB_SRCDIR from the first paragraph (i.e. use the .git subdirectory
as a reference) is only done if the option --no-git is not specified.
If --no-git is specified, then the second paragraph is in effect: then
$GNULIB_SRCDIR is supposed to contain the gnulib sources.

So we have an environment variable whose meaning depends on the options
that the user passes to 'bootstrap'.

That is terrible.

It is terrible
1) because environment variables are typically set for a longer duration
   than for invoking a single program,
2) because it leads to practical nonsense:
   Suppose you have a gnulib checkout in /src/gnulib, up-to-date in master.
   You set GNULIB_SRCDIR=/src/gnulib.
   You have two checkouts of, say, coreutils.
   You run './bootstrap' in the first one. It will use GNULIB_SRCDIR as a
   reference and use the commit indicated in the coreutils/gnulib submodule.
   You run './bootstrap --no-git' in the second one. It will use GNULIB_SRCDIR
   for the sources, i.e. use the master (tip) of gnulib.
   => Different results, depending on a command-line option!

I understand that this behaviour is the result of several patches, on
2010-02-22, 2011-01-20, 2013-07-03. As such, it was probably unintentional.

Rather than to document this terrible behaviour, I'm fixing it.

The right fix is, of course, to use two *different* environment variables
for the two different purposes. The new environment variable is
  GNULIB_REFDIR.

When I do this, as you can see in the patch below, the "Gnulib sources can
be fetched in various ways" explanation shrinks from 5 paragraphs to 4
paragraphs. So, adding this second environment variable makes the logic
simpler to document — and so, also, simpler to understand.

This change does not need changes from the packages that use 'bootstrap'.

But some developers will need to change their habit: set GNULIB_REFDIR
instead of GNULIB_SRCDIR, if that's what they meant. I'm therefore adding
an entry to the NEWS file.


2022-07-24  Bruno Haible  <bruno@clisp.org>

        bootstrap: Obey another environment variable GNULIB_REFDIR.
        * build-aux/bootstrap (usage): Document option --gnulib-refdir and
        environment variable GNULIB_REFDIR.
        Accept option --gnulib-refdir.
        When GNULIB_SRCDIR is specified, verify that it denotes a directory.
        Use $GNULIB_REFDIR, not $GNULIB_SRCDIR, as argument to git's --reference
        option.
        Improve an error message.
        * NEWS: Mention the change.

diff --git a/NEWS b/NEWS
index e5a875021e..c3f810101d 100644
--- a/NEWS
+++ b/NEWS
@@ -3,6 +3,12 @@ Important general notes
 
 Date        Modules         Changes
 
+2022-07-24  bootstrap       To specify a reference directory for the 'gnulib'
+                            submodule, use the environment variable
+                            GNULIB_REFDIR instead of GNULIB_SRCDIR or the
+                            command-line option --gnulib-refdir instead of
+                            --gnulib-srcdir.
+
 2022-02-20  (all)           Automake >= 1.14 and Autoconf >= 2.64 are required.
 
 2021-06-04  (all)           The license notices in source files are now really
diff --git a/build-aux/bootstrap b/build-aux/bootstrap
index 86a0ac6d6f..620f90cd57 100755
--- a/build-aux/bootstrap
+++ b/build-aux/bootstrap
@@ -1,6 +1,6 @@
 #! /bin/sh
 # Print a version string.
-scriptversion=2022-07-23.11; # UTC
+scriptversion=2022-07-24.10; # UTC
 
 # Bootstrap this package from checked-out sources.
 
@@ -58,8 +58,14 @@ Options:
  --gnulib-srcdir=DIRNAME  specify the local directory where gnulib
                           sources reside.  Use this if you already
                           have gnulib sources on your machine, and
-                          do not want to waste your bandwidth downloading
-                          them again.  Defaults to \$GNULIB_SRCDIR
+                          you want to use these sources.  Defaults
+                          to \$GNULIB_SRCDIR
+ --gnulib-refdir=DIRNAME  specify the local directory where a gnulib
+                          repository (with a .git subdirectory) resides.
+                          Use this if you already have gnulib sources
+                          and history on your machine, and do not want
+                          to waste your bandwidth downloading them again.
+                          Defaults to \$GNULIB_REFDIR
  --bootstrap-sync         if this bootstrap script is not identical to
                           the version in the local gnulib sources,
                           update this script, and then restart it with
@@ -82,19 +88,16 @@ are honored.
 
 Gnulib sources can be fetched in various ways:
 
- * If this package is in a git repository with a 'gnulib' submodule
-   configured, then that submodule is initialized and updated and sources
-   are fetched from there.  If \$GNULIB_SRCDIR is set (directly or via
-   --gnulib-srcdir) and is a git repository, then it is used as a reference.
-
- * Otherwise, if \$GNULIB_SRCDIR is set (directly or via --gnulib-srcdir),
-   then sources are fetched from that local directory.  If it is a git
-   repository and \$GNULIB_REVISION is set, then that revision is checked
-   out.
+ * If \$GNULIB_SRCDIR is set (directly or via --gnulib-srcdir), then
+   sources are fetched from that local directory.  If it is a git
+   repository and \$GNULIB_REVISION is set, then that revision is
+   checked out.
 
  * Otherwise, if this package is in a git repository with a 'gnulib'
-   submodule configured, then that submodule is initialized and updated and
-   sources are fetched from there.
+   submodule configured, then that submodule is initialized and updated
+   and sources are fetched from there.  If \$GNULIB_REFDIR is set
+   (directly or via --gnulib-refdir) and is a git repository, then it
+   is used as a reference.
 
  * Otherwise, if the 'gnulib' directory does not exist, Gnulib sources are
    cloned into that directory using git from \$GNULIB_URL, defaulting to
@@ -463,6 +466,8 @@ do
     ;;
   --gnulib-srcdir=*)
     GNULIB_SRCDIR=${option#--gnulib-srcdir=};;
+  --gnulib-refdir=*)
+    GNULIB_REFDIR=${option#--gnulib-refdir=};;
   --skip-po)
     SKIP_PO=t;;
   --force)
@@ -480,8 +485,10 @@ do
   esac
 done
 
-$use_git || test -d "$GNULIB_SRCDIR" \
-  || die "Error: --no-git requires --gnulib-srcdir"
+$use_git || test -n "$GNULIB_SRCDIR" \
+  || die "Error: --no-git requires --gnulib-srcdir or \$GNULIB_SRCDIR"
+test -z "$GNULIB_SRCDIR" || test -d "$GNULIB_SRCDIR" \
+  || die "Error: --gnulib-srcdir or \$GNULIB_SRCDIR is specified, but does not 
denote a directory"
 
 if test -n "$checkout_only_file" && test ! -r "$checkout_only_file"; then
   die "Bootstrapping from a non-checked-out distribution is risky."
@@ -732,6 +739,8 @@ fi
 
 cleanup_gnulib() {
   status=$?
+  # XXX It's a bad idea to erase the submodule directory if it contains local
+  #     modifications.
   rm -fr "$gnulib_path"
   exit $status
 }
@@ -741,63 +750,26 @@ git_modules_config () {
 }
 
 if $use_gnulib; then
-  if $use_git; then
+  if test -n "$GNULIB_SRCDIR"; then
+    # Use GNULIB_SRCDIR directly.
+    # We already checked that $GNULIB_SRCDIR references a directory.
+    # Verify that it contains a gnulib checkout.
+    test -f "$GNULIB_SRCDIR/gnulib-tool" \
+      || die "Error: --gnulib-srcdir or \$GNULIB_SRCDIR is specified, but does 
not contain gnulib-tool"
+  elif $use_git; then
     gnulib_path=$(git_modules_config submodule.gnulib.path)
     test -z "$gnulib_path" && gnulib_path=gnulib
-  fi
-
-  # Get gnulib files.  Populate $GNULIB_SRCDIR, possibly updating a
-  # submodule, for use in the rest of the script.
 
-  case ${GNULIB_SRCDIR--} in
-  -)
-    # Note that $use_git is necessarily true in this case.
-    if git_modules_config submodule.gnulib.url >/dev/null; then
-      echo "$0: getting gnulib files..."
-      git submodule init -- "$gnulib_path" || exit $?
-      git submodule update -- "$gnulib_path" || exit $?
+    # Get gnulib files.  Populate $gnulib_path, possibly updating a
+    # submodule, for use in the rest of the script.
 
-    elif [ ! -d "$gnulib_path" ]; then
-      echo "$0: getting gnulib files..."
-
-      trap cleanup_gnulib 1 2 13 15
-
-      shallow=
-      if test -z "$GNULIB_REVISION"; then
-        git clone -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 2'
-        git clone $shallow ${GNULIB_URL:-$default_gnulib_url} "$gnulib_path" \
-          || cleanup_gnulib
-      else
-        git fetch -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 2'
-        mkdir -p "$gnulib_path"
-        # Only want a shallow checkout of $GNULIB_REVISION, but git does not
-        # support cloning by commit hash. So attempt a shallow fetch by commit
-        # hash to minimize the amount of data downloaded and changes needed to
-        # be processed, which can drastically reduce download and processing
-        # time for checkout. If the fetch by commit fails, a shallow fetch can
-        # not be performed because we do not know what the depth of the commit
-        # is without fetching all commits. So fallback to fetching all commits.
-        git -C "$gnulib_path" init
-        git -C "$gnulib_path" remote add origin \
-            ${GNULIB_URL:-$default_gnulib_url}
-        git -C "$gnulib_path" fetch $shallow origin "$GNULIB_REVISION" \
-          || git -C "$gnulib_path" fetch origin \
-          || cleanup_gnulib
-        git -C "$gnulib_path" reset --hard FETCH_HEAD
-      fi
-
-      trap - 1 2 13 15
-    fi
-    GNULIB_SRCDIR=$gnulib_path
-    ;;
-  *)
-    # Use GNULIB_SRCDIR directly or as a reference.
-    if $use_git && test -d "$GNULIB_SRCDIR"/.git && \
-          git_modules_config submodule.gnulib.url >/dev/null; then
+    if test -n "$GNULIB_REFDIR" && test -d "$GNULIB_REFDIR"/.git \
+       && git_modules_config submodule.gnulib.url >/dev/null; then
+      # Use GNULIB_REFDIR as a reference.
       echo "$0: getting gnulib files..."
       if git submodule -h|grep -- --reference > /dev/null; then
         # Prefer the one-liner available in git 1.6.4 or newer.
-        git submodule update --init --reference "$GNULIB_SRCDIR" \
+        git submodule update --init --reference "$GNULIB_REFDIR" \
           "$gnulib_path" || exit $?
       else
         # This fallback allows at least git 1.5.5.
@@ -807,18 +779,59 @@ if $use_gnulib; then
         else
           # Older git can't clone into an empty directory.
           rmdir "$gnulib_path" 2>/dev/null
-          git clone --reference "$GNULIB_SRCDIR" \
+          git clone --reference "$GNULIB_REFDIR" \
             "$(git_modules_config submodule.gnulib.url)" "$gnulib_path" \
             && git submodule init -- "$gnulib_path" \
             && git submodule update -- "$gnulib_path" \
             || exit $?
         fi
       fi
-      GNULIB_SRCDIR=$gnulib_path
+    else
+      # GNULIB_REFDIR is not set or not usable. Ignore it.
+      if git_modules_config submodule.gnulib.url >/dev/null; then
+        echo "$0: getting gnulib files..."
+        git submodule init -- "$gnulib_path" || exit $?
+        git submodule update -- "$gnulib_path" || exit $?
+
+      elif [ ! -d "$gnulib_path" ]; then
+        echo "$0: getting gnulib files..."
+
+        trap cleanup_gnulib 1 2 13 15
+
+        shallow=
+        if test -z "$GNULIB_REVISION"; then
+          git clone -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 
2'
+          git clone $shallow ${GNULIB_URL:-$default_gnulib_url} "$gnulib_path" 
\
+            || cleanup_gnulib
+        else
+          git fetch -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 
2'
+          mkdir -p "$gnulib_path"
+          # Only want a shallow checkout of $GNULIB_REVISION, but git does not
+          # support cloning by commit hash. So attempt a shallow fetch by 
commit
+          # hash to minimize the amount of data downloaded and changes needed 
to
+          # be processed, which can drastically reduce download and processing
+          # time for checkout. If the fetch by commit fails, a shallow fetch 
can
+          # not be performed because we do not know what the depth of the 
commit
+          # is without fetching all commits. So fallback to fetching all 
commits.
+          git -C "$gnulib_path" init
+          git -C "$gnulib_path" remote add origin \
+              ${GNULIB_URL:-$default_gnulib_url}
+          git -C "$gnulib_path" fetch $shallow origin "$GNULIB_REVISION" \
+            || git -C "$gnulib_path" fetch origin \
+            || cleanup_gnulib
+          git -C "$gnulib_path" reset --hard FETCH_HEAD
+        fi
+
+        trap - 1 2 13 15
+      fi
     fi
-    ;;
-  esac
+    GNULIB_SRCDIR=$gnulib_path
+    # Verify that the submodule contains a gnulib checkout.
+    test -f "$gnulib_path/gnulib-tool" \
+      || die "Error: $gnulib_path is supposed to contain a gnulib checkout, 
but does not contain gnulib-tool"
+  fi
 
+  # XXX Should this be done if $use_git is false?
   if test -d "$GNULIB_SRCDIR"/.git && test -n "$GNULIB_REVISION" \
      && ! git_modules_config submodule.gnulib.url >/dev/null; then
     (cd "$GNULIB_SRCDIR" && git checkout "$GNULIB_REVISION") || cleanup_gnulib
@@ -1018,7 +1031,7 @@ bootstrap_post_import_hook \
 # Uninitialized submodules are listed with an initial dash.
 if $use_git && git submodule | grep '^-' >/dev/null; then
   die "some git submodules are not initialized. "     \
-      "Run 'git submodule update --init' and bootstrap again."
+      "Either use option '--no-git', or run 'git submodule update --init' and 
bootstrap again."
 fi
 
 # Remove any dangling symlink matching "*.m4" or "*.[ch]" in some






reply via email to

[Prev in Thread] Current Thread [Next in Thread]