Re: a saner bootstrap script

[Gary V. Vaughan]

[[Bruno, I've kept you in for the gnulib-tool or autopoint bug I'm
  intimating near the bottom of this message.]]

Hi Jim,

On 16 Oct 2011, at 04:15, Jim Meyering wrote:
> Gary V. Vaughan wrote:
>> Is there anything else I can do to help you incorporate this, and the
>> matching bootstrap.conf I wrote for you into coreutils now that the
>> release is out?
> 
> Thanks for persevering.  I have just tried it.

Excellent, thanks for that, and for the feedback.

> Please make at least a token effort to minimize differences in
> bootstrap.conf.  That makes it a lot easier to compare to the original.

Apologies, I was starting from the Zile bootstrap.conf that I wrote,
and copying over the various variable settings from the old coreutils
bootstrap.conf, and not editing the coreutils bootstrap.conf itself,
so I didn't think about needing to run diff between branches.

> Here is a new version with fewer gratuitous white space changes and
> that avoids changing the copyright header.


Thanks, I've applied most of that, although I routinely use diff -Ebu
to see differences in code so as not to form the habit of avoiding
consistent formatting purely for the sake of making diff work without
a couple of extra flags.

Also, if you're not opposed to the 'Written by Gary V. Vaughan' line
I'd rather keep that, since I did port the old bootstrap.conf to work
with my saner bootstrap script after all.  On the other hand, I'm far
from rabid about it, so if it annoys you, or you feel that the work
that came before outweighs my contribution enough to make it inaccurate,
then feel free to remove it when (and if) you merge.

Also, I kept the timestamp header line so that the emacs variable
settings from the bottom of my version of the file correctly update
the header timestamp on save.  Similarly, feel free to change back to
your non-timestamped header comment and remove the emacs goo from
the bottom when you merge if you'd prefer.

> It looks like your coreutils working directory must have contained
> an intl/ directory, but most people don't have that.  Since I don't
> have one, I get lots of these when compiling in lib/:
> 
>    cc1: error: ../intl: No such file or directory [-Werror]

No intl/ directory here, and I can't reproduce your failure.  Did you
copy my bootstrap and bootstrap.conf into your own working directory,
or clone my github mirror?

Here's an edited typescript of running directly with a fresh clone of
github (from the revision before patching bootstrap.conf whitespace as
you requested, not that it should matter):

address@hidden/Devo
├!10154│^2=git clone address@hidden:gvvaughan/GNU-coreutils.git
Cloning into GNU-coreutils...
remote: Counting objects: 152310, done.
remote: Compressing objects: 100% (37473/37473), done.
remote: Total 152310 (delta 114614), reused 152219 (delta 114531)
Receiving objects: 100% (152310/152310), 27.17 MiB | 36 KiB/s, done.
Resolving deltas: 100% (114614/114614), done.

address@hidden/Devo
├!10155│^2=cd GNU-coreutils

address@hidden/Devo/GNU-coreutils                             │master│8887350│
├!10156│^2=gco gary/bootstrap
Branch gary/bootstrap set up to track remote branch gary/bootstrap from origin.
Switched to a new branch 'gary/bootstrap'

address@hidden/Devo/GNU-coreutils                     │gary/bootstrap│030770c│
├!10157│^2=PATH=~/Devo/autotools-latest/bin:/usr/local/Cellar/gettext/0.18.1.1/bin:$PATH
                              

address@hidden/Devo/GNU-coreutils                     │gary/bootstrap│030770c│
├!10158│^2=./bootstrap --gnulib-srcdir=../gnulib
bootstrap: `./bootstrap' differs from `./gnulib/build-aux/bootstrap', 
bootstrap: please consider adopting the canonical version from gnulib.
bootstrap: getting translations into po/.reference for coreutils...   
receiving file list ... done
./
af.po
[[...]]
bootstrap: running: autopoint --force
Copying file ABOUT-NLS               
Copying file build-aux/config.rpath
[[...]]
bootstrap: running: git clone --reference '../gnulib' 
'git://git.sv.gnu.org/gnulib.git' 'gnulib'
Cloning into gnulib...
remote: Counting objects: 1476, done.
remote: Compressing objects: 100% (522/522), done.
remote: Total 1271 (delta 1108), reused 871 (delta 749)
Receiving objects: 100% (1271/1271), 974.96 KiB | 8 KiB/s, done.
Resolving deltas: 100% (1108/1108), completed with 154 local objects.
bootstrap: running: git submodule init
Submodule 'gnulib' (git://git.sv.gnu.org/gnulib.git) registered for path 
'gnulib'
bootstrap: running: git submodule update
Submodule path 'gnulib': checked out '244794a7887f13d9cdb91fed96932cc479905b96'
bootstrap: running: gnulib/gnulib-tool --no-changelog --avoid=canonicalize-lgpl 
--avoid=dummy --with-tests --aux-dir=build-aux --m4-base=m4 --lib=libcoreutils 
--local-dir=gl --makefile-name=gnulib.mk --tests-base=gnulib-tests --no-libtool 
--symbolic --import acl alignof...
Module list with included dependencies (indented):
    accept
    accept-tests
  acl
    acl-tests
[[...]]
bootstrap: edited `lib/Makefile.am' (backup in `lib/Makefile.am~')
bootstrap: running: autoreconf --symlink --install
[[...]]
bootstrap: linking file `gnulib/build-aux/compile'                    
bootstrap: linking file `gnulib/build-aux/install-sh'                 
bootstrap: linking file `gnulib/build-aux/missing'                    
bootstrap: linking file `gnulib/build-aux/mdate-sh'                   
bootstrap: linking file `gnulib/build-aux/texinfo.tex'                
bootstrap: linking file `gnulib/build-aux/depcomp'                    
bootstrap: linking file `gnulib/build-aux/config.guess'               
bootstrap: linking file `gnulib/build-aux/config.sub'                 
bootstrap: linking file `gnulib/doc/INSTALL'                          
bootstrap: Please set copyright_holder explicitly in `bootstrap.conf';
bootstrap: defaulting to `Free Software Foundation, Inc.'.            
bootstrap: Creating po/Makevars from po/Makevars.template ...         
bootstrap: If your pofiles are up-to-date, you can rerun bootstrap    
bootstrap: as `bootstrap --skip-po' to avoid redownloading.           
bootstrap: Done.  Now you can run './configure'.

address@hidden/Devo/GNU-coreutils                   *?│gary/bootstrap│030770c│
├!10160│^2=find . -name '*intl*' |grep -v /gnulib/
./m4/intl.m4~
./m4/intldir.m4~
./m4/intlmacosx.m4
./m4/intlmacosx.m4~                                                             
                                      

address@hidden/Devo/GNU-coreutils                   *?│gary/bootstrap│030770c│
├!10161│^2=./configure
checking for a BSD-compatible install... /usr/bin/install -c     
checking whether build environment is sane... yes                
checking for a thread-safe mkdir -p... build-aux/install-sh -c -d
[[...]]
configure: creating ./config.status                      
config.status: creating Makefile                         
config.status: creating doc/Makefile                     
config.status: creating lib/Makefile                     
config.status: creating man/Makefile                     
config.status: creating po/Makefile.in                   
config.status: creating src/Makefile                     
config.status: creating tests/Makefile                   
config.status: creating gnulib-tests/Makefile            
config.status: creating lib/config.h                     
config.status: executing depfiles commands               
config.status: executing po-directories commands         
config.status: creating po/POTFILES                      
config.status: creating po/Makefile                      
                                                         
address@hidden/Devo/GNU-coreutils                   *?│gary/bootstrap│030770c│
├!10162│^2=make
  GEN    .version
make  all-recursive
Making all in lib
  GEN    alloca.h
  GEN    c++defs.h
  GEN    warn-on-use.h
  GEN    arg-nonnull.h
  GEN    arpa/inet.h
  GEN    configmake.h
mv -f configmake.h-t configmake.h
  GEN    ctype.h
  GEN    dirent.h
  GEN    fcntl.h
  GEN    fnmatch.h
  GEN    getopt.h
  GEN    iconv.h
gperf -m 10 ./iconv_open-aix.gperf > ./iconv_open-aix.h-t
mv ./iconv_open-aix.h-t ./iconv_open-aix.h
[[...]]
  GEN    yes.1
Making all in po
Making all in tests
make[2]: Nothing to be done for `all'.
Making all in gnulib-tests
  GEN    arg-nonnull.h
  GEN    c++defs.h
  GEN    unused-parameter.h
  GEN    warn-on-use.h
make  all-recursive
Making all in .
make[4]: Nothing to be done for `all-am'.
make[2]: Nothing to be done for `all-am'.

address@hidden/Devo/GNU-coreutils                   *?│gary/bootstrap│030770c│
├!10163│^2=make check
  GEN    public-submodule-commit
make  check-recursive
Making check in lib
make  check-am
[[...]]
FAIL: misc/tty-eof (exit: 1)
============================

F: 1:
F: 1: a b
F: 1:
F: 1:
F: 1:
F: 1: a b
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
F: 1:
tty-eof: unexpand didn't exit after ^D from standard input
F: 1:
F: 1:
F: 1:

[[...]]
======================================
1 of 369 tests failed
(91 tests were not run)
See tests/test-suite.log
Please report to address@hidden
======================================
make[4]: *** [test-suite.log] Error 1
make[3]: *** [check-TESTS] Error 2
make[2]: *** [check-am] Error 2
make[1]: *** [check-recursive] Error 1
make: *** [check] Error 2

The test failure is the same one I get when bootstrapped with the
incumbent script.

> Merely creating the directory gets past that, and the build
> did complete, but obviously I don't want to have to do that.
> I used AM_GNU_GETTEXT([external]... after all.
> 
> Not surprisingly, the part of the current bootstrap script that
> takes care of that transformation is absent from your version:
> 
> -      if test $file = Makefile.am && test "X$gnulib_mk" != XMakefile.am; then
> -        copied=$copied${sep}$gnulib_mk; sep=$nl
> -        remove_intl='/^[^#].*\/intl/s/^/#/;'"s!$bt_regex/!!g"
> -        sed "$remove_intl" $1/$dir/$file |
> -        cmp - $dir/$gnulib_mk > /dev/null || {
> -          echo "$me: Copying $1/$dir/$file to $dir/$gnulib_mk ..." &&
> -          rm -f $dir/$gnulib_mk &&
> -          sed "$remove_intl" $1/$dir/$file >$dir/$gnulib_mk &&
> -          gnulib_mk_hook $dir/$gnulib_mk
> -        }
> -      elif { test "${2+set}" = set && test -r $2/$dir/$file; } ||
> -           version_controlled_file $dir $file; then
> -        echo "$me: $dir/$file overrides $1/$dir/$file"
> -      else
> -        copied=$copied$sep$file; sep=$nl
> -        cp_mark_as_generated $1/$dir/$file $dir/$file
> -      fi || exit
> 

As far as I can see, that code is not relevant except when creating and
merging two directories... something that I've not done with my bootstrap
and which is responsible for the bulk of the time savings compared to the
incumbent bootstrap script.  However, slurp() is more or less uncommented,
and even after staring at it for several months, I'm still not sure I
properly understand it, so I could easily be wrong.  Nonetheless, my saner
bootstrap seems to be working correctly on coreutils without it as far is
I'm able to check.  What am I missing?

If there is a bug in gnulib-tool, or autopoint that puts unnecessary
'intl/' references into Makefiles when the presence of
AM_GNU_GETTEXT_VERSION in configure.ac is a declaration that says there
is no need for local 'intl/', then I think the bug should be fixed there
rather than patched around in bootstrap.  Unfortunately, some difference
in our environments is preventing me from exploring further since I can't
reproduce your error on my machine :(

Are you running on Linux? I could build a Linux VM and try to reproduce
it again there if you don't have time to investigate it yourself.

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)