[PATCH 0/8] Towards a Zsh-compatible Automake testsuite

[Stefano Lattarini]

***  INTRODUCTION  ***

This is a reproposal of an older patch series aimed at making the
Automake testsuite fully compatible with Zsh (both versions 3 and 4).

The main problem that is tackled by these patches is the peculiar way
Zsh handles stderr redirections with the `xtrace' a.k.a. `set -x' shell
flag is on.

This new series also offer some additional cleanups, refactorings and
improvements over the old series.

***  DESCRIPTION OF THE PROBLEM  ***

If the Automake testsuite is forced to to run with thr Zsh shell 
(version 4 or later), e.g. by doing:
 $ make TEST_LOG_COMPILER='zsh' check
or (to enable better Bourne compatibility in Zsh from the very
beginning):
 $ ln -s `which zsh` ./sh && make TEST_LOG_COMPILER="`pwd`/sh" check
some tests experience spurious failures.  Here is a (probably not
complete) list of the failing tests:
FAIL: aclocal8.test (exit: 1)
FAIL: acsilent.test (exit: 1)
FAIL: depcomp2.test (exit: 1)
FAIL: getopt.test (exit: 1)
FAIL: location.test (exit: 1)
FAIL: missing3.test (exit: 1)
FAIL: output-order.test (exit: 1)
FAIL: parallel-tests4.test (exit: 2)
FAIL: unused.test (exit: 1)
FAIL: warnopts.test (exit: 1)

It turns out that these failures are due to the fact that Zsh (version
4 or later) handles the `xtrace' a.k.a. `set -x' shell option in a IMHO
totally brain-damaged way.  In details: if a command which is run
when `xtrace' is active has its standard error redirected to a file,
then zsh writes *also the trace* for the command into that file,
instead that into the original standard error.

The following examples should help making things clearer (warning:
I tried these only on my system, thus YMMV):
 
 $ bash -x -c 'echo bash >out 2>&1'
 + echo bash
 $ cat out
 bash
 
 $ dash -x -c 'echo dash >out 2>&1'
 + echo dash
 $ cat out
 dash
 
 $ ksh -x -c 'echo ksh >out 2>&1'
 + echo ksh
 + 1> out 2>& 1
 $ cat out
 ksh
 
 $ zsh3 -x -c 'echo zsh3 >out 2>&1'
 + echo zsh3
 $ cat out
 zsh3
 
 $ zsh4 -x -c 'echo zsh4 >out 2>&1'
 $ cat out
 +zsh:1> echo zsh
 zsh4

I found no transparent workaround for this behaviour.  So I came up
with this patch series to solve the problem.

***  STATUS OF THE PATCH SERIES  ***

Since the patches in this series introduced some invasive changes
to many test scripts, I tried to do extensive teststing with various
configurations (different shells, different make programs, etc), to
minimize the likeliness that I had introduced regressions or bugs.

A detailed description of the outcome of such tests (done after the
application of *all* the patches) follows.

== System Details ==
  
 - Debian GNU/Linux (squeeze/sid)
 - `uname -s -r -m` = Linux 2.6.30-2-686 i686
 - Perl, v5.10.1 (*) built for i486-linux-gnu-thread-multi
 - Autoconf 2.65 (debian package version: 2.65-2)
 - libtool 2.2.6 (debian package version: 2.2.6a-4)
 - GCC, GCJ and G++: version 4.3.4
 - No g77, only gfortran (4.3.4) and fort77 (1.15)
 - Bison version 2.4.1

 == Varying configurations ==
 
 [1]
    Zsh, 3.0.8
    GNU Make, 3.81
    2 parallel make jobs
 [2]
    Zsh, 4.3.10 
    GNU Make, 3.81
    2 parallel make jobs
 [3]
    Heirloom Sh, "@(#)sh.sl 1.55 (gritter) 7/6/05"
    Heirloom Make, "@(#)make.sl 1.40 (gritter) 3/15/07"
    2 parallel jobs
    Prefer Heirloom tools over system tools. More info about the Heirloom
    project:  <http://heirloom.sourceforge.net/>
 [4]
    Dash, Debian package "dash 0.5.5.1-3"
    GNU Make, 3.75
    2 parallel make jobs
 [5]
    Ksh, "Version JM 93t+ 2009-05-01"
    FreeBSD Make port, Debian package "freebsd-buildutils 7.2-1"
    1 make job
 [6]
    Heirloom Sh, "@(#)sh.sl 1.55 (gritter) 7/6/05"
    GNU Make, 3.75
    2 parallel make jobs

 == Outcomes ==

The good news is that no regression showed up w.r.t. the master branch.
The bad news is that the following relevant failures were encountered in
configurations [3], [4] and [5]:
 
 (3) FAILED: cscope instspc parallel-tests9 silent5 subobj9 yacc8
     SKIPPED: 39 tests were not run (mostly due to the fact tha $MAKE
              is not GNU make)
 
 (4) FAILED: parallel-tests3 (a spurious failure due to the fact that
             GNU make 3.75 fails to run the test scripts in parallel, even
             with `-j4' option and `parallel-tests' Automake's option
             active; this is probably not worth investigating or fixing,
             since GNU make 3.75 is at least 13 years old)
 
 (5) FAILED: instspc man4 silent8 subpkg
     SKIPPED: 38 tests were not run (mostly due to the fact tha $MAKE
              is not GNU make)

The detailed logs of testsuite runs for configurations 3, 4 and 5 are
attacched.

****  ****  ****  ****

I will soon post all the patches in this same thread.

Regards,
   Stefano

conf4-testsuite.log.bz2
Description: application/bzip

conf5-testsuite.log.bz2
Description: application/bzip

conf3-testsuite.log.bz2
Description: application/bzip