[FYI] {testsuite-work} tests: tweak and improve tests on "Simple Tests" driver

[Stefano Lattarini]

* tests/check.test: Make grepping of generated Makefile.in
slightly stricter.  Add trailing `:' command.
* tests/check2.test: Ensure verbose printing of captured make
output.  Make grepping of generated Makefile.in and of make
output stricter.  Add trailing `:' command.
* tests/check3.test: Likewise.  Also, fix minor error in a
comment.
* tests/check4.test: Stricter grepping of make output.  Use
`using_gmake' subroutine instead of copy&paste from defs.in
to determine whether $MAKE is GNU make.  Add a trailing `:'
command.
* tests/check6.test: Add trailing `:' command.
* tests/check10.test: Prefer redirection `>' over `>>' when
creating new files.  Ensure verbose printing of captured make
output.
* tests/check11.test: Likewise.
* tests/check7.test: Likewise.  Make grepping of make output
stricter.  Prefer "make distcheck" over a simple "make check"
in a VPATH build. Add trailing `:' command.
(a.c): Format following GNU coding standards.
(Makefile.am): Explicitly declare target `print-xfail-tests'
as PHONY.  Distribute $(check_SCRIPTS), for distcheck.
* tests/parallel-tests.test: Prefer redirection `>' over `>>'
when creating new files.
* tests/parallel-tests2.test: Likewise.
* tests/parallel-tests9.test: Likewise.
* tests/parallel-tests8.test: Likewise.  Make grepping of
automake stderr stricter.
* tests/check8.test: Likewise.  Make formatting of created
*.c files more consistent with GNU Coding Standards.
* tests/check-subst-prog.test (configure.in): Break overly
long lines.
(pass-prog.c, xfail-prog.c): Format them more consistently
with GNU Coding Standards.
* tests/parallel-tests3.test: Use a shorter and clearer skip
message.  Use the `unindent' subroutine to improve readability
of indented loop bodies.  Simplify the hack used to please
maintainer-check.
* tests/parallel-tests5.test: Make formatting of created *.c
files more consistent with GNU Coding Standards.  Use the
`unindent' subroutine to improve readability of indented loop
bodies.
* tests/parallel-tests10.test: Add blank line, for clarity.
Prefer trailing `:' over trailing `Exit 0', for consistency
with other tests.
---
 ChangeLog                   |   49 +++++++++++++++++++++++++++++++++++++++++++
 tests/check-subst-prog.test |   14 ++++++++----
 tests/check.test            |    8 ++++--
 tests/check10.test          |    8 +++---
 tests/check11.test          |    6 ++--
 tests/check2.test           |   14 ++++++++----
 tests/check3.test           |   16 ++++++++-----
 tests/check4.test           |   34 +++++++++++++++---------------
 tests/check6.test           |    2 +
 tests/check7.test           |   25 ++++++++++++---------
 tests/check8.test           |   20 ++++++++--------
 tests/parallel-tests.test   |    6 ++--
 tests/parallel-tests10.test |    3 +-
 tests/parallel-tests2.test  |    7 +++--
 tests/parallel-tests3.test  |   16 +++++++-------
 tests/parallel-tests5.test  |   16 +++++++-------
 tests/parallel-tests8.test  |    6 ++--
 tests/parallel-tests9.test  |    7 +++--
 18 files changed, 164 insertions(+), 93 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 39a2066..04e8921 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,52 @@
+2011-05-23  Stefano Lattarini  <address@hidden>
+
+       tests: tweak and improve tests on "Simple Tests" driver
+       * tests/check.test: Make grepping of generated Makefile.in
+       slightly stricter.  Add trailing `:' command.
+       * tests/check2.test: Ensure verbose printing of captured make
+       output.  Make grepping of generated Makefile.in and of make
+       output stricter.  Add trailing `:' command.
+       * tests/check3.test: Likewise.  Also, fix minor error in a
+       comment.
+       * tests/check4.test: Stricter grepping of make output.  Use
+       `using_gmake' subroutine instead of copy&paste from defs.in
+       to determine whether $MAKE is GNU make.  Add a trailing `:'
+       command.
+       * tests/check6.test: Add trailing `:' command.
+       * tests/check10.test: Prefer redirection `>' over `>>' when
+       creating new files.  Ensure verbose printing of captured make
+       output.
+       * tests/check11.test: Likewise.
+       * tests/check7.test: Likewise.  Make grepping of make output
+       stricter.  Prefer "make distcheck" over a simple "make check"
+       in a VPATH build. Add trailing `:' command.
+       (a.c): Format following GNU coding standards.
+       (Makefile.am): Explicitly declare target `print-xfail-tests'
+       as PHONY.  Distribute $(check_SCRIPTS), for distcheck.
+       * tests/parallel-tests.test: Prefer redirection `>' over `>>'
+       when creating new files.
+       * tests/parallel-tests2.test: Likewise.
+       * tests/parallel-tests9.test: Likewise.
+       * tests/parallel-tests8.test: Likewise.  Make grepping of
+       automake stderr stricter.
+       * tests/check8.test: Likewise.  Make formatting of created
+       *.c files more consistent with GNU Coding Standards.
+       * tests/check-subst-prog.test (configure.in): Break overly
+       long lines.
+       (pass-prog.c, xfail-prog.c): Format them more consistently
+       with GNU Coding Standards.
+       * tests/parallel-tests3.test: Use a shorter and clearer skip
+       message.  Use the `unindent' subroutine to improve readability
+       of indented loop bodies.  Simplify the hack used to please
+       maintainer-check.
+       * tests/parallel-tests5.test: Make formatting of created *.c
+       files more consistent with GNU Coding Standards.  Use the
+       `unindent' subroutine to improve readability of indented loop
+       bodies.
+       * tests/parallel-tests10.test: Add blank line, for clarity.
+       Prefer trailing `:' over trailing `Exit 0', for consistency
+       with other tests.
+
 2011-05-21  Stefano Lattarini  <address@hidden>
 
        tests/README: small consistency improvement
diff --git a/tests/check-subst-prog.test b/tests/check-subst-prog.test
index bda0629..b8931d5 100755
--- a/tests/check-subst-prog.test
+++ b/tests/check-subst-prog.test
@@ -22,9 +22,13 @@ required=native
 
 cat >> configure.in << 'END'
 AC_PROG_CC
-AC_SUBST([script_tests], ['subst-pass-script.test subst-xfail-script.test'])
-AC_SUBST([prog_tests], ['subst-pass-prog.test$(EXEEXT) 
subst-xfail-prog.test$(EXEEXT)'])
-AC_SUBST([xfail_tests], ['xfail-script.test subst-xfail-script.test 
xfail-prog$(EXEEXT) subst-xfail-prog.test$(EXEEXT)'])
+AC_SUBST([script_tests],
+         ['subst-pass-script.test subst-xfail-script.test'])
+AC_SUBST([prog_tests],
+         ['subst-pass-prog.test$(EXEEXT) subst-xfail-prog.test$(EXEEXT)'])
+AC_SUBST([xfail_tests],
+         ['xfail-script.test subst-xfail-script.test]dnl
+         [ xfail-prog$(EXEEXT) subst-xfail-prog.test$(EXEEXT)'])
 AC_OUTPUT
 END
 
@@ -48,11 +52,11 @@ cp pass-script.test subst-pass-script.test
 cp xfail-script.test subst-xfail-script.test
 
 cat > pass-prog.c <<'END'
-int main() { return 0; }
+int main (void) { return 0; }
 END
 cat > xfail-prog.c <<'END'
 #include <stdlib.h>
-int main() { return EXIT_FAILURE; }
+int main (void) { return EXIT_FAILURE; }
 END
 # The .test extension is removed for the default source file name:
 cp pass-prog.c subst-pass-prog.c
diff --git a/tests/check.test b/tests/check.test
index 54432ec..2f9f8b1 100755
--- a/tests/check.test
+++ b/tests/check.test
@@ -28,11 +28,13 @@ $ACLOCAL
 $AUTOMAKE
 
 grep '^check-TESTS:' Makefile.in
-grep '^check-DEJAGNU' Makefile.in && Exit 1
+grep 'check-DEJAGNU' Makefile.in && Exit 1
 
 # check-TESTS is phony.
-sed -n '/^\.PHONY:/,/^$/p' Makefile.in | grep check-TESTS
+sed -n '/^\.PHONY:/,/^$/p' Makefile.in | $EGREP '(^| )check-TESTS($| )'
 
 # check should depend directly on check-am (similar tests are
 # in check2.test and check3.test).
-grep '^check: check-am' Makefile.in
+$EGREP '^check:.* check-am( |$)' Makefile.in
+
+:
diff --git a/tests/check10.test b/tests/check10.test
index 7025b98..0705188 100755
--- a/tests/check10.test
+++ b/tests/check10.test
@@ -27,15 +27,15 @@ TESTS = fail pass skip xfail xpass fail2 pass2 skip2 xfail2 
xpass2
 XFAIL_TESTS = xfail xpass xfail2 xpass2
 END
 
-cat >>pass <<'END'
+cat > pass <<'END'
 #! /bin/sh
 exit 0
 END
-cat >>fail <<'END'
+cat > fail <<'END'
 #! /bin/sh
 exit 1
 END
-cat >>skip <<'END'
+cat > skip <<'END'
 #! /bin/sh
 exit 77
 END
@@ -69,7 +69,7 @@ $AUTOMAKE
   env TESTS='pass skip xfail' $MAKE -e check
   $MAKE check
   :
-) >stdout
+) >stdout || { cat stdout; Exit 1; }
 cat stdout
 
 grep '1 [tT]ests' stdout && Exit 1
diff --git a/tests/check11.test b/tests/check11.test
index 912b619..5305c96 100755
--- a/tests/check11.test
+++ b/tests/check11.test
@@ -26,7 +26,7 @@ cat > Makefile.am << 'END'
 TESTS = skip skip2
 END
 
-cat >>skip <<'END'
+cat > skip <<'END'
 #! /bin/sh
 exit 77
 END
@@ -38,11 +38,11 @@ $AUTOCONF
 $AUTOMAKE -a
 
 ./configure
-env TESTS=skip $MAKE -e check >stdout
+env TESTS=skip $MAKE -e check >stdout || { cat stdout; Exit 1; }
 cat stdout
 grep '1.*passed' stdout && Exit 1
 
-env TESTS="skip skip2" $MAKE -e check >stdout
+env TESTS="skip skip2" $MAKE -e check >stdout || { cat stdout; Exit 1; }
 cat stdout
 grep '2.*passed' stdout && Exit 1
 
diff --git a/tests/check2.test b/tests/check2.test
index 9b8bfb1..d215518 100755
--- a/tests/check2.test
+++ b/tests/check2.test
@@ -47,16 +47,20 @@ $ACLOCAL
 $AUTOCONF
 $AUTOMAKE
 ./configure
-$MAKE check >stdout
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
 cat stdout
-grep 'PASS: subrun.sh' stdout
+grep '^PASS: subrun\.sh *$' stdout
+grep 'PASS.*echo\.sh' stdout && Exit 1
 
 # check should depend directly on check-am (similar tests are
 # in check.test and check3.test).
-grep 'check: check-recursive' Makefile.in
-grep 'check: check-am' dir/Makefile.in
+$EGREP '^check:.* check-recursive( |$)' Makefile.in
+$EGREP '^check:.* check-am( |$)' dir/Makefile.in
 
 # Make sure subrun.sh is still on its line as above.  This means Automake
 # hasn't rewritten the TESTS line unnecessarily (we can tell, because all
 # Automake variables are reformatted by VAR_PRETTY).
-grep '  subrun.sh' Makefile.in
+grep '^  subrun\.sh$' Makefile.in
+
+:
diff --git a/tests/check3.test b/tests/check3.test
index be40e3f..b4f4f9d 100755
--- a/tests/check3.test
+++ b/tests/check3.test
@@ -56,14 +56,16 @@ $ACLOCAL
 $AUTOCONF
 $AUTOMAKE
 ./configure --prefix "`pwd`/inst"
-$MAKE check >stdout
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
 cat stdout
-grep 'PASS: subrun.sh' stdout
+grep '^PASS: subrun\.sh *$' stdout
+grep 'PASS.*echo\.sh' stdout && Exit 1
 
-# check should depend directly on check-am (similar tests are
-# in check.test and check2.test).
-$FGREP 'check: $(BUILT_SOURCES)' Makefile.in
-$FGREP 'check: $(BUILT_SOURCES)' dir/Makefile.in
+# check should depend directly on $(BUILT_SOURCES) (similar tests
+# are in check.test and check2.test).
+$EGREP '^check:.* \$\(BUILT_SOURCES\)( |$)' Makefile.in
+$EGREP '^check:.* \$\(BUILT_SOURCES\)( |$)' dir/Makefile.in
 
 $MAKE clean
 # Sanity checks
@@ -73,3 +75,5 @@ test ! -f dir/command2.inc
 $MAKE install
 test -f command1.inc
 test -f dir/command2.inc
+
+:
diff --git a/tests/check4.test b/tests/check4.test
index 99adb23..417b2e4 100755
--- a/tests/check4.test
+++ b/tests/check4.test
@@ -48,35 +48,35 @@ $ACLOCAL
 $AUTOCONF
 $AUTOMAKE
 ./configure --prefix "`pwd`/inst"
+
 $MAKE check >stdout && { cat stdout; Exit 1; }
 cat stdout
-grep 'FAIL: fail.sh' stdout
-grep 'PASS: ok.sh' stdout && Exit 1
+grep '^FAIL: fail\.sh *$' stdout
+grep '^PASS: ok\.sh *$' stdout && Exit 1
 
 # The exit status of `make -k' can be anything
 # (depending on the Make implementation)
 $MAKE -k check >stdout || :
 cat stdout
-grep 'FAIL: fail.sh' stdout
-grep 'PASS: ok.sh' stdout
+grep '^FAIL: fail\.sh *$' stdout
+grep '^PASS: ok\.sh *$' stdout
 
 # should also works when -k is not in first position
 $MAKE -s -k check >stdout || :
 cat stdout
-grep 'FAIL: fail.sh' stdout
-grep 'PASS: ok.sh' stdout
+grep '^FAIL: fail\.sh *' stdout
+grep '^PASS: ok\.sh *' stdout
 
 # The rest of the test is for GNU Make.
 
-# Use --version AND -v, because SGI Make doesn't fail on --version.
-# Also grep for GNU because newer versions of FreeBSD make do
-# not complain about `--version' (they seem to silently ignore it).
-($MAKE --version -v | grep GNU) || Exit 0
+if using_gmake; then
+  # Try with a long-option that do not have a short option equivalent
+  # (here, --no-print-directory).  That should cause all options to
+  # appear verbatim in MAKEFLAGS.
+  $MAKE --no-print-directory -k check >stdout || :
+  cat stdout
+  grep '^FAIL: fail\.sh *$' stdout
+  grep '^PASS: ok\.sh *$' stdout
+fi
 
-# Try with a long-option that do not have a short option equivalent
-# (here, --no-print-directory).  That should cause all options to
-# appear verbatim in MAKEFLAGS.
-$MAKE --no-print-directory -k check >stdout || :
-cat stdout
-grep 'FAIL: fail.sh' stdout
-grep 'PASS: ok.sh' stdout
+:
diff --git a/tests/check6.test b/tests/check6.test
index b3ba9d8..982ffbf 100755
--- a/tests/check6.test
+++ b/tests/check6.test
@@ -50,3 +50,5 @@ mkdir build
 cd build
 ../configure
 $MAKE check
+
+:
diff --git a/tests/check7.test b/tests/check7.test
index 5c5f4f3..39c9d77 100755
--- a/tests/check7.test
+++ b/tests/check7.test
@@ -28,20 +28,25 @@ TESTS = $(XFAIL_TESTS)
 XFAIL_TESTS = a b c d
 check_PROGRAMS = a c d
 check_SCRIPTS = b
+EXTRA_DIST = $(check_SCRIPTS)
 
+.PHONY: print-xfail-tests
 print-xfail-tests:
        @echo BEG: $(XFAIL_TESTS) :END
 END
 
-cat >>b <<'END'
+cat > b <<'END'
 #! /bin/sh
 exit 1
 END
 chmod a+x b
 
-cat >>a.c <<'END'
+cat > a.c <<'END'
 #include <stdlib.h>
-int main() { return EXIT_FAILURE; }
+int main (void)
+{
+  return EXIT_FAILURE;
+}
 END
 
 cp a.c c.c
@@ -53,12 +58,10 @@ $AUTOMAKE -a
 
 ./configure
 $MAKE check
-EXEEXT=.bin $MAKE -e print-xfail-tests >foo
-cat foo
-grep 'BEG: a.bin b c.bin d.bin :END' foo
+EXEEXT=.bin $MAKE -e print-xfail-tests >stdout || {  cat stdout; Exit 1; }
+cat stdout
+$FGREP 'BEG: a.bin b c.bin d.bin :END' stdout
 
-$MAKE distclean
-mkdir build
-cd build
-../configure
-$MAKE check
+$MAKE distcheck
+
+:
diff --git a/tests/check8.test b/tests/check8.test
index c8711e8..ae9d375 100755
--- a/tests/check8.test
+++ b/tests/check8.test
@@ -34,29 +34,29 @@ END
 
 mkdir sub
 
-cat >>foo <<'END'
+cat > foo <<'END'
 #! /bin/sh
 test -f "$srcdir/Makefile.am"
 END
-cat >>sub/foo <<'END'
+cat > sub/foo <<'END'
 #! /bin/sh
 test -f "$srcdir/Makefile.am"
 END
 chmod a+x foo sub/foo
 
-cat >>bar.c <<'END'
-int main() { return 0; }
+cat > bar.c <<'END'
+int main (void) { return 0; }
 END
-cat >>sub/bar.c <<'END'
-int main() { return 0; }
+cat > sub/bar.c <<'END'
+int main (void) { return 0; }
 END
-cat >>baz.c <<'END'
+cat > baz.c <<'END'
 #include <stdlib.h>
-int main() { return EXIT_FAILURE; }
+int main (void) { return EXIT_FAILURE; }
 END
-cat >>sub/baz.c <<'END'
+cat > sub/baz.c <<'END'
 #include <stdlib.h>
-int main() { return EXIT_FAILURE; }
+int main (void) { return EXIT_FAILURE; }
 END
 
 $ACLOCAL
diff --git a/tests/parallel-tests.test b/tests/parallel-tests.test
index da961e1..ea92ae7 100755
--- a/tests/parallel-tests.test
+++ b/tests/parallel-tests.test
@@ -42,19 +42,19 @@ END
 # foo.test and bar.test sleep to ensure their logs are always strictly newer
 # than the logs of their prerequisites, for HP-UX make.  The quoting pleases
 # maintainer-check.
-cat >>foo.test <<'END'
+cat > foo.test <<'END'
 #! /bin/sh
 echo "this is $0"
 sleep '1'
 exit 0
 END
-cat >>bar.test <<'END'
+cat > bar.test <<'END'
 #! /bin/sh
 echo "this is $0"
 sleep '1'
 exit 99
 END
-cat >>baz.test <<'END'
+cat > baz.test <<'END'
 #! /bin/sh
 echo "this is $0"
 exit 1
diff --git a/tests/parallel-tests10.test b/tests/parallel-tests10.test
index cdf148d..5718611 100755
--- a/tests/parallel-tests10.test
+++ b/tests/parallel-tests10.test
@@ -20,6 +20,7 @@
 
 parallel_tests=yes
 . ./defs || Exit 1
+
 cat >> configure.in << 'END'
 AC_OUTPUT
 END
@@ -43,4 +44,4 @@ $AUTOMAKE -a
 ./configure
 $MAKE check
 
-Exit 0
+:
diff --git a/tests/parallel-tests2.test b/tests/parallel-tests2.test
index 8fe5d30..85256c8 100755
--- a/tests/parallel-tests2.test
+++ b/tests/parallel-tests2.test
@@ -35,18 +35,18 @@ bla:
 CLEANFILES = bla
 END
 
-cat >>foo.test <<'END'
+cat > foo.test <<'END'
 #! /bin/sh
 echo "this is $0"
 test -f bla || exit 1
 exit 0
 END
-cat >>bar.test <<'END'
+cat > bar.test <<'END'
 #! /bin/sh
 echo "this is $0"
 exit 99
 END
-cat >>baz.test <<'END'
+cat > baz.test <<'END'
 #! /bin/sh
 echo "this is $0"
 exit 1
@@ -84,4 +84,5 @@ env TESTS=foo.test $MAKE -e recheck-html
 test -f bla
 test ! -f foo.log
 test -f mylog.html
+
 :
diff --git a/tests/parallel-tests3.test b/tests/parallel-tests3.test
index 1d621eb..a8004e7 100755
--- a/tests/parallel-tests3.test
+++ b/tests/parallel-tests3.test
@@ -22,7 +22,7 @@ required=GNUmake
 . ./defs || Exit 1
 
 case $MAKE in
-  *\ -j*) skip_ "this test does not work well if \$MAKE contains \`-j'";;
+  *\ -j*) skip_ "\$MAKE contains \`-j'";;
 esac
 
 cat >> configure.in << 'END'
@@ -35,13 +35,12 @@ END
 
 for i in 1 2 3 4 5 6 7 8; do
   echo "TESTS += foo$i.test" >> Makefile.am
-  cat >foo$i.test <<'END'
-#! /bin/sh
-echo "this is $0"
-# hack around maintainer-check check:
-sleep='sleep '1
-$sleep
-exit 0
+  unindent >foo$i.test <<'END'
+    #! /bin/sh
+    echo "this is $0"
+    # FIXME: spurious quote to please maintainer-check
+    sleep '1'
+    exit 0
 END
   chmod a+x foo$i.test
 done
@@ -78,4 +77,5 @@ do
   $sleep
 done
 $sleep
+
 :
diff --git a/tests/parallel-tests5.test b/tests/parallel-tests5.test
index 4d981c3..b1bc5b0 100755
--- a/tests/parallel-tests5.test
+++ b/tests/parallel-tests5.test
@@ -41,16 +41,16 @@ END
 for i in 1 2 3 4 5 6 7 8; do
   echo "scripts += foo$i.test" >> Makefile.am
   echo "programs += foo$i.prog" >> Makefile.am
-  cat >foo$i.test <<'END'
-#! /bin/sh
-echo "this is $0"
-exit 0
+  unindent >foo$i.test <<'END'
+    #! /bin/sh
+    echo "this is $0"
+    exit 0
 END
   cat >foo$i.c <<'END'
-int main()
-{
-  return 0;
-}
+    int main (void)
+    {
+      return 0;
+    }
 END
   chmod a+x foo$i.test
 done
diff --git a/tests/parallel-tests8.test b/tests/parallel-tests8.test
index d1d5b74..84ac460 100755
--- a/tests/parallel-tests8.test
+++ b/tests/parallel-tests8.test
@@ -44,7 +44,7 @@ EXTRA_DIST = foo.in foo.test
 DISTCLEANFILES = foo.test
 END
 
-cat >>foo.in <<'END'
+cat > foo.in <<'END'
 #! /bin/sh
 echo "this is $0"
 exit 0
@@ -53,8 +53,8 @@ END
 $ACLOCAL
 $AUTOCONF
 AUTOMAKE_fails -a
-grep '(srcdir.*bar' stderr
-grep 'top_srcdir.*baz' stderr
+grep '$(srcdir).*TESTS.*bar\.test' stderr
+grep '$(top_srcdir).*TESTS.*baz\.test' stderr
 
 sed '/^TESTS +=.*srcdir/d' < Makefile.am > t
 mv -f t Makefile.am
diff --git a/tests/parallel-tests9.test b/tests/parallel-tests9.test
index e6191d9..b06d364 100755
--- a/tests/parallel-tests9.test
+++ b/tests/parallel-tests9.test
@@ -33,18 +33,18 @@ bla:
        echo bla > $@
 END
 
-cat >>foo.test <<'END'
+cat > foo.test <<'END'
 #! /bin/sh
 echo "this is $0"
 test -f bla || exit 1
 exit 0
 END
-cat >>bar.test <<'END'
+cat > bar.test <<'END'
 #! /bin/sh
 echo "this is $0"
 exit 99
 END
-cat >>baz.test <<'END'
+cat > baz.test <<'END'
 #! /bin/sh
 echo "this is $0"
 exit 1
@@ -83,4 +83,5 @@ test ! -f foo.log
 test ! -f bar.log
 test ! -f baz.log
 test -f mylog.log
+
 :
-- 
1.7.2.3