[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [GSoC] Early design discussion for TAP/SubUnit support in automake.
|
From: |
Stefano Lattarini |
|
Subject: |
Re: [GSoC] Early design discussion for TAP/SubUnit support in automake. |
|
Date: |
Sun, 22 May 2011 11:38:13 +0200 |
|
User-agent: |
KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) |
[Adding automake-patches in CC:]
Reference:
<http://lists.gnu.org/archive/html/automake/2011-05/msg00017.html>
On Friday 20 May 2011, Stefano Lattarini wrote:
> Reference:
> <http://lists.gnu.org/archive/html/automake/2011-04/msg00069.html>
>
> On Friday 29 April 2011, Stefano Lattarini wrote:
> >
> > First Tentative Decisions and Questions on the User Interface
> > =============================================================
> >
> > [CUT]
> >
> > 2. New automake option `tests-protocol'
> > ---------------------------------------
> >
> > The Tap/SubUnit support in the Automake-generated testsuite drivers
> > should be enabled by a new (argument-requiring) option `tests-protocol',
> > that will be used to specify the level of support for, detection of, and
> > enforcing of SubUnit/TAP streams.
> >
> > The possible values for `tests-protocol' will be:
> > - tests-protocol=tap
> > All test scripts are expected to use the TAP protocol.
> > - tests-protocol=subunit
> > All test scripts are expected to use the SubUnit protocol.
> > - tests-protocol=adaptive
> > Each test script is expected to print on its first line of output
> > which protocol it uses (the exact format of this special line is
> > still to be determined); if this line is unrecognized, the driver
> > should assume that the test script uses no protocol. Also, in
> > this case, we should continue to honour XFAIL_TESTS. All of this
> > should help to maximize backward-compatibility.
> >
> Thinking again about this, it might be worth trying to be even more consistent
> with the existing parallel-tests functionality, and use an `ext_TEST_PROTOCOL'
> variable (or similar) instead of a global `tests-protocol' option. With some
> tweaking to the post-processing of `.log' files done in `lib/am/check.am' (to
> generate `$(TEST_SUITE_LOG)'), this might allow greater code reuse and a more
> consistent API.
>
> I've started experimenting with this idea, and I'm not seeing any obvious
> shortcoming right now. I'm hoping I'll be able to post some experimental
> patches soon enough.
>
Here are the promised patches. They are still rough, somewhat lacking in
testsuite exposure, and completely lacking in documentation, but they might
be a good first step IMHO.
> As usual, if you have suggestions, ideas or objection, I'd ble glad to hear
> them.
>
Ping :-)
> >
> > [CUT]
> >
> > 4. RST support and HTML generation: should be dropped?
> > ------------------------------------------------------
> >
> > I haven't really looked into this yet, so I don't know how much additional
> > work (if any!) would be to continue to support generation of HTML reports
> > from the testsuite-generated `.log' files in the new TAP/SubUnit testsuite
> > driver. Still, one question that bugs me is: how much is this HTML report
> > generation used in Real Life? Is it really worth to support it out of the
> > box, or could we leave it to the interested used to write their own rules
> > (maybe in a reusable `*.am' fragment)?
> >
> BTW, I also hope that new interface I'm planning will make it easier to
> implement HTML report generation also for TAP and SubUnit tests, with a
> consistent reuse of the existing code. In which case my considerations
> above will become moot.
>
I haven't looked into this yet, but from a quick glance it seems that a careful
use of reST comments in generated test logs and some simple extensions to the
$(TEST_SUITE_LOG) recipe in `lib/am/check.am' might do the trick. I'll get back
on this once I have a working prototype, maybe in a form of a testsuite
addition.
Regards,
Stefano
From 20c4c761e3cad44c328ac16f39796689412cdaa7 Mon Sep 17 00:00:00 2001
Message-Id: <address@hidden>
From: Stefano Lattarini <address@hidden>
Date: Tue, 10 May 2011 11:30:05 +0200
Subject: [PATCH 1/2] parallel-tests: add auxiliary script 'pt-wrap', refactor
---
ChangeLog | 54 ++++++++++++++
automake.in | 8 ++-
lib/Automake/tests/Makefile.in | 66 ++++++-----------
lib/Makefile.am | 2 +-
lib/Makefile.in | 2 +-
lib/am/check.am | 73 ++++++++-----------
lib/am/check2.am | 8 ++-
lib/pt-wrap | 130 ++++++++++++++++++++++++++++++++++
tests/Makefile.am | 2 +
tests/Makefile.in | 70 ++++++++----------
tests/check-no-ptwrap.test | 33 +++++++++
tests/check.test | 2 +
tests/check10.test | 4 +
tests/check2.test | 4 +
tests/check3.test | 2 +-
tests/check4.test | 4 +
tests/color.test | 10 +++-
tests/color2.test | 4 +-
tests/comment9.test | 2 +
tests/dejagnu.test | 2 +
tests/exeext4.test | 2 +-
tests/maken3.test | 2 +-
tests/maken4.test | 2 +-
tests/parallel-tests-interrupt.test | 16 ++++-
tests/parallel-tests-ptwrap.test | 93 ++++++++++++++++++++++++
tests/parallel-tests-reset-term.test | 4 +-
tests/posixsubst-tests.test | 2 +-
tests/repeated-options.test | 2 +-
28 files changed, 462 insertions(+), 143 deletions(-)
create mode 100755 lib/pt-wrap
create mode 100755 tests/check-no-ptwrap.test
create mode 100755 tests/parallel-tests-ptwrap.test
diff --git a/ChangeLog b/ChangeLog
index 4c436e5..ebc3bf7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,59 @@
2011-05-20 Stefano Lattarini <address@hidden>
+ parallel-tests: add auxiliary script 'pt-wrap', refactor
+ This refactoring should cause no API of functionality change, and
+ is meant only to simplify the future implementation of TAP and
+ SubUnit testsuite drivers.
+ TODO: add a more in-depth high-level description of the changes
+ TODO: introduced by this commit.
+ * lib/pt-wrap: New auxiliary script.
+ * lib/Makefile.am (dist_SCRIPT_DATA): Add it.
+ * automake.in (handle_tests): Require the new auxiliary script
+ `pt-wrap'. Define new makefile variable `$(am__pt_wrap)', used
+ to run that script in a proper way. Perform new substitution on
+ `WRAPPER' when processing the `check2.am' file.
+ * lib/check.am (am__tty_colors): Define new shell variable
+ `$am__color_tests'.
+ (am__rst_section): Removed, its role taken over by the `pt-wrap'
+ script.
+ (am__check_wrapper_options): New variable, contains the command
+ line options passed to `pt-wrap'.
+ (am__check_pre): Do not deal with temporary files and exit
+ traps anymore, as the `pt-wrap' script takes care of that now.
+ Define shell variable `$am__enable_hard_errors', used by
+ `$(am__check_wrapper_options)'. Reorder so that we don't
+ need to save and restore the value of the `TERM' environment
+ variable anymore.
+ Other related adjustments.
+ (am__check_post): Remove, as its role has been completely taken
+ over by the `pt-wrap' script.
+ * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
+ ?!GENERIC?%OBJ%): Call the test script through the Automake
+ substituted `%WRAPPER%', and honoring the command-line options
+ in `$(am__check_wrapper_options)'. Do not call the obsoleted
+ `$(am__check_post)' anymore.
+ * tests/check.test: Adjust.
+ * tests/check2.test : Likewise.
+ * tests/check3.test : Likewise.
+ * tests/check4.test : Likewise.
+ * tests/check10.test: Likewise.
+ * tests/color.test: Likewise.
+ * tests/color2.test: Likewise.
+ * tests/comment9.test: Likewise.
+ * tests/dejagnu.test: Likewise.
+ * tests/exeext4.test: Likewise.
+ * tests/maken3.test: Likewise.
+ * tests/maken4.test: Likewise.
+ * tests/parallel-tests-interrupt.test: Likewise.
+ * tests/parallel-tests-reset-term.test: Likewise.
+ * tests/posixsubst-tests.test: Likewise.
+ * tests/repeated-options.test: Likewise.
+ * tests/check-no-ptwrap.test: New test.
+ * tests/parallel-tests-ptwrap.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-20 Stefano Lattarini <address@hidden>
+
testsuite: avoid re-running few tests with 'parallel-tests' option
Some tests in our testsuite use the 'simple-tests' driver only
marginally, or simply as a mean to conveniently check unrelated
diff --git a/automake.in b/automake.in
index a1c6f30..211ee3c 100644
--- a/automake.in
+++ b/automake.in
@@ -4989,8 +4989,12 @@ sub handle_tests
append_exeext { exists $known_programs{$_[0]} } 'XFAIL_TESTS'
if (var ('XFAIL_TESTS'));
- if (option 'parallel-tests')
+ if (my $parallel_tests = option 'parallel-tests')
{
+ require_conf_file ($parallel_tests->{position}, FOREIGN, 'pt-wrap');
+ define_variable ('am__pt_wrap',
+ "\$(SHELL) $am_config_aux_dir/pt-wrap",
+ INTERNAL);
define_variable ('TEST_SUITE_LOG', 'test-suite.log', INTERNAL);
define_variable ('TEST_SUITE_HTML', '$(TEST_SUITE_LOG:.log=.html)',
INTERNAL);
my $suff = '.test';
@@ -5041,6 +5045,7 @@ sub handle_tests
GENERIC => 0,
OBJ => $obj,
SOURCE => $val,
+ WRAPPER => '$(am__pt_wrap)',
COMPILE =>'$(' . $compile . ')',
EXT => '',
am__EXEEXT => 'FALSE');
@@ -5080,6 +5085,7 @@ sub handle_tests
GENERIC => 1,
OBJ => '',
SOURCE => '$<',
+ WRAPPER => '$(am__pt_wrap)',
COMPILE => '$(' . $compile .
')',
EXT => $test_suffix,
am__EXEEXT => $am_exeext);
diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
index 4eb7c6a..df93aa5 100644
--- a/lib/Automake/tests/Makefile.in
+++ b/lib/Automake/tests/Makefile.in
@@ -75,11 +75,13 @@ DIST_SOURCES =
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='[0;31m'; \
grn='[0;32m'; \
lgn='[1;32m'; \
@@ -107,9 +109,8 @@ am__nobase_list = $(am__nobase_strip_setup); \
am__base_list = \
sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
-# Restructured Text title and section.
+# Restructured Text title.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
# Put stdin (possibly several lines separated by ". ") in a box.
am__text_box = $(AWK) '{ \
n = split($$0, lines, "\\. "); max = 0; \
@@ -125,60 +126,40 @@ am__text_box = $(AWK) '{ \
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+am__check_wrapper_options = \
+ --test-name "$$f" \
+ --test-dir "$$dir" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --xfail-tests '$(XFAIL_TESTS)'
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f address@hidden; \
-am__trap='rm -f '\''$(abs_builddir)/address@hidden'\''; (exit $$st); exit
$$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->address@hidden 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
-case " $(XFAIL_TESTS) " in \
- *[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat address@hidden >>$@; \
-rm -f address@hidden
+tst=$$dir$$f; log='$@'; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
+am__pt_wrap = $(SHELL) $(top_srcdir)/lib/pt-wrap
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
am__test_logs1 = $(TESTS:=.log)
@@ -471,7 +452,8 @@ recheck recheck-html:
list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS)
TEST_LOGS="'"$$list"'"'
.pl.log:
- @p='$<'; $(am__check_pre) $(PL_LOG_COMPILE) "$$tst" $(am__check_post)
+ @p='$<'; $(am__check_pre) \
+ $(am__pt_wrap) $(am__check_wrapper_options) -- $(PL_LOG_COMPILE) "$$tst"
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 5bdc02e..40104f9 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -29,7 +29,7 @@ dist_pkgvdata_DATA = COPYING INSTALL texinfo.tex ansi2knr.c
ansi2knr.1 \
scriptdir = $(pkgvdatadir)
dist_script_DATA = config.guess config.sub install-sh mdate-sh missing \
mkinstalldirs elisp-comp ylwrap acinstall depcomp compile py-compile \
- symlink-tree ar-lib
+ symlink-tree ar-lib pt-wrap
EXTRA_DIST = gnupload
diff --git a/lib/Makefile.in b/lib/Makefile.in
index 9195976..7159e00 100644
--- a/lib/Makefile.in
+++ b/lib/Makefile.in
@@ -239,7 +239,7 @@ dist_pkgvdata_DATA = COPYING INSTALL texinfo.tex ansi2knr.c
ansi2knr.1 \
scriptdir = $(pkgvdatadir)
dist_script_DATA = config.guess config.sub install-sh mdate-sh missing \
mkinstalldirs elisp-comp ylwrap acinstall depcomp compile py-compile \
- symlink-tree ar-lib
+ symlink-tree ar-lib pt-wrap
EXTRA_DIST = gnupload
all: all-recursive
diff --git a/lib/am/check.am b/lib/am/check.am
index 97ecb68..2eb5926 100644
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -20,11 +20,13 @@ if %?COLOR%
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='[0;31m'; \
grn='[0;32m'; \
lgn='[1;32m'; \
@@ -32,8 +34,7 @@ test "X$(AM_COLOR_TESTS)" != Xno \
std='[m'; \
}
else !%?COLOR%
-am__tty_colors = \
-red=; grn=; lgn=; blu=; std=
+am__tty_colors = red= grn= lgn= blu= std= am__color_tests=no
endif !%?COLOR%
.PHONY: check-TESTS
@@ -70,9 +71,8 @@ include inst-vars.am
## test is XFAIL or not. You can disable this feature by setting the
## variable DISABLE_HARD_ERRORS to a nonempty value.
-# Restructured Text title and section.
-am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
+# Restructured Text title.
+am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
# Put stdin (possibly several lines separated by ". ") in a box.
am__text_box = $(AWK) '{ \
@@ -91,59 +91,46 @@ am__text_box = $(AWK) '{ \
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to all log compiler wrappers.
+am__test_wrapper_flags = \
+ --test-name "$$f" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
+
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f address@hidden; \
-am__trap='rm -f '\''$(abs_builddir)/address@hidden'\''; (exit $$st); exit
$$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->address@hidden 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
+## The use of $dir below is required to account for VPATH
+## rewriting done by Sun make.
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat address@hidden >>$@; \
-rm -f address@hidden
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
$(TEST_SUITE_LOG): $(TEST_LOGS)
@$(am__sh_e_setup); \
diff --git a/lib/am/check2.am b/lib/am/check2.am
index 054c62d..2de4760 100644
--- a/lib/am/check2.am
+++ b/lib/am/check2.am
@@ -1,5 +1,5 @@
## automake - create Makefile.in from Makefile.am
-## Copyright (C) 2008, 2009 Free Software Foundation, Inc.
+## Copyright (C) 2008, 2009, 2011 Free Software Foundation, Inc.
## This program is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
@@ -17,7 +17,8 @@
## From a test file to a log file.
?GENERIC?%EXT%.log:
?!GENERIC?%OBJ%: %SOURCE%
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) \
+ %WRAPPER% $(am__test_wrapper_flags) -- %COMPILE% "$$tst"
## If no programs are built in this package, then this rule is removed
## at automake time. Otherwise, %am__EXEEXT% expands to a configure time
@@ -25,5 +26,6 @@
## conflict with the previous one.
if %am__EXEEXT%
?GENERIC?%EXT%$(EXEEXT).log:
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) \
+ %WRAPPER% $(am__test_wrapper_flags) -- %COMPILE% "$$tst"
endif %am__EXEEXT%
diff --git a/lib/pt-wrap b/lib/pt-wrap
new file mode 100755
index 0000000..9f3b1ac
--- /dev/null
+++ b/lib/pt-wrap
@@ -0,0 +1,130 @@
+#! /bin/sh
+# pt-wrap - basic wrapper for test script invocations within the
+# `parallel-tests' testsuite driver.
+
+scriptversion=2011-05-20.xxx; # UTC
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# As a special exception to the GNU General Public License, if you
+# distribute this file as part of a program that contains a
+# configuration script generated by Autoconf, you may include it under
+# the same distribution terms that you use for the rest of that program.
+
+# This file is maintained in Automake, please report
+# bugs to <address@hidden> or send patches to
+# <address@hidden>.
+
+# Make unconditional expansion of undefined variables an error. This
+# helps a lot in preventing typo-related bugs.
+set -u
+
+fatal ()
+{
+ echo "$0: fatal: $*" >&2
+ exit 1
+}
+
+usage_error ()
+{
+ echo "$0: $*" >&2
+ print_usage >&2
+ exit 2
+}
+
+print_usage ()
+{
+ cat <<END
+Usage:
+ pt-wrap [--help|--version] --test-name=NAME --log-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--] TEST-SCRIPT
+The \`--test-name' and \`--log-file' options are mandatory.
+END
+}
+
+# Restructured Text section.
+rst_section () { sed 'p;s/./=/g;p;g'; }
+
+# TODO: better error handling in option parsing (in particular, ensure
+# TODO: $logfile and $test_name are defined).
+test_name= # Used for reporting.
+logfile= # Where to save the result and output of the test script.
+expect_failure=no
+color_tests=no
+enable_hard_errors=yes
+while test $# -gt 0; do
+ case $1 in
+ --help) print_usage; exit $?;;
+ --version) echo "pt-wrapt $scriptversion"; exit $?;;
+ --test-name) test_name=$2; shift;;
+ --log-file) logfile=$2; shift;;
+ --color-tests) color_tests=$2; shift;;
+ --expect-failure) expect_failure=$2; shift;;
+ --enable-hard-errors) enable_hard_errors=$2; shift;;
+ --) shift; break;;
+ -*) usage_error "invalid option: '$1'";;
+ esac
+ shift
+done
+
+if test $color_tests = yes; then
+ red='[0;31m' # Red.
+ grn='[0;32m' # Green.
+ lgn='[1;32m' # Light green.
+ blu='[1;34m' # Blue.
+ std='[m' # No color.
+else
+ red= grn= lgn= blu= std=
+fi
+
+tmpfile=$logfile-t
+do_exit='rm -f $tmpfile; (exit $st); exit $st'
+trap "st=129; $do_exit" 1
+trap "st=130; $do_exit" 2
+trap "st=141; $do_exit" 13
+trap "st=143; $do_exit" 15
+rm -f $tmpfile
+
+# Test script is run here.
+"$@" >$tmpfile 2>&1
+estatus=$?
+if test $enable_hard_errors = no && test $estatus -eq 99; then
+ estatus=1
+fi
+
+case $estatus:$expect_failure in
+ 0:yes) col=$red; res=XPASS;;
+ 0:*) col=$grn; res=PASS ;;
+ 77:*) col=$blu; res=SKIP ;;
+ 99:*) col=$red; res=FAIL ;;
+ *:yes) col=$lgn; res=XFAIL;;
+ *:*) col=$red; res=FAIL ;;
+esac
+echo "${col}${res}${std}: $test_name"
+echo "$res: $test_name (exit: $estatus)" | rst_section > $logfile
+cat $tmpfile >> $logfile
+rm -f $tmpfile
+
+# Local Variables:
+# mode: shell-script
+# sh-indentation: 2
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "scriptversion="
+# time-stamp-format: "%:y-%02m-%02d.%02H"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
+# End:
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 1c8e48c..a86f884 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -234,6 +234,7 @@ check12.test \
check-exported-srcdir.test \
check-tests-in-builddir.test \
check-tests_environment.test \
+check-no-ptwrap.test \
checkall.test \
clean.test \
clean2.test \
@@ -717,6 +718,7 @@ parallel-tests-unreadable-log.test \
parallel-tests-subdir.test \
parallel-tests-interrupt.test \
parallel-tests-reset-term.test \
+parallel-tests-ptwrap.test \
parse.test \
percent.test \
percent2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 5b90af6..a9dd0eb 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -80,11 +80,13 @@ DIST_SOURCES =
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='[0;31m'; \
grn='[0;32m'; \
lgn='[1;32m'; \
@@ -112,9 +114,8 @@ am__nobase_list = $(am__nobase_strip_setup); \
am__base_list = \
sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
-# Restructured Text title and section.
+# Restructured Text title.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
# Put stdin (possibly several lines separated by ". ") in a box.
am__text_box = $(AWK) '{ \
n = split($$0, lines, "\\. "); max = 0; \
@@ -130,58 +131,43 @@ am__text_box = $(AWK) '{ \
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to all log compiler wrappers.
+am__test_wrapper_flags = \
+ --test-name "$$f" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f address@hidden; \
-am__trap='rm -f '\''$(abs_builddir)/address@hidden'\''; (exit $$st); exit
$$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->address@hidden 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat address@hidden >>$@; \
-rm -f address@hidden
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
TEST_SUITE_LOG = test-suite.log
@@ -189,6 +175,7 @@ TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
TEST_EXTENSIONS = .test
am__test_logs1 = $(TESTS:=.log)
TEST_LOGS = $(am__test_logs1:.test.log=.log)
+TEST_LOG_WRAPPER = $(SHELL) $(top_srcdir)/lib/pt-wrap
TEST_LOG_COMPILE = $(TEST_LOG_COMPILER) $(AM_TEST_LOG_FLAGS) \
$(TEST_LOG_FLAGS)
TEST_LOGS_TMP = $(TEST_LOGS:.log=.log-t)
@@ -498,6 +485,7 @@ check12.test \
check-exported-srcdir.test \
check-tests-in-builddir.test \
check-tests_environment.test \
+check-no-ptwrap.test \
checkall.test \
clean.test \
clean2.test \
@@ -981,6 +969,7 @@ parallel-tests-unreadable-log.test \
parallel-tests-subdir.test \
parallel-tests-interrupt.test \
parallel-tests-reset-term.test \
+parallel-tests-ptwrap.test \
parse.test \
percent.test \
percent2.test \
@@ -1484,7 +1473,8 @@ recheck recheck-html:
list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS)
TEST_LOGS="'"$$list"'"'
.test.log:
- @p='$<'; $(am__check_pre) $(TEST_LOG_COMPILE) "$$tst" $(am__check_post)
+ @p='$<'; $(am__check_pre) \
+ $(TEST_LOG_WRAPPER) $(am__test_wrapper_flags) -- $(TEST_LOG_COMPILE)
"$$tst"
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
diff --git a/tests/check-no-ptwrap.test b/tests/check-no-ptwrap.test
new file mode 100755
index 0000000..214abfd
--- /dev/null
+++ b/tests/check-no-ptwrap.test
@@ -0,0 +1,33 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'pt-wrap' doesn't get needlessly installed
+# when the 'parallel-tests' option is not used.
+
+parallel_tests=no
+. ./defs || Exit 1
+
+echo 'TESTS = foo.test' > Makefile.am
+
+$ACLOCAL
+
+for opts in '' '-a' '--add-missing --copy'; do
+ $AUTOMAKE $opts
+ grep '[ ]pt.*wrap' Makefile.in && Exit 1
+ find . | grep 'pt.*wrap' && Exit 1
+done
+
+:
diff --git a/tests/check.test b/tests/check.test
index 54432ec..cda84ac 100755
--- a/tests/check.test
+++ b/tests/check.test
@@ -22,6 +22,8 @@ cat > Makefile.am << 'END'
TESTS = frob.test
END
+test x"$parallel_tests" != x"yes" || : > pt-wrap
+
: > frob.test
$ACLOCAL
diff --git a/tests/check10.test b/tests/check10.test
index 7025b98..e98093e 100755
--- a/tests/check10.test
+++ b/tests/check10.test
@@ -48,6 +48,10 @@ cp fail fail2
cp xfail xfail2
cp skip skip2
+if test x"$parallel_tests" = x"yes"; then
+ cp "$top_testsrcdir/lib/pt-wrap" .
+fi
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
diff --git a/tests/check2.test b/tests/check2.test
index 9b8bfb1..b52c2bc 100755
--- a/tests/check2.test
+++ b/tests/check2.test
@@ -43,6 +43,10 @@ echo.sh:
CLEANFILES = echo.sh
END
+if test x"$parallel_tests" = x"yes"; then
+ cp "$top_testsrcdir/lib/pt-wrap" .
+fi
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
diff --git a/tests/check3.test b/tests/check3.test
index be40e3f..ccfb923 100755
--- a/tests/check3.test
+++ b/tests/check3.test
@@ -54,7 +54,7 @@ END
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure --prefix "`pwd`/inst"
$MAKE check >stdout
cat stdout
diff --git a/tests/check4.test b/tests/check4.test
index 99adb23..efd06d6 100755
--- a/tests/check4.test
+++ b/tests/check4.test
@@ -44,6 +44,10 @@ END
chmod +x ok.sh dir/fail.sh
+if test x"$parallel_tests" = x"yes"; then
+ cp "$top_testsrcdir/lib/pt-wrap" .
+fi
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
diff --git a/tests/color.test b/tests/color.test
index 570b21d..2bf5b6d 100755
--- a/tests/color.test
+++ b/tests/color.test
@@ -67,8 +67,14 @@ cp pass xpass
chmod +x pass fail skip xpass xfail
$ACLOCAL
-$AUTOMAKE
$AUTOCONF
+
+if test x"$parallel_tests" = x"yes"; then
+ $AUTOMAKE --add-missing
+else
+ $AUTOMAKE
+fi
+
./configure
test_color ()
@@ -95,3 +101,5 @@ test_no_color ()
AM_COLOR_TESTS=always $MAKE -e check >stdout && { cat stdout; Exit 1; }
cat stdout
test_color
+
+:
diff --git a/tests/color2.test b/tests/color2.test
index eb9c527..4f8862e 100755
--- a/tests/color2.test
+++ b/tests/color2.test
@@ -80,7 +80,7 @@ cp pass xpass
chmod +x pass fail skip xpass xfail
$ACLOCAL
-$AUTOMAKE
+$AUTOMAKE -a
$AUTOCONF
./configure
@@ -120,3 +120,5 @@ AM_COLOR_TESTS=no MAKE=$MAKE expect -f expect-make >stdout \
|| { cat stdout; Exit 1; }
cat stdout
test_no_color
+
+:
diff --git a/tests/comment9.test b/tests/comment9.test
index e95d99b..9023958 100755
--- a/tests/comment9.test
+++ b/tests/comment9.test
@@ -30,6 +30,8 @@ TESTS = \
7.test
EOF
+: > pt-wrap
+
$ACLOCAL
$AUTOMAKE
diff --git a/tests/dejagnu.test b/tests/dejagnu.test
index df385b0..1998941 100755
--- a/tests/dejagnu.test
+++ b/tests/dejagnu.test
@@ -25,6 +25,8 @@ AUTOMAKE_OPTIONS = dejagnu
TESTS = frob.test
END
+test x"$parallel_tests" != x"yes" || : > pt-wrap
+
$ACLOCAL
$AUTOMAKE
diff --git a/tests/exeext4.test b/tests/exeext4.test
index c3f968c..2ec3bca 100755
--- a/tests/exeext4.test
+++ b/tests/exeext4.test
@@ -50,7 +50,7 @@ END
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --add-missing --copy
./configure
$MAKE print-bin > output
cat output
diff --git a/tests/maken3.test b/tests/maken3.test
index 5a77f1a..eb54d35 100755
--- a/tests/maken3.test
+++ b/tests/maken3.test
@@ -149,7 +149,7 @@ check_targets ()
done
}
-$AUTOMAKE -Wno-override
+$AUTOMAKE -a -Wno-override
./configure
check_targets || Exit 1
diff --git a/tests/maken4.test b/tests/maken4.test
index 5c21943..4a11910 100755
--- a/tests/maken4.test
+++ b/tests/maken4.test
@@ -154,7 +154,7 @@ check_targets ()
done
}
-$AUTOMAKE -Wno-override
+$AUTOMAKE -a -Wno-override
./configure
check_targets || Exit 1
diff --git a/tests/parallel-tests-interrupt.test
b/tests/parallel-tests-interrupt.test
index 45e55d4..3bc5dff 100755
--- a/tests/parallel-tests-interrupt.test
+++ b/tests/parallel-tests-interrupt.test
@@ -28,12 +28,20 @@ END
cat > Makefile.am << 'END'
TESTS = foo.test
-## Ugly, but required by foo.test. See below.
-TEST_LOG_COMPILER = echo $$$$ > pid && exec 9>&2 && $(SHELL) -x
+## Provide more debugging info.
+AM_TEST_LOG_COMPILER = $(SHELL) -x
+## Rut required by foo.test; see below.
+AM_TESTS_ENVIRONMENT = exec 9>&2;
END
# This is hacky and ugly, but has the great advantage of avoiding us a lot
# of pain with background processes and related synchronization issues.
+
+cat - "$top_testsrcdir"/lib/pt-wrap > pt-wrap <<'END'
+#!/bin/sh
+echo $$ > pid
+END
+
cat > foo.test << 'END'
#!/bin/sh
exec 2>&9
@@ -48,7 +56,7 @@ chmod a+x foo.test
$ACLOCAL
$AUTOCONF
-$AUTOMAKE -a
+$AUTOMAKE
./configure
@@ -60,6 +68,8 @@ for signum in $trapped_signals; do
rm -f pid fail *.log *.log-t
env signum=$signum $MAKE check && { ls -l; Exit 1; }
ls -l
+ # These files shouldn't exit, but in case they do, their content might
+ # provide helpful information about the causes of the failure(s).
cat foo.log-t || :
cat foo.log || :
cat test-suite.log || :
diff --git a/tests/parallel-tests-ptwrap.test b/tests/parallel-tests-ptwrap.test
new file mode 100755
index 0000000..408233e
--- /dev/null
+++ b/tests/parallel-tests-ptwrap.test
@@ -0,0 +1,93 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'pt-wrap' gets automatically installed
+# in the correct directory by 'parallel-tests' option.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+: Try first with parallel-tests defined in AM_INIT_AUTOMAKE.
+
+mkdir am-init-automake
+cd am-init-automake
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([my_aux_dir])
+AM_INIT_AUTOMAKE([parallel-tests])
+AC_CONFIG_FILES([Makefile sub/Makefile])
+AC_OUTPUT
+END
+
+mkdir sub my_aux_dir
+
+cat > Makefile.am <<END
+SUBDIRS = sub
+TESTS = foo.test
+END
+
+cat > sub/Makefile.am <<END
+TESTS = bar.test
+END
+
+$ACLOCAL
+$AUTOMAKE -a 2>stderr || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+
+ls -l . sub my_aux_dir # For debugging.
+test -f my_aux_dir/pt-wrap
+test ! -r pt-wrap
+test ! -r sub/pt-wrap
+
+grep '^configure\.in:3:.*installing.*my_aux_dir/pt-wrap' stderr
+
+cd ..
+
+: Now try with parallel-tests defined in AUTOMAKE_OPTIONS.
+
+mkdir automake-options
+cd automake-options
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([build-aux])
+AM_INIT_AUTOMAKE
+AC_CONFIG_FILES([dir/GNUmakefile])
+AC_OUTPUT
+END
+
+mkdir build-aux dir
+
+cat > dir/GNUmakefile.am <<END
+TESTS = foo.test
+AUTOMAKE_OPTIONS = parallel-tests
+TESTS += bar.test
+END
+
+$ACLOCAL
+$AUTOMAKE --add-missing --copy dir/GNUmakefile 2>stderr \
+ || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+
+ls -l . dir build-aux # For debugging.
+test -f build-aux/pt-wrap
+test ! -r pt-wrap
+test ! -r dir/pt-wrap
+
+grep '^dir/GNUmakefile\.am:2:.*installing.*build-aux/pt-wrap' stderr
+
+:
diff --git a/tests/parallel-tests-reset-term.test
b/tests/parallel-tests-reset-term.test
index 3a222c0..844f64c 100755
--- a/tests/parallel-tests-reset-term.test
+++ b/tests/parallel-tests-reset-term.test
@@ -61,9 +61,11 @@ mkcheck ()
return $rc
}
+cp "$top_testsrcdir"/lib/pt-wrap .
+
$ACLOCAL
$AUTOCONF
-$AUTOMAKE -a
+$AUTOMAKE
./configure
TERM=ansi; export TERM
diff --git a/tests/posixsubst-tests.test b/tests/posixsubst-tests.test
index 26ae69f..3f45c72 100755
--- a/tests/posixsubst-tests.test
+++ b/tests/posixsubst-tests.test
@@ -50,7 +50,7 @@ chmod +x foo1.test bary
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure
$MAKE check >out 2>&1 && { cat out; Exit 1; }
diff --git a/tests/repeated-options.test b/tests/repeated-options.test
index c3c2e29..1475f6c 100755
--- a/tests/repeated-options.test
+++ b/tests/repeated-options.test
@@ -58,7 +58,7 @@ int main (void)
}
END
-cp $testsrcdir/../lib/compile .
+cp "$top_testsrcdir/lib/compile" "$top_testsrcdir/lib/pt-wrap" .
$ACLOCAL
$AUTOMAKE --foreign --foreign -Wall 2>stderr || { cat stderr >&2; Exit 1; }
--
1.7.2.3
From d083852bece1cac0ba3601a954d47d551b5f9ecf Mon Sep 17 00:00:00 2001
Message-Id: <address@hidden>
In-Reply-To: <address@hidden>
References: <address@hidden>
From: Stefano Lattarini <address@hidden>
Date: Fri, 20 May 2011 21:45:51 +0200
Subject: [PATCH 2/2] parallel-tests: allow custom wrapper scripts
---
ChangeLog | 37 ++++++
automake.in | 33 +++++-
lib/Automake/tests/Makefile.in | 27 +++--
lib/am/check2.am | 6 +-
tests/Makefile.in | 3 +-
tests/parallel-tests-no-ptwrap.test | 64 +++++++++++
tests/parallel-tests-wrapper-xfail-tests.test | 146 +++++++++++++++++++++++++
tests/parallel-tests-wrapper.test | 140 ++++++++++++++++++++++++
8 files changed, 437 insertions(+), 19 deletions(-)
create mode 100755 tests/parallel-tests-no-ptwrap.test
create mode 100755 tests/parallel-tests-wrapper-xfail-tests.test
create mode 100755 tests/parallel-tests-wrapper.test
diff --git a/ChangeLog b/ChangeLog
index ebc3bf7..8e51dcf 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,42 @@
2011-05-20 Stefano Lattarini <address@hidden>
+ parallel-tests: allow custom wrapper scripts
+
+ !!TODO!!: more in-depth and clear description
+
+ Allow extension-based definition of "wrapper script" for test
+ scripts and "log compilers". These wrapper script will be
+ responsible of running the tests, interpreting the test results,
+ and writing the `.log' files.
+
+ This new feature is implemented by allowing the developer to define
+ make variables `<ext>_LOG_WRAPPER' and `AM_<ext>_LOG_WFLAGS' (this
+ last one overridable by the user through `<ext>_LOG_WFLAGS').
+ Similarly for what is currently done `LOG_COMPILE' & Company,
+ we also allow for extension-agnostic variables `LOG_WRAPPER',
+ `AM_LOG_WFLAGS' and `LOG_WFLAGS'.
+
+ This new API should allow easy and flexible use of different test
+ protocols in the future; in particular, we plan to use it will be
+ used to implement TAP and SubUnit harnesses.
+
+ * automake.in (handle_tests): Define default for $(LOG_WRAPPER),
+ and, for any registered text extension `<ext>', define defaults
+ for $(<ext>_LOG_WRAPPER). Substitute %WRAPPER% using these new
+ variables properly, instead of the old internal $(am__pt_wrap).
+ When processing check2.am, also substitute %WRAPPER_FLAGS%.
+ Require auxiliary script `pt-wrap' only if no wrapper has been
+ explicitly defined for the test script kinds.
+ * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
+ ?!GENERIC?%OBJ%): Pass the %WRAPPER_FLAGS% to the %WRAPPER% call.
+ !!! TODO !!! ADD MORE TEST CASES.
+ * tests/parallel-tests-no-ptwrap.test: New test.
+ * tests/parallel-tests-wrapper-xfail-tests.test: Likewise.
+ * tests/parallel-tests-wrapper.test: Likewise.
+ * tests/Makefile.am (TESTS): !!!TODO!!!
+
+2011-05-20 Stefano Lattarini <address@hidden>
+
parallel-tests: add auxiliary script 'pt-wrap', refactor
This refactoring should cause no API of functionality change, and
is meant only to simplify the future implementation of TAP and
diff --git a/automake.in b/automake.in
index 211ee3c..a38ba06 100644
--- a/automake.in
+++ b/automake.in
@@ -4991,10 +4991,6 @@ sub handle_tests
if (my $parallel_tests = option 'parallel-tests')
{
- require_conf_file ($parallel_tests->{position}, FOREIGN, 'pt-wrap');
- define_variable ('am__pt_wrap',
- "\$(SHELL) $am_config_aux_dir/pt-wrap",
- INTERNAL);
define_variable ('TEST_SUITE_LOG', 'test-suite.log', INTERNAL);
define_variable ('TEST_SUITE_HTML', '$(TEST_SUITE_LOG:.log=.html)',
INTERNAL);
my $suff = '.test';
@@ -5038,6 +5034,19 @@ sub handle_tests
if substr ($obj, - length ($test_suffix)) eq $test_suffix;
}
$obj .= '.log';
+ # The "log wrapper" program, deputed to handle tests protocol
used by
+ # test scripts. By default, it's assumed that no protocol is
used,
+ # so we fall back to the old "parallel-tests" behaviour,
implemented
+ # by the `pt-wrap' auxiliary script.
+ if (! var 'LOG_WRAPPER')
+ {
+ require_conf_file ($parallel_tests->{position}, FOREIGN,
'pt-wrap');
+ define_variable ('LOG_WRAPPER',
+ "\$(SHELL) $am_config_aux_dir/pt-wrap",
+ INTERNAL);
+ }
+ my $wrapper = '$(LOG_WRAPPER)';
+ my $wrapper_flags = '$(AM_LOG_WFLAGS) $(LOG_WFLAGS)';
my $compile = 'LOG_COMPILE';
define_variable ($compile,
'$(LOG_COMPILER) $(AM_LOG_FLAGS)
$(LOG_FLAGS)', INTERNAL);
@@ -5045,7 +5054,8 @@ sub handle_tests
GENERIC => 0,
OBJ => $obj,
SOURCE => $val,
- WRAPPER => '$(am__pt_wrap)',
+ WRAPPER => $wrapper,
+ WRAPPER_FLAGS => $wrapper_flags,
COMPILE =>'$(' . $compile . ')',
EXT => '',
am__EXEEXT => 'FALSE');
@@ -5076,6 +5086,16 @@ sub handle_tests
{
(my $ext = $test_suffix) =~ s/^\.//;
$ext = uc $ext;
+ # See comments about definition of LOG_WRAPPER, above.
+ if (! var "${ext}_LOG_WRAPPER")
+ {
+ require_conf_file ($parallel_tests->{position}, FOREIGN,
'pt-wrap');
+ define_variable ("${ext}_LOG_WRAPPER",
+ "\$(SHELL) $am_config_aux_dir/pt-wrap",
+ INTERNAL);
+ }
+ my $wrapper = '$(' . $ext . '_LOG_WRAPPER)';
+ my $wrapper_flags = '$(AM_' . $ext . '_LOG_WFLAGS) $(' . $ext
. '_LOG_WFLAGS)';
my $compile = $ext . '_LOG_COMPILE';
define_variable ($compile,
'$(' . $ext . '_LOG_COMPILER) $(AM_' . $ext
. '_LOG_FLAGS)'
@@ -5085,7 +5105,8 @@ sub handle_tests
GENERIC => 1,
OBJ => '',
SOURCE => '$<',
- WRAPPER => '$(am__pt_wrap)',
+ WRAPPER => $wrapper,
+ WRAPPER_FLAGS =>
$wrapper_flags,
COMPILE => '$(' . $compile .
')',
EXT => $test_suffix,
am__EXEEXT => $am_exeext);
diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
index df93aa5..56f60ea 100644
--- a/lib/Automake/tests/Makefile.in
+++ b/lib/Automake/tests/Makefile.in
@@ -126,13 +126,13 @@ am__text_box = $(AWK) '{ \
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
-am__check_wrapper_options = \
+# Default flags passed to all log compiler wrappers.
+am__test_wrapper_flags = \
--test-name "$$f" \
- --test-dir "$$dir" \
--log-file '$@' \
--color-tests "$$am__color_tests" \
--enable-hard-errors "$$am__enable_hard_errors" \
- --xfail-tests '$(XFAIL_TESTS)'
+ --expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
@@ -145,25 +145,31 @@ $(am__sh_e_setup);
\
$(am__vpath_adj_setup) $(am__vpath_adj) \
$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-if test -n '$(DISABLE_HARD_ERRORS)'; then \
- am__enable_hard_errors=no; \
-else \
- am__enable_hard_errors=yes; \
-fi; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
+case " $(XFAIL_TESTS) " in \
+ *[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
-am__pt_wrap = $(SHELL) $(top_srcdir)/lib/pt-wrap
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
am__test_logs1 = $(TESTS:=.log)
TEST_LOGS = $(am__test_logs1:.pl.log=.log)
+PL_LOG_WRAPPER = $(SHELL) $(top_srcdir)/lib/pt-wrap
PL_LOG_COMPILE = $(PL_LOG_COMPILER) $(AM_PL_LOG_FLAGS) $(PL_LOG_FLAGS)
TEST_LOGS_TMP = $(TEST_LOGS:.log=.log-t)
DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST)
@@ -453,7 +459,8 @@ recheck recheck-html:
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS)
TEST_LOGS="'"$$list"'"'
.pl.log:
@p='$<'; $(am__check_pre) \
- $(am__pt_wrap) $(am__check_wrapper_options) -- $(PL_LOG_COMPILE) "$$tst"
+ $(PL_LOG_WRAPPER) $(am__test_wrapper_flags) $(AM_PL_LOG_WFLAGS)
$(PL_LOG_WFLAGS) -- \
+ $(PL_LOG_COMPILE) "$$tst"
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
diff --git a/lib/am/check2.am b/lib/am/check2.am
index 2de4760..3dbce09 100644
--- a/lib/am/check2.am
+++ b/lib/am/check2.am
@@ -18,7 +18,8 @@
?GENERIC?%EXT%.log:
?!GENERIC?%OBJ%: %SOURCE%
@p='%SOURCE%'; $(am__check_pre) \
- %WRAPPER% $(am__test_wrapper_flags) -- %COMPILE% "$$tst"
+ %WRAPPER% $(am__test_wrapper_flags) %WRAPPER_FLAGS% -- \
+ %COMPILE% "$$tst"
## If no programs are built in this package, then this rule is removed
## at automake time. Otherwise, %am__EXEEXT% expands to a configure time
@@ -27,5 +28,6 @@
if %am__EXEEXT%
?GENERIC?%EXT%$(EXEEXT).log:
@p='%SOURCE%'; $(am__check_pre) \
- %WRAPPER% $(am__test_wrapper_flags) -- %COMPILE% "$$tst"
+ %WRAPPER% $(am__test_wrapper_flags) %WRAPPER_FLAGS% -- \
+ %COMPILE% "$$tst"
endif %am__EXEEXT%
diff --git a/tests/Makefile.in b/tests/Makefile.in
index a9dd0eb..4054c67 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1474,7 +1474,8 @@ recheck recheck-html:
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS)
TEST_LOGS="'"$$list"'"'
.test.log:
@p='$<'; $(am__check_pre) \
- $(TEST_LOG_WRAPPER) $(am__test_wrapper_flags) -- $(TEST_LOG_COMPILE)
"$$tst"
+ $(TEST_LOG_WRAPPER) $(am__test_wrapper_flags) $(AM_TEST_LOG_WFLAGS)
$(TEST_LOG_WFLAGS) -- \
+ $(TEST_LOG_COMPILE) "$$tst"
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
diff --git a/tests/parallel-tests-no-ptwrap.test
b/tests/parallel-tests-no-ptwrap.test
new file mode 100755
index 0000000..c9ef5f4
--- /dev/null
+++ b/tests/parallel-tests-no-ptwrap.test
@@ -0,0 +1,64 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'pt-wrap' doesn't get needlessly installed
+# if it's not used, i.e., if the user has defined his own *LOG_WRAPPER
+# variables.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<'END'
+AC_PROG_CC
+AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
+AC_SUBST([X_LOG_WRAPPER], [none])
+AC_OUTPUT
+END
+
+mkdir sub1 sub2
+
+cat > Makefile.am <<'END'
+SUBDIRS = sub1 sub2
+LOG_WRAPPER =
+TEST_LOG_WRAPPER = :
+TESTS = foo bar.test
+END
+
+cat > sub1/Makefile.am <<'END'
+TEST_EXTENSIONS = .x .sh .pl
+SH_LOG_WRAPPER = dummy1
+PL_LOG_WRAPPER = dummy2
+TESTS = a.pl b.sh c.x
+END
+
+cat > sub2/Makefile.am <<'END'
+TEST_EXTENSIONS = .bar
+LOG_WRAPPER = x
+BAR_LOG_WRAPPER = y
+TESTS = 1 2.bar 3.test 4.t 5.tt $(check_PROGRAMS)
+check_PROGRAMS = p1 p2$(EXEEXT) p3.bar p4.suf
+END
+
+$ACLOCAL
+
+for opts in '' '--add-missing' '-a -c'; do
+ $AUTOMAKE $opts
+ $EGREP '(^| | )pt.*wrap' \
+ Makefile.in sub1/Makefile.in sub2/Makefile.in && Exit 1
+ find . | grep 'pt.*wrap' && Exit 1
+done
+
+:
diff --git a/tests/parallel-tests-wrapper-xfail-tests.test
b/tests/parallel-tests-wrapper-xfail-tests.test
new file mode 100755
index 0000000..8b8cb6f
--- /dev/null
+++ b/tests/parallel-tests-wrapper-xfail-tests.test
@@ -0,0 +1,146 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests wrappers: XFAIL_TESTS support.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<'END'
+AC_SUBST([nihil], [])
+AC_SUBST([ac_xfail_tests], ['x5.test x6$(test_suffix)'])
+AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
+AC_OUTPUT
+END
+
+cat > Makefile.am <<'END'
+SUBDIRS = . sub1 sub2
+TEST_LOG_WRAPPER = $(srcdir)/tw
+TESTS = pass.test xfail.test
+XFAIL_TESTS = xfail.test
+END
+
+mkdir sub1 sub2
+
+cat > sub1/Makefile.am <<END
+empty =
+
+TEST_LOG_WRAPPER = \$(top_srcdir)/tw
+
+# XFAIL_TESTS should gracefully handle TAB characters, and multiple
+# whitespaces.
+XFAIL_TESTS =\$(empty)${tab}x1.test x2.test${tab}x3.test${tab}\
+x4.test ${tab} x5.test x6.test${tab}\$(empty)
+
+TESTS = pass.test x1.test x2.test x3.test x4.test x5.test x6.test
+END
+
+cat > sub2/Makefile.am <<'END'
+AUTOMAKE_OPTIONS = -Wno-portability-recursive
+
+TEST_LOG_WRAPPER = $(srcdir)/../tw
+
+# XFAIL_TESTS should gracefully AC_SUBST @substitution@ and
+# make variables indirections.
+an_xfail_test = x1.test
+test_suffix = .test
+v0 = x3.test
+v1 = v
+v2 = 0
+XFAIL_TESTS = $(an_xfail_test) x2.test @nihil@ x3${test_suffix}
+XFAIL_TESTS += $($(v1)$(v2)) x4.test @ac_xfail_tests@
+
+TESTS = pass.test x1.test x2.test x3.test x4.test x5.test x6.test
+END
+
+cat > pass.test <<'END'
+#!/bin/sh
+exit 0
+END
+
+cat > xfail.test <<'END'
+#!/bin/sh
+exit 1
+END
+
+chmod a+x pass.test xfail.test
+
+cp pass.test sub1/pass.test
+cp pass.test sub2/pass.test
+
+for i in 1 2 3 4 5 6; do
+ cp xfail.test sub1/x$i.test
+ cp xfail.test sub2/x$i.test
+done
+
+cat > tw <<'END'
+#! /bin/sh
+set -e
+test_name=INVALID
+log_file=/dev/null
+expect_failure=no
+while test $# -gt 0; do
+ case $1 in
+ --test-name) test_name=$2; shift;;
+ --expect-failure) expect_failure=$2; shift;;
+ --log-file) log_file=$2; shift;;
+ # Ignored.
+ --color-tests) shift;;
+ --enable-hard-errors) shift;;
+ # Explicitly terminate option list.
+ --) shift; break;;
+ # Shouldn't happen
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+st=0
+"$@" || st=$?
+rm -f "$log_file"
+case $st,$expect_failure in
+ 0,no) echo "PASS: $test_name"; exit 0;;
+ 1,no) echo "FAIL: $test_name"; exit 1;;
+ 0,yes) echo "XPASS: $test_name"; exit 1;;
+ 1,yes) echo "XFAIL: $test_name"; exit 0;;
+ *) echo "UNEXPECTED OUTCOME: $test_name"; exit 99;;
+esac | tee "$log_file"
+END
+chmod a+x tw
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+VERBOSE=yes $MAKE check
+test -f test-suite.log
+test -f sub1/test-suite.log
+test -f sub2/test-suite.log
+
+for dir in sub1 sub2; do
+ cd $dir
+ cp pass.test x1.test
+ cp x2.test pass.test
+ VERBOSE=yes $MAKE check && Exit 1
+ grep '^FAIL: pass\.test$' test-suite.log
+ grep '^XPASS: x1\.test$' test-suite.log
+ test `grep -c '^FAIL' test-suite.log` -eq 1
+ test `grep -c '^XPASS' test-suite.log` -eq 1
+ cd ..
+done
+
+:
diff --git a/tests/parallel-tests-wrapper.test
b/tests/parallel-tests-wrapper.test
new file mode 100755
index 0000000..5537aab
--- /dev/null
+++ b/tests/parallel-tests-wrapper.test
@@ -0,0 +1,140 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests: per-extension test wrappers.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+# Automake shouldn't match the '/test' in 'sub/test' as '.test' suffix.
+TESTS = 1.chk 2.test 3 4.c.chk 5.suf sub/test
+
+TEST_EXTENSIONS = .chk .test
+
+CHK_LOG_WRAPPER = ./chk-wrapper
+TEST_LOG_WRAPPER = $(SHELL) $(srcdir)/test-wrapper
+LOG_WRAPPER = noext-wrapper
+
+AM_CHK_LOG_WFLAGS = --am-chk
+CHK_LOG_WFLAGS = --chk
+AM_TEST_LOG_WFLAGS = -am-test
+TEST_LOG_WFLAGS = -test
+AM_LOG_WFLAGS = am
+LOG_WFLAGS = _
+END
+
+mkdir sub bin
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH; export PATH
+
+cat > wrapper.skel <<'END'
+#! /bin/sh
+set -e
+
+me=`echo "$0" | sed 's,^.*/,,'`
+if test -z "$me"; then
+ echo "$0: cannot determine program name" >&2
+ exit 99
+fi
+
+am_log_wflags='@am_log_wflags@'
+log_wflags='@log_wflags@'
+
+test_name=INVALID
+log_file=BAD.log
+extra_opts=
+while test $# -gt 0; do
+ case $1 in
+ --test-name) test_name=$2; shift;;
+ --log-file) log_file=$2; shift;;
+ # Ignored.
+ --expect-failure) shift;;
+ --color-tests) shift;;
+ --enable-hard-errors) shift;;
+ # Remembered in the same order they're passed in.
+ $am_log_wflags|$log_wflags) extra_opts="$extra_opts $1";;
+ # Explicitly terminate option list.
+ --) shift; break;;
+ # Shouldn't happen
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+
+echo "$me" "$test_name" $extra_opts > "$log_file"
+
+exec "$@"
+exit 127
+END
+
+sed -e 's|@am_log_wflags@|--am-chk|' \
+ -e 's|@log_wflags@|--chk|' \
+ < wrapper.skel > chk-wrapper
+
+sed -e 's|@am_log_wflags@|-am-test|' \
+ -e 's|@log_wflags@|-test|' \
+ < wrapper.skel > test-wrapper
+
+sed -e 's|@am_log_wflags@|am|' \
+ -e 's|@log_wflags@|_|' \
+ < wrapper.skel > bin/noext-wrapper
+
+# `test-wrapper' is deliberately not made executable.
+chmod a+x chk-wrapper bin/noext-wrapper
+
+# Not needed anymore.
+rm -f wrapper.skel
+
+cat > 1.chk << 'END'
+#! /bin/sh
+exit 0
+END
+chmod a+x 1.chk
+cp 1.chk 2.test
+cp 1.chk 3
+cp 1.chk 4.c.chk
+cp 1.chk 5.suf
+cp 1.chk sub/test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+$MAKE
+VERBOSE=yes $MAKE check
+ls -l . sub
+
+test ! -r BAD.log
+
+echo 'chk-wrapper 1.chk --am-chk --chk' > 1.exp
+echo 'test-wrapper 2.test -am-test -test' > 2.exp
+echo 'noext-wrapper 3 am _' > 3.exp
+echo 'chk-wrapper 4.c.chk --am-chk --chk' > 4.c.exp
+echo 'noext-wrapper 5.suf am _' > 5.suf.exp
+echo 'noext-wrapper sub/test am _' > sub/test.exp
+
+st=0
+for x in 1 2 3 4.c 5.suf sub/test; do
+ cat $x.log
+ diff $x.exp $x.log || st=1
+done
+
+Exit $st
--
1.7.2.3