Prevent spurious failures with ./testsuite -x

[Noah Misch]

Back in http://lists.gnu.org/archive/html/autoconf/2004-06/msg00121.html, I
described one cause of spurious test suite failures with -x and proposed
disabling shell traces of test commands that do not ignore their stderr.
Unfortunately, that solution would have disabled tracing for most test cases in
the Autoconf test suite.

This patch adds some basic analysis on the commands to determine whether they
could yield a multiline shell trace, and only if they can will the test suite
not trace them.  This only disables tracing for 23 Autoconf tests, 19 of which
were failing under -x before.  With this patch, ./testsuite -v -x passes fully.

As included, this patch depends on my just-posted Autotest test case patch, but
you can delete the chunks in autotest.at and use it independently.

The patch applies to HEAD.

2004-10-23  Noah Misch  <address@hidden>

        * lib/autotest/general.m4 [--trace] (_AT_CHECK): Do not enable
          shell tracing on a command that could contain multiple lines.
        * doc/autoconf.text: Document that fact and its implications.
        * lib/m4sugar/m4sh.m4 (AS_ESCAPE_FOR_EXPAND): New macro.
        * tests/autotest.at (Multiline backquote command substitution,
          Multiline parameter expansion, Literal multiline command,
          Multiline parenthetical command substitution): Remove XFAIL.

diff -X dontdiff -urp ac-testat/doc/autoconf.texi ac-testtrace/doc/autoconf.texi
--- ac-testat/doc/autoconf.texi Wed Oct 13 16:29:02 2004
+++ ac-testtrace/doc/autoconf.texi      Sun Oct 24 00:40:59 2004
@@ -15410,6 +15410,29 @@ commands @var{run-if-pass}.
 The @var{commands} @emph{must not} redirect the standard output, nor the
 standard error.
 
+If the @var{commands} may contain a newline, @command{testsuite} will
+not enable shell tracing for them when the user passes @option{-x}.
+Since unquoted, unescaped literal newlines are never necessary to
+achieve particular semantics, escape them.  For example, avoid this:
+
address@hidden
+cat LOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOONG |
+  grep fairly-long-in-its-own-right
+echo "A sentence."
address@hidden display
+
+Write this instead:
+
address@hidden
+cat LOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOONG | \
+  grep fairly-long-in-its-own-right ; \
+echo "A sentence."
address@hidden display
+
+The presence in @var{commands} of any command substitution or a
+parameter expansion yielding multiple lines also inhibits shell tracing,
+but those constructs are rarely so easy to remove.
+
 If @var{status}, or @var{stdout}, or @var{stderr} is @samp{ignore}, then
 the corresponding value is not checked.
 
diff -X dontdiff -urp ac-testat/lib/autotest/general.m4 
ac-testtrace/lib/autotest/general.m4
--- ac-testat/lib/autotest/general.m4   Sat Oct 23 03:49:33 2004
+++ ac-testtrace/lib/autotest/general.m4        Sun Oct 24 00:47:39 2004
@@ -1204,7 +1204,10 @@ m4_define([AT_CHECK_NOESCAPE],
 # but we must group COMMANDS as it is not limited to a single command, and
 # then the shells will save the traces in at-stderr. So we have to filter
 # them out when checking stderr, and we must send them into the test suite's
-# stderr to honor -x properly.
+# stderr to honor -x properly. Since only the first line of the trace of a
+# multiline command starts with a `+', and I know of no straightforward way to
+# filter out the unadorned trace lines, we disable shell tracing entirely for 
+# commands that could span multiple lines.
 #
 # Limiting COMMANDS to a single command is not good either, since them
 # the user herself would use {} or (), and then we face the same problem.
@@ -1221,10 +1224,35 @@ m4_define([_AT_CHECK],
 [$at_traceoff
 echo "AT_LINE: AS_ESCAPE([$1])"
 echo AT_LINE >"$at_check_line_file"
-( $at_traceon; $1 ) >"$at_stdout" 2>"$at_stder1"
-at_status=$?
-grep '^ *+' "$at_stder1" >&2
-grep -v '^ *+' "$at_stder1" >"$at_stderr"
+
+at_trace_this=
+if test -n "$at_traceon"; then
+    at_lf='
+'
+    at_cmd_expanded="AS_ESCAPE_FOR_EXPAND([$1])"
+    case "$at_cmd_expanded" in
+        *$\(*\)*)          at_reason='a $(...) command substitution' ;;
+        *\`*\`*)           at_reason='a `...` command substitution' ;;
+        *[[^\\]]"$at_lf"*) at_reason='an embedded newline' ;;
+        *)                 at_reason= ;;
+    esac
+    if test -n "$at_reason"; then
+        echo "Not enabling shell tracing (command contains $at_reason)"
+    else
+        at_trace_this=yes
+    fi
+fi
+
+if test -n "$at_trace_this"; then
+    ( $at_traceon; $1 ) >"$at_stdout" 2>"$at_stder1"
+    at_status=$?
+    grep '^ *+' "$at_stder1" >&2
+    grep -v '^ *+' "$at_stder1" >"$at_stderr"
+else
+    ( $1 ) >"$at_stdout" 2>"$at_stderr"
+    at_status=$?
+fi
+
 at_failed=false
 dnl Check stderr.
 m4_case([$4],
diff -X dontdiff -urp ac-testat/lib/m4sugar/m4sh.m4 
ac-testtrace/lib/m4sugar/m4sh.m4
--- ac-testat/lib/m4sugar/m4sh.m4       Sat Oct 23 03:49:34 2004
+++ ac-testtrace/lib/m4sugar/m4sh.m4    Sat Oct 23 23:36:47 2004
@@ -483,6 +483,15 @@ m4_define([AS_ESCAPE],
             m4_ifval([$2], [[\([$2]\)]], [[\([\"$`]\)]]),
             [\\\1])])
 
+# AS_ESCAPE_FOR_EXPAND(STRING)
+# ----------------------------
+# Escape characters in STRING that have special meaning to the shell
+# within double quotes, but leave parameter expansions active.
+# These substitutions come from sed_double_backslash in GNU Libtool.
+m4_define([AS_ESCAPE_FOR_EXPAND],
+[m4_bpatsubsts([AS_ESCAPE([$1], [`"\])],
+               [^\(\(\\\\\\\\\)*\\\\\)\$], [\1\\$],
+           [\([^\\]\(\\\\\\\\\)*\\\\\)\$], [\1\\$])])
 
 # _AS_QUOTE_IFELSE(STRING, IF-MODERN-QUOTATION, IF-OLD-QUOTATION)
 # ---------------------------------------------------------------
diff -X dontdiff -urp ac-testat/tests/autotest.at ac-testtrace/tests/autotest.at
--- ac-testat/tests/autotest.at Sat Oct 23 19:25:21 2004
+++ ac-testtrace/tests/autotest.at      Sat Oct 23 23:26:35 2004
@@ -78,14 +78,14 @@ AT_CHECK_AT_TEST([Literal multiline comm
   [AT_CHECK([echo Auto'
 'conf], 0, [Auto
 conf
-], [])], [:])
+], [])])
 
 AT_CHECK_AT_TEST([Multiline parameter expansion],
   [FOO='one
 two'
    AT_CHECK([echo "$FOO"], 0, [one
 two
-], [])], [:])
+], [])])
 
 AT_CHECK_AT_TEST([Backquote command substition],
   [AT_CHECK([echo `echo hi`], 0, [hi
@@ -97,7 +97,7 @@ bar
 ])
    AT_CHECK([echo "`cat myfile`"], 0, [foo
 bar
-], [])], [:])
+], [])])
 
 AT_CHECK_AT_TEST([Parenthetical command substition],
   [AT_CHECK([echo $(echo hi)], 0, [hi
@@ -109,7 +109,7 @@ bar
 ])
    AT_CHECK([echo "$(cat myfile)"], 0, [foo
 bar
-], [])], [:])
+], [])])
 
 ## ------------------------------- ##
 ## Funny characters in test names. ##