Autotest: enable colored test results.

[Ralf Wildenhues]

The logical next step for Autotest to be on par with Automake's
parallel-test.

Unlike Automake, the testsuite author does not have to do anything for
the user to be able to use color; AT_COLOR_TESTS only changes the
default to yes.  We could easily let it accept an optional argument, if
you think it is useful.

The ANSI escape sequences may be unportable and brittle wrt. manual
editing of the generated testsuite, esp. with the embedded literal
escape character (same with tests/autotest.at of course).  I can't do
good portability testing at the moment, for I'm lacking access to some
hosts, but since similar code has survived in Automake, so this may be
good enough.

One small inconsistency is that, if the testsuite exits with an error,
then the error summary is given on standard error not standard output;
that is fine per se but stderr might not go to a terminal at the same
time as stdout.  Oh well.

Testsuite coverage is not complete, AT_COLOR_TESTS is not tested and the
terminal vs. non-terminal and --color=ARG is not fully covered yet.  I
tested that manually, but all on GNU/Linux with a gnome term terminal
only.

What do you think?

Thanks,
Ralf

    Autotest: enable colored test results.
    
    * lib/autotest/general.m4 (AT_INIT): Accept --color and
    --color=no|yes|always.  If desired, colorize test results and
    testsuite summary on standard output.
    (AT_COLOR_TESTS): New macro, set the default for color to yes.
    * doc/autoconf.texi (Writing Testsuites): Document it.
    (testsuite Invocation): Document --color* options.
    * tests/local.at: Call AT_COLOR_TESTS for Autoconf's testsuite.
    * tests/autotest.at (color test results): New test, mirroring
    color.test from Automake.
    * NEWS: Update.

diff --git a/NEWS b/NEWS
index f1fdefb..dcb1fde 100644
--- a/NEWS
+++ b/NEWS
@@ -24,6 +24,8 @@ GNU Autoconf NEWS - User visible changes.
 ** Autotest testsuites accept an option --recheck to rerun tests that
    failed or passed unexpectedly during the last non-debug testsuite run.
 
+** Autotest testsuites may optionally provide colored test results.
+
 * Major changes in Autoconf 2.65 (2009-11-21) [stable]
   Released by Eric Blake, based on git versions 2.64.*.
 
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index ca0d85c..b76970d 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -23413,6 +23413,12 @@ Writing Testsuites
 several times.
 @end defmac
 
address@hidden AT_COLOR_TESTS
address@hidden
+Enable colored test results by default when the output is connected to
+a terminal.
address@hidden defmac
+
 Autotest test suites rely on @env{PATH} to find the tested program.
 This avoids the need to generate absolute names of the various tools, and
 makes it possible to test installed programs.  Therefore, knowing which
@@ -23795,6 +23801,13 @@ testsuite Invocation
 Force more verbosity in the detailed output of what is being done.  This
 is the default for debugging scripts.
 
address@hidden --color
address@hidden address@hidden@r{|address@hidden|address@hidden
+Enable colored test results.  Test results are colored if the argument
address@hidden is given, or if standard output is connected to a terminal
+and either the macro @code{AT_COLOR_TESTS} is used or the argument
address@hidden is not given.
+
 @item --debug
 @itemx -d
 Do not remove the files after a test group was performed ---but they are
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 1e38abe..86c27ee 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -389,6 +389,8 @@ at_jobs=1
 at_traceon=:
 at_trace_echo=:
 at_check_filter_trace=:
+# Whether to enable colored test results.
+at_color=m4_ifdef([AT_color], [AT_color], [no])
 
 # Shall we keep the debug scripts?  Must be `:' when the suite is
 # run by a debug script, so that the script doesn't remove itself.
@@ -492,6 +494,13 @@ do
        at_clean=:
        ;;
 
+    --color )
+       at_color=yes
+       ;;
+    --color=* )
+       at_color=$at_optarg
+       ;;
+
     --debug | -d )
        at_debug_p=:
        ;;
@@ -663,6 +672,17 @@ else
   # Sort the tests, removing duplicates.
   at_groups=`AS_ECHO(["$at_groups"]) | tr ' ' "$as_nl" | sort -nu`
 fi
+
+if test x"$at_color" = xalways \
+   || { test x"$at_color" = xyes && test -t 1; }; then
+  at_red='@<:@0;31m'
+  at_grn='@<:@0;32m'
+  at_lgn='@<:@1;32m'
+  at_blu='@<:@1;34m'
+  at_std='@<:@m'
+else
+  at_red= at_grn= at_lgn= at_blu= at_std=
+fi
 m4_divert_pop([PARSE_ARGS_END])dnl
 m4_divert_push([HELP])dnl
 
@@ -704,6 +724,8 @@ dnl extra quoting prevents emacs whitespace mode from 
putting tabs in output
 Execution tuning:
   -C, --directory=DIR
 [                 change to directory DIR before starting]
+      --color[[=yes|no|always]]
+[                 enable colored test results on terminal, or always]
   -j, --jobs[[=N]]
 [                 Allow N jobs at once; infinite jobs with no arg (default 1)]
   -k, --keywords=KEYWORDS
@@ -1156,40 +1178,46 @@ _ATEOF
        at_msg='FAILED ('`cat "$at_check_line_file"`')'
        at_res=fail
        at_errexit=$at_errexit_p
+       at_color=$at_red
        ;;
     yes:0)
        at_msg="UNEXPECTED PASS"
        at_res=xpass
        at_errexit=$at_errexit_p
+       at_color=$at_red
        ;;
     no:0)
        at_msg="ok"
        at_res=pass
        at_errexit=false
+       at_color=$at_grn
        ;;
     *:77)
        at_msg='skipped ('`cat "$at_check_line_file"`')'
        at_res=skip
        at_errexit=false
+       at_color=$at_blu
        ;;
     yes:*)
        at_msg='expected failure ('`cat "$at_check_line_file"`')'
        at_res=xfail
        at_errexit=false
+       at_color=$at_lgn
        ;;
     no:*)
        at_msg='FAILED ('`cat "$at_check_line_file"`')'
        at_res=fail
        at_errexit=$at_errexit_p
+       at_color=$at_red
        ;;
   esac
   echo "$at_res" > "$at_job_dir/$at_res"
   # In parallel mode, output the summary line only afterwards.
   if test $at_jobs -ne 1 && test -n "$at_verbose"; then
-    AS_ECHO(["$at_desc_line $at_msg"])
+    AS_ECHO(["$at_desc_line $at_color$at_msg$at_std"])
   else
     # Make sure there is a separator even with long titles.
-    AS_ECHO([" $at_msg"])
+    AS_ECHO([" $at_color$at_msg$at_std"])
   fi
   at_log_msg="$at_group. $at_desc ($at_setup_line): $at_msg"
   case $at_status in
@@ -1490,12 +1518,14 @@ if $at_errexit_p && test $at_unexpected_count != 0; then
     at_result="$at_result $at_were run, one failed"
   fi
   at_result="$at_result unexpectedly and inhibited subsequent tests."
+  at_color=$at_red
 else
   # Don't you just love exponential explosion of the number of cases?
+  at_color=$at_red
   case $at_xpass_count:$at_fail_count:$at_xfail_count in
     # So far, so good.
-    0:0:0) at_result="$at_result $at_were successful." ;;
-    0:0:*) at_result="$at_result behaved as expected." ;;
+    0:0:0) at_result="$at_result $at_were successful." at_color=$at_grn ;;
+    0:0:*) at_result="$at_result behaved as expected." at_color=$at_lgn ;;
 
     # Some unexpected failures
     0:*:0) at_result="$at_result $at_were run,
@@ -1543,10 +1573,10 @@ $at_skip_count tests were skipped." ;;
 esac
 
 if test $at_unexpected_count = 0; then
-  echo "$at_result"
+  echo "$at_color$at_result$at_std"
   echo "$at_result" >&AS_MESSAGE_LOG_FD
 else
-  echo "ERROR: $at_result" >&2
+  echo "${at_color}ERROR: $at_result$at_std" >&2
   echo "ERROR: $at_result" >&AS_MESSAGE_LOG_FD
   {
     echo
@@ -1759,6 +1789,12 @@ m4_define([AT_COPYRIGHT],
 [m4_default([$2], [m4_newline])([$1])])])# AT_COPYRIGHT
 
 
+# AT_COLOR_TESTS
+# --------------
+# Enable colored test results if standard error is connected to a terminal.
+m4_define([AT_COLOR_TESTS],
+[m4_define([AT_color], [yes])])
+
 # AT_SETUP(DESCRIPTION)
 # ---------------------
 # Start a group of related tests, all to be executed in the same subshell.
diff --git a/tests/autotest.at b/tests/autotest.at
index ad38b7a..cef0fef 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -1402,6 +1402,91 @@ AT_CHECK([test -s sigpipe-stamp || test ! -f 
micro-suite.dir/7/micro-suite.log],
 
 AT_CLEANUP
 
+
+# --color
+AT_CHECK_AT_TEST([colored test results],
+  [AT_CHECK([:])
+   AT_CLEANUP
+   AT_SETUP([fail])
+   AT_CHECK([exit 1])
+   AT_CLEANUP
+   AT_SETUP([xpass])
+   AT_XFAIL_IF([:])
+   AT_CHECK([:])
+   AT_CLEANUP
+   AT_SETUP([xfail])
+   AT_XFAIL_IF([:])
+   AT_CHECK([exit 1])
+   AT_CLEANUP
+   AT_SETUP([skip])
+   AT_CHECK([exit 77])
+   AT_CLEANUP
+   AT_SETUP([hardfail])
+   AT_XFAIL_IF([:])
+   AT_CHECK([exit 99])
+], [], [], [], [], [], [
+
+TERM=ansi
+export TERM
+
+red='@<:@0;31m'
+grn='@<:@0;32m'
+lgn='@<:@1;32m'
+blu='@<:@1;34m'
+std='@<:@m'
+
+# Check that grep can parse nonprinting characters.
+# BSD 'grep' works from a pipe, but not a seekable file.
+# GNU or BSD 'grep -a' works on files, but is not portable.
+AT_CHECK([case `echo "$std" | grep .` in #'' restore font-lock
+           $std) :;;
+           *) Exit 77;;
+         esac], [], [ignore], [],
+        [echo "grep can't parse nonprinting characters" >&2])
+
+if echo 'ab*c' | grep -F 'ab*c' >/dev/null 2>&1; then
+  FGREP="grep -F"
+else
+  FGREP=fgrep
+fi
+
+# No color.
+AT_CHECK([$CONFIG_SHELL ./micro-suite], [1], [stdout], [stderr])
+for color in "$red" "$grn" "$lgn" "$blu"; do
+  AT_CHECK([cat stdout stderr | $FGREP "$color"], [1])
+done
+
+# Color of test group results.
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always], [1], [stdout], [stderr])
+AT_CHECK([cat stdout | grep " only " | $FGREP "$grn"], [], [ignore])
+AT_CHECK([cat stdout | grep " fail " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stdout | grep " xfail " | $FGREP "$lgn"], [], [ignore])
+AT_CHECK([cat stdout | grep " xpass " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stdout | grep " skip " | $FGREP "$blu"], [], [ignore])
+AT_CHECK([cat stdout | grep " hardfail " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+
+# The summary is green if all tests were successful, light green if all
+# behaved as expected, and red otherwise.
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always 1 -k skip],
+        [0], [stdout])
+AT_CHECK([cat stdout | grep 'test.*successful' | $FGREP "$grn"],
+        [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always 1 -k xfail -k skip],
+        [0], [stdout])
+AT_CHECK([cat stdout | grep 'as expected' | $FGREP "$lgn"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k fail],
+        [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k xpass],
+        [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k hardfail],
+        [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+], [1])
+
+
 ## ------------------- ##
 ## srcdir propagation. ##
 ## ------------------- ##
diff --git a/tests/local.at b/tests/local.at
index a812c43..39360ef 100644
--- a/tests/local.at
+++ b/tests/local.at
@@ -25,6 +25,8 @@ m4_pattern_allow([^m4_(define|shift)$])
 # Programs this package provides
 AT_TESTED([autom4te autoconf autoheader autoupdate autoreconf ifnames])
 
+# Enable colored test output.
+AT_COLOR_TESTS
 
 ## ---------------- ##
 ## Utility macros.  ##