Re: [PATCH 5/5] {test-protocols} tap: add experimental TAP-aware driver

[Stefano Lattarini]

On Friday 15 July 2011, Stefano Lattarini wrote:
> * doc/automake.texi (Using the TAP test protocol): New section.
> (Overview of Custom Test Drivers Support): Minor updates.
> * lib/tap-driver: New script, TAP-aware test driver for Automake;
> implemented in perl and based on TAP::Parser.
> * lib/Makefile.am (dist_script_DATA): Add it.
> * tests/tap-autonumber.test: New test.
> * tests/tap-bailout.test: Likewise.
> * tests/tap-basic.test: Likewise.
> * tests/tap-deps.test: Likewise.
> * tests/tap-diagnostic.test: Likewise.
> * tests/tap-empty.test: Likewise.
> * tests/tap-empty-diagnostic.test: Likewise.
> * tests/tap-escape-directive.test: Likewise.
> * tests/tap-exit.test: Likewise.
> * tests/tap-fancy.test: Likewise.
> * tests/tap-fancy2.test: Likewise.
> * tests/tap-global-result.test: Likewise.
> * tests/tap-html.test: Likewise.
> * tests/tap-log.test: Likewise.
> * tests/tap-merge-stdout-stderr.test: Likewise.
> * tests/tap-more.test: Likewise.
> * tests/tap-more2.test: Likewise.
> * tests/tap-no-merge-stdout-stderr.test: Likewise.
> * tests/tap-no-spurious-summary.test: Likewise.
> * tests/tap-no-spurious.test: Likewise.
> * tests/tap-not-ok-skip.test: Likewise.
> * tests/tap-numeric-description.test: Likewise.
> * tests/tap-out-of-order.test: Likewise.
> * tests/tap-passthrough.test: Likewise.
> * tests/tap-plan.test: Likewise.
> * tests/tap-plan-errors.test: Likewise.
> * tests/tap-plan-corner.test: Likewise.
> * tests/tap-realtime.test: Likewise.
> * tests/tap-recheck-logs.test: Likewise.
> * tests/tap-recheck.test: Likewise.
> * tests/tap-skip-whole.test: Likewise.
> * tests/tap-summary.test: Likewise.
> * tests/tap-todo-skip.test: Likewise.
> * tests/tap-todo-skip-together.test: Likewise.
> * tests/tap-todo-skip-whitespace.test: Likewise.
> * tests/tap-skipall-whitespace.test: Likewise.
> * tests/tap-unplanned.test: Likewise.
> * tests/tap-whitespace-normalization.test: Likewise.
> * tests/tap-with-and-without-number.test: Likewise.
> * tests/tap-xfail-tests.test: Likewise.
> * tests/tap-bad-prog.test: New xfailing test.
> * tests/tap-color.test: Likewise.
> * tests/tap-plan-corner2.test: Likewise.
> * tests/tap-message-0.test: Likewise.
> * tests/tap-signal.test: Likewise.
> * tests/Makefile.am (TESTS, XFAIL_TESTS): Update.
> ---
>
And consider this squashed in:

-*-*-*-

diff --git a/tests/tap-color.test b/tests/tap-color.test
index 8ed7265..82c0b51 100755
--- a/tests/tap-color.test
+++ b/tests/tap-color.test
@@ -52,7 +52,8 @@ AUTOMAKE_OPTIONS = color-tests
 AM_TEST_LOG_DRIVER_FLAGS = --comments
 TEST_LOG_COMPILER = cat
 TEST_LOG_DRIVER = $(PERL) $(srcdir)/tap-driver
-TESTS = all.test skip.test bailout.test plan-errors.test
+TESTS = all.test skip.test bail.test badplan.test noplan.test \
+        few.test many.test order.test
 END
 
 cat > all.test << 'END'
@@ -66,15 +67,41 @@ ok 5 - zardoz # TODO
 END
 
 cat > skip.test << 'END'
-1..0 # Whole script not run
+1..0 # SKIP whole script
 END
 
-cat > errors.test << 'END'
+cat > bail.test << 'END'
 1..1
 ok 1
 Bail out!
 END
 
+cat > badplan.test << 'END'
+# foo
+1..1
+ok 1
+END
+
+cat > noplan.test << 'END'
+ok 1
+END
+
+cat > few.test << 'END'
+1..2
+ok 1
+END
+
+cat > many.test << 'END'
+1..1
+ok 1
+ok 2
+END
+
+cat > order.test << 'END'
+1..1
+ok 5
+END
+
 $ACLOCAL
 $AUTOCONF
 $AUTOMAKE
@@ -92,9 +119,15 @@ test_color ()
   cat stdout | grep "^${blu}SKIP${std}: all\.test 3 - baz # SKIP sk$"
   cat stdout | grep "^${red}FAIL${std}: all\.test 4 - quux$"
   cat stdout | grep "^${red}XPASS${std}: all\.test 5 - zardoz # TODO$"
-  cat stdout | grep "^${blu}SKIP${std}: skip\.test # Whole script not run$"
-  cat stdout | grep "^${grn}PASS${std}: bailout\.test 1$"
-  cat stdout | grep "^${mgn}ERROR${std}: all\.test - Bail out!$"
+  cat stdout | grep "^${blu}SKIP${std}: skip\.test - whole script$"
+  cat stdout | grep "^${grn}PASS${std}: bail\.test 1$"
+  cat stdout | grep "^${mgn}ERROR${std}: bail\.test - Bail out!$"
+  cat stdout | grep "^${mgn}ERROR${std}: badplan\.test - test plan in middle 
of output$"
+  cat stdout | grep "^${mgn}ERROR${std}: noplan\.test - missing test plan$"
+  cat stdout | grep "^${mgn}ERROR${std}: few.test - too few tests run 
(expected 2, got 1)$"
+  cat stdout | grep "^${mgn}ERROR${std}: many.test - too many tests run 
(expected 1, got 2)$"
+  cat stdout | grep "^${mgn}ERROR${std}: many.test 2 # UNPLANNED$"
+  cat stdout | grep "^${mgn}ERROR${std}: order.test 5 # OUT-OF-ORDER 
(expecting 1)$"
   # Diagnostic messages shouldn't be colorized.
   cat stdout | grep "^# all\.test: Hi! I shouldn't be colorized!$"
   :

-*-*-*-

Sorry for the noise,
  Stefano