autoconf-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] Keep testsuite files on unexpected pass.


From: Ralf Wildenhues
Subject: Re: [PATCH] Keep testsuite files on unexpected pass.
Date: Mon, 16 Aug 2010 21:08:15 +0200
User-agent: Mutt/1.5.20 (2010-04-22)

* Eric Blake wrote on Sat, Aug 14, 2010 at 08:06:18PM CEST:
> On 08/14/2010 12:55 AM, Ralf Wildenhues wrote:
> > Also I just noticed that --errexit is broken wrt. XPASS tests, but
> > that's not new with your patch either.  The following seems to fix
> > it but I don't have testsuite exposure yet.  One thing to note for
> > the latter is that the "test ... and inhibited subsequent tests."
> > notice is not necessarily true, in that the testsuite does run
> > subsequent tests after an XPASS.
> 
> Good catch.  Are you planning on cleaning up this diff into a full
> patch, or should I apply it as is?

Proposed full patch,  also adding coverage for other test results.

Thanks,
Ralf

    Fix Autotest --errexit to exit after XPASSing tests.
    
    * lib/autotest/general.m4 (AT_INIT) <at_fn_group_postprocess>:
    Exit after an unexpected passing test if $at_errexit.
    * tests/autotest.at (errexit): Also try tests that xpass, skip,
    xfail, or fail hard.

diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 343871f..ea71700 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -1245,6 +1245,9 @@ _ATEOF
       # or the success was unexpected.
       if $at_debug_p || test $at_res = xpass; then
        at_fn_create_debugging_script
+       if test $at_res = xpass && $at_errexit; then
+         echo stop > "$at_stop_file"
+       fi
       else
        if test -d "$at_group_dir"; then
          find "$at_group_dir" -type d ! -perm -700 -exec chmod u+rwx \{\} \;
diff --git a/tests/autotest.at b/tests/autotest.at
index 0546163..f4a6f0a 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -368,12 +368,41 @@ AT_CHECK_AT_TEST([errexit],
   [AT_CHECK([false])
    AT_CLEANUP
    AT_SETUP([test that should not be run])
-   AT_CHECK([:])],
+   AT_CHECK([:])
+   AT_CLEANUP
+   AT_SETUP([xpassing test])
+   AT_XFAIL_IF([:])
+   AT_CHECK([:])
+   AT_CLEANUP
+   AT_SETUP([another test that should not be run])
+   AT_CHECK([:])
+   AT_CLEANUP
+   AT_SETUP([skipping test])
+   AT_CHECK([exit 77])
+   AT_CLEANUP
+   AT_SETUP([xfailing test])
+   AT_XFAIL_IF([:])
+   AT_CHECK([false])
+   AT_CLEANUP
+   AT_SETUP([a test that should be run])
+   AT_CLEANUP
+   AT_SETUP([hard failure])
+   AT_XFAIL_IF([:])
+   AT_CHECK([exit 99])
+   AT_CLEANUP
+   AT_SETUP([yet another test that should not be run])],
   [], [1], [stdout], [stderr], [],
   [AT_CHECK([test -f micro-suite.log], [1])
    touch micro-suite.log # shut up AT_CAPTURE_FILE.
    AT_CHECK([grep "should not be run" stdout], [1])
-   AT_CHECK([grep "1 .* inhibited subsequent" stderr], [], [ignore])],
+   AT_CHECK([grep "1 .* inhibited subsequent" stderr], [], [ignore])
+   AT_CHECK([$CONFIG_SHELL ./micro-suite --errexit 3-], [1], [stdout], 
[stderr])
+   AT_CHECK([grep "should not be run" stdout], [1])
+   AT_CHECK([grep "1 .* inhibited subsequent" stderr], [], [ignore])
+   AT_CHECK([$CONFIG_SHELL ./micro-suite --errexit 5-], [1], [stdout], 
[stderr])
+   AT_CHECK([grep "should be run" stdout], [0], [ignore])
+   AT_CHECK([grep "should not be run" stdout], [1])
+   AT_CHECK([grep "inhibited subsequent" stderr], [], [ignore])],
   [--errexit])
 
 AT_CHECK_AT_TEST([unquoted output],



reply via email to

[Prev in Thread] Current Thread [Next in Thread]