[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] Keep testsuite files on unexpected pass.
From: |
Ralf Wildenhues |
Subject: |
Re: [PATCH] Keep testsuite files on unexpected pass. |
Date: |
Mon, 16 Aug 2010 21:08:15 +0200 |
User-agent: |
Mutt/1.5.20 (2010-04-22) |
* Eric Blake wrote on Sat, Aug 14, 2010 at 08:06:18PM CEST:
> On 08/14/2010 12:55 AM, Ralf Wildenhues wrote:
> > Also I just noticed that --errexit is broken wrt. XPASS tests, but
> > that's not new with your patch either. The following seems to fix
> > it but I don't have testsuite exposure yet. One thing to note for
> > the latter is that the "test ... and inhibited subsequent tests."
> > notice is not necessarily true, in that the testsuite does run
> > subsequent tests after an XPASS.
>
> Good catch. Are you planning on cleaning up this diff into a full
> patch, or should I apply it as is?
Proposed full patch, also adding coverage for other test results.
Thanks,
Ralf
Fix Autotest --errexit to exit after XPASSing tests.
* lib/autotest/general.m4 (AT_INIT) <at_fn_group_postprocess>:
Exit after an unexpected passing test if $at_errexit.
* tests/autotest.at (errexit): Also try tests that xpass, skip,
xfail, or fail hard.
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 343871f..ea71700 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -1245,6 +1245,9 @@ _ATEOF
# or the success was unexpected.
if $at_debug_p || test $at_res = xpass; then
at_fn_create_debugging_script
+ if test $at_res = xpass && $at_errexit; then
+ echo stop > "$at_stop_file"
+ fi
else
if test -d "$at_group_dir"; then
find "$at_group_dir" -type d ! -perm -700 -exec chmod u+rwx \{\} \;
diff --git a/tests/autotest.at b/tests/autotest.at
index 0546163..f4a6f0a 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -368,12 +368,41 @@ AT_CHECK_AT_TEST([errexit],
[AT_CHECK([false])
AT_CLEANUP
AT_SETUP([test that should not be run])
- AT_CHECK([:])],
+ AT_CHECK([:])
+ AT_CLEANUP
+ AT_SETUP([xpassing test])
+ AT_XFAIL_IF([:])
+ AT_CHECK([:])
+ AT_CLEANUP
+ AT_SETUP([another test that should not be run])
+ AT_CHECK([:])
+ AT_CLEANUP
+ AT_SETUP([skipping test])
+ AT_CHECK([exit 77])
+ AT_CLEANUP
+ AT_SETUP([xfailing test])
+ AT_XFAIL_IF([:])
+ AT_CHECK([false])
+ AT_CLEANUP
+ AT_SETUP([a test that should be run])
+ AT_CLEANUP
+ AT_SETUP([hard failure])
+ AT_XFAIL_IF([:])
+ AT_CHECK([exit 99])
+ AT_CLEANUP
+ AT_SETUP([yet another test that should not be run])],
[], [1], [stdout], [stderr], [],
[AT_CHECK([test -f micro-suite.log], [1])
touch micro-suite.log # shut up AT_CAPTURE_FILE.
AT_CHECK([grep "should not be run" stdout], [1])
- AT_CHECK([grep "1 .* inhibited subsequent" stderr], [], [ignore])],
+ AT_CHECK([grep "1 .* inhibited subsequent" stderr], [], [ignore])
+ AT_CHECK([$CONFIG_SHELL ./micro-suite --errexit 3-], [1], [stdout],
[stderr])
+ AT_CHECK([grep "should not be run" stdout], [1])
+ AT_CHECK([grep "1 .* inhibited subsequent" stderr], [], [ignore])
+ AT_CHECK([$CONFIG_SHELL ./micro-suite --errexit 5-], [1], [stdout],
[stderr])
+ AT_CHECK([grep "should be run" stdout], [0], [ignore])
+ AT_CHECK([grep "should not be run" stdout], [1])
+ AT_CHECK([grep "inhibited subsequent" stderr], [], [ignore])],
[--errexit])
AT_CHECK_AT_TEST([unquoted output],