Re: test failures on AIX 5.3

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Ralf Wildenhues on 6/30/2009 3:21 PM:
> On AIX 5.3, the testsuite currently has 3 failures for me:
> 
>>  111: autotest.at:266    Syntax error
>>       autotest

Let's deal with this one first.

>> ../../autoconf/tests/autotest.at:266: exit code was 2, expected 1
> 
> I've seen similar failures with other shells.  I think the test is too
> strict, we cannot assume a shell will fail the way bash does.

I'm not sure - the point of the test was that the outer testsuite shell
gracefully recovers from a syntax error in a test (since the test is run
in a subshell).  But according to the log, there was no recovery under
your shell.  Compare your stderr:

> stderr:
> ./micro-suite[1709]: syntax error at line 16 : `)' unexpected
> stdout:

with bash's:

> stderr:
> /home/eblake/autoconf/tests/testsuite.dir/111/micro-suite.dir/at-groups/2/test-source:
>  line 16: syntax error near unexpected token `)'
> /home/eblake/autoconf/tests/testsuite.dir/111/micro-suite.dir/at-groups/2/test-source:
>  line 16: `) >>"$at_stdout" 2>>"$at_stderr"'
> micro-suite: WARNING: unable to parse test group: 2
> ERROR: All 3 tests were run,
> 1 failed unexpectedly.
> stdout:

POSIX states this:

A syntax error in a special built-in utility may cause a shell executing
that utility to abort, while a syntax error in a regular built-in utility
shall not cause a shell executing that utility to abort. (See Consequences
of Shell Errors for the consequences of errors on interactive and
non-interactive shells.) If a special built-in utility encountering a
syntax error does not abort the shell, its exit value shall be non-zero.
http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14

I can reproduce this sort of shell difference with something even simpler
(which also points out a zsh bug, at least for 4.3.9):

$ echo 'if' > tmp
$ bash -c '. ./tmp; echo $?'
./tmp: line 2: syntax error: unexpected end of file
1
$ ash -c '. ./tmp; echo $?'
./tmp: 2: Syntax error: end of file unexpected (expecting "then")
$ pdksh -c '. ./tmp; echo $?'
pdksh: ./tmp[1]: syntax error: `if' unmatched
1
$ zsh -c '. ./tmp; echo $?'
0
$ zsh -c 'emulate sh; . ./tmp; echo $?'
./tmp:2: parse error near `\n'
0

So ash is like AIX, where the syntax error encountered during `.' exited
the entire script rather than just the . builtin, but my reading of POSIX
says that they are allowed to do that.  Meanwhile, zsh is buggy for
keeping $? at 0 even after a syntax error in `.'.

One possible solution - throw more forks at the problem:

$ ash -c '(. ./tmp); echo $?'
./tmp: 2: Syntax error: end of file unexpected (expecting "then")
2

Does adding the subshell fix things for AIX like it did for ash?  If so,
then we can modify lines 1364 and 1399 of lib/autotest/general.m4 to
source "$at_test_source" from within a subshell, and that should clean up
this testsuite failure (at the cost of one more fork per test, but when
has that stopped us before?).  I won't push this commit until I get some
feedback from you.

But that still doesn't avoid the zsh bug.  Maybe the test should be
modified to SKIP on shells that can't detect syntax errors on buggy shells?

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpLXZwACgkQ84KuGfSFAYAjHwCcD1FbolO0xxZEn09NI8JR0Qlj
SyAAnjkgTq5c3nBdh9j0hPE29fjkMnB2
=T7q8
-----END PGP SIGNATURE-----

>From 38d4a349fa3f98bf41526f737a71158308cb96d5 Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Wed, 1 Jul 2009 06:45:17 -0600
Subject: [PATCH] Make autotest more tolerant to syntax errors.

* lib/autotest/general.m4 (AT_INIT) <driver loop>: Source
questionable text in a subshell, to avoid killing outer shell.
Reported by Ralf Wildenhues.

Signed-off-by: Eric Blake <address@hidden>
---
 ChangeLog               |    7 +++++++
 lib/autotest/general.m4 |    4 ++--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 8f0b5ae..b5f4360 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2009-07-01  Eric Blake  <address@hidden>
+
+       Make autotest more tolerant to syntax errors.
+       * lib/autotest/general.m4 (AT_INIT) <driver loop>: Source
+       questionable text in a subshell, to avoid killing outer shell.
+       Reported by Ralf Wildenhues.
+
 2009-06-30  Jan Madzik  <address@hidden>  (tiny change)
            Ralf Wildenhues  <address@hidden>

diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index ddc2270..62e7d60 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -1361,7 +1361,7 @@ dnl           kill -13 $$
       at_fn_group_prepare
       if cd "$at_group_dir" &&
         at_fn_test $at_group &&
-        . "$at_test_source" # AT_JOB_FIFO_FD<&-
+        (. "$at_test_source") # AT_JOB_FIFO_FD<&-
       then :; else
        AS_WARN([unable to parse test group: $at_group])
        at_failed=:
@@ -1396,7 +1396,7 @@ else
     at_fn_group_prepare
     if cd "$at_group_dir" &&
        at_fn_test $at_group &&
-       . "$at_test_source"; then :; else
+       (. "$at_test_source"); then :; else
       AS_WARN([unable to parse test group: $at_group])
       at_failed=:
     fi
-- 
1.6.3.3.334.g916e1