Re: [RFC] Autotest: AT_TEST and XFAILs

[Akim Demaille]

Hi Paulo,

Sorry for the delays.

 > In the attempt to add expected failures to Autotest (after which I
 > might use it in GNU Smalltalk!),

Cool feature!

 > I found out that there is no clear place to do initializations that
 > ought to be *outside* the subshell. 

Could you make this clearer to me?  What kind of initialization are
you referring to?  See for instance in foreign.at what are *user*
initializations to me.  I'm not sure the user should have this.

 > For this reason I am submitting this patch, which adds another
 > argument to AT_SETUP which does those initializations: you can use
 > it for AT_KEYWORDS and for my macro AT_XFAIL_IF.

I think this should remain private to Autotest, the user does not need
access to the top level.  This is breaking our abstraction.

 > In addition, I am providing an alternative interface than
 > AT_SETUP...AT_CLEANUP: a single macro called AT_TEST that receives
 > description, commands and initializations (like AC_CONFIG_FILES for
 > example).  The attached patch also changes the whole test suite to use
 > it -- it is quite mechanical to do that with "perl -i -pe".

Sorry, but I don't really like this part.  The SETUP/CLEANUP version
is much clearer to me.  In addition, I don't like the small initial
part being rejected to the end.  It should be the opposite IMHO.

I'm ok with AT_TEST if you think this is good, but please, don't
deprecate the pair.

 > Two words on expected failures.  First, a test can be marked to fail
 > conditionally, since AT_XFAIL_IF expects a shell expression: this is
 > quite hard to do with Automake's XFAIL_TESTS.  Second, the code to
 > write the results summary was completely redone to include the wealth
 > of possibilities that you can have with expected failures.

Great material!