Re: patch to add AT_CHECK_EUNIT autotest macro

[Ralf Wildenhues]

Hello,

the idea seems ok to me.  Please test distcheck before committing.
Minor nits below.

* Romain Lenglet wrote on Fri, Jul 31, 2009 at 03:24:04PM CEST:

> +AT_CHECK_EUNIT([testme_testsuite], address@hidden, address@hidden,
> +               [-pa "address@hidden@}/src"])
> address@hidden example
> +
> +This macro must be invoked in between @code{AT_SETUP} and @code{AT_CLEANUP}.

AT_CHECK has measures to complain if it is not between AT_SETUP and
AT_CLEANUP.  Can you please check that AT_CHECK_EUNIT complains, too
(it should, because it uses AT_CHECK)?  Thanks.

Would users want to write tests that are expected to fail/exit with a
nonzero status (from one of the involved tools)?

> +Variables @code{ERL}, @code{ERLC}, and (optionally) @code{ERLCFLAGS}
> +must be defined as the path of the Erlang interpreter, the path of the
> +Erlang compiler, and the command-line flags to pass to the compiler,
> +respectively.  Otherwise, the test group is skipped.

I'm not sure the semantics in that last sentence are best; but that is a
detail we can fix later.

> +If the testsuite is run in verbose mode, with option @option{--verbose},
> +EUnit is also run in verbose mode to output more details about
> +individual unittests.

s/unittests/unit tests/  ?   (more instances below)

> --- a/lib/autoconf/autotest.m4
> +++ b/lib/autoconf/autotest.m4
> @@ -84,6 +84,14 @@ at_top_builddir=\$at_top_build_prefix
>  AUTOTEST_PATH='m4_default([$2], [$1])'
>  
>  SHELL=\${CONFIG_SHELL-'$SHELL'}
> +
> +# Required to run EUnit unittests.
> +ERL='$ERL'
> +ERLC='$ERLC'
> +ERLCFLAGS='$ERLCFLAGS'
>  ATEOF
> -])
> +],
> +[ERL="$ERL"
> +ERLC="$ERLC"
> +ERLCFLAGS="$ERLCFLAGS"])

Can we make this code addition conditional to test suites in which
AT_CHECK_EUNIT is used?  (Diversions ought to help here.)
We don't want unnecessary code expansions.

> --- /dev/null
> +++ b/lib/autotest/erlang.m4

> +_AT_DEFINE_SETUP([AT_CHECK_EUNIT],
> +[AT_CHECK([test -f "$ERL" && test -f "$ERLC" || exit 77])

AT_SKIP_IF has lower overhead.

> +AT_CHECK(["$ERLC" $ERLCFLAGS -b beam $1.erl])

Does that never produce any output?  Wow.

> --- a/tests/autotest.at
> +++ b/tests/autotest.at
[...]
> +cd pkg
> +AT_CHECK_CONFIGURE
> +cd ..
> +
> +AT_CHECK([grep '^ERL='\''.*'\' pkg/t/atconfig], [], [ignore])
> +AT_CHECK([grep '^ERLC='\''.*'\' pkg/t/atconfig], [], [ignore])
> +AT_CHECK([grep '^ERLCFLAGS='\''.*'\' pkg/t/atconfig], [], [ignore])
> +
> +cd pkg/s
> +AT_CHECK([/bin/sh ./compile])

$CONFIG_SHELL, please.

> +cd ../..
> +
> +cd pkg/t
> +AT_CHECK([./suite], [0], [ignore])
> +cd ../..
> +
> +AT_CLEANUP

Thanks,
Ralf