Re: automake 1.11.3 check-TESTS and command line length

[Tim Landscheidt]

Bob Friesenhahn <address@hidden> wrote:

>> Are you referring to the GraphicsMagick testsuite?  If yes, it seems to me
>> that it could benefit greatly from the use of TAP once automake 1.12 is out;
>> for example, all the 314 'tests/rwfile*.sh' tests could be rewritten as one
>> single TAP test (and similarly for many other auto-generated tests), thus
>> removing the need for a huge number of files without reducing granularity
>> of the results.

> I took a look at what "TAP" is and see that I have already
> been using it already (not knowing what it was called) for
> Perl-related tests. The fundamental problem with TAP is that
> documented implementations
> (http://testanything.org/wiki/index.php/TAP_Consumers)
> require additional packages on the system such as Perl,
> Ruby, Javascript, Java, etc.  I can not expect users who
> want to verify their build with 'make check' to install an
> additional language interpreter.

> It seems that (while I was sleeping) Automake 1.12 will be
> providing support for TAP tests
> (http://lists.gnu.org/archive/html/automake/2011-04/msg00069.html). Such
> support will not be worthwhile if it depends on Perl or some
> other interpreter since this might not be installed on the
> end-user's machine.  If it is based on portable C
> (e.g. http://www.eyrie.org/~eagle/software/c-tap-harness/)
> or portable shell script then it will be sufficiently
> usable.

> So what is the story about TAP in Automake 1.12?  Is it time
> to describe the new feature to us mere mortals (i.e. people
> on address@hidden) rather than Automake developers?

Development automake comes with consumers in awk and Perl.
I have recently used the former (*1) with automake 1.11.1.
It involved copying lib/tap-driver.sh from a development
version to my project.  tests/Makefile.am then just read:

| TESTS_ENVIRONMENT = PERL5LIB=$(top_srcdir)/lib; export PERL5LIB;
| TEST_LOG_DRIVER = env AM_TAP_AWK='$(AWK)' $(SHELL) 
$(top_srcdir)/build-aux/tap-driver.sh
| TESTS = a.test b.test c.test

I believe later automake versions prefer
AM_TESTS_ENVIRONMENT for the first line, but updating would
have been a major headache.

  Note though that IIRC if you were to combine (all) tests
in one test, you'd lose the ability to run them in parallel
(or you would have to implement that yourself).

Tim

(*1)   In a project that uses Perl, using awk isn't very ra-
       tional, but I wanted to try it out.