Re: Tests fail due to argument list too long

[Ralf Wildenhues]

* Bob Friesenhahn wrote on Fri, Jul 31, 2009 at 07:29:31PM CEST:
> On Fri, 31 Jul 2009, Ralf Wildenhues wrote:
> 
> >You should be able to work around it by splitting tests in pieces,
> >and running things piecewise only for now; assuming the *_TESTS
> >variables are all nonempty:

> > check-wand:
> >       $(MAKE) $(AM_MAKEFLAGS) check-TESTS \
> >               TESTS='$(WAND_TESTS)' TEST_SUITE_LOG=test-suite-wand.log

> Are you saying that the user would then need to explicitly invoke
> 'check-wand' or is it possible to create a 'check' dependency so
> that all of these would be implicitly invoked in turn?

I think it should be possible to create a 'check' target that invokes
all bits in turn (ignore automake target override warnings with
-Wno-override).

> It looks
> like there would be a bunch of mini-test reports and so the user is
> likely to miss most of the reports since they would have scrolled on
> by.

You could error out after the first failure; or provide a summary at the
end, manually.  As I said, it's a hack for now (and it also uses
internal details like check-TESTS).  I still need to think of a way to
fix this issue properly in a future release.

> It is likely that the number of tests will quadruple over time since
> currently the tests are the bare minimum (only 785 tests) for my
> software.  I don't mind explicitly splitting internally if there is
> still a way for the user to do the standard 'make check' and 'make
> distcheck' (with reliable ultimate failure on test fail) and see a
> total report for all of the tests at the end.

You could use multiple Makefile.am files (gasp!) with one test suite in
each.  Or you could merge multiple tests in one test script/program of
yours.  All hacks, yes, but at least they should get you going for now.

Cheers,
Ralf