[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Extended test suites

From: Ralf Wildenhues
Subject: Re: Extended test suites
Date: Fri, 28 Aug 2009 20:20:31 +0200
User-agent: Mutt/1.5.20 (2009-08-09)

Hello Martin,

* Martin Quinson wrote on Thu, Aug 27, 2009 at 06:33:05PM CEST:
> I already know about the XFAIL variable and the possibility to mark that
> some tests are known to fail (and unfortunately, we also have to use
> this). But I'm here speaking of something like a TEST_EXTRA variable and
> a check-extra: make target testing the base tests plus the extra ones.

You could let some tests skip (exit 77) if some condition (in a file) is
or is not set.

> I tried to do it with something like the following chunk:
> TESTS?=regular mandatory tests
> TESTS_EXTRA=other, more fragile tests.
> check-extra: $(TESTS_EXTRA)
>       for d in $(SUBDIRS) ; do $(MAKE) check-extra $$d ; done
>       TESTS="$(TESTS) $(TESTS_EXTRA)" $(MAKE) check

Why does this not work?

TESTS = regular mandatory tests
TESTS_EXTRA = other, more fragile tests.
check-extra: $(TESTS_EXTRA)
        for d in $(SUBDIRS) ; do $(MAKE) check-extra $$d ; done
        $(MAKE) check TESTS="$(TESTS) $(TESTS_EXTRA)"

It should be portable.  If you want to override portably from
environment variables, you need to use 'make -e'.

Do you know the 'parallel-tests' option from Automake 1.11?  It allows
you to run tests in parallel, and allows some more features like recheck
and partial check runs based on dependencies.  You could use those too
to avoid checking some fragile tests.

Hope that helps.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]