[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: serial-tests option and backwards compatibility

From: Stefano Lattarini
Subject: Re: serial-tests option and backwards compatibility
Date: Mon, 04 Feb 2013 14:31:43 +0100

On 02/04/2013 01:16 AM, Luke Mewburn wrote:
> On Wed, Jan 16, 2013 at 12:12:13PM +0100, Stefano Lattarini wrote:
>   | On 01/15/2013 07:32 PM, Brandon Black wrote:
>   | > So my conundrum is this: in automake-1.13, the default switched from 
> serial
>   | > testing to parallel testing.  The only way I can see to disable this is 
> to
>   | > add "serial-tests" to my AM_INIT_AUTOMAKE.  However, my AM_INIT_AUTOMAKE
>   | > also only requires version 1.11.6, which I'd like to continue 
> supporting,
>   | > and these older versions of automake will barf on the invalid option
>   | > "serial-tests" (I also tried "no-parallel-tests", but that's also 
> invalid).
>   | > 
>   | Yes, the "serial-tests" option was unfortunately only introduced in
>   | Automake 1.12.
> We use autotest extensively, and our test scenario is rather complex
> and relies upon the existing serial behaviour;
I don't follow; if you use autotest, why you also need to rely on the
AUtomake-provided test harnesses?  None of the GNU projects using
autotest and that I know of (tar, bison, autoconf, libtool) does so.

> we setup file systems
> and databases, we start daemons, etc.  We can't use parallel tests
> at this time.
> If parallel tests supported a dependency graph a la make
But it does; in fact, it relies on make to honour such a dependency graph;
quoting from section 15.2.3 "Parallel Test Harness" of the Automake manual

  In order to guarantee an ordering between tests even with make -jN,
  dependencies between the corresponding .log files may be specified
  through usual make dependencies. For example, the following snippet
  lets the test named foo-execute.test depend upon completion of the
  test foo-compile.test:

     TESTS = foo-compile.test foo-execute.test
     foo-execute.log: foo-compile.log

  Please note that this ordering ignores the results of required tests,
  thus the test foo-execute.test is run even if the test foo-compile.test
  failed or was skipped beforehand. Further, please note that specifying
  such dependencies currently works only for tests that end in one of
  the suffixes listed in TEST_EXTENSIONS.

Or am I missing something else?

> or tsort then we could possibly take advantage of that.
> In my humble opinion, providing an option to control a feature in
> a major release (1.12.x) then change the default for that feature
> in the next major release (1.13.x) 7 months later is too short a period.
With this I agree, in retrospect.  But it's too late to change that
(reverting the default-switching only to re-introduce it in a later
release would just exacerbate confusion at this point, IMHO).

> Especially when the time between previous major releases was 2.5 years.
> Examining the Changelog and release dates:
>  2009-12-08  Release automake 1.11.1
>  2012-02-21  Add "serial-tests" support (in "HEAD")
>  2012-04-13  Release automake 1.11.5 (without "serial-tests")
>  2012-05-18  Parallel tests now the default (in "HEAD", not 1.11.x)
>  2012-06-01  Release automake 1.12.1 (with "serial-tests")
When I did this, I should really have published a 1.11.x release offering
this same option as well; that would have removed all confusion.  Sigh,
such a low-hanging fruit not picked :-(

>  2013-01-01  Release automake 1.13.1 (parallel tests now default)
> This isn't the only backwards incompatible change made recently,
> and in my humble opinion I think the timeframes introducing
> backwards incompatibility are too aggressive.
You are not the only one to think so, and I've come to agree (at least
partially); for some more discussions and background, see:


So things should proceed more carefully in the future (I hope).

> From a backwards-compatibility point of view, I think the default
> should be reverted to serial tests, and make it clearer that
> parallel tests are available as an option.
With this I must disagree, sorry.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]