[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: serial-tests option and backwards compatibility

From: Bob Friesenhahn
Subject: Re: serial-tests option and backwards compatibility
Date: Sat, 19 Jan 2013 10:02:11 -0600 (CST)
User-agent: Alpine 2.01 (GSO 1266 2009-07-14)

On Thu, 17 Jan 2013, Stefano Lattarini wrote:
(I know future is to use TAP log protocol, but that's not an excuse
to break working package tests.)

Actually, TAP makes sense in a lot of situations, but not always,
an maybe not even most of the time.  First, several TAP tests in
a single script are usually run serially (most TAP-producing libraries
out there are not multi-threaded), and on fast and many-cores machines
this can be a real performance killer if you implement your testsuite
in just a handful of TAP scripts.  In addition, TAP is overkill for
many testing scenarios, not worth the trouble.  The good thing of the
parallel Automake harness is that it allows you to freely mix "classic"
test scripts and TAP based test script, allowing you to use the best
of both worlds (Automake' own test suite makes an heavy use of this

My own experience with TAP is that the per-test overhead was reduced so that the tests actually ran considerably faster (particularly under Microsoft Windows). If your test software runs really slow, then the experience may be different. There is a lot of overhead in the parallel-tests framework.

For parallel builds, just make sure that you have enough independent TAP test scripts to keep the system busy when running parallel tests. Only very CPU-bound single-threaded tests will be sped-up much by the parallel-tests framework. If your software is already threaded, then parallel-tests may lengthen the test time due to creating too many threads on the system.

Of course, TAP does not solve the backward-compatibility problem.

Bob Friesenhahn
GraphicsMagick Maintainer,

reply via email to

[Prev in Thread] Current Thread [Next in Thread]