automake
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: test setup advice


From: Marc Alff
Subject: Re: test setup advice
Date: Wed, 29 Mar 2006 00:04:01 -0700
User-agent: Mozilla Thunderbird 1.0.7 (X11/20060327)


Hi Tom,


tom fogal wrote:

I'm looking for advice on how to structure a build system using
automake.

Thanks for any solutions / ideas / comments.

-tom

These are just random thoughts / ideas that I used on C++ projects,
just my two cents (and in all cases, my personal opinion, your mileage may vary). It's not automake specific and more high level, but I implemented these with automake and it worked well.

Any developer will use an excuse that tests are slow to build / difficult to maintain to actually not maintain them, so being sensitive to that is very healthy. However, it's not limited to tests alone but true for the entire code base, so having code that builds fast and lean improves development
in general.
To achieve that, a couple of techniques do work in C++ :
- try to limit the #includes dependency hell. Sometimes a forward (class foo;) is much better that actually
including foo.h
- avoid including /usr/include/xxx.h (C stdio, C++ stream, etc) in header files, include that in .cpp files instead. - once in a while review the dependencies generated by automake on header files : it's very instructive and shows you what the compiler has to deal with. I personally prefer to cut dependencies rather that using
pre-compiled headers of other things like that.
- templates are powerful, but can become quickly hell when abused.

Tests do influence design : for the production code itself, the better it's designed, the easier it is to test. Writing tests at the same time of the production code actually influences the code to a better karma. Writing tests after the facts on spaghetti code is just damage control, not to mention it's very difficult.

Separating "production" and "test" code in different directories is very healthy, for many reasons : - all the test code can be instantly unplugged to make sure that the production code builds without using test code while compiling against dummy header files, or worst linking against test stubs. - porting your code to a different platform can be done for the production code alone if needed,
in case the test code uses tools not available everywhere.
- looking from the CVS point of view, someone checking in a change in production code makes me more nervous that someone checking in a new test, and having a simple way to find out which code is which
really helps.

On test themselves, I make the distinction between :
- test drivers (code that you actually have to write for a new family of test cases), - test cases (just a description of the test itself, might not be C++ code but just a text or xml file).

The idea is to write a test driver that reads an external script that tells it what test to execute, or what function to call with what value. This code is typically more complex to write, but is written once.
A test case (typically a script) is when written for each test.
Adding a test case does not involves any build, and can be done at any time.
It's also very useful for debugging, since anyone (including non programmers) can write a new case at anytime
(provided of course that it uses the feature of existing drivers).

If each test consists of code then yes, it takes time to build, and effort to maintain. If each test case consist of a script that talks to a test driver, building is very fast,
maintenance can be a breeze of hell, depending on the design.
A word of warning, good planning is the key there.

In general, I prefer "small" test binaries and not linking to everything.a, so that once the test
gives the correct result, the test driver can be leveraged to do :
- performance profiling (gprof),
- code coverage (tcov)
- or dynamic analysis (purify like tools).
The idea is that since someone made an effort to write it (it's automated in the makefile, right ?),
it might as well be used to it's full value.

tcov is very important, since a good test suite will try to improve coverage,
not beat the same function to death while ignoring all the others,
and it's very hard to access coverage without a tool (take a wild guess, divide by 10, that's the real coverage).

That's all for now, sorry if it's a bit off list.

I hope this helps with ideas.

Cheers,
Marc Alff.





reply via email to

[Prev in Thread] Current Thread [Next in Thread]