[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: test setup advice
Re: test setup advice
Wed, 29 Mar 2006 00:04:01 -0700
Mozilla Thunderbird 1.0.7 (X11/20060327)
tom fogal wrote:
I'm looking for advice on how to structure a build system using
Thanks for any solutions / ideas / comments.
These are just random thoughts / ideas that I used on C++ projects,
just my two cents (and in all cases, my personal opinion, your mileage
It's not automake specific and more high level, but I implemented these
with automake and it worked well.
Any developer will use an excuse that tests are slow to build /
difficult to maintain to actually
not maintain them, so being sensitive to that is very healthy. However,
it's not limited to tests
alone but true for the entire code base, so having code that builds fast
and lean improves development
To achieve that, a couple of techniques do work in C++ :
- try to limit the #includes dependency hell. Sometimes a forward (class
foo;) is much better that actually
- avoid including /usr/include/xxx.h (C stdio, C++ stream, etc) in
header files, include that in .cpp files instead.
- once in a while review the dependencies generated by automake on
header files : it's very instructive
and shows you what the compiler has to deal with. I personally prefer to
cut dependencies rather that using
pre-compiled headers of other things like that.
- templates are powerful, but can become quickly hell when abused.
Tests do influence design : for the production code itself, the better
it's designed, the easier it is to test.
Writing tests at the same time of the production code actually
influences the code to a better karma.
Writing tests after the facts on spaghetti code is just damage control,
not to mention it's very difficult.
Separating "production" and "test" code in different directories is very
healthy, for many reasons :
- all the test code can be instantly unplugged to make sure that the
production code builds without using
test code while compiling against dummy header files, or worst linking
against test stubs.
- porting your code to a different platform can be done for the
production code alone if needed,
in case the test code uses tools not available everywhere.
- looking from the CVS point of view, someone checking in a change in
production code makes me more
nervous that someone checking in a new test, and having a simple way to
find out which code is which
On test themselves, I make the distinction between :
- test drivers (code that you actually have to write for a new family of
- test cases (just a description of the test itself, might not be C++
code but just a text or xml file).
The idea is to write a test driver that reads an external script that
tells it what test to execute,
or what function to call with what value. This code is typically more
complex to write, but is written once.
A test case (typically a script) is when written for each test.
Adding a test case does not involves any build, and can be done at any time.
It's also very useful for debugging, since anyone (including non
programmers) can write a new case at anytime
(provided of course that it uses the feature of existing drivers).
If each test consists of code then yes, it takes time to build, and
effort to maintain.
If each test case consist of a script that talks to a test driver,
building is very fast,
maintenance can be a breeze of hell, depending on the design.
A word of warning, good planning is the key there.
In general, I prefer "small" test binaries and not linking to
everything.a, so that once the test
gives the correct result, the test driver can be leveraged to do :
- performance profiling (gprof),
- code coverage (tcov)
- or dynamic analysis (purify like tools).
The idea is that since someone made an effort to write it (it's
automated in the makefile, right ?),
it might as well be used to it's full value.
tcov is very important, since a good test suite will try to improve
not beat the same function to death while ignoring all the others,
and it's very hard to access coverage without a tool (take a wild guess,
divide by 10, that's the real coverage).
That's all for now, sorry if it's a bit off list.
I hope this helps with ideas.