Re: test setup advice

[Ralf Wildenhues]

Hi Tom,

* tom fogal wrote on Mon, Mar 27, 2006 at 11:17:51PM CEST:
> 
> I'm writing a C++ project which I currently have just under 30
> automake-based tests (e.g. check_PROGRAMS, TESTS) created.  The
> structure of my build system is that all of my source files for the
> program (under /src) build a `noinst' static library.  Then I compile
> the file with main() and link in this `everything' library.
> 
> My /tests directory then uses that same `everything' library to link
> all of the test programs against.  However the build trees for /src and
> /tests are separate; the toplevel Makefile.am uses SUBDIRS to descend
> into each of them.  The advantage of this whole setup is that adding a
> test is very simple:

Just FYI: if you used a non-recursive setup, all you'd need to change
would be (don't forget subdir-objects!):

>     1) write the test into e.g. "test.cpp"
>     2) in /tests/Makefile.am, append "test" to a variable I have
>        defined.

In Makefile.am, append "tests/test" instead.

>     3) in /tests/Makefile.am, add a "test_SOURCES=test.cpp" line.

In Makefile.am, add a "tests_tests_SOURCES = test.cpp" line.

> Unfortunately this lacks a lot of dependency information.  My new
> "test" depends on the `everything' library, which is correct, but it
> *really* only depends on one or two files from my /src tree.

So I assume you have "LDADD = ../src/libeverything.a".

> The
> upshot is that if I change anything from my /src tree, all of my tests
> are recompiled.  This wouldn't be such a big deal, but C++ is
> relatively slow to compile / link.  When I get 80 or so tests (which
> seems like it will be quite reasonable), I imagine I won't even want to
> bother running 'make check' anymore, and this will defeat the whole
> purpose of my unit testing.
> 
> The solution, of course, is to drop the `everything' library and
> explicitly code in the dependencies on particular files for every
> individual test.  This is going to cause /tests/Makefile.am to become
> very large and intricate though, and thus I believe adding a test will
> be a very error prone activity.

You could change incrementally, you could use more than one convenience
library to have more fine-grained control, and you could even list
sources explicitly.

Let me show you an example with C code:

TESTS = $(check_PROGRAMS)
check_PROGRAMS = tests/a$(EXEEXT) tests/b$(EXEEXT) tests/c$(EXEEXT)
noinst_LIBRARIES = src/libeverything.a src/libsomething.a
src_libeverything_a_SOURCES = src/s1.c src/s2.c
src_libsomething_a_SOURCES = src/s2.c
tests_a_SOURCES = tests/a.c
tests_b_SOURCES = tests/b.c src/s1.c
tests_c_SOURCES = tests/c.c
LDADD = src/libeverything.a
tests_b_LDADD =
tests_c_LDADD = src/libsomething.a

Here, by default, all programs will be linked against libeverything.
That is overridden for tests/b and tests/c: the former only uses s2.c,
while the latter only uses libsomething.

The respective nonrecursive setup looks like this:
$ cat Makefile.am
SUBDIRS = src tests
$ cat src/Makefile.am
noinst_LIBRARIES = libeverything.a libsomething.a
libeverything_a_SOURCES = s1.c s2.c
libsomething_a_SOURCES = s2.c
$ cat tests/Makefile.am
TESTS = $(check_PROGRAMS)
check_PROGRAMS = a$(EXEEXT) b$(EXEEXT) c$(EXEEXT)
a_SOURCES = a.c
b_SOURCES = b.c ../src/s1.c
c_SOURCES = c.c
LDADD = ../src/libeverything.a
b_LDADD =
c_LDADD = ../src/libsomething.a

(and obviously you need to adjust the list of files in AC_CONFIG_FILES
in configure.ac, too).  Does that help?

Cheers,
Ralf