Re: Unit testing

[Chris Yate]

On 1 Nov 2016 01:03, "Carl Sorensen" <address@hidden> wrote:
>
>
>
> The only thing that I can think of for unit testing is to do the
following:
>
> 1) For each file, define a set of regression tests that the file affects.
> The set of regression tests should exercise all known functionality of the
> files.
>
> 2) Get a known good, "gold-standard" definition of each of those
> regression tests.
>
> 3) Assert that when each of the regression tests is run, the output should
> pixel-to-pixel identical to the gold standard.
>
> 4) If the assertion in 3 is met for all reg-tests identified in step 1,
> the file passes.
>
> Note that we have a mechanism similar to this in place right now, with
> make test-baseline and make check.  I suspect it could be cleaned up with
> some work to provide the desired unit test functionality.
>

It's probably pedantry, but "unit tests" by most good definitions ought not
to touch stuff like network, threads, filesystems and databases. If we're
testing the graphic output it's too high level.

As I think the actual drawing stage is after we've positioned all the
objects - a really simple example is an imaginary C++ function to place a
note (function signature is almost certainly wrong):

   int PlaceNote( Staff& staff, const std::string& notevalue)

Unit tests would directly examine the staff object after the function call
for correct note placement, and the return value in exceptional cases.

No mucking about with pdf creation or parsing, so it's about a thousand
times faster (or possibly much more) to run the test suite.

As I said before, I've no idea how to do this directly in Scheme, but I
imagine the Scheme interpreter executes in a C++ process anyway, so it'd
probably be feasible to run the tests through that in some way.

Though this approach also typically relies on the ability to inject mock
objects/services into the subject code to stub out filesystem access etc.
This might be a problem, I don't know the Lilypond code base well enough to
say.

Chris