Re: Unit testing

[Carl Sorensen]

On 10/29/16 6:57 PM, "Chris Yate" <address@hidden> wrote:
>I think the problem is, how would you define an assertion, and what are
>you intending to test (i.e. what's the Subject Under Test?). I think
>something like Lilypond might require some quite elaborate test fixtures
>/ fakes.
>
>There are probably C++ functions you could test with a C++ assert based
>test framework - Catch for example - or something simpler.  Obviously
>anything doing a bit of maths is easy to test; the interesting functions
>tend to manipulate "Grob" objects; would the
> tests be examining their properties after the function call?
>
>But what about Scheme code?...


The only thing that I can think of for unit testing is to do the following:

1) For each file, define a set of regression tests that the file affects.
The set of regression tests should exercise all known functionality of the
files.

2) Get a known good, "gold-standard" definition of each of those
regression tests.

3) Assert that when each of the regression tests is run, the output should
pixel-to-pixel identical to the gold standard.

4) If the assertion in 3 is met for all reg-tests identified in step 1,
the file passes.

Note that we have a mechanism similar to this in place right now, with
make test-baseline and make check.  I suspect it could be cleaned up with
some work to provide the desired unit test functionality.

HTH,

Carl

P.S. We could do the same for midi output, as well