Re: create code tests

[Carl Sorensen]

Dan,

This is a great explanation.

Can you add this to the CG?  Anywhere that it seems to make sense.

Thanks,

Carl


On 11/29/19, 3:18 PM, "lilypond-devel on behalf of Dan Eble" 
<lilypond-devel-bounces+c_sorensen=address@hidden on behalf of address@hidden> 
wrote:

    On Nov 29, 2019, at 16:20, <address@hidden> <address@hidden> wrote:
    >  
    > I am familiar with the use of makefile. So if you can give me directions 
with some placeholder commands, or comment at the right place in the makefile 
(s) 
    
    Well, start at the "test" target in the top-level GNUmakefile.in.  It runs 
make in a number of subdirectories.  I would start by adding something like 
this there:
    
        $(MAKE) -C input/regression/ly2musicxml out=test local-test
    
    You're going to need to create that ly2musicxml directory (or another name, 
if you have a better idea) and put a GNUmakefile there.  I would copy a 
GNUmakefile from a nearby directory and figure out what needs to be changed and 
what unnecessary things can be removed.
    
    The target "local-test" should run your scripts.  Ideally, this should 
involve rules that regenerate the files only when they are out of date.  You've 
said that expectations can be tested automatically, so I think that, for now, 
you should do that immediately when the output files are generated, and cause 
make to fail if the expectations are not met.  (Later, we might want to allow 
all tests to run and summarize the failures at the end, but this seems 
simpler.)  Arrange the makefile so that if make is run again immediately 
without fixing anything, it will fail again in the same way; test this.
    
    Once a test has passed, no subsequent "make test" should rerun it until 
either its prerequisites are modified or "make test-clean" has been run to 
remove the test results.
    
    The "test-clean" target is defined in the top-level GNUmakefile.in.  It 
runs "make clean" in input/regression, and input/regression/GNUmakefile defines 
SUBDIRS.  If you add your new subdirectory to SUBDIRS there and have placed all 
your output files in $(outdir), the existing infrastructure should then remove 
them automatically.
    — 
    Dan