Re: RFC: docker for CI

[Dan Eble]

On Feb 7, 2020, at 15:21, Han-Wen Nienhuys <address@hidden> wrote:
>>>   * use a headless browser to take a image snapshot of the top of regtest
>>>  result page.
>>> 
>> Sounds convoluted.  Why not attach the difference images directly?
>  
> Those are potentially 1372 images to attach if you made a change with global 
> impact.

Why not attach the N images with the greatest differences directly?

More generally, I'd want a digest of the results (not all of which are visual) 
that is as useful as possible for the size we are willing to post to the 
review.  We control output-distance.py, so we could generate something new that 
fits this case.

>> Are full logs and test results retained, or does a developer need to 
>> reproduce the test locally to get them?
> 
> You'd retain the full logs and results as part of the docker image. Currently 
> my checkout is about 1.8G of data, and a lilypond docker image itself would 
> be close  to that too.

This approach is new to me.  I'm used to CI systems that are configured to 
archive particular files from the workspace (e.g., the regtest output tree, the 
final docs) and full build log for a limited time (days to weeks).  I think it 
balances the types of things you can investigate without reproducing the build 
yourself against retaining a huge amount of data.

Can you expand on the purpose of saving the full Docker image--which is not 
just the LilyPond workspace but the OS too, correct?  Are you thinking that 
someone would prefer to download it and debug in a container rather than 
reproduce the build in their usual development environment?
— 
Dan