Re: testing out Docker CI scripts?

[David Kastrup]

Han-Wen Nienhuys <address@hidden> writes:

> On Sun, Feb 23, 2020 at 12:16 AM David Kastrup <address@hidden> wrote:
>>
>> Yes.  But if every developer tests on the same platform, we will have
>> to email back and forth with users to figure out what is going on
>> when the stuff does not blow up on our unified platform code.  We
>> have had that situation with floating point on Windows (or rather
>> 32bit platforms generally) just now.  Windows-only problems are
>> really tricky things.  So I am skeptical that a unification of test
>> platforms among developers will make it easier rather than harder to
>> track down problems among us.
>
> So in summary, there is James and you running slightly different
> flavors of Linux, doing testing on pending patches (james) and staging
> => master integrations (you).
>
> This means that errors specific to your system (eg. Pango 1.42) are
> caught late, when the staging => master push happens.

This one was not caught by James because of the autogen.sh problem if I
remember correctly.  But yes, this can happen.  Our systems are not
failsafe, but that's not because we are lacking perfect reproducibility.

> When we have a docker-based testing, everyone can easily these flavors
> of linux, and they can be tested much earlier, for example when the
> contributor prepares a patch. This is also especially important once
> GUILE 2 moves to a supported state, because we'll have 2 pretty
> different platforms.

If GUILE 2+ ever gets to the state of being well-supported to a degree
where we make it the default platform we distribute without misgivings,
Guile 1 will be dead.  We won't test for it, and nobody will use it.

> Moreover, when we discover some other older or newer flavor of Linux
> that we also want to support, we can add a Docker image for that too,
> and widen our coverage of platforms.

10 years ago VM technology existed, and LilyDev existed.  The bulk of
our heavy-hitting system integrators and contributors come from a
Windows background.  I don't think anybody of them uses Windows and/or
VM for his development-based work but rather runs native versions of
Linux for the development in spite of continuing to use Windows for
their personal work with LilyPond.

That's because VMs came at a cost.  Docker images do, too.  Somebody is
going to have to shoulder that cost, either in terms of the lost
performance or in terms of having to shell out money for running the CI.

Again: in a corporate setting, this would be a no-brainer.  But we have
to deal with the reality of LilyPond being a resource-hungry pig in a
setting where we want to rely on free services.

> This doesn't solve the problem of getting reliable windows builds, or
> usable windows bug reports. The issue of windows support is unrelated
> to how well we test on Linux, so I propose we don't discuss it here or
> now.

-- 
David Kastrup
My replies have a tendency to cause friction.  To help mitigating
damage, feel free to forward problematic posts to me adding a subject
like "timeout 1d" (for a suggested timeout of 1 day) or "offensive".