Re: Add a cooperative FS lock to lilypond-book. (issue 555360043 by address@hidden)

[dak]

On 2020/02/26 08:28:33, hahnjo wrote:
> On 2020/02/26 08:19:39, hahnjo wrote:

> > > On a philosophical level, it is a lilypond-book implementation
detail
> > > that it can't deal with concurrent invocation, so the remediation
for
> > > this problem should be in lilypond-book too.
> > 
> > Let me disagree: It's an implementation detail of make that it runs
things in
> > parallel. IMHO a build system should ensure that the result of
running with
> > multiple jobs is the same as a sequential run.
> 
> That said: I'm also fine if some other developer accepts this patch.
See my
> timing data above to get to your own conclusion. After all, my opinion
is just
> one of a larger range.

My take on this is that this "implementation detail" of parallel
invocation resulting in awkward breakage is something that warrants
fixing irrespective of our build system.  All that the UG states here is

‘--lily-output-dir=DIR’
     Write lily-XXX files to directory DIR, link into ‘--output’
     directory.  Use this option to save building time for documents in
     different directories which share a lot of identical snippets.

It doesn't state at all what happens in cases of contentions.  Fixing
contentions with a lock is a brute-force solution just not allowing for
parallelism, but it is a solution to the contention problem.

It is not a solution to lilypond-book starting more jobs than Make knows
about.  Or to all but one lilypond-book invocation not doing any
progress and blocking Make which could instead start other actual
single-process tasks.  So I see this patch and its approach as an
improvement to lilypond-book.  I don't see that it solves the parallel
build carnage: it just scales down the impact from having to choose
between complete serialization and database failure.

https://codereview.appspot.com/555360043/