Re: make doc timings

[David Kastrup]

Han-Wen Nienhuys <address@hidden> writes:

> I tried what happens if one concats all texi/tely files together, and
> runs lp-book on them.
>
> The result is 25 minutes of purely CPU bound grinding (this is with
> Guile 2.2). Then building the remaining docs takes about 15 minutes.
> In this last phase, there is some inefficiency: we process the
> documents per language directory, but for each directory there is a
> bunch of small files, and a humongous notation manual. We can't move
> on to the next directory until the notation manual PDF file finishes.

Where would the point be in moving on to the next directory when
CPU_COUNT processors are already working on the notation manual?

The startup time of .scm files is not a lot of trouble with respect to
the spent user time since LilyPond forks into separate processes _after_
already having started up.  While it starts up under control of
LilyPond-book, it does so with a single CPU: I'll admit that.  But if we
are going to end up paying for CPU time, I prefer that we don't have
independent processes all starting up on their own CPU.  That would take
about the same real time but quite more user time.

> In order to fix this, we would have to reorganize the build system so
> it builds everything out of one directory.

Or basically have a jobserver structure for LilyPond-book.  We don't
need to flatten the directory structure but just the organisation of the
workload.

That will buy us smaller realtime.  It will not buy us significantly
smaller billable user time.

So its main payoff would be if we keep doing the bulk of our
test/integration work on private computers rather than virtualised CPUs.

-- 
David Kastrup
My replies have a tendency to cause friction.  To help mitigating
damage, feel free to forward problematic posts to me adding a subject
like "timeout 1d" (for a suggested timeout of 1 day) or "offensive".