Re: make doc timings

[David Kastrup]

David Kastrup <address@hidden> writes:

> Han-Wen Nienhuys <address@hidden> writes:
>
>> I tried what happens if one concats all texi/tely files together, and
>> runs lp-book on them.
>>
>> The result is 25 minutes of purely CPU bound grinding (this is with
>> Guile 2.2). Then building the remaining docs takes about 15 minutes.
>> In this last phase, there is some inefficiency: we process the
>> documents per language directory, but for each directory there is a
>> bunch of small files, and a humongous notation manual. We can't move
>> on to the next directory until the notation manual PDF file finishes.
>
> Where would the point be in moving on to the next directory when
> CPU_COUNT processors are already working on the notation manual?
>
> The startup time of .scm files is not a lot of trouble with respect to
> the spent user time since LilyPond forks into separate processes _after_
> already having started up.  While it starts up under control of
> LilyPond-book, it does so with a single CPU: I'll admit that.  But if we
> are going to end up paying for CPU time, I prefer that we don't have
> independent processes all starting up on their own CPU.  That would take
> about the same real time but quite more user time.
>
>> In order to fix this, we would have to reorganize the build system so
>> it builds everything out of one directory.
>
> Or basically have a jobserver structure for LilyPond-book.  We don't
> need to flatten the directory structure but just the organisation of the
> workload.
>
> That will buy us smaller realtime.  It will not buy us significantly
> smaller billable user time.
>
> So its main payoff would be if we keep doing the bulk of our
> test/integration work on private computers rather than virtualised CPUs.

Actually, keeping a pre-initialized process of LilyPond around which
forks off work processes on-demand that can immediately do actual work
would likely make Johannes Feulner (Scorio) happy.  That would be more a
LilyPond job server than a LilyPond-book job server, but it's not like
you couldn't funnel one through the other.

-- 
David Kastrup