Re: testing out Docker CI scripts?

[Han-Wen Nienhuys]

On Sun, Feb 23, 2020 at 11:44 AM Han-Wen Nienhuys <address@hidden> wrote:
> and initially it is working as expected (4 cpus.)
>
> 11:12:40 3.23  Making Documentation/out-www/music-glossary.pdf < texi
> 11:12:41 3.30  Making Documentation/out-www/notation.pdf < texi
> 11:12:47 3.43  Making Documentation/out-www/snippets.pdf < texi
>
> but later in the process, I get
>
> 11:17:54 2.07  Making Documentation/de/out-www/notation.pdf < texi
> ..
> 11:18:46 1.85  Making Documentation/es/out-www/notation.texi < tely
> 11:19:25 2.51  Making Documentation/es/out-www/learning.texi < tely
>
> ie. the build process for other languages is leaving ~2 CPUs idle, for
> about 15 minutes. This means it should be doable to shave off 7.5
> minutes off the make doc build.

I should have taken more detailed notes on the first runtime.

I tried some more work with parallelization, which yields an overall
runtime of 36m, which shows that I haven't been able to disprove
David. It's a ~5 min improvement. Probably we should look more deeply
into making lp-book not be exclusive.

15:17:11 2.73  Making Documentation/de/out-www/learning.pdf < texi
15:17:17 2.91  Making Documentation/de/out-www/notation.pdf < texi
..
15:18:13 2.26  Making Documentation/es/out-www/learning.texi < tely
15:18:14 2.26  Making Documentation/es/out-www/notation.texi < tely

real 36m12.256s
user 79m24.432s
sys 10m36.464s

For posterity, during the lp-book processing for doc/web, lilypond is
busy at about 40%. The rest presumably goes to ghostscript and image
scaling.

Compared to just the regtests, that is 2.5x more files and 2.5x more
expensive processing = 6x more expensive overall. Since the regtest
completes in about 3 minutes, the equivalent for "make doc" would be
18 mins. So we still are missing a factor two. It looks like the poor
parallelism of the build process can explain that remaining factor.

-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen