Re: Moving away from make

[Han-Wen Nienhuys]

On Sun, Sep 25, 2011 at 4:39 AM, Graham Percival
<address@hidden> wrote:
> Yes, various expert FLOSS members (such as Reinhold, Carl, and
> IIRC yourself) have stepped forward to fix a few things in the
> builds -- but the only people who are "working" on the build
> system "full time" are windows users.
> (that said, Phil recently bought a fast computer so that he can
> make builds faster and thus experiment more easily)
>
>
> Don't tell me that experienced hackers will take care of the build
> system.  They aren't.

The real problem is that there are no build systems that do not suck:

* you have to discouple build rule organization from directory
hierarchy.  To keep our sanity, we organize around per-directory
recursive makes. This means that lily/out/lilypond can not directly
depend on flower/path.cc; instead we have to run a recursive make just
to figure it hasn't changed.

* it has to run as a server, and keep the tree state in memory.
Otherwise, incremental builds will be dog-slow.  Scons got this wong.

* it has to be programmable in a sane language (make fails this badly)

Make does suck, but at least, it is simple, universal and
well-understood. This is why we should stick to it until something
significantly better comes along. If we're sticking to make, I doubt
that we will end up with something better than what we currently have.

There is some argument to use makefile generators instead of our own
system, eg. CMake has some promise, but most of the lilypond build
involves custom tools, so we'd need to write a lot of cmake templates
from scratch. It's not clear to me this will be much better than the
make templates we use right now.

The only system I know that does not suck is not open source, unfortunately:
http://google-engtools.blogspot.com/2011/08/build-in-cloud-how-build-system-works.html.
Sometimes my hands itch to rewrite and open source a functional subset
of it, but there is only have so much time in a day.

Regarding parallelism and lilypond-book: AFAICS - running parallel
builds should work correctly right now, although the combination of
make -jX and lilypond -djob-count=X is not pretty.  It would be an
interesting exercise to see if lilypond-book coud use a recursive make
invocation to schedule document sub-jobs.

--
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen