bug#46402: Cuirass rebuilds the same packae multiple times

[Ludovic Courtès]

Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

>> here is an example: https://ci.guix.gnu.org/build/287478/details and
>> https://ci.guix.gnu.org/build/287476/details are being built at the
>> same time, and the logs currently show they are both running the test
>> phase of the php package.
>
> Thanks for the report. This problem has been briefly discussed
> yesterday. It has been introduced by the new remote building mechanism
> in Cuirass. Hydra solves it by breaking each build into buildsteps
> corresponding to the derivation inputs.
>
> The buildsteps are then submitted to the workers in a logical order. I
> proposed to introduce a similar mechanism in Cuirass but Ludo expressed
> doubts. Ludo do you think this problem could be solved otherwise?

I’m not sure exactly but I can share my feelings.  :-)

Seems to me that ‘BuildSteps’ is an orthogonal concern that has little
to do with Cuirass’ job and with its data model.  In Hydra I saw that as
a (necessary) kludge.

I like the way the Coordinator does it, and AIUI it’s pretty much the
same as what the daemon is doing: submit build requests in topological
order, such that when a derivation build is submitted, its prerequisites
are known to be built already.

I suppose what makes it more difficult here is that we have this extra
“job” abstraction on top of derivations; everything in Cuirass revolves
around jobs, which leads to this impedance mismatch.

If Cuirass would instead delegate derivation build requests to a
Coordinator/daemon-like thing, it wouldn’t have to worry about those
details.  That would better separate concerns.

This is quite a hand-wavy reply but I hope it’s useful!

Thanks,
Ludo’.