Re: Turning off tests leads to a different store item

[Suhail]

Simon Tournier <zimon.toutoune@gmail.com> writes:

> On Thu, 02 Nov 2023 at 15:25, Suhail <suhail@bayesians.ca> wrote:
>
>> Yes, with the test derivation being something like a "fixed-output
>> derivation". [[info:guix#Derivations][From the manual]]:
>
> No, it cannot be a “fixed-output” derivation…
>
> …because we cannot know in advance the expected content hash of the
> tests output.

Yes, hence the "something like". It's similar, but also different in
important ways.

> And from my understanding, one solution would be to have something as
> below.  One “object” for building and another “object” for testing.
>
>     ( Here, I am using the same object namely <package> for the both;
> just to make concrete the discussion. :-) )
>
> The point is: we have two derivations; one for the build (without tests)
> and another for the tests (without build).
>
> ...
>
> Somehow, we could have a “build” build-system and a “test” build-system.
> And the “build object” would be an inputs of the “test object“.

You are trying to reconcile our discussion with what you know about Guix
internals which is both useful and necessary (eventually). However,
since I have less knowledge of said internals at present, I am not able
to meaningfully contribute to what the implementation may look like
yet. What I can do, instead, is to articulate some
invariances/properties that I believe are both desirable and reasonable
(without considering how feasible or not they may be).

First some notation. Let's say the "test metadata" captures the
information of interest: were the tests run, and if so what was their
outcome. A very simple (but sufficient for our purposes) datatype would
be the union of null, #t and #f (where the test outcome is a
boolean). It's possible that in practice, "test metadata" may have
additional information (for instance a reference to the "build output"
in the store), but we'll ignore that for now. I'll use "test output" to
mean the combination of "test metadata" and "build output" where "build
output" is also the input to the "thing that generates the 'test
metadata'". So we have the property that the "test output" extends the
"build output" by providing some companion information (i.e., the "test
metadata").

The invariants of interest are about what things are considered
equivalent. To my understanding this is captured in the Guix notion of
what is and isn't considered a Substitute.

1. It should be possible to discard test metadata: We should be able to
   use "test output" as a substitute for "build output". I.e., a
   derivation that doesn't demand that we run the tests ought not to
   care whether or not we did.

2. "build output" can be used as a substitute for "test output" with
   null "test metadata".

3. It shouldn't be possible to vacuously manufacture test outcomes:
   "build output" cannot be used as a substitute for "test output" with
   either #t or #f "test metadata".

If our hypothetical build system (say, ds-build-system) were to admit
the above invariances, do you foresee some complications that may arise
that need to be addressed?

> Well, somehow perhaps some revamp of the <package> record.

Perhaps, but I am not quite there yet to consider how it might be
implemented, because what "it" is is still not sufficiently clear to me.

-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an
intention to enter into an agreement, has no operative effect until a
definitive agreement is signed in writing by both parties, and that no
party should act in reliance on the email or any representations of the
sender until a definitive agreement is signed in writing by both
parties.

This email may contain information that is privileged, confidential
and/or exempt from disclosure.  No waiver whatsoever is intended by
sending this e-mail which is intended only for the named recipient(s).
Unauthorized use, dissemination or copying is prohibited.  If you
receive this email in error, please notify the sender and destroy all
copies of this email.