[bug#50620] [PATCH 1/2] guix: packages: Document 'computed-origin-method'.

[Liliana Marie Prikler]

Hi,

Am Mittwoch, den 29.09.2021, 19:48 +0200 schrieb zimoun:
> Hi,
> 
> On Wed, 29 Sept 2021 at 16:36, Liliana Marie Prikler
> <liliana.prikler@gmail.com> wrote:
> 
> > > Perhaps I am wrong about option (2) -- my claim is that
> > > computed-origin-method is *always* used with a promise so it is
> > > for sure an half-baked guess but enough; and it avoids to hard
> > > code the modules from where the packages come from.  Therefore,
> > > option (2) does not improve, IMHO.
> > 
> > The probability of having a promise when using computed-origin-
> > method is 100%.  What is the probability of having computed-origin-
> > method when you see a promise?  The answer is: we don't know.  We
> > can see from the
> 
> You mean, what is the probability of having a computed-origin-method
> when the origin-uri is a promise?  We do not know, but pragmatically,
> for now 100%. :-)
> 
> Option (2) is:
> 
>  ___ (or (eq? method (@@ (gnu packages gnuzilla) computed-origin-
> method))
>  _______ (eq? method (@@ (gnu packages linux) computed-origin-
> method)))
> 
> then I ask you similarly: what is the probability of having packages
> using computed-origin-method in these 2 modules only?  We do not
> know,
> but pragmatically, for now 100%. :-)
> 
> The hypothetical probabilities to evaluate are:
> 
>  - what would be the probability that a new package having a promise
> as origin-uri is not indeed a package with a computed-origin-method?
> vs
>  - what would be the probability that a new package using
> computed-origin-method is not part of either (gnu packages gnuzilla)
> or (gnu packages linux)?
> 
> Anyway!  Well, I am not convinced that it is worth to tackle these
> hypothetical issues. :-)
I meant that only in reference to a comparison of your original patch
in #50515 vs. option (2).  Option (2) is conservative, it only detects
computed origin which it knows how to handle.  Your original patch is
more liberal in that it detects anything that has a promise as uri as a
computed origin, which might however not have the same semantics as
those two copies of the same code.  Obviously, both (3) and (3a) don't
have this problem, but they're also conservative in that e.g. I could
roll my own channel with the exact same computed-origin-method
copypasta'd once more and it wouldn't be detected, though that's
probably off-topic.[1]

> That's why the option (3):
> 
>    (eq? method (@@ (guix packages) computed-origin-method))
> 
> which means refactorize*.  It is somehow the two worlds: check i.e.,
> safer, no modules hard-coded and keep private the time to have The
> Right Plan for this computed-origin-method.
I was a little confused when I read factorize from Ludo or refactorize
from you.  I know this technique under the name "refactoring".

> *refactorize: I think (guix packages) is better because it defines
> '<origin>' and other tooling friends.  Because
> 'computed-origin-method' is somehow a temporary tool about origin,
> i.e., a mechanism to define packages, it makes sense to me to put it
> there.  However, (gnu packages) is about tools to deal with packages,
> not to define them; although one could argue that 'search-patch' is
> there is used to define package. For what my rationale behind the
> choice of (guix packages) is worth.  And indeed, I have only
> half-mentioned this rationale.
To that I would counter, that (guix packages) only defines package and
origin records and how to compile them to bags and derivations.  All
the methods through which those fields are filled are defined outside,
be it the occasional local-file used in "Build it with Guix"-style
recipes or the closer to our methods svn-fetch and git-fetch.

Were it not for the fact that it uses procedures defined in (guix
packages), you might have a better time reasoning that this should be a
hidden part of (guix gexp), but since it does, it would introduce a
cycle if you did.  While (gnu packages) does offer "tools to deal with
packages" as you put it, I'd argue the way it does is in establishing a
namespace for them (fold-packages etc.) and that this namespace (the
"GNU namespace") is the best location for computed-origin-method.

The reason we use computed-origin-method at all, is because as GNU Guix
we take a hard stance on the FSDG and do our damndest not to lead users
to nonfree software.  This includes producing clean --source tarballs. 
Were it not for that, we could easily deblob at build time, and this is
an important observation to make, because it means that projects that
don't need this level of control can "safely" defer it the way we do
currently for "unpack more after unpack".[2]  In other words, I
strongly believe that users of compute-origin-method do the hard work
of computing origins to follow GNU standards and will thereby have no
issue referencing the GNU namespace to get to it.

> As I said, generating this sources.json file by the website is
> clunky.
> Somehow, it is a quick hack to have something up waiting The Right
> Way; the long-term generations should be done through the Data
> Service, as it had been already discussed but not yet implemented.
> Help welcome. :-)
I have no opinion on that as I don't work on the Data Service.

> > > About update guix package [2/2], it has to be done, IIUC.  The
> > > file
> > > sources.json contains what the package guix provides, not what
> > > the
> > > current Guix has.  The website -- built using the Guile tool
> > > haunt --
> > > uses Guix as a Guile library.  Maybe I miss something.
> > 
> > What I was trying to say was that you wouldn't need to rebuild the
> > guix package before applying the 50515 changes, which this one
> > seems to require.  Again, I'm not 100% sure that's the case, but
> > IIUC (gnu packages) is its own realm in this regard.
> 
> Hum, maybe there is a misunderstanding here.  On one hand 50620
> applies to the guix.git repo and on the other hand 50515 applies to
> guix-artwork.git repo.
Oh, I somehow missed that completely.  Oops.

> To have the sources of linux-libre and icecat reported in
> sources.json and thus to get a chance to have them archived by SWH,
> we need:
> 
>  a- if computed-origin-method is factorized then update the guix
> package (Guix as a library) else do nothing; patch applied to
> guix.git
>  b- tweak how sources.json is built; patch applied to guix-
> artwork.git
> 
> Well, the aim of 50620 is to deal with a) whereas the aim of 50515 is
> to deal with b).  Note that 50515 requires a v2 if
> computed-origin-method is factorized.
> 
> Maybe I miss something.  From my understanding, all the modules are
> part of Guix as a library.  Therefore, it does not depends on where
> we refactorize.
No, no, I was wrongly under the impression that this sources.json would
be generated by Guix itself what with all the other SWH interaction we
already have.  Again, a mistake on my part.

> To be honest, I thought that this tiny improvement of the SWH
> coverage would have been much more easier and that that trivial task
> would not have taken more than 15 days with lengthy discussions. :-)
To be honest, part of the lengthy discussion was me being confused
about your intent – in multiple ways.  If you wanted a "quick and dirty
fix" you could have went with checking those two modules explicitly on
the guix-artwork side and it would have had a fairly small impact. 
Reading this patch first and the discussion second, I had assumed your
intent was rather to formalize a method that had hitherto only been
used informally and the move to the guix namespace amplifies that imo.

All the best,
Liliana

[1] The only solution is of course to compare via equal? instead of
eq?, which if implemented for functions would be Turing-hard :)
[2] Of course, this assumes that other projects don't want to actually
unpack dependencies when using --source, but I think it's likelier they
will just use recursive submodule checkout.