bug#47336: Disarchive as a fallback for downloads

[Timothy Sample]

Hello,

This patch series adds Disarchive assembly (backed by SWH lookup) as a
fallback for downloads.

To try it, make sure you are running the daemon in an environment with
Disarchive available:

    $ ./pre-inst-env guix environment --ad-hoc guile disarchive
    # ./pre-inst-env guix-daemon --build-users-group=guixbuild

Don’t forget to stop your existing Guix Daemon.  :)

You also need to make sure that regular downloads are unavailable.  I do
this by adjusting the “try” loop at the end of “url-fetch” in
“guix/build/download.scm”.  I replace the usual list of URLs with ‘()’:

    (let try ((uri (append uri content-addressed-uris)))
      (match '() ; uri
        ...))

Now you can ask Guix for a recent .tar.gz source package:

    $ ./pre-inst-env guix build --no-substitutes -S python-httpretty

You should see:

    Trying to use Disarchive to assemble 
/gnu/store/kbcnm57y2q1jvhvd8zw1g5vdiwlv19y9-httpretty-1.0.5.tar.gz
    Assembling the directory httpretty-1.0.5
    Downloading from Software Heritage...
    7903d608efc89c14afb4d692a3721156e31a43e2/
    7903d608efc89c14afb4d692a3721156e31a43e2/httpretty-1.0.5/
    7903d608efc89c14afb4d692a3721156e31a43e2/httpretty-1.0.5/COPYING
    [...]
    Checking httpretty-1.0.5 digest... ok
    Assembling the tarball httpretty-1.0.5.tar
    Checking httpretty-1.0.5.tar digest... ok
    Assembling the Gzip file httpretty-1.0.5.tar.gz
    Checking httpretty-1.0.5.tar.gz digest... ok
    Copying result to 
/gnu/store/kbcnm57y2q1jvhvd8zw1g5vdiwlv19y9-httpretty-1.0.5.tar.gz
    successfully built 
/gnu/store/k0b3c7kgzyn1nlyhx192pcbcgbfnhnwa-httpretty-1.0.5.tar.gz.drv

There’s lots to talk about though....

First, it looks up the metadata on my server.  This is fine for a demo,
but not what we want forever.  The patch series supports adding several
mirrors for looking up the metadata.  In the past, we talked about
putting everything on one or a few of the big Git hosting platforms like
GitHub or Gitlab.  That way, it would be easily picked up by SWH and
archived “forever”.  Right now, I have Cuirass set up to build the
metadata, and a little script that moves it from the build server to my
Web server.  It would be simple enough to adjust that script to push it
to a remote Git repo.  (Of course, the next step is to move this setup
to Guix infrastructure.)  Thoughts?

On the code level, there were two things I couldn’t figure out for
myself.

I made the mirror list just simple strings.  AIUI, the client and the
daemon have to agree about the format of the mirror list.  Given that
running old daemons is common, changing the format is difficult.  Is it
worth it to copy the more flexible interface used by the content
addressed mirrors?  If yes, do I have to do the same ‘module-autoload!’
dance to use ‘bytevector->base16-string’?  :)  (I probably would have
just copied it, but that part confused me a bit.)

I imported some modules from “guix/build/download.scm” (well, just
“base16” and “swh”).  It feels weird to use a bunch of host-side modules
from what’s nominally a “guix/build” module.  This is okay because
“guix/build/download.scm” is not /really/ build-side code.  It’s more
like daemon (-ish) code that just happens to live in “guix/build”, which
is why importing host-side modules is OK... right?

Hopefully everything else is more-or-less fine.  :)


-- Tim