Re: Guix v1.1.0 (~2020) and Software Heritage & friends

[Ludovic Courtès]

Hello!

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> Here I would like to report my attempt to simulate a “network collapse”
> – as discussed with Mathieu at Guix Days. :-)

Great experiment!

> Using this,
>
>     archive.softwareheritage.org
>     disarchive.guix.gnu.org
>     git.savannah.gnu.org
>
> is not enough.  It fails for many.  For instance,
>
>     ed-1.15
>     gcc-core-2.95.3
>     ghostscript-9.27
>     guile-2.2.6
>     linux-libre-4.19.56-gnu
>     linux-libre-headers-stripped-4.14.67-i686-linux
>     mes-0.19
>     mes-minimal-stripped-0.19-i686-linux
>
> do not seem backed up with SWH + Disarchive.

It’d be great to identify the root cause for each of them, with a script
to do that.  But note that 1.1.0 did *not* support Disarchive; it can
only look up SWH or one of the content-addressed mirrors.

That said, it’s true that disarchive.guix lacks coverage for some of
these.  For ed, that’s because it’s lzip:

--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=v1.1.0 -- build -S ed
guile: warning: failed to install locale
0.1 MB will be downloaded:
   /gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz
substituting /gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz...
downloading from 
https://ci.guix.gnu.org/nar/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz...
 ed-1.15.tar.lz  66KiB                                  1.3MiB/s 00:00 
[##################] 100.0%
/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz

$ wget  https://disarchive.guix.gnu.org/sha256/$(guix hash -f hex $(guix 
time-machine --commit=v1.1.0 -- build -S ed))
guile: warning: failed to install locale
--2023-03-06 21:27:53--  
https://disarchive.guix.gnu.org/sha256/ad4489c0ad7a108c514262da28e6c2a426946fb408a3977ef1ed34308bdfd174
Resolving disarchive.guix.gnu.org (disarchive.guix.gnu.org)... 141.80.181.40
Connecting to disarchive.guix.gnu.org 
(disarchive.guix.gnu.org)|141.80.181.40|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-06 21:27:53 ERROR 404: Not Found.
--8<---------------cut here---------------end--------------->8---

For Guile it’s probably because it’s xz and predates xz support in
Disarchive:

--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=v1.1.0 -- build -Sd guile@2.2.6
guile: warning: failed to install locale
/gnu/store/19mwlfc1092vipdka1y0l6v92ahkhl52-guile-2.2.6.tar.xz.drv
$ guix gc --references 
/gnu/store/19mwlfc1092vipdka1y0l6v92ahkhl52-guile-2.2.6.tar.xz.drv |grep 
tar.xz.drv
/gnu/store/pf9acfsb3zkmvfqcny4b8hrjif9a1kb8-guile-2.2.6.tar.xz.drv
$ wget -O-  https://disarchive.guix.gnu.org/sha256/$(guix hash -f hex $(guix 
build /gnu/store/pf9acfsb3zkmvfqcny4b8hrjif9a1kb8-guile-2.2.6.tar.xz.drv))
10.9 MB will be downloaded:
  /gnu/store/zfp7d4wr5hbl5lrnzs8c15bsc3ygq74g-guile-2.2.6.tar.xz
substituting /gnu/store/zfp7d4wr5hbl5lrnzs8c15bsc3ygq74g-guile-2.2.6.tar.xz...
downloading from 
https://ci.guix.gnu.org/nar/zfp7d4wr5hbl5lrnzs8c15bsc3ygq74g-guile-2.2.6.tar.xz 
...
 guile-2.2.6.tar.xz  10.4MiB         24.3MiB/s 00:00 ▕██████████████████▏ 100.0%

--2023-03-06 21:30:41--  
https://disarchive.guix.gnu.org/sha256/b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988
Resolving disarchive.guix.gnu.org (disarchive.guix.gnu.org)... 141.80.181.40
Connecting to disarchive.guix.gnu.org 
(disarchive.guix.gnu.org)|141.80.181.40|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-06 21:30:41 ERROR 404: Not Found.

$ wget -qO-  https://disarchive.ngyro.com/sha256/$(guix hash -f hex $(guix 
build /gnu/store/pf9acfsb3zkmvfqcny4b8hrjif9a1kb8-guile-2.2.6.tar.xz.drv)) 
|head -4
(disarchive
  (version 0)
  (xz-file
    (name "guile-2.2.6.tar.xz")
--8<---------------cut here---------------end--------------->8---

> Well, basically after these, I stop because it’s boring and I add
> ftp.gnu.org to the list.  It still fails,
>
>     nyacc-0.86.0
>     static-binaries
>
> First lesson: we have holes in sources.json and maybe Disarchive too; I
> will give a look.

Some are known issues: lack of lzip support in Disarchive, lack of
SVN/SWH integration, missing entries in the Disarchive database for old
packages, etc.

One thing that’s easily done is retroactively populating the Disarchive
database.  It’s a job that Timothy did on their side:
<https://disarchive.ngyro.com> has more “old” entries, which we should
copy at disarchive.guix.

We should also reuse the tools that Timothy wrote:
<https://git.ngyro.com/>.  In particular, we should use project
infrastructure to compute a new Preservation of Guix report, and ideally
automate the whole process.

Maybe we should write down a road map and “recruit” on these items?

Thanks,
Ludo’.