Re: Introducing Guix to HPC at my institution

[Sébastien Lerique]

Hello!

On 01 Apr 2021 at 17:35, Ludovic Courtès 
<ludovic.courtes@inria.fr> wrote:

> > Case 1: with substitutes, on a local (non-NFS) folder
> > -----------------------------------------------------
> > [snip]
>
> Can you run guix-daemon like so?
>
>   LC_ALL=en_US.utf8 guix-daemon --disable-chroot &

Then:

 $ guix install hello --no-offload
 [... dowloading ...]
 13.3 MB will be downloaded
 substitution of 
 /gnu/store/395pvii4bcjqxvdv7h0drq10lxi01sv1-glibc-utf8-locales-2.31 
 failed
 guix install: error: some substitutes for the outputs of 
 derivation 
 `/gnu/store/b2jkmg71m0dpf3i6hvskb32ra48lls28-glibc-utf8-locales-2.31.drv' 
 failed (usually happens due to networking issues); try 
 `--fallback' to build derivation from source

Restarting from scratch and using `--fallback` then leads to the 
bzip2-mesboot failure of Case 2.

If, instead of resarting from scratch, I just run again `guix 
install hello --no-offload`, I then get:

 guix install: error: got unexpected path 
 `/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash: 
 warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)' 
 from substituter

And indeed once again lib/locale is a dead symlink, as the 
glibc-utf8-locales packaged in my tar.gz has been removed from the 
store. Why is that? I had a look through the same commands with 
--debug=4 or 5, but to no insight (aside from seeing locale 
derivations being deleted at some stage). I posted the end of the 
log produced by `guix install hello --no-offload --debug=4`, 
starting with the failed substitution, here: 
https://paste.debian.net/1191991/ .

> The “unexpected substitute message” thing is a bug: it turns out 
> that
> stderr of ‘guix substitute’ is consumed directly by the daemon 
> at this
> point, which it shouldn’t (similar to the issue fixed by
> ee3226e9d54891c7e696912245e4904435be191c).

Ok, I'll file a bug for that then :)

> > Case 2: from source (no substitutes), on a local (non-NFS) 
> > folder
> > -----------------------------------------------------------------
> > [snip]
>
> The problem is that ‘--disable-chroot’ is a bit of the wild 
> west: build
> processes can access the whole file system and what you do as a 
> user can
> interfere with them.
>
> It could be that the bzip2 build failure above is just that: the 
> build
> process picks something from /usr/lib or /usr/bin, and that 
> breaks
> everything.
>
> I think ‘--disable-chroot’ is OK if you’re going to use 
> substitutes for
> almost everything.  Otherwise, it’s not good.  Your use case 
> calls for
> built-in support; that way, the daemon could take still 
> advantage of
> user namespaces to set up a chroot and all.

I see. By the way, starting guix-daemon without `--disable-chroot` 
(and without substitutes) leads to:

 guix build: error: cannot change ownership of 
 ‘/gnu/store/0dn61y4n8ig333b23hmc80hvlcy8gdli-guile-bootstrap-2.0.drv.chroot’: 
 Invalid argument

so --disable-chroot is indeed still necessary! I am interested in 
working on this built-in support once I get the substitutes case 
working (although I will probably come ask for guidance for that) 
:)

Thanks!
Sébastien