bug#49439: grafts cause “guix environment” to get killed with OOM

[Ludovic Courtès]

Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

> Thinking about it, the grafts code depends on what’s in the store: when
> nothing is in the store, it bounces to the “build handler”, which
> accumulates the list of missing store items, until it starts building
> them.

So I can reproduce the problem Ricardo and you initially reported by
running:

  ./pre-inst-env guix environment pigx-scrnaseq --search-paths

after removing some of the ungrafted store items with:

  guix gc -D $(guix build r-rlang --no-grafts) \
    $(guix gc --references $(guix build pigx-scrnaseq --no-grafts))

The seemingly endless CPU usage and unbound memory use comes from the
‘build-accumulator’ build handler.  Here, the graph of ‘pigx-scrnaseq’
has many nodes, and many of them depend on, say, ‘r-rlang’.  Thus, when
‘r-rlang’ is not in the store, the grafting code keeps asking for it by
calling ‘build-derivations’, which aborts to the build handler; the
build handler saves the .drv file name and the continuation and keeps
going.  But since the next package also depends on ‘r-langr’, we abort
again to the build handler, and so on.

The end result is a very long list of <unresolved> nodes, probably of
this order in this case:

  $ guix graph -t reverse-package r-rlang |grep 'label = "'|wc -l
  594

Presumably, the captured continuations occupy quite a bit of memory,
hence the quick growth.

I suppose one solution is to fire suspended builds when the build
handler sees a build request for a given derivation for the second time.
It needs more thought and testing…

Ludo’.

PS: Did you know ‘pigx-scrnaseq’ has twice as many nodes as
    ‘libreoffice’?

      $ guix graph -t bag pigx-scrnaseq |grep 'label = "'|wc -l
      1359
      $ guix graph -t bag libreoffice |grep 'label = "'|wc -l
      699

    That makes it a great example to study and fix scalability issues!