bug#59365: make-dynamic-linker-cache OOMs for LLVM 15 on i686-linux

[Ludovic Courtès]

Hi,

(Cc: Maxim and Greg for LLVM packaging questions below.)

Marius Bakke <marius@gnu.org> skribis:

> LLVM 15.0.4 fails on i686-linux:
>
>   https://ci.guix.gnu.org/build/1702995/details
>
> Because the 'make-dynamic-linker-cache' phase runs out of memory:
>
>   starting phase `make-dynamic-linker-cache'
>   GC Warning: Repeated allocation of very large block (appr. size 268439552):
>       May lead to memory leak and poor performance
>   GC Warning: Repeated allocation of very large block (appr. size 134221824):
>       May lead to memory leak and poor performance
>   GC Warning: Repeated allocation of very large block (appr. size 268439552):
>       May lead to memory leak and poor performance
>   GC Warning: Failed to expand heap by 285216768 bytes
>   GC Warning: Failed to expand heap by 268439552 bytes
>   GC Warning: Out of Memory! Heap size: 3620 MiB. Returning NULL!
>   Warning: Unwind-only out of memory exception; skipping pre-unwind handler.
>   Warning: Unwind-only out of memory exception; skipping pre-unwind handler.
>   Warning: Unwind-only out of memory exception; skipping pre-unwind handler.
>
> (excerpt from https://ci.guix.gnu.org/build/1702995/log/raw)
>
> Not sure why this phase uses so much memory.  Ideas?

Yes: the gremlin.scm code uses ‘file-dynamic-info’, which loads the
whole file in memory.  Ridiculous.

We should instead mmap it (but there are no ‘mmap’ bindings in Guile,
yet) or arrange to load just the relevant parts (we’ll have to check but
maybe ‘file-dynamic-info’ can find everything it needs at the beginning
of a file, the PT_DYNAMIC segment.)

For example, with the patch below, things still appear to be fine with
LLVM:

--8<---------------cut here---------------start------------->8---
scheme@(guix build gremlin)> (file-dynamic-info 
"/gnu/store/mj14k58lfc88jhcn6va0s2fpwkv3s35c-llvm-13.0.1/lib/libLLVMScalarOpts.so")
$11 = #<<elf-dynamic-info> soname: "libLLVMScalarOpts.so.13" needed: 
("libLLVMAggressiveInstCombine.so.13" "libLLVMInstCombine.so.13" 
"libLLVMTransformUtils.so.13" "libLLVMAnalysis.so.13" "libLLVMCore.so.13" 
"libLLVMSupport.so.13" "libstdc++.so.6" "libm.so.6" "libgcc_s.so.1" "libc.so.6" 
"ld-linux-x86-64.so.2") rpath: () runpath: 
("/gnu/store/mj14k58lfc88jhcn6va0s2fpwkv3s35c-llvm-13.0.1/lib" 
"/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib" 
"/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib" 
"/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/10.3.0/../../..")>
scheme@(guix build gremlin)> (file-dynamic-info 
"/gnu/store/mj14k58lfc88jhcn6va0s2fpwkv3s35c-llvm-13.0.1/lib/libLLVMX86CodeGen.so.13")
$12 = #<<elf-dynamic-info> soname: "libLLVMX86CodeGen.so.13" needed: 
("libLLVMAsmPrinter.so.13" "libLLVMX86Desc.so.13" "libLLVMX86Info.so.13" 
"libLLVMGlobalISel.so.13" "libLLVMCFGuard.so.13" "libLLVMSelectionDAG.so.13" 
"libLLVMCodeGen.so.13" "libLLVMTarget.so.13" "libLLVMTransformUtils.so.13" 
"libLLVMAnalysis.so.13" "libLLVMProfileData.so.13" "libLLVMMC.so.13" 
"libLLVMCore.so.13" "libLLVMSupport.so.13" "libstdc++.so.6" "libm.so.6" 
"libgcc_s.so.1" "libc.so.6" "ld-linux-x86-64.so.2") rpath: () runpath: 
("/gnu/store/mj14k58lfc88jhcn6va0s2fpwkv3s35c-llvm-13.0.1/lib" 
"/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib" 
"/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib" 
"/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/10.3.0/../../..")>
--8<---------------cut here---------------end--------------->8---

We could temporarily delete this phase for all 32-bit builds of LLVM.

But the crux of the problem is that llvm@15 has a single huge shared
library, unlike previous versions:

--8<---------------cut here---------------start------------->8---
$ du -hL  /gnu/store/bgqdvvi7k6l255332rfawgjmn2hpn13r-llvm-15.0.4/lib/*.so
133M    
/gnu/store/bgqdvvi7k6l255332rfawgjmn2hpn13r-llvm-15.0.4/lib/libLLVM-15.0.4.so
96K     /gnu/store/bgqdvvi7k6l255332rfawgjmn2hpn13r-llvm-15.0.4/lib/libLTO.so
16K     
/gnu/store/bgqdvvi7k6l255332rfawgjmn2hpn13r-llvm-15.0.4/lib/libRemarks.so
--8<---------------cut here---------------end--------------->8---

(It also has tons of .a files, which shouldn’t be there.)

Is that big LLVM.so due to different build options on our side?  Or is
it a radical upstream change (sounds unlikely, but who knows)?

Thanks,
Ludo’.

diff --git a/guix/build/gremlin.scm b/guix/build/gremlin.scm
index 2a74d51dd9..8a38dde1eb 100644
--- a/guix/build/gremlin.scm
+++ b/guix/build/gremlin.scm
@@ -250,7 +250,11 @@ (define (file-dynamic-info file)
 info."
   (call-with-input-file file
     (lambda (port)
-      (elf-dynamic-info (parse-elf (get-bytevector-all port))))))
+      (elf-dynamic-info (parse-elf
+                         ;; Read at most 10 MiB in memory, which should be
+                         ;; enough to get the PT_DYNAMIC segment.
+                         ;; TODO: mmap the whole file instead.
+                         (get-bytevector-n port (* 10 (expt 2 20))))))))
 
 (define (file-runpath file)
   "Return the DT_RUNPATH dynamic entry of FILE as a list of strings, or #f if