[bug#38546] [00/11] Update Julia, Fix precompilation, add HTTP.jl

[Nicolò Balzarotti]

Hi Simon,

Sorry, I forgot to send the dsfmt patch.  Also, julia's
SOURCE_DATE_EPOCH patch was named differently. I've fixed this in theattached 
patches.  You need to apply Add-dsfmt.patch, Update-to-1.3.1
and then julia-use-SOURCE_DATE_EPOCH.

About reproducibility: if I'm not wrong, sys.so contains Base library
precompiled ([1]).  Precompilation is still non deterministic (here's
[2] an issue on github). Something I did to check precompilation:

Since I'm tired of this, I've started investigating better.

Loaded a guix container
#+begin_src bash
mkdir -p /tmp/jlprecompiled/
./pre-inst-env guix environment -u nixbld --no-cwd --container --network 
--expose=$(realpath determinism) --share=/tmp/jlprecompiled/ --ad-hoc julia 
coreutils nss-certs curl tar gzip libfaketime
#+end_src

from this container, run some time this:

#+begin_src bash
export SSL_CERT_DIR="$GUIX_ENVIRONMENT/etc/ssl/certs"
export SSL_CERT_FILE="$SSL_CERT_DIR/ca-certificates.crt"
export GIT_SSL_CAINFO="$SSL_CERT_FILE"

SOURCE_DATE_EPOCH=1 julia --startup-file=no -q -e 'using Pkg; Pkg.instantiate()'
SOURCE_DATE_EPOCH=1 faketime '2008-12-24 08:15:42' julia --startup-file=no -q 
-e 'using Random; using Pkg; Pkg.add("Compat"); Random.seed!(0); using Compat'

cp -r ~/.julia/compiled/v1.3/Compat/ /tmp/jlprecompiled/$RANDOM
sha256sum ~/.julia/compiled/v1.3/Compat/*
# 6c10d65f6cd9cdd5b98e4d5e2b82b7bd00508f0daccef0967dec92cb6468052e  
/home/nixbld/.julia/compiled/v1.3/Compat/GSFWK_Pua4I.ji
ls -lah ~/.julia/compiled/v1.3/Compat/*
# -rw------- 1 nixbld users 102K Jan 21 07:39 
/home/nixbld/.julia/compiled/v1.3/Compat/GSFWK_Pua4I.ji
#+end_src

I could not get the same results twice (also, size differs).  I'll work
on this on some spare time (for example, there are other places where
SOURCE_DATE_EPOCH can be used, but this [3] is a problem I need to
solve first).

Maybe 1.3.1 (when reviewed) can be merged, since we have
the same problem with julia 1.1, but we can wait for the
source-date-epoch and julia-xyz patches until we solve this.

What do you think?

Thanks, Nicolò

[1] https://docs.julialang.org/en/v1/devdocs/sysimg/
[2] https://github.com/JuliaLang/julia/issues/25900
[3] https://github.com/JuliaLang/julia/issues/34115

0001-gnu-Add-dsfmt.patch
Description: Text Data

0001-gnu-julia-Update-to-1.3.1.patch
Description: Text Data

0001-gnu-julia-use-SOURCE_DATE_EPOCH-for-precompilation-t.patch
Description: Text Data

zimoun <address@hidden> writes:

> Hi Nicolò,
>
> Thank you for working on this.
> Julia rocks! ;-)
>
>
> However, I am lost with your patch set.
>
> I have tried only the first patch you attached. I hit this error:
>
> --8<---------------cut here---------------start------------->8---
> error: dsfmt: unbound variable
> --8<---------------cut here---------------end--------------->8---
>
> So something is missing. :-)
>
> Then I notice that you change the list of 'patches' and you "forgot"
> to send them too. :-)
>
>
> Almost related, does this update fix the reproducibility bug [1]? For example,
>
> --8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix build julia
> ./pre-inst-env guix build julia --check --no-grafts -K
> --8<---------------cut here---------------end--------------->8---
>
>
> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22304
>
>
> All the best,
> simon