[bug#38546] [00/11] Update Julia, Fix precompilation, add HTTP.jl

[Nicolò Balzarotti]

Hi Simon,

zimoun <address@hidden> writes:

> Hi Nicolò,
>
> Cool that you figured out a source of non-reproducibility.
>
>
> On Tue, 21 Jan 2020 at 14:45, Nicolò Balzarotti <address@hidden> wrote:
>
>> Sorry, I forgot to send the dsfmt patch.  Also, julia's
>> SOURCE_DATE_EPOCH patch was named differently. I've fixed this in 
>> theattached patches.  You need to apply Add-dsfmt.patch, Update-to-1.3.1
>> and then julia-use-SOURCE_DATE_EPOCH.
>
> This patch 'julia-SOURCE_DATE_EPOCH-mtime.patch' is the one you
> mentioned here [#], right?
>
> Could you send it as an upstream PR?
>
> [#] https://github.com/JuliaLang/julia/issues/34115#issuecomment-568171025
>
>
>> About reproducibility: if I'm not wrong, sys.so contains Base library
>> precompiled ([1]).  Precompilation is still non deterministic (here's
>> [2] an issue on github). Something I did to check precompilation:
>
> I am not sure to well understand the source of non-determinism.
>
> Does the patch about SOURCE_DATE_EPOCH fix the issue of [1] and [2]?
> Or is it something else?
>

The first patch (the one I mention in [#]) fixes _one source of_
non-determinism (the stored mtime inside the precompile cache).  But
others are present (precompiling the same file 2 times lead to
differences even in file size, something I reported on [2]).  What I'm
working on is trying to find out what other sources are and patch them,
too.  I patched some c code (src/support/timefuncs.c) so that it follows
SOURCE_DATE_EPOCH too.  This finally lead to same-size files.  I'm now
working on parsing binary (.ji) files directly with julia built in
function to have a textual representation of the differences.

I can't send SOURCE_DATE_EPOCH patches upstream until [3] is merged, as
patching timefuncts leads to an endless test suite (as they were using
time() to check for passed time. If time() always returns 1, tests
sleeps forever).  Yesterday evening I completed that patch (and has been
approved).  My plans are on finding the hopefully last source of
non-determinism, patch it and send everything upstream (hoping they
care).

>
>> I could not get the same results twice (also, size differs).  I'll work
>> on this on some spare time (for example, there are other places where
>> SOURCE_DATE_EPOCH can be used, but this [3] is a problem I need to
>> solve first).
>
> Is the problem [3] not solved by 'julia-SOURCE_DATE_EPOCH-mtime.patch'?
>
>
>
>> Maybe 1.3.1 (when reviewed) can be merged, since we have
>> the same problem with julia 1.1, but we can wait for the
>> source-date-epoch and julia-xyz patches until we solve this.
>
> My opinion is: if a patch is floating around to fix the
> source-date-epoch issue, let try to push it upstream. If it is
> rejected, let talk later if Guix will include it or not. And in the
> meantime, I will try to review the 1.3.1 because yes I agree that it
> should be included even if we know it is not reproducible -- the
> package Guitarix [@] is updated and not reproducible neither.
>

Thanks! I hope in a 1.4 release where everything is fixed ;)

Nicolò

> [@] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21803
>
>
> Thank you for working on this.
>
>
> All the best,
> simon
>
>
>> [1] https://docs.julialang.org/en/v1/devdocs/sysimg/
>> [2] https://github.com/JuliaLang/julia/issues/25900
>> [3] https://github.com/JuliaLang/julia/issues/34115