Re: reproducible .pdmp generation

[Bernhard M. Wiedemann]

On 01/10/2024 18.32, Daniel Colascione wrote:
> On October 1, 2024 9:04:26 AM PDT, Eli Zaretskii <eliz@gnu.org> wrote:
> > > Date: Tue, 1 Oct 2024 17:08:30 +0200
> > > From: "Bernhard M. Wiedemann" <bernhardout@lsmod.de>
> > >
> > > But there might be other issues involved.
> > > One pair of .pdmp files had this noteworthy hunk in the diff:
> > >
> > > >   009ca3a0  73 20 74 6f 20 64 6f 63  20 73 74 72 69 6e 67 73  |s to doc strings|
> > > >   009ca3b0  2e 2e 2e 64 6f 6e 65 0a  50 75 72 65 2d 68 61 73  |...done.Pure-has|
> > > >   009ca3c0  68 65 64 3a 20 31 35 39  35 37 20 73 74 72 69 6e  |hed: 15957 strin|
> > > > -009ca3d0  67 73 2c 20 31 39 35 33  20 76 65 63 74 6f 72 73  |gs, 1953 vectors|
> > > > +009ca3d0  67 73 2c 20 31 39 34 37  20 76 65 63 74 6f 72 73  |gs, 1947 vectors|
> > > >   009ca3e0  2c 20 34 39 35 33 31 20  63 6f 6e 73 65 73 2c 20  |, 49531 conses, |
> > > >   009ca3f0  31 32 33 33 20 62 79 74  65 63 6f 64 65 73 2c 20  |1233 bytecodes, |
> > > >   009ca400  33 37 31 20 6f 74 68 65  72 73 0a 44 75 6d 70 69  |371 others.Dumpi|
> > >
> > > Is there a tool to convert .pdmp files into some readable text form, so
> > > that I can better understand what differs there?
> >
> > Not that I know of.  Maybe Daniel (CC'ed) does.
> >
> > > What other approaches could help to track down the source of
> > > non-determinism here?
> >
> > Ditto.
> >
> > > Are these files actually needed to use emacs?
> >
> > You mean, the .pdmp files?  Without them, you can only start temacs
> > and load all the preloaded Lisp files from scratch, before you can
> > have a fully-functional Emacs.
> >
> > If you meant other files, please name them.
>
> pdmp files are basically packed Emacs heap images with embedded relocations. If 
> you want to make them deterministic, make the temacs run that produces them 
> deterministic. For example, you can try turning off ASLR and making sure that 
> if anything is using a PRNG, it's seeded. Above, it seems like the string table 
> is differing somewhat between two runs. I'd start with tracking down the 
> origins of the strings.

ASLR and parallelism were already disabled in my example reproducer via 
setarch -R and taskset 1
I tried to make it deterministic with code patches, but even in the best 
(random) case, there were still 15 random bytes left in 4 locations.

See the attached patch for what I tried.

Something like objdump -D would be nice to have for .pdmp files to 
better understand where it differs.

I uploaded two such .pdmp files from emacs-29.4 to
http://rb.zq1.de/other/emacs/
in case someone wants to have a closer look.

Ciao
Bernhard M.

test.patch
Description: Text Data

OpenPGP_signature.asc
Description: OpenPGP digital signature