Re: On elisp running native

[Andrea Corallo]

Eli Zaretskii <address@hidden> writes:

>> From: Andrea Corallo <address@hidden>
>> Cc: address@hidden, address@hidden
>> Date: Sat, 28 Dec 2019 11:57:36 +0000
>> 
>> Every .el file is compiled into a .eln.  This is technically a shared
>> library.
>> 
>> Into the .eln Emacs is expecting to find a bunch of symbols.  Two of
>> these are used for relocating objects and Emacs primitive functions.  So
>> Emacs essentially writes some pointers into them.
>
> Can you tell more about this part?  What does this relocation include,
> and to what extent the details depend on the .eln file being in ELF
> format?

Sure.  About these two:

- A pointer has to be set in the shared library to point to the array
  containing all the function pointers of Emacs C primitives and
  functions used to support the run-time.  Lets call it 'function link
  table'.

  There is only one copy of this structure and is malloced by Emacs.

  This is used by the code into the .eln file in order to jump into
  Emacs.  All elns share the same 'function link table'.

- The other structure is an array of lisp objects that are the objects
  coming from the 'constant vectors' of every byte-compiled function.

  This is allocated as static in every eln and initialized during the
  eln load process.

  To recreate the objects Emacs read a string from the eln it-self and
  run the reader.  When the objects are revived into memory these are
  set into the array one by one.

I do not depend on ELF.  I'm already using the functions wrapped by our
'dynlib' in order to execute this machinery.

>> Then there's 'top_level_run'.  This is a real function that is responsible 
>> for
>> modifying the environment as the execution of the various top level forms
>> on the original .el file would do.
>> 
>> Emacs jumps into 'top_level_run' and this will call back into Emacs for
>> evaluating forms and/or defining native functions (effectively new
>> subrs).  This function is obviously not called when resuming from an
>> image dump.
>
> So when Emacs calls a Lisp function foo-bar, that call will produce a
> call to 'top_level_run' function of the corresponding .eln file, and
> then 'top_level_run' of that .eln will call the code produced from
> 'foo-bar'?  How does Emacs know which .eln "provides" foo-bar?

Sorry I wasn't clear.

'top_level_run' is run once during file load.  'top_level_run' calls
back into Emacs to register 'foo-bar' (as a lightly special subr).

So after that 'foo-bar' is called as every other subr is.

> And how would the code in the .eln know to call back into Emacs?  When

See above

> you compile the native code, do you leave the symbols of Emacs
> functions unresolved, and rely on the dynamic linker to resolve them?
> If so, this doesn't work on Windows: you will get link errors.

I just use dynlib_sym to obtain the addresses of these few symbols.  The
dynamic linker does not come into play and the rest is handled by the
described mechanism.  The fact that the list of primitives expected by
the eln matches the one provided by the system is checked comparing two
hashes duiring the load process.  If the eln is not compatible an error
is raised.

> Thanks.

I guess it's easier to read in code than to explain (is really not
much).  In case you can find 'load_comp_unit' into:

https://gitlab.com/koral/gccemacs/raw/dev/src/comp.c

Please let me know if I can better clarify some of these points.

Andrea

-- 
address@hidden