Re: [feature/native-comp] breakage on build

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: phillip.lord@russet.org.uk,  emacs-devel@gnu.org
>> Date: Mon, 01 Feb 2021 11:01:04 +0000
>> 
>> each .eln while executing must indeed call into Emacs C code.  Call
>> happens through function pointers, say we must call Fmessage, something
>> like this will be rendered into the eln:
>> 
>> freloc_link_table->Fmessage (...)
>> 
>> 'freloc_link_table' is a pointer statically allocated within the eln
>> object, it gets set while loading to point to the right data structure
>> in memory.  This data structure is a C structure holding all function
>> pointers to all Emacs primitive functions.
>> 
>> For this reason we must be sure that the definition of
>> 'freloc_link_table' use while compiling the eln matches the one offered
>> by the Emacs loading the eln file.
>> 
>> This is the main reason eln are not portable between different Emacs
>> configurations.
>> 
>> HASH1 has the duty to disambiguates this and everything else that could
>> reflect into an incompatibility of the eln (`system-configuration`
>> `emacs-version`).
>> 
>> As from time to time we can refine mechanisms like the load mechanism or
>> anything else and consequentially generating another incompatibility the
>> macro ABI_VERSION also contribute to HASH1.  I bump a new number there
>> each time I realize I'm changing something that introduce and
>> incompatibility.
>> 
>> To summarize eln was never designed for compatibility across versions or
>> configurations and HASH1 is made to guard against that.
>
> OK, thanks.  I understand the reasons now.  However, I'm not sure we
> should encode all this information in the .eln file's name.  For
> starters, some of those reasons are also true for *.elc files, right?
> For example, suppose that a primitive that the .el file calls changed
> its signature in backward-incompatible ways -- the .el files which
> call it need to be recompiled, or else they will fail, either silently
> or noisily.  Right?  But we don't encode those details in the names of
> the *.elc files, do we?  Why should the *.eln files be different?

I think with eln the situation is considerably more sentitive than the
elc one in this respect.  The reason is that we (on purpose) skip
Ffuncall to call directly into C code, as a consequence an
incompatibility will most certainly lead to a crash and not a runtime
error.

> Next, why not encode this information in some data in the *.eln files,
> and have the code which loads the *.eln files check the compatibility
> and reject the load if they don't match?  Why encode this information
> in the file's name?

We can imagine two alternative scenarios:

1- We merge HASH1 and HASH3

  - In this case everything will work as of now but certain operations
    will become more difficult, the typical example is manually removing
    all the eln left by an old Emacs configuration that is not anymore
    in use (me and others do this quite often).

2- We move HASH1 into the eln as metadata.

  - The limitation of 1 persists

  - To check that each eln is loadable one has: to 'dlopen' it, read the
    metadata ('dlsym'), verify and in case reject.  This is certainly
    way more expansive than directly searching for a given filename and
    might impact negatively startup time, especially on certain OSes
    where opening files is an expensive operation.

  - We'd have name clashes for different eln coming from different
    Emacs configurations/versions.

>> > And in any case, if the triplet is in HASH1, why not remove its
>> > human-readable representation and save a few bytes?
>> 
>> At the time was thought to be convinient to have the triplet also as
>> human readable.  Indeed no problem from my side on removing it if this
>> is problematic (wasn't aware of this Windows limitation).
>> 
>> That said I think the main cure for this Windows file length issue is to
>> diet down the hash length to something more reasonable (16 or 8
>> characters?).
>
> I think we should do both.  The chances that the same cache directory
> will be used for different architectures is pretty low anyway.

Okay.

Shall we pick a length?  8 characters is probably more than okay but we
can go for 16 if we prefer to stay on the safe side.

Thanks

  Andrea