bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation

[Stefan Monnier]

Eli Zaretskii [2022-05-16 05:31:01] wrote:
> My worry is about packages that do NOT depend on such macros.  Calling
> package-activate-all will activate all the packages on the user's
> system, and there's no way of knowing what those packages do at
> activation time.  They can change variables, advise functions,
> redefine commands, etc.

Yup.   I agree it can be worrisome, but:
- Those packages are presumably already activated in the current session
  and hence similarly affected the generation of the `.elc` files if the
  `.elc` files were generated by `package-install`.
- Preserving the `load-path` like we already do also exposes similar
  problems (some files may shadow the ones we need).

> We have no idea what will be the state of the session after
> doing that.

After the files are compiled, the session is killed (we're talking about
an Emacs sub-process performing native compilation asynchronously), so
I don't think we need to worry too much about that.

>> I'm not claiming that calling `package-activate-all` is right for
>> reasons of principle.  We sadly never clearly defined what it is that
>> a package can count on.
> Then we should do that _before_ we propose solutions that rely on
> what's there, knowing that what's there was never intended to solve
> this particular issue.

I don't understand what you're saying, here.

There is a de-facto definition of what a package can count on (which is
that its autoloads file has been loaded), which has been used ever since
the inception of `package.el`.

We can't go back to a time before that.

I don't know enough of what you mean by "this particular issue" to judge
whether this design was made to solve this issue or not, but I do know
that the autoloads file is loaded on purpose before compiling the `.elc`
files so as to make it more convenient to write the code of a package,
just like we happily rely on `loaddefs.el` being loaded when working on
Emacs's bundled code.

> This cure is worse than the disease.

My proposed patch is not a cure.  The cure would be to compile from the
`.elc` file (since the root of the problem is that we need those
autoloads and the `load-path` to macroexpand the code, and the only
reason why the native-compiler needs to macroexpand the code is because
it (re)starts from the `.el` file instead of using the `.elc` file
where no macro-expansion need to take place).


        Stefan