[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: native compilation units
From: |
Andrea Corallo |
Subject: |
Re: native compilation units |
Date: |
Wed, 08 Jun 2022 06:56:11 +0000 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) |
Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> Performance issues with read access to directories containing less than
>>> 10K files seems like something that was solved last century, so
>>> I wouldn't worry very much about it.
>> Per my response to Eli, I see (network) directories become almost unusable
>> somewhere around 1000 files,
>
> I don't doubt there are still (in the current century) cases where
> largish directories get slow, but what I meant is that it's now
> considered as a problem that should be solved by making those
> directories fast rather than by avoiding making them so large.
>
>>> [ But that doesn't mean we shouldn't try to compile several ELisp files
>>> into a single ELN file, especially since the size of ELN files seems
>>> to be proportionally larger for small ELisp files than for large
>>> ones. ]
>>
>> Since I learned of the native compiler in 28.1, I decided to try it out and
>> also "throw the spaghetti at the wall" with a bunch of packages that
>> provide features similar to those found in more "modern" IDEs. In terms of
>> startup time, the normal package system does not deal well with hundreds of
>> directories on the load path, regardless of AOR native compilation, so I'm
>> tranforming the packages to install in the version-specific load path, and
>> compiling that ahead of time. At least for the ones amenable to such
>> treatment.
>
> There are two load-paths at play (`load-path` and
> `native-comp-eln-load-path`) and I'm not sure which one you're taking
> about. OT1H `native-comp-eln-load-path` should not grow with the number
> of packages so it typically contains exactly 2 entries, and definitely
> not hundreds. OTOH `load-path` is unrelated to native compilation.
>
> I also don't understand what you mean by "version-specific load path".
>
> Also, what kind of startup time are you talking about?
> E.g., are you using `package-quickstart`?
>
>> Given I'm compiling all the files AOT for use in a common installation
>> (this is on Linux, not Windows), the natural question for me is whether
>> larger compilation units would be more efficient, particularly at startup.
>
> It all depends where the slowdown comes from :-)
>
> E.g. `package-quickstart` follows a similar idea to the one you propose
> by collecting all the `<pkg>-autoloads.el` into one bug file, which
> saves us from having to load separately all those little files. It also
> saves us from having to look for them through those hundreds
> of directories.
>
> I suspect a long `load-path` can itself be a source of slow down
> especially during startup, but I haven't bumped into that yet.
> There are ways we could speed it up, if needed:
>
> - create "meta packages" (or just one containing all your packages),
> which would bring together in a single directory the files of several
> packages (and presumably also bring together their
> `<pkg>-autoloads.el` into a larger combined one). Under GNU/Linux we
> could have this metapackage be made of symlinks, making it fairly
> efficient an non-obtrusive (e.g. `C-h o` could still get you to the
> actual file rather than its metapackage-copy).
> - Manage a cache of where are our ELisp files (i.e. a hash table
> mapping relative ELisp file names to the absolute file name returned
> by looking for them in `load-path`). This way we can usually avoid
> scanning those hundred directories to find the .elc file we need, and
> go straight to it.
>
>> I posed the question to the list mostly to see if the approach (or similar)
>> had already been tested for viability or effectiveness, so I can avoid
>> unnecessary experimentation if the answer is already well-understood.
>
> I don't think it has been tried, no.
>
>> I don't know enough about modern library loading to know whether you'd
>> expect N distinct but interdependent dynamic libraries to be loaded in as
>> compact a memory region as a single dynamic library formed from the same
>> underlying object code.
>
> I think you're right here, but I'd expect the effect to be fairly small
> except when the .elc/.eln files are themselves small.
>
>> It's not clear to me whether those points are limited to call
>> sites or not.
>
> I believe it is: the optimization is to replace a call via `Ffuncall` to
> a "symbol" (which looks up the value stored in the `symbol-function`
> cell), with a direct call to the actual C function contained in the
> "subr" object itself (expected to be) contained in the
> `symbol-function` cell.
>
> Andrea would know if there are other semantic-non-preserving
> optimizations in the level 3 of the optimizations, but IIUC this is very
> much the main one.
Correct that's the main one: it does that for all calls to C primitives
and for all calls to lisp function defined in the same compilation unit.
Other than that speed 3 enables pure function optimization and self tail
recursion optimization.
Andrea
- Re: native compilation units, (continued)
- Re: native compilation units, Lynn Winebarger, 2022/06/06
- Re: native compilation units, Lynn Winebarger, 2022/06/07
- Re: native compilation units, Stefan Monnier, 2022/06/07
- Re: native compilation units, Eli Zaretskii, 2022/06/07
- Re: native compilation units, Lynn Winebarger, 2022/06/14
- Re: native compilation units, Stefan Monnier, 2022/06/14
- Re: native compilation units, Lynn Winebarger, 2022/06/14
- Re: native compilation units,
Andrea Corallo <=
- Re: native compilation units, Lynn Winebarger, 2022/06/11
- Re: native compilation units, Stefan Monnier, 2022/06/11
- Re: native compilation units, Lynn Winebarger, 2022/06/11
- Re: native compilation units, Stefan Monnier, 2022/06/11
- Re: native compilation units, Lynn Winebarger, 2022/06/12
- Re: native compilation units, Stefan Monnier, 2022/06/12
- Re: native compilation units, Lynn Winebarger, 2022/06/13
- Re: native compilation units, Stefan Monnier, 2022/06/13
- Re: native compilation units, Lynn Winebarger, 2022/06/14
- Re: native compilation units, Stefan Monnier, 2022/06/15