Re: Project local variables.

[Dmitry Gutov]

On 08.11.2020 17:21, Ergus wrote:

> > Something like project-set and project-get could be arranged (or 
> > perhaps just a project-session hash table). But given that the values 
> > won't persist between Emacs sessions, we'd have to consider to use 
> > cases first.
> Yes, I was not thinking in variables that persist among sessions; but
> only in the active session, so initialize them once.

Sounds like maybe "project session cache" might be an appropriate name.

> Let's say; when a
> file is open Emacs now checks if it belongs to a project; such a project
> is detected (AFAIK) looking up for a vc directory like .git or something
> similar. Is exactly in this case where some environment could be
> shared/scoped to the current project. In LSP I see that most of the
> checks are made every time a file is opened; when using ctags it asks
> every time for the tags file to use instead of reusing "what could be
> inferred" from the current project information if there are other files
> open. To persist among session in the worst case we can add some
> declarations in dir-locals... but I was not thinking on that.

lsp-mode doesn't always use project.el for projects, though. Eglot does, 
but then I'm not sure it's a good idea for it in general (after all, it 
might have to support more project markers purely for internal purposes).

In any case, both of them could use a hash table keyed on a project 
root, for instance. Which would be the simplest version of the "project 
session cache" I mentioned above.

> > Two things to consider: the performance of the operation you're trying 
> > to "cache", and the performance of 'project-current'.
> >
> > The latter is not "free". And my near-term plan is to make 
> > project-try-vc slower by removing the vc-file-getprop/vc-file-setprop 
> > dance because it can lead to outdated information. Or at least try 
> > that and see which problems that brings.
> >
> > In any case, what I'm saying is, 'project-current' on a remote host 
> > might not be fast either. Though it could be cached to at least only 
> > do the search once per user command.
> I don't really know how both performance will compare indeed. In general
> I would expect that looking into the local memory hash-table will be
> faster that looking in a remote file system among the network for an
> executable/file or path. Ex: parallel files-systems, slow networks, or
> experimental boards with slow processors, big processes and so on.

Hash tables are fast. What I meant is, whether the original operation, 
which we hope to speed up with said cache, is going to slower than the 
'project-current' call. The latter is not free either (we currently 
cache the result indefinitely, but we should do less of that in the future).

I suppose whether it's feasible depends on how one would organize the 
code. I.e. whether one could call project-current once, and then do a 
bunch of lookups inside its session cache. If so, it should be an 
improvement.

> I mean; after opening a file, of course a check in the file system will
> be needed to know if that file is in a current opened project (I use
> project-root).

project.el doesn't have a global variable which tracks the "current 
opened project". It just knows how to map a file (or, usually, its 
parent directory) to a structure that knows how to describe the project 
that contains it.

Even if we do add something like that to serve as another kind of cache, 
a file-in-directory-p check wouldn't suffice because we want to support 
nested projects, too.

> > Have you tried using (file-remote-p buffer-file-name) as the hash key?
> I am using project-root and setting a buffer local variables to use as
> the key.

The value of project-root is probably good enough. I was going to 
suggest to key by the project instance itself, but really, it's very 
unlikely to have two different project instances that have the same root 
(even from different backends). Though not impossible, of course.

> I set the variable after opening a file (in a hook); the rest
> of my variables are set lazily... if-set->use else set-and-use. And also
> added a function hook to check if other files of the same projects
> remain open when I kill a buffer; else I delete the info from the hash 
> table".

Unless the cached data structures are really big, I probably wouldn't 
bother with eviction/garbage collection.

> > If the operation to be sped up is really (executable-find "cat"), the 
> > result is really project-independent and should only depend on the host.
>   Every file I open in one of the projects may share the project-root at
> least. executable-find will be host dependent, but TAGS file, a build
> directory, compile_commands.json, root to build or open a shell/vterm,
> maybe some modes that I want to enable temporally for this project (ex:
> lsp, company, langtool for Latex).
> > If there are other, actually project-dependent examples, the 
> > project-local variables (or "session cache", rather) could be 
> > implemented as a hash of hashes, keyed by project instance. There 
> > implementation seems like it will be rather trivial, but first I would 
> > like to know the use cases.
> Se above; maybe this is not enough to justify the implementation of a
> new level of scopes in emacs; I am just mentioning the use cases I had
> in mind and facing very often.

As discussed, the implementation of that new feature seems pretty trivial.

The questions are whether it does save you on CPU time, whether it will 
fit the goals even with not-immediate project-current, and whether we do 
need an implementation of it in project.el itself.

If any project can add a hash table and key the data by project-root, 
that sounds trivial, maybe that's good enough? Unless we identify some 
common pitfalls which we could help people avoid by writing some extra 
code here.