Re: progmodes/project.el and search paths

[Dmitry Gutov]

Sorry for the late reply.

On 08/03/2015 04:19 AM, Eric Ludlam wrote:

> I apologize for not being up on the full thread.  It was very long and I
> didn't really get what the goals were.

I would be lying if I said I got everything figured out. One "unified 
project interface" proposal surfaced a while back, and eventually 
fizzled out. I decided to revisit the subject when I was writing a 
couple of commands that needed some information about the current 
project, and here we are.

Those turned out to be pretty useful, by the way.

> One of my hopes for Emacs is that it will keep tabs on what I'm up to to
> provide useful context relevant feedback when I ask for it.

That sounds like something Flymake or Flycheck would do. Or a completion 
system.

> My
> experience has been that tools that depend on a project to ask what the
> current project is do so a lot.  As such, I cache the current project,
> and current root project as buffer local variables.  Those project
> structures then have relevant meta data as well.  Another tactic is
> initializing each buffer into a project as it is created.  That moves
> search time into a different timing that is less noticeable than
> on-demand when the user makes a request.

IME, (vc-call-backend backend 'root dir) takes a negligible amount of 
time, so we might as well call it once per command. Of course, it might 
be different when using Tramp, but both Tramp and VC do some caching of 
their own. Other project implementations are free to employ their own 
caching methods (might be buffer-local, or if the implementation 
considers that only one project might be enabled at a time, simply in 
its own variables).

> Also, identifying projects should be a service of the core project
> system.  Dependent projects need to provide detection data without
> having Emacs load the full project definition.  That way you can have
> lots of projects types without forcing Emacs to load piles of code that
> you don't need.

Each project-find-functions element is free to only contain the bare 
necessary logic to identify its project type, and autoload the rest only 
as required.

> By way of example, EDE's autoloader does this well, and is wrapped in a
> complex test suite to make sure things don't get loaded when detecting
> projects of a different type.  I re-wrote most of it last year and it is
> much better than it was previously.  Use of 'locate-dominating-file' was
> a big win.

From a quick glance, it defines some restrictions on how a project 
could be detected, and then adds some reusable logic to optimize that. 
I'm not really clear on what those restrictions are. Maybe we'll want 
them, maybe not, we'll have to see.

> Anyway, all that is just about matching projects.  Projects then also
> have meta data.  Things like compilation configuration, compile
> commands, release commands, include paths, classpaths, and whatnot. I
> don't know what you intend with project meta data.  If there was a
> thread about that, I missed it, sorry.

Probably a generic function called project-atrributes that returns a 
hash. Some keys could be standardized.

includes path and class path roughtly correspond to project-search-path.

> Thanks for the pointer.  My Emacs25 didn't compile when I pulled from
> git today, so I didn't get to try it or go very far beyond reading the
> latest in project.el.

That's odd, I didn't have any particular problems compiling it recently.

> For your implementation in ede.el, you are welcome to use what I wrote.

I'm not sure I understand. There's an implementation there already. Is 
your code better? How?

> I think EDE has a generic API for defining a project.  It probably looks
> big on the outside if you just read the doc where it starts with
> "ede-new",

There is a lot to read there. One of the value propositions of 
project.el is that it's easier to grasp.

Another distinction is that project.el is not an API for defining a 
project. It's an API for doing something with an already-defined project.

> but for most cases, if you enable EDE, it will find your
> Emacs src, Linux src, Cmake based build, etc, and it will magically
> provide that to Semantic's parser, to location functions (if enabled)
> and to taggers like global (if enabled.)   The reason it looks big is
> because those are the things that were needed while I was working on
> other tools that needed project support.  I would guess your system will
> grow to a similar state if you keep adding features needed by tools that
> want a project.  It is a really big task.

Maybe. So far, out of that list, the "information for taggers" seems 
like something that could be exposed in the API. The information that 
Semantic needs is probably not generic enough.

The rest are just project types, with different source directories.

> Someone pointed out on the cedet mailing list recently that the doc (and
> probably the website) is too focused on how EDE can create makefiles
> which most people don't want.  Most questions on my mailing list ask how
> to get some other tool like smart completion in semantic working, and
> when they find out that they just need to turn on EDE, and not use the
> full project management system, they are pretty happy.

That's good. In my experience, the CEDET tools are quite dependent on 
each other. Semantic doesn't work for the languages I use, so a lot of 
value proposition goes out of the window.

> > probably because it doesn't match every kind of project). Projectile
> > seems to the most popular "project implementation" out there.
>
> Projectile is a nice interface for moving around between files.

Some other commands it defines:

projectile-run-shell-command-in-root
projectile-find-tag
projectile-multi-occur
projectile-replace
projectile-regenerate-tags
projectile-grep
projectile-test-project
projectile-run-project

I figure most of them could be re-implemented on top of a generic API.

> Projectile has definitions for
> projects EDE does not have because various CEDET developers haven't
> worked with those tools.  The opposite is probably true too.

Probably. Projectile is also 2675 lines long, in total.

> If EDE is considered specific because it is called ede, or because it
> uses eieio, that would be too bad.  It's purpose is to generically
> detect projects such that anyone could create a new project definition,
> find project metadata, and make it accessible to other tools.  This has
> already happened many times.  Supporting EDE is more difficult than just
> providing a dominating file name because it also wants to find extra
> meta data.  I think that is a pretty generic task also.

All I know for sure is that I find EDE's code hard to read (and CEDET as 
a whole, to be honest), and there are not too many third-party packages 
that have decided to build on EDE. Maybe a simpler API will sway a 
larger crowd.

> One way EDE could be made more generic is to make the 'target' concept
> optional.  Simple ede projects also have to define targets as part of
> the detection and matching system, so making them optional would
> simplify creating new projects types in EDE.

It's pretty hard to take something out of an existing API, or a class 
hierarchy. And you can say these methods are optional, but they still 
take up space, and possible consumers are still going to see them.

> Another way EDE could be made more generic is by having built-in tools
> like the 'compile' command ask EDE how to compile, or by having flymake
> use EDE to autodetect how to run make, or perhaps etags could ask EDE
> where to run etags from, and a host of other options.  I had pulled
> those features out into ede named features to keep things independent
> when CEDET was an independent tool.  I don't see why naming conventions
> couldn't change to make things better.

These sounds like good possible applications for project.el.