Re: Unified project interface

[Dmitry Gutov]

Hi Stephen,

Sorry for the delay; I had to think on it.

On 07/25/2015 03:55 AM, Stephen Leake wrote:

> I would suggest the terminology "main project" vs "included projects"
> for this; the "main project" is the code you own; the "included
> projects" are all the others (included the system includes).

There's sense in this, but is "an included project you don't own, which 
is not in system includes" a significant use case?

In my own experience, either the projects on which the current one 
depends are installed in system directories, and thus are 
indistinguishable from system includes (such as Ruby gems, installed via 
"gem install"), or the linked project's tree lives near the current one, 
and we can consider it a part of the main project, for the purposes of 
editing.

Because even if the second project is a third-party dependency, 
sometimes you want to make changes to it anyway, and the user might get 
miffed if xref-query-replace prohibits that.

> So this would be an argument to xref-find-regexp;
>
> (defun xref-find-regexp (regexp &optional not-included-projects)
> "Search all files in all project directories for REGEXP.
> If NOT-INCLUDED-PROJECTS is non-nil (default nil), search only the main
> project. Otherwise, search the main project and included projects."

It should probably be a new command, because otherwise specifying that 
second argument interactively would be a pain (C-u already makes 
xref-find-regexp ask a bunch of questions).

> To implement this, project-search-path might want to be an alist of:
>
> (PROJECT-NAME <search-path>)
>
> where the first element is the main project and the rest are the
> included ones.

Maybe do it differently?

If we do have linked projects (and the separation between 
main/included), some generic function, probably named differently, would 
return a list of those project instances. Then the caller will ask each 
instance for its root (or the directories).

There's no need for the main project's search-path entry there, 
xref-find-regexp can just use its root (or the directories) as well.

> > We have that in project-ignores now.
>
> Right. That's not consistent with current things like load-path. I

Indeed. But maybe supporting ignores for project-search-path is not a 
big priority. There's not much to ignore in /usr/include, for instance. 
The directories in load-path also usually don't contain weird files or 
build artefacts (and when they do, it's not like elisp-search-path 
really can account for those). So maybe a good first approximation would 
be to simply not apply the ignores list to the project-search-path 
elements outside of the project tree.

If that's not good enough, project-ignores can accept a directory as its 
argument (it must be among those returned by project-directories, or 
project-search-path), and return the appropriate ignores list.

This will definitely help if each element in project-directories is a 
separate (e.g. Git) repository, with its own .gitignore.

> should think at some point in the implementation of any "search project"
> function you will want an explicit list of directories (and maybe files)
> to search, which will be the union of project-search-path minus
> project-ignores. So it seems simpler to specify that list directly.

I'm not sure about that. It seems then the implementation will have to 
walk the whole project tree itself, and collect all directories that 
don't match the ignores (because .gitignore syntax allows not-anchored 
entries for directories). And if we're to support ignoring files (and 
obvious use case), it'll become a list of files, which can grow pretty 
big. I think it's better to delegate this job to 'find', as long as that 
doesn't force us to compromise on functionality.

> The user interface (the syntax of the project file) could be structured
> as includes and ignores (excludes).

.gitignore also supports whitelisting entries (which override the 
excludes). I haven't gotten around to writing support for those.

> > You're probably aware of similar tools.
>
> monotone, git, subversion ... :).

Not sure about the other two, but Git doesn't need to be told the 
project root. It can be called from any directory, and works with 
relative paths quite well.

> Recently I've been playing with Google's Android Studio (shudder; only
> to make Emacs do what it does, I promise :). It defines a project root
> by a dominating file.

Of course. Why else one would be playing with it? ;)

In this case, project-directories can return a list with one element.

> On the other hand, AdaCore GPS (an Ada-specific IDE) defines a project
> by a project file that specifies various lists of directories (source,
> object, executable); there is no root directory. The project file itself
> can be anywhere, it doesn't have to be in one of the listed directories.
> That's what Emacs Ada mode does as well.

In this case, it seems, the relevant project implementation will be 
based on user explicitly telling it where the project file lives (by 
calling a particular command, with a prompt), and then the project will 
be enabled globally, for the whole Emacs session.

project-directories then will return some of the directories specified 
in the project file (except the object ones, maybe, if there's nothing 
we can edit there), but probably (?) not the one where the project file 
lies, if it's somewhere separately.

Would there be any significant value in project-root returning the 
directory of the project file?

> I think they are disjoint concepts. Nothing wrong with keeping both;
> some tools will need them, some won't.

That would mean documenting the distinction, so that implementers know 
what to do. I'm increasingly less clear on it.

> If project-directories contains all directories (not just root ones),
> this is just (file-name-directory path) . (file-name-non-directory
> path).

It would be. But I expect project-directories to only include the roots. 
IOW, no project-directories element should be a subdirectory of another.

> > Then instead of project-root, a consumer will use (car
> > (project-relativize-path default-directory)).
>
> I don't follow; that means the value of 'root directory' changes
> depending on where the current file is.

That's correct. If the file is inside the "main" project directory, it 
will use the main project root. If the file is in another directory, 
it's in a "linked" body of code (which would often be called a project 
as well), and its root is used.

Atom has taken this approach: each element in 
atom.project.getDirectories() corresponds to a Git repository. See the 
link I sent previously, and also https://github.com/atom/atom/pull/3691.

This may look wrong for certain configurations (your AdaCore example 
comes to mind), but would the concept of root be even useful in that case?

> Given the variable project-root, it makes sense to define
> project-relativize-path to return the portion of the path that is
> relative to project-root, and error if there is no such portion (ie an
> absolute path outside the root tree). Monotone has such a function in
> the C++ code.

Sure. But if project-root is not among project-directories, I'm not sure 
when, and for what cases, that directory can be used.