bug#61817: 30.0.50; Project.el finds incorrect project roots in git worktrees

[Arthur Miller]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Depending on how we might define a "project parent" in the future (probably as
> "project of root's parent directory", though), and how the worktrees reside on
> disk relative to the parent, these notions won't necessarily match.

Sure, but I don't think they have to. I am just telling that we need both; my
suggestion is to use project-vc-root, as the king of all roots; regardless of
the shape of folder structures; and current-project-root for worktrees and what
not. Just as a suggestion, more about this one little further below.

>> If you have a branch "main", work in a branch X, and would like to test/fix
>> something unrelated to X, wouldn't you create a new workree Y from clean 
>> "main"?
>> It is a bit special case, which I personally use, so to automate this, 
>> "worktree"
>> as root is not enough, I need the actual root. For other purposes I need the
>> worktree. So as said, both are needed. Of course, worktree will be used more
>> frequently for natural reasons.
>
> That sounds like a fine approach, though not the only possible one.

Yeas; of course; there is never one approach only.

> have a limited number of worktrees: some constant, and some for temporary
> branches. I suppose it depends whether you need to create some configuration
> files specific to the local system when deploying and testing a new repository
> checkout.

Exactly. The more general approach is to list branches with completing read and
let user choose one to create a patch to. But the one I described is just a
little quick-fix. M-x add-new-patch, and it does what I described above.

>>> Whenever one chooses a different shape, though, they should consider how 
>>> that
>>> will affect the common uses.
>> It is not about the shape; you can put a worktree as a subdirectory in a main
>> repository, it does not need to be out of source tree; project.el will still 
>> not
>> find the real git root. You can test it yourself in Emacs source:
>
> Indeed, it won't, because nobody told it it should traverse up to "worktree
> parent", always or under certain condition.

Exactly, so once in a worktree, current project.el can't find the "actual git
root" as you called it. I would like it to be able too, so that I can use
project.el as a "generic" library independent of VC system.

> That structure would also be quite weird, though: root in dir A, but all 
> project
> files residing in some sub-sub-directory D of A. Doesn't that sound weird?

Yes, it is very weird, and that why I too also make worktrees out of the main 
repo,
parallel to it. I took it up because you wrote about choosing different shapes
for directory structures and consequences, but I do same as you.

> Then the question would be how to provide both of those pieces of information 
> to
> callers. If we add the constraint that it has to be in some abstract way
> (independent from Git), then the answer is probably "no way".

I am not sure myself; but obviously for the VC that does not support the
feature, project-vc-root could return just what it returns now, and just act as
a synonym for project-root, while for systems that do (git) it will return
whatever actual root is? 

> I do believe that it's a "useful middleware". And a trait of such middlewares 
> is
> they usually try to present some abstraction over the implementation.

Ok, can we then have some useful stuff like: project-add-new-patch and
project-add-new-target or similar (I welcome better ideas); where at least some
common actions are automated? project-vc-dir already does some VC stuff (push,
pull etc), it can well provide some other useful abstractions? 

I can help to donate those, at least for Git, but the Git part is easy to write
anyway.

> If, perhaps, we would add a command doing something like the above to
> project.el, or we envision somebody doing it in some popular extension, *and* 
> it
> would be possible to do with several project backends (maybe real and 
> potential
> ones), then that would sound like a feature for project.el.
>
> If a caller, OTOH, needs to know that they're working with Git, and with a
> particular project backend, to do what they wanted, then that would be a poor
> fit for the API. But we could try to fit in features in some other way. Adding
> them to the VC package, perhaps.

Exactly; I totally agree; that is why I have started discussion. I can do
git-based everything easily myself, but instead of doing it git-based per
definition I wanted to use project.el as an abstraction so my little project
stuff can work in possibly other scenarios then just with Git.

> I honestly don't understand EDE enough to have a solid opinion. But it did 
> seem
> to have some pronounced pieces that didn't make sense for projects I regularly
> interact with.

EDE seems interesting to me, because of the integration with rest of Emacs,
speedbar etc, but I can't say I have much understanding of Ede myself.

> That sounds like a decent feature, though one that is often taken up by 
> command
> line tools. It not quite clear what Emacs integration would add to
> that. Besides, perhaps being able to write templates in Org, or something.

As said: automation. I have nothing against command line, and I can certainly
just write a little shell or elisp script to create a folder, copy some files,
init git, generate some pieces etc. But everytime I do, manually, or writing a
new script, I am repeating a set of dull steps, which are totally unnecessary
and just take focus from what I really want to do. 

>       Besides, perhaps being able to write templates in Org, or something.

Well, I wouldn't say it is just "or something" and you don't need to use it in
Org at all :).

Org-capture does setup a small gui (read-key) and is easy to re-use. More
important it provides a relatively familiar interface to writing Lisp property
lists (capture templates) even to non-lispers :). org-capture is quite well
accepted in Org community, I believe; and there are a lots of non-programmers
which use it for wrting notes, mailes etc.

I don't know; it is really small, already included, and usually already loaded
into Emacs; and lots easier to use than some other alternatives to provide some
basic gui and entry point for customization. I have attached a toy example for 
an
illustration. I am sure I could resue transient, hydra, or just write my own 
too,
but I want something that comes with Emacs which Hydra does not, and am not so
familiar with transient myself, and I believe holds for lots of other people 
too.

> But the beauty of it, it doesn't have to be strongly bound to project.el. It 
> can
> *use* project-current/project-root, but be a package of its own, with separate
> naming/features/hooks and language/framework detection.

Yes, of course, I know it can; that is what I already do. It does not even need
to use Emacs, I can write it in Bash or as a script for cmd.exe; but that is not
the quesiton. Currently I don't even use project.el, but as said, if it will
provide more backends and abstractions, I would prefer to use it than to make my
own git-specific everything.

>> 2) better integration with tools for common workflow(s). Adding
>> features, fixes, tests etc, is mostly about creating a new patch, which with
>> addition of worktrees in Git can be super nicely automated. For example: add 
>> new
>> library, or add new executable, or add new fix, it all boiles down to add new
>> branch which leads to add new worktree, add a possible target in Makefile,
>> create possibly a test file, probably some other thing. It would be nice to 
>> be
>> able to say: M-x add-new-shared-library or M-x add-new-executable, or M-x
>> add-new-fix and Emacs will do at least initial boilerplate.
>> If you plan to add build tool support to project.el, it would be welcome :).
>
> Hmm, as I see it, this is split into two different parts:
>
> - Do a Git-specific thing (create a worktree), once per body of work.

Yes, and to me this feels like a good target for an abstraction. Shouldn't
something like that be in project-vc-dir app?

> - Do a build tool specific thing, probably several.

Again yes. That is the harder part, but there is a lot of freedom to implement
just parts of it, and successively per tool, and so on, and the entry point 
could
be the abstraction above.

> The thing about the latter, you have described pretty well the general 
> direction
> of what features a hypothetical buildtool.el might contain. Alas, I have 
> little
> experience with languages where you regularly do things like that (adding new
> shared libraries, new executables, other build artefacts), and it seems to be 
> an
> open question whether there is point in adding a language agnostic API on top 
> of
> that, or if people should just create c++-buildtool.el, java-buildtool.el, 
> etc,
> separately without trying to unify the UI.

That does not have to be a build artefact, like .dll or .jar or .exe. It is just
a "target". That is the beauty of Makefile which works just with targets and
dependencies, and does not care what a target is. "clean" target as we know it
from C or C++, or start or stop some web or system services are just as normal
as building artefacts.

> If somebody else wanted to work on this, though, probably a fair amount of
> people on this list would welcome that, and help with advice/review/some later
> contributions.

Sure, I am happy to hear suggestions and opinions; especially if someone can
help me to understand and re-use Ede for the tool integration.

However; I am even more happy if you add something like project-vc-root (or
choose any name you want), and some hooks or other abstraction for
project-add/remove-* where * is any of patch, feature, target, ... something
similar, I guess you get the idea :).

Anyway, thank you for the kind answers and interesting discussion, and hope you
don't mind I have deleted older parts; it was a bit hard tp read. so I have left
just your latest and my answers (mostly); hope it is ok.

project-capture.el
Description: Text document