On generating compilation databases

[Arsen Arsenović]

Evening,

Could, and how should, (auto)make be instructed to output compile
commands?

Following the example of ninja[1], I tried to teach make to generate
makefile commands; however, I ran into a snag.  Automake generates:

  $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) \
  $(tangle_CPPFLAGS) $(CPPFLAGS) $(tangle_CXXFLAGS) $(CXXFLAGS) -MT \
  src/tangle-main.o -MD -MP -MF src/$(DEPDIR)/tangle-main.Tpo -c -o \
  src/tangle-main.o \
  `test -f 'src/main.cpp' || echo '$(srcdir)/'`src/main.cpp

(wrapping mine)

... which relies on a shell construct, making it hard to 'deduce' from
the point of a pure make processor (note that I didn't actually go too
far implementing this, since I discovered this quite early on in my
gmake hacking).

While I feel like make would be a good place to implement this, I don't
think it's general enough to always use.  The automake example above is
already problematic.

At this point, before taking this further, I went searching over MLs,
and I came across these threads:
https://lists.gnu.org/archive/html/automake/2018-07/msg00006.html
https://lists.gnu.org/archive/html/bug-make/2018-07/msg00049.html

... however, these appear to have been inconclusive.

An alternative that I thought of is setting CC, CXX and other
"conventional" compile variables (and maybe some others as defined by a
special piece of syntax?) to a tool (e.g. $LIBEXECDIR/make/report-calls)
that emits postprocessed command variables to some side channel (e.g. a
socket inherited in program invocations and recursive makes), which then
gets collected by the root make invocation (with the caveats that the
default target runs without dependency checking except for ordering) and
dumped into compile_commands.json.  This seems doable, but how fragile
it might be worries me (namely, due to stamp-* targets and similar).
This option is also somewhat akin to what Bear[2] does (and through my
experience with it, I feel these drawbacks are significant enough,
though the approach I described could be a bit better, since it ignores
dependency age checking, and hence would "catch" files that don't get
rebuilt too).

Another possibility that came to mind was having automake (or, rather,
an automake-powered autotools build system) generate
compile_commands.json at configure time, if enabled.  This seems to
avoid the "unwanted command" problem from above, but wouldn't work for
some parts of some projects (for instance, gcc.git/gcc/Makefile.in is
*not* automake'd).

Maybe it'd be worth it to do both?  With automake being preferred (by
the user).

To justify this, I think it'd be valuable for contributors, even if
support for consuming these DBs is lacking in the toolchain.  I can
definitely imagine that some would be discouraged from working on
important parts of the GNU toolchain due to a lack of their usual guard
rails, which are especially useful in large codebases, to help one ease
into an unknown environment.  Not to mention that Emacs recently merged
Eglot, so there seems to be acceptance of LSP (and in turn, LSP servers
that utilize compile commands) in other parts of GNU.  I'd like to
eventually see a GNU LSP implementation generic across all supported
languages, but I'm not sure I'm in the right to ask for that, since I
definitely wouldn't know where to even start implementing that yet ;).
There has been some effort by RMS to push GCC contributors towards
implementing a language server, but I can't find archives of this
discussion (maybe it was in private?), as well as an effort by dmalcom
to implement such a thing in GCC before, though I'm not sure where that
ended up going, the archives don't show much.

So, in conclusion, what do you think is the right approach?  Do you
think a feature like this should even exist in the GNU Build System (or,
Make more broadly)?  Do you have any implementation ideas?

[1] https://ninja-build.org/manual.html#_extra_tools (compdb)
[2] https://github.com/rizsotto/Bear

Hope to hear from you soon, have a lovely night.
-- 
Arsen Arsenović

signature.asc
Description: PGP signature