Re: new feature idea: ingesting processed rulesets

[David A. Wheeler]

> 
> On Tue, 23 May 2023 12:07:51 -0400
> "David A. Wheeler" <dwheeler@dwheeler.com> wrote: 
>> The solution is to *NOT* use recursive make. Have *ONE* process run the
>> makefile, with the correct data. Now you can enable parallel jobs, and have
>> it run really quickly, because the make process has the correct information.
>> Using this approach you can routinely run large make jobs in a fraction of a
>> second.

> On May 23, 2023, at 1:16 PM, Henrik Carlqvist <hc94@poolhem.se> wrote:
> If you prefer to do it that way, you can have your top Makefile include all
> sub.mk files in the directory structure that it is able to find.

Yes, absolutely. The point is that you want a single process to have all the 
information,
so it can perform the correct actions... multiple *files* is fine. As noted in
"Recursive Make Considered Harmful" by Peter Miller
<"Recursive Make Considered Harmful" by Peter Miller, e.g.:
https://accu.org/journals/overload/14/71/miller_2004//>:
"The above [long list of problems] is based on one simple action: the DAG was 
artificially separated into incomplete pieces. This separation resulted in all 
of the problems familiar to recursive make builds.
Did make get it wrong? No. This is a case of the ancient GIGO principle: 
Garbage In, Garbage Out . Incomplete Makefiles are wrong Makefiles.
To avoid these problems, don’t break the DAG into pieces; instead, use one 
Makefile [process] for the entire project...
modern make implementations have include statements. . By including a relevant 
fragment from each module, the total size of the Makefile and its include files 
need be no larger than the total size of the Makefile s in the recursive case."

I find that "whole project" Makefiles are *incredibly* fast. I auto-generate 
dependencies from source in larger projects, but store them in ".d" files that 
depend on their sources; thus, regeneration of dependencies only happens when 
it needs to happen. Modern filesystems (with modern caches) can load all those 
make fragments in a blink of an eye. And they completely eliminate the endless 
problems of recursive make approaches.

Here are some things you might find useful for using make well:
"A Super-Simple Makefile for Medium-Sized C/C++ Projects" 
https://spin.atomicobject.com/2016/08/26/makefile-c-projects/
http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
My make-booster: https://github.com/david-a-wheeler/make-booster

--- David A. Wheeler