Re: [gnu-prog-discuss] Could automake-generated Makefiles required GNU make?

[Warren Young]

On 11/22/2011 8:50 AM, Bob Friesenhahn wrote:
> It would be useful to enumerate the user-visible benefits if Automake
> can depend on using GNU make.

- For me, the biggest potential benefit isn't to Automake, but to 
Autoconf: if it were reimplemented in terms of GNU make, you'd then get 
parallel ./configure for free.  (e.g. gmake -j config)

One way to do that would be for each autoconf test to write out a file 
containing the results of the test, so make's dependency graph walker 
could figure out which tests can be run independently of each other and 
correctly order the rest.  I'm envisioning that you'd have a 
subdirectory (e.g. "accache") with one file in it per configure test, 
essentially replacing config.cache.  That in turn means that this 
hypothetical "Autoconf 3: Return of the Code Jedi" would give you test 
result caching for free.

Keep in mind, we aren't far from seeing 8-core *laptops*.  My simplest 
autotools based projects take about 5 seconds to reconfigure on my 
fastest box.  It would be nice if I could get that down below a second.


- Reduced build overhead.  GNU make can do many things internally that 
standard make(1) needs to shell out to do.  For example, given a list of 
*.cpp names, generate *.o:

    OBJECTS=$(SOURCES:.cpp=.o)

You might think this is a bad example in the context of this discussion, 
since Automake can do that conversion itself, since it makes you give it 
the list of source files.  However, I expect that compressing the same 
effect into a single line would save time due to the lower I/O and 
parsing overheads.

These overheads can stack up.

One case that's so easy to see it would be a straw-man if it weren't for 
the fact that so many people use it, is Cygwin.  The autotools are 
*sssslloooww* on Cygwin due to the lack of a native fork() type facility 
in Windows.  A lot of that overhead would go away if the generated 
Makefiles didn't have to shell out as often to get things done, because 
they could use internal GNU make facilities instead.

Even on a fast modern Unixy box, you can see measurable overheads due to 
the autotools.  You've probably watched some source-based package system 
at work before.  (e.g. FreeBSD ports, Gentoo Linux, Fink, MacPorts, 
Homebrew...)  Wouldn't you say the autotools are responsible for ~5% of 
the time spent in such systems?  With OSes that ship binary packages 
instead, well, *someone* has to build them.  I seem to recall that 
full-featured modern OSes like RHEL take days, maybe weeks to completely 
rebuild from source.  Outside that rarefied world, you have more common 
things like continuous integration servers.  If someone knocks 5% down 
to 4%, they'd save the world hours or days per year per machine.

There are probably dozens of other ways to reduce the size, parse time, 
and I/O overhead of generated Makefiles, once the autotools can count on 
having GNU make features.  Some follow:


- Build commands could use the $^ or $+ automatic variable instead of an 
explicit repetition of the prereq list.


- Some variables could use GNU make's "simply-expanded variable" syntax 
(FOO := value) to avoid having to reparse and recalculate the value on 
each use.  This can give big savings, particularly when the value is 
calculated by an external shell script and the value can be counted on 
not to change during a given build.


- You can save one shell command in this common pattern in recursive 
Makefiles by use of GNU make's nonstandard -C flag:

    sometarget:
        cd subdir ; $(MAKE) sometarget


- GNU make's text transformation functions can substitute for many 
common uses of sed, awk and handrolled shell code:

    https://www.gnu.org/s/hello/manual/make/Functions.html#Functions

Again, this is not necessarily just of value to those developing the 
autotools.  Fewer fork()s mean the autotools run faster, which can add 
up to user-visible build time savings in many cases.


- GNU make makes it easier to do things like specify the source files 
for a build with wildcards, rather than the current method where you 
have to list them separately.  I've read emails on this list a few times 
where people tried to justify this current requirement, but they've been 
so unconvincing I don't even remember the justifications.  It would be 
much nicer for us Automake users if we could say something like this:

    myprog_SOURCES=myprog/%.cpp myprog/%.h

Not only is that shorter than actually listing all the files, it means 
that whenever I add a new module to myprog/, it gets built and linked in 
automatically.


- GNU make's more powerful '%' form of build rules would let Automake 
implement some things as easy defaults.  For example, there's the common 
wish for object files to go somewhere other than the default.  Automake 
offers subdir-objects and separate builddirs as two ways to achieve 
this, but I think it would be nicer if Automake generated something like 
this by default instead:

    .objs/%.o: src/%.cpp
        $(CC) $(CFLAGS) -o $@ $<

(If you think that's wrong, why is libtool's .libs correct?)

It would also be easy to implement separate debug and release builds 
this way.  The best way I can think of to do that with the current 
autotools is to have two separate builddirs.  Bleah.

It might also be possible to make cross-compilation cleaner in a similar 
way.  (e.g. *.o goes to .objs/system-triplet-here/*.o)

Defaults matter.  The fact that you *can* make the autotools do a thing 
today is a poor substitute for a system that falls out of the box doing 
the right thing from the start.

GNU make's more powerful syntaxes would likely change how Automake gets 
developed, resulting in features that wouldn't have been created otherwise.



IMHO, the main reason not to do any of this is that the BSDs still 
default to BSD make.

Everywhere else, GNU make is the default, as evidenced by the fact that 
the O'Reilly book on make stopped covering anything but GNU make in its 
third edition.  That book came out 7 years ago, and was late to the 
party, as books often are.