Re: Dump the database to a makefile and invoke make on the dumped makefile.

[Tim Murphy]

Hi,

That's well understood and the problem at the moment is that $(info)
statements etc are executed and "pollute" the output.

It does point out an area of scalability though because in our testing
it made an immense difference.   Some of our macros are perhaps 50k
(effectively although that's really a bunch of macros) and might get
invoked perhaps in the low 10s of thousands of times.  Parsing the
makefile becomes a bottleneck because make doesn't do this in parallel
and parsing get a worse and worse problem the larger your build if you
want make to have a "total dependency view".

In an ideal world, make would have a total view of dependencies but
read and parse makefiles in parallel.  It would also have ways of
"generalising" dependencies (trading precise-incrementalism for lower
memory usage and complexity as packaging systems do when any change
however insubstantial in package A triggers a rebuild of package B -
not precise but easy to compute). You'd choose what you wanted when
you started your build - it might even adapt depending on how big the
build turned out to be.

Another capability would be to learn how to deal cleverly with common
dependencies to save processing and memory.   e.g. when we use cpp to
generate deps on our code, make chokes with the huge number of them
but since many header files are common to all our code, one can chop
memory usage by about 5 if one finds the common dependencies and puts
them behind a phony target  and then only supplies "target: prereq"
rules for the dependencies that are unique to a particular file. e.g.

COMMON_HEADERS: frodo.h stdio.h
 myfile1.cpp: COMMON_HEADERS fred.h bob.h
 myfile2.cpp: COMMON_HEADERS alice.h jane.h tiffany.h
 myfile3.cpp: COMMON_HEADERS alice.h jane.h tiffany.h


This is also an example of an instance where one could trade precision
for lower complexity (big O) - one could make all headers common -
would cause your build system to rebuild some things totally
unnecessarily but it would require very little dependency checking and
in a large enough build with the right structure that might be best.
It would be more sophisticated to have different gradations e.g.
COMMON_H BOY_H GIRL_H and so on.  It would be much more useful if the
build tool could  work out how to make efficient sets like this
automatically. e.g.

COMMON_H: frodo.h stdio.h
BOY_H: fred.h bob.h
GIRL_H alice.h jane.h tiffany.h
 myfile1.cpp: COMMON_H BOY_H
 myfile2.cpp: COMMON_H GIRL_H
 myfile3.cpp: COMMON_H GIRL_H

If you know that program X uses library Y and you also know that
library Y "owns" a set of header files then you could have a great win
by introducing a dependency that says "object files of library X
depend on the headers of library Y".  This is also a generalisation
which  will often be untrue (not all headers in library Y affect all
object files in X) in a conservative way but it cuts down on the
amount of decisions f make has to manage by a lot and at a certain
scale it looks like a great idea even if people who build little
programs with 100 compiles would not be impressed with what it did to
their incremental builds.

One can do a lot of this my designing makefiles in a certain way but
then you are committed. Working well on the micro scale is sometimes
at odds with working well at the macro scale and an ideal make tool
would somehow help one to deal with that without one having to
maintain different and very complicated makefiles for each situation.

Regards,

Tim




On 26 February 2012 19:51, Paul Smith <address@hidden> wrote:
> On Sat, 2012-02-18 at 18:46 +0000, Tim Murphy wrote:
>> The option you need is:
>> -p, --print-data-base       Print make's internal database.
>
> This is the (only) place to start but just to warn you: the output of
> this option was not designed to be used this way and we don't guarantee
> that the format etc. will not change in future versions of make.  This
> output is intended for human consumption and debugging, not specifically
> for further automated processing.
>
> --
> -------------------------------------------------------------------------------
>  Paul D. Smith <address@hidden>          Find some GNU make tips at:
>  http://www.gnu.org                      http://make.mad-scientist.net
>  "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
>



-- 
You could help some brave and decent people to have access to
uncensored news by making a donation at:

http://www.thezimbabwean.co.uk/friends/