[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Undesirable overhead in GNU Make 4.2.1.

From: Paul Smith
Subject: Re: Undesirable overhead in GNU Make 4.2.1.
Date: Tue, 06 Feb 2018 09:38:59 -0500

On Tue, 2018-02-06 at 15:12 +0100, Hans Ottevanger wrote:
> It appears that this behavior is caused by the need to maintain the
> variable MAKEFILE_LIST. The names of all included makefiles are
> appended one by one to the ever growing string representing this
> variable.This also explains why memcpy() and strlen() champion the
> profiled runs we did and also the quadratic behavior we observed.
> When we change the source (read.c) so that the MAKEFILE_LIST is not
> maintained at all, the overhead is gone and we are back to about 12
> seconds again.

Another option would be to read all the files in as a single bloc.

What about doing something like this; replace whatever loads all the .d
files, maybe something like this:

  include $(wildcard *.d)

with something like this instead:

  _ := $(shell cat *.d > $(TMP)/all.d)
  include $(TMP)/all.d

I haven't tried this but it should work, I think.

> Are there any ideas (or opinions) about how to get rid of this type
> of undesirable overhead? One of our own ideas is to provide a special
> variant of -include (called e.g. dinclude?), intended to include just
> .d files, without maintaining MAKEFILE_LIST. On the other hand it
> might be feasible to improve the efficiency of appending to a
> variable. Any suggestions?

My preference would be to allow MAKEFILE_LIST to be "special" such that
it's updated when expanded, rather than being updated as files are read
in.  That way the value will always be accurate but the price for
computing the string won't be paid until/unless the variable is

Of course, we'd still need to maintain an ordered list of pointers to
files that will be added to MAKEFILE_LIST when it's requested, but we
already need to maintain file structures for each of those so that
won't be too much overhead.

The tricky part is creating an infrastructure for these types of
variables.  There are other similar ones, such as .VARIABLES, which
could benefit from similar treatment.  We have a flag that marks
variables that behave specially when set; this would be a flag that
marks variables that behave specially when expanded.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]