Re: Any way to get rid of -MP parameter to gcc for dependency creation?

[Xan Lopez]

On Thu, Jan 6, 2011 at 11:08 PM, Ralf Wildenhues <address@hidden> wrote:
>> If I'm right this means I've moved from having the vast majority of
>> the time in user CPU time to a 60/40 split, which I guess means by now
>> we are spending a lot of time stating files (?).
>
> Can you send sysperf output for this as well, please?  Thanks.
>

Here it is http://people.gnome.org/~xan/make2.txt.gz

>
> The makefile + included files that I've looked at (where not all of
> webkit is built) already weigh in at some 90 MB total size, the very
> bulk of which in the .deps/*.P*o files just as you mentioned.  Getting
> that down to 50some MB by not using -MP seems desirable iff make really
> is spending most of its time reading, parsing, and not stating.
> Of course it also seems desirable to not lose the deleted header hack.

Perhaps a silly question: could it be easier to just post-process the
include of all the Plo files to get rid of the duplicates? Since the
vast majority of the data is duplicated among the files it seems a
reasonable thing to do.

> This is not the first issue where it would be nice to special-case GNU
> make for better efficiency and better warnings.  I think Automake should
> provide a special GNU_MAKE conditional (maybe name it AM_GNU_MAKE or so,
> since the other name is already taken by the gnulib module gnu-make),
> and allow for some special behavior being triggered by that:
>

(...)

>
> I think I might be able to do a proto patch over the weekend.

In any case if there are other areas where the output could be
optimized for GNU make systems I think that makes a lot of sense too
(and I recall it was something already requested in, say, that old
thread about huge projects).

>
>
> For the remaining 14s, there is one more uuugly hack I can think of:
> get rid of the recursive make invocation that is done in the 'all' rule.
> The reason it exists is so $(BUILT_SOURCES) and 'autotoolsconfig.h' are
> updated before the "normal" prerequisites of 'all' are dealt with.
> Can you require GNU make 3.80 or newer?  Then I think the following
> (completely untested!) addition should cut the time almost in half:
>
> # Ugly hack: override automake's definition of 'all' in order to
> # avoid a make recursion.
> # This requires order-only prerequisites (GNU make 3.80 or newer).
> all: all-am
> %.$(OBJEXT): | $(BUILT_SOURCES) autotoolsconfig.h
> %.lo: | $(BUILT_SOURCES) autotoolsconfig.h

I've tried this and I see no significant win, perhaps a solid 0.2,
0.15 difference (the timing is variable so it's difficult to measure
with precision wins that small).

>
>
> After that, it's probably time to start writing a beta build system.
>
> BTW, it seems webkit also has a cmake-based build system (right?).
> Does what it creates fare better?

I haven't tested it personally, but I can ask. What I know is that
Chromium uses gyp, which on Linux generates Makefiles, and they claim
their null-build time is pretty much zero (not sure on which machine,
though, so perhaps that's only the case for some huge iron). So it
would seem that whatever our problems GNU make shouldn't be one of
them.

What I can tell you is that our autotools setup is known for being the
slowest of the lot :/ (vs Xcode, VisualStudio, CMake, Gyp, ...)

>
> Cheers,
> Ralf
>