Re: Using -MMD instead of -MD for depndency generation

[Holger Freyther]

On 10/25/2010 08:09 AM, Ralf Wildenhues wrote:
> Hello Holger,
> 
> thanks for the report.
> 
> * Holger Freyther wrote on Sat, Oct 23, 2010 at 10:47:47PM CEST:
>> when typing make in the build directory of WebKit/GTK+ it takes multiple
>> minutes until the first file is compiled.
> 
> Can you go into (a lot) more detail here?  What happens during those
> minutes, which processes are running (look at ps output etc.)?

I am talking about executing make after the source has been built once. E.g.
there is only one file changed and I want to rebuild the library with this
single new file. My benchmark is to type 'time make' on unchanged source. From
my expectation it will parse the GNUmakefile and all the files (the .Plo, Po
with the dependencies) included by it, then it needs to verify the
dependencies of all these rules for the 'all' target and then exits. Right now
this takes 5:42m usertime (+- 10 seconds).


Is this
> cold-cache or hot-cache?  Only the first time the code is compiled, or
> the build tree is used, i.e., does it happen again after 'make clean'?
> Or are you talking about configure (-q?) run time here?


My system is a MSI X340 with a U7300  @ 1.30GHz and had about 2GB of RAM
available, it is running 2.6.36rc8 (but all kernels have shown this behavior)
on a Fedora F14 installation. GNU make is of version 3.82, automake is 1.11.1.
This is with a 'hot-cache' but with only 2GB of RAM not every file will fit
into the page cache. The disk speed is quite okay, slowed down a bit due 
dm-crypt.

The system is not loaded at all, it runs a wm, some gnome-terminals with
remote ssh sessions, when executing make it will be the process that takes 99%
of the CPU.


> 
> What system is this on, is it very loaded with other processes?
> Can you please post a link to the tarball you're compiling from, so we
> can try to reproduce this?

Just grab a recent WebKit/GTK+ tarball[1], and run configure on the sources,
the build will take quite a bit of time.

> 
>> I would like to reduce the size of the
>> .Plo (dep files) in my build tree and build them with -MMD instead of -MD. Is
>> there a way to instruct automake to use -MMD for gcc3 like depmode?
> 
> No, there isn't, but also, in my experience, that will not help all that
> much at all.  All files listed as dependencies with -MD will be opened
> anyway by the compiler eventually, and a list of files usually won't
> amount to sizeable file size.
> 
> I'm still guessing that it's something else that slows down your build
> by so much.

Sure, it is not the only reason why it is slow. I have made a micro benchmark
with -MD vs. -MMD and typing 'time make' on a changed source tree..

-MD:
real    5m37.810s
user    0m51.572s
sys     4m45.353s

(best measurement, worst is 5m42.810s)

-MMD:
real    4m16.368s
user    0m41.604s
sys     3m33.983s


So it appears by reducing the number of rules and files things will be faster
(no real surprise here). It would be nice if you could help us to reduce the
complexity of the rules created, but on the other hand we really do have many
many files.

any help is more than appreciated
        holger





[1] http://webkitgtk.org/webkit-1.3.5.tar.gz