Re: Library dependency files

[Todd Showalter]

On Thu, Apr 29, 2010 at 1:57 PM, Erik Rull <address@hidden> wrote:

> Very interesting - how does your make rule look like for this case?

@makedepend $(CFLAGS) $(SRCS)

    The single invocation of makedepend with all sources allows it to
cache aggressively.

> My "doing nothing" takes currently ~ 4 seconds - how large is your project?

    ~200K lines of source across ~500 .c and .h files.  Any given
build uses roughly 95% of the sources.

> I'm compiling also for different cpu targets - a short taste of your
> makefile would be interesting.

    I think I've described the scheme here before, but the basic idea
is that we have our source tree organized in a specific way:

Kernel/
   Graphics/
      arch.OpenGL/
      arch.NintendoWii/
      arch.SonyPSP/
      ...
   ...

    Each directory has a file called "files.make", which contains a
list of the source files that are valid for that directory.

    We have a tool ("BuildList").  You tell it where the source tree
is, and what architecture flags it should use.  It walks the source
tree looking for files.make files, and builds a list of sources.  If
(for example) we have a case like this:

Math/Matrix44.c
Math/arch.NintendoDS/Matrix44.c

    and "NintendoDS" is one of the architecture flags set for this
build, "Math/Matrix44.c" is discarded and
"Math/arch.NintendoDS/Matrix44.c" is added to the list of sources.  If
no architecture flags match the subdirectories, the file in the root
is taken.  This lets us share most of the source code between our
architectures without having to resort to #ifdef very often.

    Once the source list is built, we hand that to makedepend to
generate a dependency file, and append the source list to it.  Note
that we don't call makedepend per-file, we call it once with the
complete source list so it doesn't have to re-parse all the header
files for each source.

    At this point, we have a complete file containing (1) all the
source files, (2) all the object files, which we've marked up to build
in $(OUTDIR)/$(ARCH)/Objects, [1], and (3) all the dependency info.
We recurse to another instance of make, and hand it the complete
recipe along with the arch-specific build instructions.

    So, makedepend gets invoked once with all the sources, and when
make goes to actually build the project it has everything it needs in
one file.  This has given us fairly quick builds, even on cygwin.[2]

                                                                  Todd.

[1] We actually do slightly more than this; the object files also get
tagged with their debug level, so multiple versions can coexist.  We
mirror the source tree in the object tree as well.  So, we'd have
(say) Output/arch.NintendoDS/Objects/Graphics/Sprite.opt.o

[2] I don't know why (though I have my suspicions) but process
creation time on cygwin seems to be dismal even on a fast machine with
no virus scanner running and the directory mounted exec.

-- 
 Todd Showalter, President,
 Electron Jump Games, Inc.