[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: automake -vs- huge projects (1st patch)
From: |
Alexandre Duret-Lutz |
Subject: |
Re: automake -vs- huge projects (1st patch) |
Date: |
Sun, 28 Dec 2003 03:25:05 +0100 |
User-agent: |
Gnus/5.1003 (Gnus v5.10.3) Emacs/21.3 (gnu/linux) |
>>> "adl" == Alexandre Duret-Lutz <address@hidden> writes:
[...]
adl> 1999-11-22 Tom Tromey <address@hidden>
adl>
adl> * automake.in (handle_single_transform_list): Generate explicit
adl> rule for subdir objects. Fixes new addition to subobj.test.
[...]
adl> The other side of the coin is that dependency tracking will not
adl> work anymore, because the dependency stuff for subdir/X.o should
adl> go into subdir/.deps/X.Po but the default suffix rule will put
adl> it in ./.deps/subdir/X.Po.
[...]
Tom> Hmm, maybe this is the issue from way back.
adl> I'm not sure about this. In the past dependencies were put in
adl> .deps/subdir/X.Po (i.e. where the suffix rule--at least
adl> today--would put them). You moved them to subdir/.deps/X.Po on
adl> 2002-01-20 for PR/224.
I've just noticed that the above change of 1999-11-22 occurs
just after the merge of user-dep-gen-branch. So perhaps you
were right to think the issue was related to dependency
tracking...
Anyway, here is a first patch that disables explicit rules for
subdir-objects without per-target flags.
Random notes:
- folding the depfile&tmpdepfile computation into depcomp shorten
the "screen garbage" resulting from the invocation of depcomp
by one line. This applies to all non-fastdep compilation,
not only subdir-objects.
- computing $depbase on-the-fly in fastdep rules adds one line
of output, so I've arranged things so that it is only used
when subdir-objects is set.
- the test suite passes with GNU Make, and I've run the 14 tests
that uses subdir-objects with BSD Make (will start a full
check momently). Apart from this I have not yet tested this
on a real project. I'm hoping you can test this on libgcj.
- If you omit the NEWS chunk, the patch should also apply to
automake-1.8 vanilla.
--
Alexandre Duret-Lutz
2003-12-28 Alexandre Duret-Lutz <address@hidden>
* automake.in (handle_languages): Define %DEPBASE% conditionally
on subdir-objects. Define SUBDIROBJ. Do not clean *_.c files
here ...
(lang_c_finish): ... do it here.
(handle_single_transform_list): Do not output specific rules for
subdir-objects files which are not renamed. This should reduce
the size of Makefiles with lots of subdirectory sources.
* lib/depcomp: Use $DEPDIR to compute dependency directories.
* lib/am/depend2.am (%EXT%.o, %EXT%.obj, %EXT%.lo): Adjust call
to depcomp. Compute depbase on-the-fly in generic fastdep rules
for subdir-objects.
* tests/ansi9.test: Do not grep for an explicit rule that
we no longer expect. Really run $MAKE to make sure the chain
of rules works.
* tests/yacc5.test: Do not grep for an explicit rule that
we no longer expect. Adjust to use set -e.
Index: NEWS
===================================================================
RCS file: /cvs/automake/automake/NEWS,v
retrieving revision 1.258
diff -u -r1.258 NEWS
--- NEWS 26 Dec 2003 03:57:59 -0000 1.258
+++ NEWS 28 Dec 2003 01:54:04 -0000
@@ -1,6 +1,9 @@
New in 1.8a:
-* Nothing yet.
+* Inference rules are used to compile sources in subdirectories when the
+ `subdir-objects' option is used and no per-target flags are used. This
+ should reduce the size of some projects a lot, because Automake used to
+ output an explicit rule for each such object in the past.
New in 1.8:
Index: automake.in
===================================================================
RCS file: /cvs/automake/automake/automake.in,v
retrieving revision 1.1528
diff -u -r1.1528 automake.in
--- automake.in 27 Dec 2003 15:39:45 -0000 1.1528
+++ automake.in 28 Dec 2003 01:54:09 -0000
@@ -1087,6 +1087,27 @@
# This is not used by depend2.am.
my $der_ext = (&{$lang->output_extensions} ($ext))[0];
+ # When we output an inference rule like `.c.o:' we
+ # have two cases to consider: either subdir-objects
+ # is used, or it is not.
+ #
+ # In the latter case the rule is used to build objects
+ # in the current directory, and dependencies always
+ # go into `./$(DEPDIR)/'. We can hard-code this value.
+ #
+ # In the former case the rule can be used to build
+ # objects in sub-directories too. Dependencies should
+ # go into the appropriate sub-directories, e.g.,
+ # `sub/$(DEPDIR)/'. The value of this directory
+ # need the be computed on-the-fly.
+ #
+ # DEPBASE holds the name of this directory, plus the
+ # basename part of the object file (extensions Po, TPo,
+ # Plo, TPlo will be added later as appropriate). It is
+ # either hardcoded, or a shell variable (`$depbase') that
+ # will be computed by the rule.
+ my $depbase =
+ option ('subdir-objects') ? '$$depbase' : '$(DEPDIR)/$*';
$output_rules .=
file_contents ($rule_file,
new Automake::Location,
@@ -1095,11 +1116,7 @@
'DERIVED-EXT' => $der_ext,
- # In this situation we know that the
- # object is in this directory, so
- # $(DEPDIR) is the correct location for
- # dependencies.
- DEPBASE => '$(DEPDIR)/$*',
+ DEPBASE => $depbase,
BASE => '$*',
SOURCE => '$<',
OBJ => '$@',
@@ -1108,7 +1125,8 @@
COMPILE => '$(' . $lang->compiler . ')',
LTCOMPILE => '$(LT' . $lang->compiler . ')',
- -o => $output_flag);
+ -o => $output_flag,
+ SUBDIROBJ => !! option 'subdir-objects');
}
# Now include code for each specially handled object with this
@@ -1209,11 +1227,6 @@
if $source !~ /\$U/;
(my $source_ = $source) =~ s/\$U/_/g;
- # Explicitly clean the _.c files if they are in
- # a subdirectory. (In the current directory they get
- # erased by a `rm -f *_.c' rule.)
- $clean_files{$source_} = MOSTLY_CLEAN
- if $objdir ne '.';
# Output an additional rule if _.c and .c are not in
# the same directory. (_.c is always in $objdir.)
if ($objdir ne $srcdir)
@@ -1531,13 +1544,37 @@
$object = $directory . '/' . $object;
}
- # If doing dependency tracking, then we can't print
- # the rule. If we have a subdir object, we need to
- # generate an explicit rule. Actually, in any case
- # where the object is not in `.' we need a special
- # rule. The per-object rules in this case are
- # generated later, by handle_languages.
- if ($renamed || $directory ne '')
+ # If the object file has been renamed (because per-target
+ # flags are used) we cannot compile the file with an
+ # inference rule: we need an explicit rule.
+ #
+ # If the source is in a subdirectory and the object is in
+ # the current directory, we also need an explicit rule.
+ #
+ # If both source and object files are in a subdirectory
+ # (this happens when the subdir-objects option is used),
+ # then the inference will work.
+ #
+ # The latter case deserves a historical note. When the
+ # subdir-objects option was added on 1999-04-11 it was
+ # thought that inferences rules would work for
+ # subdirectory objects too. Later, on 1999-11-22,
+ # automake was changed to output explicit rules even for
+ # subdir-objects. Nobody remembers why, but this occured
+ # soon after the merge of the user-dep-gen-branch so it
+ # might be related. In late 2003 people complained about
+ # the size of the generated Makefile.ins (libgcj, with
+ # 2200+ subdir objects was reported to have a 9MB
+ # Makefile), so we now rely on inference rules again.
+ # Maybe we'll run across the same issue as in the past,
+ # but at least this time we can document it. However since
+ # dependency tracking has evolved it is possible that
+ # our old problem no longer exists.
+ # Using inference rules for subdir-objects has been tested
+ # with GNU make, Solaris make, Ultrix make, BSD make,
+ # HP-UX make, and OSF1 make successfully.
+ if ($renamed ||
+ ($directory ne '' && ! option 'subdir-objects'))
{
my $obj_sans_ext = substr ($object, 0,
- length ($this_obj_ext));
@@ -4935,6 +4972,12 @@
push (@objects, $base . '_.$(OBJEXT)');
push (@objects, $base . '_.lo')
if var ('LIBTOOL');
+
+ # Explicitly clean the _.c files if they are in a
+ # subdirectory. (In the current directory they get erased
+ # by a `rm -f *_.c' rule.)
+ $clean_files{$base . "_.c"} = MOSTLY_CLEAN
+ if dirname ($base) ne '.';
}
# Make all _.o (and _.lo) files depend on ansi2knr.
Index: lib/depcomp
===================================================================
RCS file: /cvs/automake/automake/lib/depcomp,v
retrieving revision 1.47
diff -u -r1.47 depcomp
--- lib/depcomp 9 Nov 2003 00:10:50 -0000 1.47
+++ lib/depcomp 28 Dec 2003 01:54:09 -0000
@@ -1,7 +1,7 @@
#! /bin/sh
# depcomp - compile a program generating dependencies as side-effects
-scriptversion=2003-11-08.23
+scriptversion=2003-12-27.19
# Copyright (C) 1999, 2000, 2003 Free Software Foundation, Inc.
@@ -43,6 +43,7 @@
depmode Dependency tracking mode.
source Source file read by `PROGRAMS ARGS'.
object Object file output by `PROGRAMS ARGS'.
+ DEPDIR directory where to store dependencies.
depfile Dependency file to output.
tmpdepfile Temporary file to use when outputing dependencies.
libtool Whether libtool is used (yes/no).
@@ -61,7 +62,6 @@
echo "depcomp: Variables source, object and depmode must be set" 1>&2
exit 1
fi
-# `libtool' can also be set to `yes' or `no'.
if test -z "$depfile"; then
base=`echo "$object" | sed -e 's,^.*/,,' -e 's,\.\([^.]*\)$,.P\1,'`
@@ -69,8 +69,7 @@
if test "$dir" = "$object"; then
dir=
fi
- # FIXME: should be _deps on DOS.
- depfile="$dir.deps/$base"
+ depfile="$dir$DEPDIR/$base"
fi
tmpdepfile=${tmpdepfile-`echo "$depfile" | sed 's/\.\([^.]*\)$/.T\1/'`}
Index: lib/am/depend2.am
===================================================================
RCS file: /cvs/automake/automake/lib/am/depend2.am,v
retrieving revision 1.56
diff -u -r1.56 depend2.am
--- lib/am/depend2.am 8 Nov 2003 14:03:35 -0000 1.56
+++ lib/am/depend2.am 28 Dec 2003 01:54:09 -0000
@@ -56,7 +56,8 @@
## compile rules so that they are output on a single line (instead of 5)
## would be a good compromise. Actually we use two line rather than one,
## because this way %SOURCE% is always located at the end of the first
-## line and is therefore easier to spot.
+## line and is therefore easier to spot. (We need an extra line when
+## depbase is used.)
?GENERIC?%EXT%.o:
?!GENERIC?%OBJ%: %SOURCE%
@@ -64,13 +65,13 @@
## In fast-dep mode, we can always use -o.
## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
?!GENERIC? if %COMPILE% -MT %OBJ% -MD -MP -MF "%DEPBASE%.Tpo" %-c% -o
%OBJ% `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE%; \
+?SUBDIROBJ??GENERIC? depbase=`echo %OBJ% | sed
's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`; \
?GENERIC? if %COMPILE% -MT %OBJ% -MD -MP -MF "%DEPBASE%.Tpo" %-c% -o
%OBJ% %SOURCE%; \
then mv -f "%DEPBASE%.Tpo" "%DEPBASE%.Po"; else rm -f "%DEPBASE%.Tpo";
exit 1; fi
else !%FASTDEP%
if %AMDEP%
source='%SOURCE%' object='%OBJ%' libtool=no @AMDEPBACKSLASH@
- depfile='%DEPBASE%.Po' tmpdepfile='%DEPBASE%.TPo' @AMDEPBACKSLASH@
- $(%FPFX%DEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ DEPDIR=$(DEPDIR) $(%FPFX%DEPMODE) $(depcomp) @AMDEPBACKSLASH@
endif %AMDEP%
if %?GENERIC%
?-o? %COMPILE% %-c% %-o% %OBJ% %SOURCE%
@@ -88,13 +89,13 @@
## In fast-dep mode, we can always use -o.
## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
?!GENERIC? if %COMPILE% -MT %OBJOBJ% -MD -MP -MF "%DEPBASE%.Tpo" %-c% -o
%OBJOBJ% `if test -f '%SOURCE%'; then $(CYGPATH_W) '%SOURCE%'; else
$(CYGPATH_W) '$(srcdir)/%SOURCE%'; fi`; \
+?SUBDIROBJ??GENERIC? depbase=`echo %OBJ% | sed
's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`; \
?GENERIC? if %COMPILE% -MT %OBJOBJ% -MD -MP -MF "%DEPBASE%.Tpo" %-c% -o
%OBJOBJ% `$(CYGPATH_W) '%SOURCE%'`; \
then mv -f "%DEPBASE%.Tpo" "%DEPBASE%.Po"; else rm -f "%DEPBASE%.Tpo";
exit 1; fi
else !%FASTDEP%
if %AMDEP%
source='%SOURCE%' object='%OBJOBJ%' libtool=no @AMDEPBACKSLASH@
- depfile='%DEPBASE%.Po' tmpdepfile='%DEPBASE%.TPo' @AMDEPBACKSLASH@
- $(%FPFX%DEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ DEPDIR=$(DEPDIR) $(%FPFX%DEPMODE) $(depcomp) @AMDEPBACKSLASH@
endif %AMDEP%
if %?GENERIC%
?-o? %COMPILE% %-c% %-o% %OBJOBJ% `$(CYGPATH_W) '%SOURCE%'`
@@ -112,15 +113,14 @@
if %FASTDEP%
## In fast-dep mode, we can always use -o.
## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
-?GENERIC? if %LTCOMPILE% -MT %LTOBJ% -MD -MP -MF "%DEPBASE%.Tpo" %-c% -o
%LTOBJ% %SOURCE%; \
-## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
?!GENERIC? if %LTCOMPILE% -MT %LTOBJ% -MD -MP -MF "%DEPBASE%.Tpo" %-c% -o
%LTOBJ% `test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE%; \
+?SUBDIROBJ??GENERIC? depbase=`echo %OBJ% | sed
's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`; \
+?GENERIC? if %LTCOMPILE% -MT %LTOBJ% -MD -MP -MF "%DEPBASE%.Tpo" %-c% -o
%LTOBJ% %SOURCE%; \
then mv -f "%DEPBASE%.Tpo" "%DEPBASE%.Plo"; else rm -f "%DEPBASE%.Tpo";
exit 1; fi
else !%FASTDEP%
if %AMDEP%
source='%SOURCE%' object='%LTOBJ%' libtool=yes @AMDEPBACKSLASH@
- depfile='%DEPBASE%.Plo' tmpdepfile='%DEPBASE%.TPlo' @AMDEPBACKSLASH@
- $(%FPFX%DEPMODE) $(depcomp) @AMDEPBACKSLASH@
+ DEPDIR=$(DEPDIR) $(%FPFX%DEPMODE) $(depcomp) @AMDEPBACKSLASH@
endif %AMDEP%
## We can always use `-o' with Libtool.
?GENERIC? %LTCOMPILE% %-c% -o %LTOBJ% %SOURCE%
Index: tests/ansi9.test
===================================================================
RCS file: /cvs/automake/automake/tests/ansi9.test,v
retrieving revision 1.2
diff -u -r1.2 ansi9.test
--- tests/ansi9.test 14 Nov 2003 21:25:58 -0000 1.2
+++ tests/ansi9.test 28 Dec 2003 01:54:09 -0000
@@ -1,5 +1,5 @@
#! /bin/sh
-# Copyright (C) 2002 Free Software Foundation, Inc.
+# Copyright (C) 2002, 2003 Free Software Foundation, Inc.
#
# This file is part of GNU Automake.
#
@@ -22,6 +22,7 @@
# can still be deansified.
# Report from Paul D. Smith.
+required=gcc
. ./defs || exit 1
set -e
@@ -30,6 +31,7 @@
AC_PROG_CC
AM_PROG_CC_C_O
AM_C_PROTOTYPES
+AC_OUTPUT
EOF
cat > Makefile.am << 'END'
@@ -38,10 +40,29 @@
loadavg_SOURCES = loadavg.c
loadavg_CFLAGS = -DTEST
sub_sub_SOURCES = sub/sub.c
+# Force ansi2knr's use, regardless of the compiler.
+U=_
+ANSI2KNR=./ansi2knr
END
+cat > loadavg.c << 'END'
+int
+main ()
+{
+ return 0;
+}
+END
+
+mkdir sub
+cp loadavg.c sub/sub.c
+
$ACLOCAL
-$AUTOMAKE --add-missing
+$AUTOCONF
+$AUTOMAKE --add-missing -Wno-override
$FGREP 'loadavg-loadavg$U.o: loadavg$U.c' Makefile.in
-$FGREP 'sub/sub$U.o: sub/sub$U.c' Makefile.in
+# The following rule should not exists, because the
+# default .o.c: inference rule is enough.
+$FGREP 'sub/sub$U.o: sub/sub$U.c' Makefile.in && exit 1
+./configure
+$MAKE sub/sub_.c
Index: tests/yacc5.test
===================================================================
RCS file: /cvs/automake/automake/tests/yacc5.test,v
retrieving revision 1.6
diff -u -r1.6 yacc5.test
--- tests/yacc5.test 14 Nov 2003 21:26:01 -0000 1.6
+++ tests/yacc5.test 28 Dec 2003 01:54:09 -0000
@@ -1,5 +1,5 @@
#! /bin/sh
-# Copyright (C) 2001, 2002 Free Software Foundation, Inc.
+# Copyright (C) 2001, 2002, 2003 Free Software Foundation, Inc.
#
# This file is part of GNU Automake.
#
@@ -23,6 +23,8 @@
. ./defs || exit 1
+set -e
+
cat > configure.in << 'END'
AC_INIT
AM_INIT_AUTOMAKE(nonesuch, nonesuch)
@@ -41,10 +43,10 @@
: > sub/maude.y
-$ACLOCAL || exit 1
-$AUTOMAKE -a || exit 1
+$ACLOCAL
+$AUTOMAKE -a
-grep '^maude\.c:' Makefile.in || exit 1
+grep '^maude\.c:' Makefile.in
## Try again with subdir-objects.
@@ -55,10 +57,11 @@
maude_SOURCES = sub/maude.y
END
-$ACLOCAL || exit 1
-$AUTOMAKE -a || exit 1
+$ACLOCAL
+$AUTOMAKE -a
-grep '^sub/maude\.c:' Makefile.in || exit 1
+# No rule needed, the default .y.c: inference rule is enough.
+grep '^sub/maude\.c:' Makefile.in && exit 1
## Try again with per-exe flags.
@@ -80,6 +83,4 @@
grep 'maudec' Makefile.in && exit 1
# Make sure the .o file is required.
-grep '^am_maude_OBJECTS.*maude' Makefile.in || exit 1
-
-exit 0
+grep '^am_maude_OBJECTS.*maude' Makefile.in
- Re: automake -vs- huge projects, (continued)
- Re: automake -vs- huge projects, Alexandre Duret-Lutz, 2003/12/16
- Re: automake -vs- huge projects, Tom Tromey, 2003/12/16
- Re: automake -vs- huge projects, Alexandre Duret-Lutz, 2003/12/17
- Re: automake -vs- huge projects, Tom Tromey, 2003/12/30
- mailing list archives (Was: Re: automake -vs- huge projects), Alexandre Duret-Lutz, 2003/12/30
- Re: mailing list archives (Was: Re: automake -vs- huge projects), Thomas Dickey, 2003/12/30
- Re: mailing list archives, Alexandre Duret-Lutz, 2003/12/30
- Message not available
- Re: automake -vs- huge projects (1st patch),
Alexandre Duret-Lutz <=
- Re: automake -vs- huge projects (1st patch), Thomas Fitzsimmons, 2003/12/30
- Re: automake -vs- huge projects (1st patch), Alexandre Duret-Lutz, 2003/12/30
Re: automake -vs- huge projects, Ralf Corsepius, 2003/12/18