Re: slow makefiles are getting even slower

[Zdenek Kabelac]

> While the complexity has been increasing over time, I'd like to point
> out that some cases have been massively improved: for example, the
> Gstreamer makefile.am used to take hours (literally) to be processed -
> this has been fixed in more recent Automakes.
> 

Well trust me I've spent weeks to give some decent speed and compatibility
to avifile makefiles - I've tried to minimalize the size and the amount
of variables - there are  no  @VARS@ anywhere in the Makefile.am
(So in this case I think it could be possible to simply skip
generation of  Makefile.in files in my eyes.)

> However, it would be nice to improve Automake's complexity in more
> general cases.

I could imagine an extra option to auto* building process
(or it could recognize this situation itself (no @VARS@).

For this case it might use completely different strategy - i.e. 
it could prepare just one  file with variable-replacement and 
this file will be joined with the rest of Makefile.in (or even
directly  Makefile.am - as configure could handle this)
And so generated Makefile.in might have just rules for
the source (and I guess this could be created on the fly - that's
why I believe no Makefile.in files are actually necessary)

Of course this would require better connection between autoconf & automake
As the  AC_OUTPUT  would be using Makefile.am directly)


To your proposition of usage of just one Makefile.am:
I think it's unacceptable to create something incompatible - I simply 
cannot force users to install the latest auto* stuff on their machines
as this would certainly broke the 'building capabilities' of their
systems - and as I do prefer that users are using CVS - as making
releases is too much time consuming - I could not say:  sorry either
install newest auto* or use old tarballs - I'd be doing nothing
else then tarballs...

I think that current slowness is somewhere inside auto* architecture -
when I run strace - I could easily see that for each created
Makefile there is opened many many files - I think the the first
steps should be made in this field instead of creating new 
incompatible Makefile layouts - also for each Makefile.am the
variables are again and again parsed and new finite automata is build - 
why ???  (thought this probably goes to autoconf/configure developers)

Also I guess that adding option for let's say  --for-GNU
or something like this would help as well - I really do not need
backward compatibility with some 15-20 year old Sparcs systems so
if that would make things easier I'd giveup this compatibility
for 30x times faster processing time - my project wouldn't run there
anyway...


There are also other numerous problems - like the libtool - it's
increasing the compilation time by factor 4

i.e. library ffmpeg compiled with libtools -  >1minute
compiled with plain Makefile (with the same gcc flags)  15 seconds
(and with dependencies of course)

I could do some tricks & magic - include all file into one and
compile the library this way - which will run ~13.5 seconds 
however then I loose the chance of the fast recompilation
when I change one file so it's usable for one-time build only.

(btw these 15 seconds are with quite fast 1.4GHz Athlon - it's much slower
with Celerons - and when you take into acount that as a developer
your are recompiling things quite often it becomes noticable)

Also it should be probably somewhat more complex debate between
auto* developer - as automake is just one part of it - but
without strong cooperation between auto* developers I don't see
any solution (well actually the one - to go in mplayer's way and
write everything myself - but as I've already spent ages with
configure.in scripts I'd like to see that it's useful for something -
and as I maintain also some other projects and the basic idea
behind auto* is good - I would rather like to help with
fixing current auto* tools.)

(BTW - configure time for mplayer  5 seconds - for avifile 30 seconds
and both are checking mostly for the same thing (thought check for Qt
is a bit longer (gcc faults)) - but anyway 20 seconds are spend with
'creating Makefile' - it's 58 x ~12KB files - 

so I could decompress about 300 DivX video frames on this box
or create about 3  12KB Makefiles per second on this Athlon 
- I guess something is not right here...

-- 
  .''`.    Zdenek Kabelac  address@hidden, users.sf.net, fi.muni.cz}
 : :' :          Debian GNU/Linux maintainer - www.debian.{org,cz}
 `. `'  Modern processors are the most advanced heating systems around.
   `-                                     www.tomshardware.com