Re: Why no dependency tracking with preprocessed Fortran?

[Ralf Wildenhues]

* Daily, Jeff A wrote on Fri, Jun 25, 2010 at 08:32:46PM CEST:
> I think if we want dependency tracking as a side-effect, then the user
> must list their sources in a dependent order (which only they would
> know).  This is what I've had to do in my projects.  The downside is
> that -j can't be used safely.

That's a big downside, I would like to avoid that.  For users who aren't
developers, this is no better than no dependency tracking at all.

Also, turning off parallel mode only works safely when it's done for the
makefile, thus potentially pessimizing unrelated targets as well.

If automatic dependencies are not feasible, then manual ones (specified
by the Makefile.am author) are still better than strictly serial
compilation.  At least, that allows partial parallelization; of course
we rely on the correctness of the rules in that case.

> But I can't see how -j could ever be
> used safely with Fortran sources unless dependency information was
> generated strictly prior to compilation.

I agree.

> Let the onus be on the
> developer such that if they change where modules are defined, they
> must also update the order of their source files in the Makefiles.
> 
> If dependency tracking could be done someone strictly prior to
> compilation, then the source file order in the Makefiles is moot since
> make will compile them in the correct order regardless.  The -j option
> might even be viable at that point.  But the "eager" problem won't be
> easy to solve, then, right?

Yes, it will be a bit complex.  It could maybe be done by something like
the BUILT_SOURCES scheme, by cheating an outer 'make', letting that call
a script to update dependencies, and thus hopefully avoiding spurious
rebuilds.  The initial setup of a dependency file (one per makefile? Or
one per object? Not sure) could be done in config.status just as in
depout.m4.

The next steps toward an implementation is usually to have a number of
example setups that we would like to work, and Makefiles (yes,
Makefiles not Makefile.am files) that work, and that we would like
automake to generate: a simple example, one with more complex
dependencies, one with generated sources, maybe one with mixed C/Fortran,
one with sources in subdirs with or without subdir-objects, one with
module dependencies across different makefiles, one with dependencies
on installed module files, one that uses modules as well as headers.
The examples should exercise not only a one-time build, but also ensure
rebuilds do not more work than necessary, cleaning and distribution
works as expected, edits cause the right rebuilds, and typical edits
that change the module dependency structure work as well; IOW, there
should be a test for the "deleted header file" problem and similar
issues.

These examples can start off being very specific, but then should be
reworked step by step to be more general, portable to various
compilers/systems, and the implementation in automake.in can follow suit
to realize that.

The steps start off being fairly easy to do even if you're not familiar
with Automake's innards.  The further you get, the better; we can help
with the later steps.  Please document your findings as you go along,
side issues and problems you encounter, compilers and systems you test,
so we have a chance to understand why something was done one way and not
another.  Please do look at current testsuite scripts for how these
things are usually set up.

Another open (to me) question is whether and how we would like to mix
and mash module dependencies and header dependencies; the latter could
still be treated similarly to C handling.  OTOH gfortran just added an
option to output them both at once.

An orthogonal question is the actual contents of the script that
extracts the dependencies from the source files.

Hope that helps.

Cheers,
Ralf

PS: my current list of documentation pages:
http://gcc.gnu.org/onlinedocs/gcc-4.5.0/gfortran/Preprocessing-Options.html#Preprocessing-Options
http://docs.cray.com/books/S-3901-55/
http://h21007.www2.hp.com/portal/site/dspp/menuitem.863c3e4cbcdc3f3515b49c108973a801/?ciid=a1080f1bace021100f1bace02110275d6e10RCRD
http://www.lahey.com/docs/lfpro72help/hh_start.htm
http://www.absoft.com/Support/Documentation/fortran_documentation.html
http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/fortran/lin/compiler_f/index.htm
http://www.pgroup.com/resources/docs.htm
http://www.nersc.gov/nusers/systems/franklin/software/ftn.php
http://publib.boulder.ibm.com/infocenter/comphelp/v111v131/index.jsp
http://docs.sun.com/app/docs/doc/820-7600/aevog?l=en&a=view
http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi/0650/bks/SGI_Developer/books/MPro7F90CD_RM/sgi_html/ch02.html