Re: Expressing dependencies

[Laurence Finston]

On Mon, 5 Jan 2004, Robert Collins wrote:

> On Mon, 2004-01-05 at 03:53, Laurence Finston wrote:
>
> > This is essentially what I tried to do by using the auxiliary program
> > `3DLDFcpl' in the rule for building the executable `3dldf' (roughly):
> >
> > 3dldf: $(3DLDF_CWEBS)
> >        3DLDFcpl
>
> Thats not quite what I was suggesting.
>

It would have worked quite nicely, though, except for the rules for the .o
files that Automake generated automatically. If I remember correctly, it
also affected the `dist' target and the various `clean' targets.


> > It's reasonable behavior for Automake to assume that the sources for a C++
> > program are called <filename>.cc, <filename>.cxx, or <filename>.c++, but it 
> > is
> > very restrictive to assume files need to be rebuilt based merely on the
> > information that their prerequisites have a more recent timestamp.
>
> As I said, thats the /only/ information (cheaply) available to automake.
> Alternatives include generating a md5 of the file and comparing that to
> a calculated one, or other such has-content-changed tests. But make
> isn't language aware - it can't tell if a file change is 'meaningful'.
>

I don't know what an md5 is.

> > I don't think the problem lies with CWEB, or Bison and Flex, for that 
> > matter.
> > CWEB isn't GNU software, and I doubt whether the authors would appreciate
> > being asked to "fix" it. We can't do anything about make, either. I think 
> > the
> > problem should be solved within Automake.
>
> The problem lies in the concept of the header 'maybe changing'. If the
> header is dependent on the .web source, then it can't be considered
> correct if it's timestamp is older than the .web source - because make
> is meant to look at just modification dates. If you want to consider
> file content changes, make has to generate some database (of sorts) to
> track file content, and while we *could* do that in GNU Make, automake
> targets posix make - so that would not be portable. So the rules that
> will most likely work well for you are - no target to build the .h file
> at all, and magic in a script to replace it if needed. Note that if the
> header is replaced, you'll need to reinvoke make on the same dir, to get
> it to notice that reliably.
>
> Rob
>

There is a mechanism for
regenerating a header file, which is needed for the case that it doesn't
exist.

I have solved the immediate problem for 3DLDF.
That's not what I'm getting at. In my opinion, it would be worthwhile
to try to implement a general solution for this problem, which
affects users of other development tools, too. I believe that the solution
should be implemented within Automake.

These are the problems that remain:

1. If a .web file changes, but these changes affect neither the .cxx file
nor the .h file, it must be touched in such a way that its timestamp
is the same as the oldest of the two associated timer files. This
makes it necessary to revert the buffer, if the .web file is currently
being edited. This is harmless, but annoying. If it's not touched,
make will run ctangle on it the next time around.

2. I wasn't able to rewrite the rules in order to make it possible for
a single .web file to write multiple output files that are used in the
rules for building the executable `3dldf'. Maybe it's possible, but it
was just too complicated. This means that the GNU build tools are
limiting my ability to use the features of CWEB.

3. It was fairly difficult for me to get the rules to work right. I
think they do now, but problems may still crop up. Solving it for
3DLDF helped me, but it doesn't help anyone else.

I submitted my package to the GNU project after it was "finished", and
then I had to go back and learn how to use the GNU build tools for it. I
could have saved myself a lot of trouble by using them in the first
place. I suspect a lot of people will make the same mistake.

If I was writing low-level system software, I would just write in C
without using CWEB, and I wouldn't have these problems with
make. However, 3DLDF isn't that kind of software, and using CWEB has
significant advantages. I think authors of GNU packages should be
encouraged to use literate programming tools rather than
discouraged. If Automake doesn't provide support for the tools
people are using, of whatever kind, I think this will tend to
discourage them from submitting their packages to the GNU project.

This isn't just a theoretical problem for me. I'm considering
implementing interfaces to packages that supply 3D-modelling and
rendering functionality via libraries. I have to be careful about
this, because of the copyright and licensing issues involved. If
possible, I'd prefer to use GNU packages, but many packages are not
GNU. I'm not saying that lack of native support for CWEB in Automake
is the reason, I'm sure it's not. However, I think it would be
worthwhile trying to make it easier for people to conform to the GNU
Coding Standards while still using the development tools they like. I
am not suggesting changing the standards.

We can't change CWEB or make. I think it would be
possible to change Automake so that it supports CWEB. For example, it
could change the way it generates the default rules if .web sources
are present. It could also be made to generate variables and timers
for expressing the dependencies among the files automatically. Then
rules using these variables and timers could be written to the
Makefiles automatically, and the individual maintainer wouldn't have
to worry about coding them. Automating this should make the rules more
reliable, and might even make it possible to have more complex
dependencies.

Laurence Finston
3DLDF maintainer
Website: http://wwwuser.gwdg.de/~lfinsto1
email: address@hidden