Re: Explicit, implicit rule chain doesn't work in created dirs

[barkalow]

On Tue, 27 Aug 2002, Paul D. Smith wrote:

> %% <address@hidden> writes:
>
>   b> dir:
>   b>  mkdir -p dir/subdir
>
> This rule is incorrectly written.  You have informed make that you will
> be creating a target "dir", but you really created a target
> "dir/subdir".  As a result, make doesn't know anything about
> "dir/subdir", it only knows about "dir".

Using dir/subdir as the target here (and below) doesn't change anything,
so that's not the problem:

------
dir/subdir:
        mkdir -p dir/subdir

dir/subdir/file.b: dir/subdir
        touch dir/subdir/file.b

#dir/subdir/%.b: dir/subdir
#       touch dir/subdir/file.b

#%/file.b: %
#       touch dir/subdir/file.b

dir/subdir/%.a: dir/subdir/%.b
        cp $< $@

all: dir/subdir/file.a
------
Either of the comment-out versions trigger, but the uncommented one
doesn't. I've also noticed that making it .PHONY doesn't change anything
either way, despite the fact that make should not care at all about the
file dir/subdir/file.b

>   b> if dir/subdir doesn't exist, "make all" fails:
>   b>  *** No rule to make target `dir/subdir/file.a', needed by `all'.
>
> This is because of GNU make's internal directory cache.  The first time
> make accesses a directory, it creates a cache of the data it finds
> there.
>
> In this case, it looks at dir when it first sees that it needs to build
> dir/subdir/file.b, and it doesn't exist.  Further, there are no rules
> defined in the makefile to build dir/subdir.  So, make gives up.

Actually, I've tracked this farther now.

make uses the dep->changed flag for dependencies of implicit rules which
are in directories which don't exist. I'm not clear why this is
sufficiently common to warrant an optimization at all, but...

The comment on line 395 of implicit.c says that we make be able to skip
looking for the file. We can skip checking whether the file exists (since
the directory doesn't exist, the file must not; even if the directory has
been created, the file won't have been created unless we created it, in
which case we believe it to exist (even if it doesn't, i.e. a .PHONY
file). But we can't be sure that we couldn't create the file (since it
could be .PHONY or we could create the directory and the file).

lookup_file() is actually checking for information on the target, not a
file; it is named oddly, considering that it is even used for things like
.SUFFIXES, which couldn't refer to an actual file. It is appropriate to
call lookup_file even if you know the file doesn't exist, so that you can
see if you could make it. So the condition here should be:

  if (lookup_file (p) != 0 ||
      ((!dep->changed || check_lastslash) && file_exists_p (p)))

(i.e., if there's info on how to make the file, or it might exist and it
does exist...)

Making this change causes my example Makefile to behave the way I would
expect, running all three commands if nothing exists.

This shouldn't be able to hurt correctness, since lookup_file (p) will
only return non-zero if there's actually a rule defined or something has
matched already; in the former case, the Makefile author intended it to be
found, presumably, and in the latter it would be found again in the
implicit rule search (which doesn't check dep->changed).

It shouldn't hurt performance significantly either, since it's just an
additional hash lookup for each possible target in a nonexistent
directory; but hardly any implicit rules look for the prerequisites in a
different directory from the target unless the different directory really
ought to exist (the exceptions being RCS and SCCS, which might not be in
use but are in the default rule set); it would still not actually look on
the disk for the file, since it does know that the file isn't there.

The documentation doesn't say you can't make targets in directories that
don't exist beforehand, and it's not even particularly unlikely that a
.PHONY target would be "in a directory that doesn't exist" (since it's not
actually a file).

> Luckily for you, what you are trying to do is a very bad idea anyway :).
> By that I mean, you should never create directories by defining them as
> targets and listing them as prerequisites.  It's just not what you want
> to do, period.
>
> Here are two good reasons:
>
>  1) First, unless you are careful to define the directory as a
>     prerequisite to _every_ target, someone will get a failure if they
>     run "make foo.o" and foo.o doesn't depend on the directory.

Same is true of any other dependency, however. At least in that case it
will say the file couldn't be created, rather than generating incorrect
results.

>  2) Second, if you do #1, then all your code will be rebuilding at odd
>     times for no discernable reason.
>
>     Remember that the modification time of a directory changes whenever
>     a file is added or removed in that directory.  So, whenever you add
>     a file to the directory "dir" above its modification time will be
>     reset; that means that any files that depend on "dir" will be
>     rebuilt--just because you added (or removed) a file in "dir".
>
>     This is almost never what you want.

That's what I was avoiding with the dir/subdir trick. Ideally it would be
possible to tell make that the date on some targets is never important,
rather than using the --old-file= parameter anytime it changes.

> The answer is very simple: always create directories first, before make
> starts trying to build targets.
>
> You can do this with simply expanded variables (which are expanded by
> make as the makefile is read in, before any targets are evaluated), and
> the $(shell ...) function:
>
>   _foo := $(shell [ -d dir/subdir ] || mkdir -p dir/subdir)

It would be nice if the documentation suggested that...

(but the "[ -d dir/subdir ] ||" is unnecessary, since mkdir -p has no
effect if the directory exists).

        -Daniel
*This .sig left intentionally blank*