Re: Idea: Automatically correct timestamps

[David A. Wheeler]

On Sun, 09 Jun 2019 22:11:32 -0400, Paul Smith <address@hidden> wrote:
> I don't think this can work, as-is.
> 
> It's a common paradigm in make (including POSIX make) where there is a
> rule that could cause a very expensive operation to be started, to
> instead make a cheap test and if it determines the expensive operation
> is not needed it will not update the target at all.

Good point.

> This problem could be addressed by adding a clause that if the
> timestamp of the target was not modified at all, it should be left
> alone.  It's hard to come up with a justifiable purpose for a target's
> mod time to be changed, but to something less than one of its
> prerequisites.

I agree.  I'm fine with that solution: "If the mod time changed, it'd better
be at least equal to its prerequisites" handles these cases.
So many systems support nanosecond time resolution values in
the filesystem that it's be pretty unlikely to have this problem
*and* happen to hit exactly the same "backwards" timestamp.

> Although doubtless someone has done it, on purpose, somewhere for some
> reason.

I can't imagine *why*.  But to prevent backwards incompatibility, this could
be strictly opt-in, and disabled by default.  Then it can't possibly hurt
anyone, and turning it on can eliminate a potential issue.
I think it will *not* negatively impact the vast majority of people, so
people will be able to just turn it on.

> Hm.  Maybe people have just given up reporting this problem but
> honestly I can't remember the last time I had anyone ask questions here
> or on StackOverflow that are related to clock skew.  I sort of thought
> it was a solved problem.

Reporting this is tricky, because it's not reproducible. Users are unlikely to
realize the cause of the problem; as far as they're concerned it's a
stray gamma ray.

If your multiple processors are sharing RAM the problem is less likely to occur.
But since speed of light isn't infinite, and clocks drift by nature, all that
the underlying system can do is try to make it less likely.

In any case, make is fundamentally dependent on the quality of its input
(garbage in, garbage out).  This would counter a potential source of garbage.

> mainly they're asking for things like rebuilding targets automatically when
> compiler options have changed etc.

Oh, I see.  Okay, this proposal won't help with *that*.  But I still think
it'd be a good thing to do.

Below is a revised proposal, given that feedback.

--- David A. Wheeler

======================================

The following proposes an opt-in mechanism enabling make
to automatically correct timestamps of targets if their modification time
changes to value that's before the creation time of their prerequisites.

If MAKETIMEADJUSTMAX is defined and set to a value > 0, then
make will try to adjust the timestamp of targets as follows
(e.g., to compensate distributed system clock skew):

First, make will check the timestamp of the non-.PHONY target(s)
before and after executing a rule, and after executing the rule
it will calculate largest dependency timestamp
(the largest timestamp of the non-.PHONY dependencies).
If the all the dependencies are .PHONY,
target existed before the rule ran and its timestamp is the same,
or the target's timestamp after executing the rule is at least
the largest dependency timestamp, no timestamp adjustment occurs.

Otherwise, make will consider adjusting the time of the target.
If the largest dependency timestamp - target's timestamp after rule
is less than or equal to MAKETIMEADJUSTMAX, it will try to make the
target's timestamp equal to the largest dependency timestamp.
If difference is larger or it fails to make the time adjustment it considers
MAKEMUSTTIMEADJUST (the problem is skipped if this variable is empty,
and it's considered a failure if the variable is not empty).

Note: When this adjustment occurs, perhaps make could report
that it's doing the adjustment to stderr (unless it's quiet).

--- David A. Wheeler