Re: timestamps vs. checksums

[Paul D. Smith]

%% Markus Werle <address@hidden> writes:

  mw> Recently I had an email discussion about build tools.  Someone
  mw> stated that make was flawed due to the fact that it uses
  mw> timestamps to decide whether to recompile a file or not.

  mw> I disagree with saying that make is flawed.

It is flawed, but it works as designed.  When make was designed 20+
years ago, the idea that compiles could complete in less than a second
or that you'd be doing builds over the network was pretty ludicrous.

  mw> Actually NFS is flawed, which I encountered today:

NFS is flawed as well, of course :).

  mw> Checking whether a file has changed is just a minor task
  mw> among others that make has to address.

You miss the bigger picture.  Sure, using md5 to find the checksum of a
file is very simple.

Here's the problem: what are you going to compare it to?

Currently make is completely stateless, because we only compare relative
timestamps between files: is file A newer than file B?  Yes?  OK, update
file B.  Make doesn't have to remember anything from one invocation to
the next because all the information it uses is maintained by the
filesystem.

If you want to implement a checksum comparison it's a huge leap, because
now make must become stateful: to know whether this file's contents have
changed you have to have state from the previous invocation of make
that remembers what the checksum (or whatever) was then, so you can
compare it to now.

Now we're talking about make keeping a database of some kind (note I'm
not talking about a fancy SQL database, or even necessarily something
like dbm--but you need _some_ kind of data storage).  Now you run into
issues like locking: what happens when someone invokes make recursively
in the same directory?  What about parallel builds?  What about
portability: not all systems have libraries like dbm--does make have to
provide its own?  Or do we store the checksum in one file per target
rather than lots of checksums in one DB file?

Etc. etc. etc.

As always, the devil is in the details.


That's not to say that this shouldn't be done: it's actually an absolute
must-have if make is to survive too much into the future: systems are
getting too fast to rely on timestamps anymore.  It's on the TODO list
but as I've said there are many considerations.  There are a number of
threads dealing with this in the archives of these lists.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <address@hidden>          Find some GNU make tips at:
 http://www.gnu.org                      http://make.paulandlesley.org
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist