Re: new module for temporary files in temporary directories

[Paul Eggert]

Bruno Haible <address@hidden> writes:

> The rule of thumb is: If a data field can be written by the main program
> and read by the signal handler, or vice versa, it needs to be marked as
> 'volatile'.

OK, thanks for explaining it.  Though I think you meant that the field
needs to be accessed only via volatile lvalues, not that the field
itself has to be volatile (an idea that cannot be combined with malloc).

> Data fields which are read-only from the moment they are entered
> into the data structure (such as the contents of strings) are not marked
> 'volatile'; I hope 'volatile' is not needed here.

Me too, though the ice is pretty thin here; please see below.

> 2) Another example is when a new list element is added to a list that
> is accessed from the signal handler. If the compiler would reorder the
> memory stores such that the list element is added to the list before it
> is fully initialized, the signal handler would likely crash.

Doesn't this contradict the previous remark "Data fields which are
read-only from the moment they are entered"?  Hence it suggests that
even these read-only data fields must be accessed only via volatile
lvalues.  So, to be safe, you'd have to declare them to be volatile as
well, which will infect more of the code with "volatile".  (But again,
please see below.)


> 3) The theoretical reason is that in ISO C, section 5.1.2.3.(5)
> guarantees to me that memory stores have been performed at sequence points
> only if I mark them 'volatile'.

I think that's a misunderstanding of that section.

>From the C standard's point of view, a volatile object (not merely an
object accessed via a volatile lvalue, but a volatile object, e.g.,
"static int volatile x;") has externally visible behavior.  You can
think of such objects as having little oscilloscope probes attached to
them, so that the user can observe every access to them, just as the
user can observe data written to output files.  This is not true of
ordinary objects accessed via volatile lvalues (which is what the new
module mostly uses); only volatile objects can be observed by the
user.

5.1.2.3.(5) talks only about volatile objects; it does not place any
constraints on accesses to ordinary objects via volatile lvalues.

> (I don't understand what 5.1.2.3.(4) is
> talking about since GCC 4 takes the freedom to reorder memory loads and
> stores arbitrarily across sequence points, if it can prove that there is
> no aliasing.)

That part of the C standard is often misunderstood.  The C standard
requires that the only object accesses that signal handlers may do
(other than to local variables) is to assign to "sig_atomic_t
volatile" objects.  If your signal handler does any other object
accesses, or if it calls any function other than _Exit or signal or
abort, behavior is undefined.  GCC 4 is entitled to do the
optimizations in question, so long as it respects this (which is
usually fairly easy to do).

Every optimizing C implementation that I know of takes advantage of
this gaping loophole, which means that for the kind of programming the
temporary-file module is trying to do, behavior is pretty much
undefined all around.

If your signal handler conforms to C, then you can rely on
5.1.2.3.(4)'s guarantees about objects' values during an interrupt.
This is true even for nonvolatile objects -- except that, because your
signal handler cannot access these nonvolatile objects, you cannot
check the guarantees!  (Tricky, huh?)

> To me, "volatile" means: "Perform the memory access here and now, without
> merging several memory accesses, without changing the memory word size
> width, and without reordering."

That's not what the standard says, I'm afraid.  For volatile
_objects_, accesses must be done before the next sequence point, but
otherwise merging, reordering, and wordsize change is allowed for
volatile objects.  Worse, volatile _lvalues_ provide no more
guarantees than ordinary lvalues when the underlying objects are not
volatile.  And in either case, programs must use only extremely
limited signal handlers in order to get defined behavior.

For this reason, I'm becoming more inclined to not use volatile
lvalues, and to rely only on volatile objects.  This is not because I
think volatile objects suffice; it's only because I doubt whether
adding "volatile" to lvalues helps, and it certainly clutters up the
code (and has my readers wonder what's going on :-).

Lest it be thought that this is merely an academic exercise:

We used to have lots of problems with signals and 'ls'.  For example,
a SIGTSTP signal can arrive at any time, and it used to cause "ls" to
output a command sequence to change the terminal back to the default
color.  But this command sequence could appear in the middle of a
multibyte character, or in the middle of another color-change
sequence; either event could mess up the screen.

In <http://lists.gnu.org/archive/html/bug-coreutils/2004-04/msg00154.html>
I fixed this problem by rewriting 'ls' to use almost-C89 signal
handling, and this fixed the problems for good.  I write "almost"
since ls's SIGTSTP handler increments a sig_atomic_t volatile counter,
rather than just set a sig_atomic_t volatile variable; but this is
good enough in practice.

Obviously this sort of change can't be done in general, but it does
suggest that perhaps the temporary file module might want to have an
API that is useful for programs that are like 'ls' and use mostly-C89
signal semantics.  Such an API wouldn't have to mention the keyword
'volatile'.  (I'm mostly just thinking aloud here -- I don't have a
specific suggestion, sorry.....)


PS.  Hmm, I guess I should write this all up somewhere.  Perhaps
the autoconf manual.

PPS.  POSIX complicates this situation quite a bit by adding threads,
which are a HUGE can of worms; and by adding a few functions that can
be called from a signal handler.  I don't think anybody in the world
actually understands all the implications of this with respect to
"volatile".  Certainly I don't.

PPPS.  Which reminds me, I assume the temporary-file module is not
intended to be thread-safe, and that only its cleanup function can be
called from a signal handler?  This should probably be documented
anyway.