Re: new module for temporary files in temporary directories

[Bruno Haible]

Paul Eggert wrote on 2006-07-05:
> > The rule of thumb is: If a data field can be written by the main program
> > and read by the signal handler, or vice versa, it needs to be marked as
> > 'volatile'.
> 
> OK, thanks for explaining it.  Though I think you meant that the field
> needs to be accessed only via volatile lvalues, not that the field
> itself has to be volatile (an idea that cannot be combined with malloc).

I wasn't aware that it makes a difference regarding the C99 wording, and I'm
still not convinced it makes a difference in practice. (See below.)

In the clean-temp.c code I tried to avoid casts, because they clutter up
the code. In the future gl_*_list.c changes I'll use only volatile lvalue
assignments at particularly selected points, because it would be an
unjustified slowdown to mark the fields 'volatile' (and thus to force all
field accesses to be 'volatile').

> > Data fields which are read-only from the moment they are entered
> > into the data structure (such as the contents of strings) are not marked
> > 'volatile'; I hope 'volatile' is not needed here.
> ...
> > 2) Another example is when a new list element is added to a list that
> > is accessed from the signal handler. If the compiler would reorder the
> > memory stores such that the list element is added to the list before it
> > is fully initialized, the signal handler would likely crash.
> 
> Doesn't this contradict the previous remark "Data fields which are
> read-only from the moment they are entered"?  Hence it suggests that
> even these read-only data fields must be accessed only via volatile
> lvalues.  So, to be safe, you'd have to declare them to be volatile as
> well, which will infect more of the code with "volatile".  (But again,
> please see below.)

In theory, yes. In practice, we can stop the propagation of 'volatile'
at function boundaries, for functions we know the compiler will not
inline. (Only if the compiler inlines a function, it has the chance to
reorder the statements in ways that were not foreseen by the programmer.)
'malloc' is one such function. Therefore writing

   volatile int *data = (volatile int *) malloc (n * sizeof (int));

is safe IMO.

> > (I don't understand what 5.1.2.3.(4) is
> > talking about since GCC 4 takes the freedom to reorder memory loads and
> > stores arbitrarily across sequence points, if it can prove that there is
> > no aliasing.)
> 
> That part of the C standard is often misunderstood.  The C standard
> requires that the only object accesses that signal handlers may do
> (other than to local variables) is to assign to "sig_atomic_t
> volatile" objects.  ... because your
> signal handler cannot access these nonvolatile objects, you cannot
> check the guarantees!  (Tricky, huh?)

OK, so essentially you are saying that 'volatile', as specified by
C99, is too weak for anything but programming of I/O ports of embedded
devices.

But there are other people who need volatile: The Linux kernel community.
When they write

              *((volatile unsigned long *)trampoline_base) = 0;

they don't mean that GCC should assume weak C99 semantics. They mean "perform
the storage of the word immediately, as a word access". And I think,
the Linux kernel community is important enough that at least GCC and
the Intel compilers will have to stick with the pragmatic definition
of 'volatile'. And so we can rely on this pragmatic definition as well,
regardless whether ISO C99 got this right.

> > To me, "volatile" means: "Perform the memory access here and now, without
> > merging several memory accesses, without changing the memory word size
> > width, and without reordering."
> 
> That's not what the standard says, I'm afraid.  For volatile
> _objects_, accesses must be done before the next sequence point, but
> otherwise merging, reordering, and wordsize change is allowed for
> volatile objects.

The kernel people will cry if a compiler does this.

> We used to have lots of problems with signals and 'ls'.  For example,
> a SIGTSTP signal can arrive at any time, and it used to cause "ls" to
> output a command sequence to change the terminal back to the default
> color.  But this command sequence could appear in the middle of a
> multibyte character, or in the middle of another color-change
> sequence; either event could mess up the screen.
> 
> In <http://lists.gnu.org/archive/html/bug-coreutils/2004-04/msg00154.html>
> I fixed this problem by rewriting 'ls' to use almost-C89 signal
> handling, and this fixed the problems for good.

Maybe one reason of this is that now you avoid doing output to file
descriptor 1 while the main program uses stdout?
  - If the previous code used fwrite or fputc inside the signal handler,
    it assumed that the stdio library code is async-signal safe - which
    it certainly is on hardly any system, not even in glibc.
  - If the previous code used write(1,...), it could indeed output
    the default colour escape sequence while the state of the terminal
    depended on where the boundary between two successive stdio buffer
    contents happen to fall: within a multibyte sequence or within an
    escape sequence.

So I agree with you that you cannot do much in a signal handler. But
volatile access to memory locations is one of the things that work in
practice, even if C99 is too weak to guarantee it.

> PPS.  POSIX complicates this situation quite a bit by adding threads,
> which are a HUGE can of worms; and by adding a few functions that can
> be called from a signal handler.

The threads are actually the solution to the problem that so few things
are allowed in a signal handler. Once you can assume a mt-safe libc -
and you can nowadays - some of these previously hard problems (like:
how to display a progress indicator) are suddenly simple.

> PPPS.  Which reminds me, I assume the temporary-file module is not
> intended to be thread-safe, and that only its cleanup function can be
> called from a signal handler?

Yes, this is true.

> This should probably be documented anyway.

So far no-one asked whether gnulib code is multi-thread safe. Until then,
i.e. until there is demand, and until we decide on a better policy, I'm
going with a default answer:
"Our code has global variables and is thus not MT-safe. If you want it
to behave MT-safe, you need to place a lock yourself."

Bruno