Re: History file clobbered by multiple simultaneous exits

[Geoff Kuenning]

Linda:

I actually don't use histappend; I like the fact that history eventually
disappears.

Chet:

I have to confess that I'm equally confused.  The probability of a
malloc failure seems vanishingly small; my bashes are only about 20M
virtual, my history size is a measly 500, and my current history is just
over 8K..

I can't think of an interleaving in either 4.2 or 4.3 that would result
in this behavior.  No matter what the interleaving, one of the instances
ought to finish last.

I wrote two test programs to try to reproduce the behavior, and neither
failed.  This makes me wonder if it has to do with how bash deals with
signals (I'm assuming a SIGHUP is involved, though it might be another).
Perhaps bash is getting a second signal during the brief period when the
buffer is being malloc'ed and the history file is being formatted, and
that second signal is killing it before it can do the write?  That seems
unlikely, but sometimes computers surprise us.

> Let's think about why and how this could happen. Maybe that will give some
> insight into how to solve the problem.
>
> The current (bash-4.3-devel) code works something like this, assuming no
> errors (lib/readline/histfile.c:history_do_write()):
>
> rename (histfile, histfile~)
> open file with O_CREAT|O_TRUNC
> malloc buffer large enough to hold all history data
> write all of the history entries in one write(2) call
> close file
> unlink (histfile~)
>
> The bash-4.2 code works the same way except that it does not back up the
> history  file.  Each shell does the same thing when it exits, assuming
> histappend is not set, as in your configuration.
>
> There are a couple of ways the history file can end up zero-length: the
> malloc can fail, or the write can fail.  In bash-4.2, it's too late to do
> anything about the truncated history file at that point.  In bash-4.3, the
> previous history file will be restored.
>
> The O_EXCL solution could work, to some extent, in bash-4.3.  It won't do
> any good in bash-4.2 because it will cause open() to fail under the
> perfectly reasonable circumstance of an existing history file.
>
> What have I missed that could cause file truncation due to a race condition?
>
> Chet
> -- 
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>                ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/
>

-- 
    Geoff Kuenning   geoff@cs.hmc.edu   http://www.cs.hmc.edu/~geoff/

If we knew what we were doing, it wouldn't be research, would it?
                -- Albert Einstein