Re: [bug #46481] install-info can "corrupt" dir file if interrupted.

[Pádraig Brady]

On 21/11/15 22:15, Karl Berry wrote:
>     ag> of its new atomic file-swap ability:
>     http://lists.gnu.org/archive/html/coreutils/2015-11/msg00068.html
> 
> Nice, but no "new" facility can be assumed for general use.  That's why
> I hoped there would be a gnulib module that make use of such fancy new
> facilities where possible, and fall back to whatever
> best-possible-attempt otherwise.  To save Gavin the time, trouble, and
> exposure of trying to implement it.  This is not a new problem.  One
> would sure think it's been solved somewhere already in GNU and can be
> reused.
> 
>     mthl> There is a big difference between the behavior expected at
>     compile time and install time.
> 
> Oh, whatever.  Using gcc was just an example.  Same thing if I type
> "make install" and hit CTRL-C (or CTRL-\ or kill -9 from another
> terminal or unplugging the computer or whatever) in the middle of
> installing some huge binary: I don't expect it to remain uncorrupted.
> And, in practice, I believe that it is likely to get corrupted.
> 
>     You will want every installation operation to be
>     atomic to garantee the integrity of the system.
> 
> Sure, I agree it is desirable, but I don't believe it is, in fact,
> implemented by much of anything, except (I suppose) at the
> package-manager level.
> 
> What's different about the dir file is that it amalgamates stuff from
> many packages, and isn't easy to reconstruct (e.g., can't just type
> "make install" again).  Thus I can see that in principle it is worth
> extra trouble to try to preserve.  But I'm not volunteering to implement
> it, so it's just an opinion ... -k

Replacing a file robustly is a tricky problem,
and rarely done correctly, with most people resorting
to a database for such functionality.

I looked at the problem recently for http://github.com/pixelb/crudini
and the implementation there uses existing file system APIs
which should work on local and most contemporary network file systems.
It's been extensively tested, small races found and fixed
and is now seen to be robust for this task. The lineage of changes are:
https://github.com/pixelb/crudini/commit/231aaafc
https://github.com/pixelb/crudini/commit/8d549be8
https://github.com/pixelb/crudini/commit/df33e66a
https://github.com/pixelb/crudini/commit/7108dd7e
https://github.com/pixelb/crudini/commit/4e1af9ff
https://github.com/pixelb/crudini/commit/10875e6a
https://github.com/pixelb/crudini/commit/3f0d2051

Now while the above works, there are caveats noted
for each of the provided replacement methods.
Using newer apis like renameat() or exchangedata()
if available would avoid most of the caveats I think.

Given how tricky all this is, it really should be done once and reused,
and I plan to incorporate the above mentioned functionality into
a rewrite utility which can be called by whatever to replace
files with ACID guarantees.

I've most of the bits figured out now, so I just need
some time to pull it together.

cheers,
Pádraig.