Editfiles Considered Harmful (was: Re: Complex Editfiles Examples)

[Eric Sorenson]

On Thu, 4 Dec 2003, Jamie Wilkinson wrote:

> This one time, at band camp, Russell Adams wrote:
> >Egad man!! I'm impressed. ;]
> >
> >I setup a similar construct last night to edit Gentoo's
> >/etc/make.conf, but its not nearly as complex.
> 
> My best editfiles is the one for /etc/ssh/sshd_config, most of my other
> configuration file edits are based on that.

It really terrifies me when I see stuff like this.

Not that your editfiles won't work (necessarily); it's just scary to think that
as a general practice, people let tools generate syntax which have no idea
themselves whether the syntax is valid.  

This is what I meant the other day by std_editfiles_rant.h, but I really would
like to know if I'm completely off-base on this so I'll write a bit about where
I'm coming from.

In my first cfengine site install, I never touched the editfiles functionality
because I simply didn't need it. I was mostly shipping binaries around to keep
a consistent gnu-tools environment on solaris in a production ISP environment
that eschewed NFS and NIS.

For the password file distribution problem, I wrote a bit of a perl frontend to
build up the appropriate passwd/shadow files and place them in the repository
for clients to pick up via 'copy:' when they updated.. (this lives on in all
its cobwebby glory at http://explosive.net/opensource/cfpasswd ) This
worked pretty well and I didn't get bit too badly by any bugs -- even at
cfengine 1.5.x!

On to my current site, where I wasn't directly involved in the first-gen
cfengine deployment. It's an internal corporate network and we were butting up
against NIS limits but had NFS available everwhere, so the main purpose for
cfengine was to build up passwd, shadow, and group from fragments over NFS. 

The implementor used editfiles for this task, and we started seeing weird edge
cases like zero-length shadow files, corrupted entries, etc. Some of these were
parse bugs or things not working as expected, some were implementation problems
in our configs -- the blame for which I still ultimately pin on editfiles'
strange set of features and commands, as has been amply discussed on this list.

But as I was trying to hone an alternative to this for our new next-gen
cfengine2-based setup, I had to pin down exactly what bugged me about the
editfiles problem in order to convince the rest of the team that we really
needed to eliminate it from our vocabulary as an operating pragma, and instead
copy whole files from a version-controlled repository.  This was the Right
Thing to Do, I felt it in my gut, but it wasn't easy to articulate.  I think
I've got it pretty well sussed, and it turns out there's really two issues: one
has to do with systemic behavior and the other is about management complexity.

First, let's look at a typical goal of an editfiles task. Usually you're
updating a system configuration file from its default state, in order to bring
the behavior of a daemon/subsystem in line with policy. After the first time it
runs, the editfiles stanza ought to be a no-op; this is the stated purpose of
commands like AppendIfNoLineMatching. The problem with this approach is that
the product of the original file state run through editfiles actions
cannot be guaranteed to be equivalent to the desired end state, if there's
no semantic rigor imposed on the actions performed. This is the idea behind
Alva Couch's "conduits", and in no sense are the regexp description and
editfiles commands a conduit for any of the things editfiles's potentially
editing. 

Put simply, there's nothing inherent in editfiles which prevents you from
accidentally scribbling nonsense all over a live file on the target machine.
This is incredibly dangerous for anything of significance! I've seen problems
where 'AppendIfNoLinesMatching' didn't match when it should have, causing
broken-record repetition. Substring matches for 'LocateLineMatching' may
match too early or not at all (usually depending on whether or not you
use ^ and $ anchors correctly).  Worse than all of this, the baseline
behavior of the subsystem in question may have changed and your edits
do not produce the intended consequences -- even though editfiles is doing
exactly what you've told it to do!

Contrast this with editing or generating a config file, testing that file on a
non-production system, and then "promoting" it into production by checking it
in the repository. Unlike a series of adds, deletes and edits to the contents
of the file, I have a high degree of certainty what the target system will look
like (and by extension, how it will behave) at the end of the cfengine run.

In a related sense, the second problem shows iteself when someone else
tries to modify a modification you've made, or when the policy changes,
necessitating a change in the system's behavior. Now you have to backtrack
from the configuration files as they exist on the production systems,
into the guts of your editfiles cleverness (which by now has descended to
mere unreadability) and reverse-engineer the adds, deletes, and edits,
in order to add a new directive to the top of the file.  

Why not just edit the file with an editor and check it in to the repository?
Let cfengine copy the new version over and restart the deamon.  I'd argue
that this is the end result that you want -- a new version of sshd_config on
all machines of a particular class (or whatever), so why not just do it?
Obviously there's been a lot of work put into making the editfiles command
set very rich, but it's never going to be able to handle all the cases
we run into without becoming (even more) hopelessly tangled, syntax-wise;
regardless, it'll never be as smart as a human with a text editor.

The final blow to the editfiles methodology should be clear in the
not-so-hypothetical situation where someone asks you to show them what changed
between last week and this week in the ssh subsystems of your machines.  This
is, and should be, a text diff between two revisions of the configuration file.

It should NOT be a text diff between two revisions of your editfiles section,
where you have to pull out the whole stanza from each version *and the original
configuration file* and compare the results of applying each stanza to a copy
of the config... That's crazy-talk, absurd on its face. 

Use CVS. Make a directory tree that looks like a filesystem, and populate each
leaf directory with the files you want to manage. Either make separate trees
per host-class or use class-keyed filename extensions in a unified tree, it
really doesn't matter. Use copy: with checksums, not date-stamps, so machines
which get installed after your latest config change still get the authoritative
version.  Use a define=className in your copy directive to specify triggers for
restarting the affected service daemon (remembering to pre-declare className
with AddInstallables in your control: section).  

Voila, all your editfiles problems just disappeared.

Seriously though, I would like to hear any criticism or feedback on the 
above -- if there's something that just doesn't fit into this model, or
maybe some loon^H^H^H^Hfine person who feels as strongly Pro-editfiles as I do
Anti-, I'd welcome the dialog.

-- 

    Eric Sorenson - EXPLOSIVE Networking - http://explosive.net