Re: Why can't root check in files?

[luke]

First off, I should thank everyone who's taken the trouble to discuss
this.  I should also say that the reason I'm still discussing it at
length is not because I'm trying to do something childish like "win an
argument".  I really am simply concerned that maybe there's a major
flaw in my approach, and wanting to find out what it might be.  Which
is why below I try to explain how none of the dangers described in the
examples, really apply to my scheme.  (Or, more accurately, how none of
the dangers are magnified by my scheme.)

I apologise for the long discussion; and really appreciate the time
spent by people in responding.  I've been lucky myself in having enough
spare time over the last few days to respond.

On 16 Oct, Jesus Manuel NAVARRO LOPEZ wrote:
>  Hi, Luke:
>  
>  address@hidden wrote:

> > Some examples would help.  I'm still drawing a blank on imagining
> > dangers.
> > 
>  
>  The easiest: a non tested checkout goes directly to the production
>  environment thus potentially giving you an unusable system

Agreed, that would be very bad.  But in my system, since it's quite
non-intrusive, it would happen only if the administrator went in and
directly messed up the files in /etc.  But since my system is designed
to replace that simple-minded system of managing the system, it's
no worse than the system it was replacing - and will even let you undo
most errors.

The only ones it won't be able to undo are things that prevent cvs
itself from working - presumably things like destroying the password
file or something.

All I'm saying is that it's a simple lightweight system that's better
than nothing.

The idea of testing changes first in a separate checkout is right and
good.  But nothing stops you from doing that in the live area either.
Testing these checkouts you're talking about must be quite tricky, 
since you have to fake up enough of the rest of the system to be able to
try out the changes, then after everything checks out, change all
your work to refer to real file locations etc. and not stuff for the
fake system, and commit those chanegs and then copy them into the live
area.  I'm thinking of things like sendmail.cf which will have
references to other files by absolute pathname.

> (specially as
>  soon as you see it's not good idea posting to the server in-file
>  passwords; then you substitute by a token like [HERE_PASSWORD];

cvs isn't running in client-server mode.  You're running cvs as root on
a root-owned repository created under root's home directory, on the
local machine, and all the files have their normal permissions.  So
unless cvs somehow broadcasts the passwords or changes the permissions
on files in the repository to something that gives ordinary users read
access, I don't see a problem.

But you probably know more about cvs' internals than I do, and I'm
willing to learn there's a problem for me there.


> next you
>  see is you forgot to manually substitute that token on any given file).

That isn't necessary in my scheme, if I understand you correctly, so it
doesn't apply.  It doesn't apply because you're working on the live
system.  Yes, I agree that that's more dangerous and probably unwise;
but if it;s what people are doing already, then my scheme doesn't make
that any more dangerous.

I see that it certainly does apply in the scheme that you are explaining
- where you have a separate checkout and a build system to install the
changes.  (This is the step I referred to as  "change all your work to
refer to real file locations etc".)

BTW, all the files are marked as -ko in my scheme, so it won't mess up
files that are under cvs control elsewhere (BSD packages are usualy
like this.)

>  Another one.  CVS will work as root; as soon as any other process/user
>  access to that CVS *server* you can expect another hole on your system.

Sorry, I don't understand.  Are you saying that if I run the non
client-server cvs command, other people can somehow "connect to it"
while my cvs commit or whatever is running, and with my privileges
access files?!

> > But this seems to me like an onerous requirement, and an intrusive
> > policy.  It means that you can't just use rpm to install packages, or
> > webmin or linuxconf or control-panel etc. etc. to help configure your
> > system.
> > 
>  
>  Well, yes you can, by proper config procedures.  You want to use rpm?
>  well then use it.  But record what's have been installed so you can see
>  what has been modified, accept/reject changes, then checkin as needed.

That's easy to do in my system, since you can at least do a cvs diff
and see what config files have changed.  (The way I've set it up, it
doesn't checkin binary files.  Probably I should provide that as an
option, too, come to think of it.)

Anyway, after the diff you can decide to undo the changes or commit
them, as you see fit.

Otherwise you'd have to hunt down the changes that rpm made (somehow),
and then compare the files changed through some other more awkward
mechanism.
 
>  About non scriptable facilities like those you mention, now you see why
>  they *seem* to be a good thing, not being so in the long run.

I see what you're getting at; but lots of people like to use them, and
it seems like a circular argument to say that your scheme is better, and
one of the reasons it's better is because it stops you from using easy
config tools, because they're bad because they make it hard to use your
scheme.

(Whew.  Well, it made sense to me as I thought it!)

> > You can't use any of those things without manually comparing those
> > changes back to what your "build" step does, and retro-fitting the
> > changes that you approve of back into your build system.
> > 
> > In other words, you lose most of the benefits of using the helper tool
> > in the first place!
> > 
>  
>  You should see if their usage proves *real* benefit.

We'd better not go into that!  :-)

> > > > Also, your method would also require a diff of each file to be
> > > > installed, against the corresponding live file - since other people in
> > > > the wheel group could make changes directly; possibly via webmin or
> > > > linuxconf or any of the other sysadmin tools floating around, or by
> > > > installing new packages.  Otherwise you risk wiping out changes
> > > > (possibly good changes that may have required lots of work).
> > >
> > >  Oh, no!  Absolutely NOT!  My method mandates that _all_ changes be made
> > >  through the CVS source files, and _only_ through the CVS source files.
> > 
> > If you insist.  It still means you have to do a diff of each file, and
> > a comparison of each file's metadata, to what is embodied in your build
> > system.  Which sounds to me to be harder than what I assumed you were
> > doing.
> > 
>  
>  Yes.  That means you have an extra layer of knowledge of your system and
>  that you avoid "magics" blindly changing your system's state.

Well, it seems to me like you don't have extra knowledge about your
system, you just have extra knowledge about this extra layer.

> > >  This is yet another reason why you _need_ an intermediate "build" step.
> > 
> > It sounds to me like an extra reason why an intermediate "build" step
> > introduces more work!  I'm not saying that it doesn't also give you
> > benefits - I';m just saying that it's more work.  Like using cfengine
> > would give you more benefits, but is also more work.
> > 
>  
>  It is not that the build step adds more work, but that you're trying to
>  save some time avoiding some steps.  You can try, but probably sooner or
>  later you will see that your shortcut not always pays.  You seem to want
>  accountability and repeatibility (that's good) but then, you try to
>  shortcut because it seems too workload.  Then review your objectives. 
>  It migth prove you don't need such a level of control of your system (it
>  comes to my mind that if you *really* would need it, tools like
>  linuxconf or webmin would be immediatly banned, and you wouldn't talk
>  about other's non-acountable changes to config files as "good changes
>  that may have requiered lots of work" instead of "dangerous unacountable
>  changes that must be avoided by any means").  Under this assumption
>  everything comes clear to me (I don't try to be rude, really).

No, that's fine, I'm not offended.  We're just trying to understand
each other, I believe.

>  You want
>  accountability and repeatibility because you feel it's good and it seems
>  cool, but you are not really convinced it's the goal to go, so as soon
>  as that control-layer suppouses more workload you feel it doesn't pay

Yes, I agree.  The benefit must be balanced against the cost.

But as well - imagine that you're working in an environment where there
is almost no change management: people just dive in and change things.
If the change breaks stuff, the users scream, and then all the
administrators work real hard and fix things.

Let's ignore the horror that you feel at this concept, and how bad it
is; I'm sure lots of places are actually run like this.  In the
simplest case, most home Unix systems would be.

Anyway, if you now want to introduce your scheme, you have to convince
the administrators and their bosses that they should stop doing what
they've been doing, and start working in a new way that makes it harder
to make changes, and takes more time to get things done.  You may not
be able to.

(Maybe you will after the whole network goes down for the 3rd time
after someone made some dumb mistake, and stayed down for days!)
I'm not saying that your solution isn't better, I'm just saying it's
more costly.

My scheme is like a baby brother to yours.  I can more easily imagine
people adopting my scheme and then eventually deciding they needed
more, and adopt your scheme.  But they might not have made the change
if they had to do it in a single step.

>  (and it migth probe so: most home systems, for instance, have great
>  benefits from using such graphical aid tools with no versionning tools
>  in use, since the extra work doesn't pay for the avoidance of mistakes
>  that you can make from time to time and that are neither critical nor
>  difficult to fix).

Exactly.

> > >  You _want_ all changes to be forced through the change control system!
> > 
> > To me the difficulty in the scheme you describe is the complexity
> > introduced by this extra level of indirection.  And not all the changes
> > can easily be forced through the change control system, since they're
> > designed to just work with /etc, and have no knowledge of the change
> > control system.
> > 
>  
>  *You* are the one with knowledge of the change control system.  I would
>  thougth that was more than enough.

No, I don't know how Greg Woods' scheme works.  I'd have to get a copy
of it and learn it, or have some design and build a scheme from
scratch, and both would take far more time and effort than adopting
my little scheme.  You have to have ordinary user knowledge of cvs, for
my scheme.  Not of a possibly-complex build system.
 
> > 
> > Okay, so we're agreed that the scheme you describe risks the loss of
> > changes.  Maybe I've also explained that it also means that lots of
> > neat system config tools (webmin etc.) become Forbidden in your scheme,
> > or at least force lots of manual work: retrofitting their changes back
> > into your build system.
> > 
>  
>  Knowing a bit Woods opinions on this and other fields, I would say
>  (well, in any case I do say for myself) that loosing unacounted changes
>  is not a real danger but for the one who made the changes (why he didn't
>  follow approved procedures?) while "hidden" non-documented non-accounted
>  changes to a live production server *is* a *real* danger.

Yes, so being able to do "cvs diff" afterwards, and reveal all those
changes is a pretty neat thing, isn't it?  :-)  (Well, it won't easily
tell you file metadata changes, but you can find it out.)

Also, Greg Woods wrote (in reply to Jesus's comments above, not mine):

> yes, exactly!  ;-)
> 
> Any real Professional Engineer with experience managing something like a
> large mainframe installation will scoff at even the processes and
> procedures you and I might use to more carefully do accountable and
> repeatable change management in a unix environment.

I can well believe that.  To my way of thinking, then, the schemes sit
like this:

No control at all --> some really simple scheme -->  my scheme -->
  the Woods scheme  -->  a mainframe scheme

> > But also mandating this mechanism as the way to enforce discipline
> > seems a bit draconian.  There are other ways - requiring a process of
> > peer review of changes by another administrator would be one; logging
> > each change would be another; using my scheme is another.
> > 
>  
>  I knew once a *good* proffesional on these fields.  He sayed to me:
>  Sysadmin is not a democracy but a dictatorship: it's the way to go, it's
>  the way that works, and that's all to be said.  My own experiences from
>  those old days to-date are that he was real real real right.

Agreed.  There's a lot to be said for that attitude.


>  Considerable burden to the process of management? Sure?  *If* you want
>  to have *any* "process of management" then you *must* control the build
>  process, or you won't have any management at all.  Full stop.

cvs gives you some pretty reasonable control.  So you do have some
management.  It's probably the key thing I like about my scheme: it's
really non-intrusive, yet it provides a useful (if not perfect) safety
net.  It's a pragmatic system.  I'm not claiming it's perfect.

> > > > Maybe the clue is in your mention of "target system(s)".  Perhaps you
> > > > were pushing out the changes to multiple machines.  My scheme is
> > > > designed to work locally, just on the local machine.
> > 
> > Okay, good, then I have understood.  You were trying to manage
> > something more complex than I am - not just the config of the local
> > machine, but of others too.  (I imagine that differences between each
> > machine would raise interesting problems, and I imagine push you towards
> > something larger and more powerful like cfengine to solve that bigger
> > problem.)
> > 
> > >  This is yet another reason why you _want_ an intermediate "build" step!
> > 
> > Yes, it's a reason why you need a build step - *if* you're managing
> > more than just the local machine.
> > 
>  
>  No.  You forget you *always* have a build stage, be you aware of it or
>  not. If you use, let's say, webmin, the build step is as simple as
>  pressing the "commit changes" button, but it's there.  If you vi-edit
>  /etc/password by hand, there you have your build stage.  Surely you can
>  use a more complex build process if you need to cope with more variables
>  (like more machines) or need more control.  As an example, I use more or
>  less the same approach Woods is talking about.  But in most cases I
>  manage (only myself) a few so different boxes that it doesn't pay to
>  have an explicit build process/scripts.  But *there is* a build process;
>  it's only that it is on my mind and/or layed out on the system's
>  documentation and it is enforced by means of my only volunt of acomplish
>  it.

I think the difference in effort of the build step is actually very
significant.  If what you're calling the build step is as easy as
clicking a button or typing ZZ, then that's significantly different to
spending an hour working out what that rpm -i did, and then modifying
the sources to do (hopefully) the same thing, then testing it, then
doing the build step and pushing those changes out.

> > [...]
> > >  No, I solved the exact same problem you're attempting to sove, just in a
> > 
> > No, you weren't solving the exact same problem.  You've just explained
> > how you were solving a superset of my problem.  Since I'm trying to
> > solve a simpler problem, I don't see why the solution can't be simpler.
> > 
>  
>  Because as you yourself are telling us, your simple solution doesn't
>  reach the goals you were trying to acomplish from the very beginning
>  (why would you need/want a source control process if you end admitting
>  you won't have acountability nor repeatibility).

Eh?  My simple solution has achieved the goals I was aiming at.  I
don't know why you think it doesn't.  My system has exactly the same
accountability as cvs + su logs have.  And I was never trying to clone
one system's configuration onto another, since I'm assuming they may
well be different.  (If that's what you mean by repeatability.)

Thanks for your helpful reply.

Regards,

luke