Re: Part 2: System Structure

[Jonathan S. Shapiro]

On Wed, 2006-05-17 at 17:20 +0200, Bas Wijnen wrote:

> Well, perhaps it is all much easier if the program cooperates in being
> debugged.  But with a transparent space bank, it is at least possible to do it
> with a program that doesn't cooperate.  If I understand you correctly, this is
> not the case with a malicious program which was originally set up to be
> debuggable.

It is not. The authority to *read* a program image does not in any way
convey the authority to *write* the program image (e.g. to insert BPT),
nor to halt the process, nor to alter the process registers.

> Ehm, ok.  Now I don't understand you at all anymore.  I thought the proble of
> dynamic libraries was that the code can be different when it's run than when
> it was tested.  That isn't the case for static libraries.  But now the problem
> is that the code is prepared at all?

We are definitely talking past each other.

The problem is that you test the code in one environment, and execute it
in another. If critical code is isolated in a separate address space,
then the *only* way it can be invoked is through its intended interface.
Stray pointers and/or changes to global variables that are intended to
be private to the implementation cannot happen.

If the critical code shares an address space with its caller --
regardless of whether the library is dynamic or static -- all of these
errors are possible.

A direct consequence is that no isolation of faults exists in principle.
If exec() screws up, you have no idea whether the problem lies in exec()
or in some weird behavior of the program.

So my argument is: if you want things to be robust, a key step is to
isolate them from the errors of other parts of the system.

You can't have this and also be in the same address space.

More precisely: routines that involve no mutable data *can* have this,
but the minute you have a mutable data item (such as a global, or a
stack frame), isolation starts to decay very quickly in practice.

> 
> > Your statement that "preparing" is sufficient is completely contrary to
> > the entire history of system building experience since the beginning of
> > computer science. The plain fact is that stray pointers exist, and there
> > is absolutely no amount of preparation that can protect you from this
> > error if you are in a single address space.
> 
> Ah, so you say the problem is that a stray pointer of the parent could mess up
> the child?

This isn't a parent/child thing. You started this discussion by saying
that you were planning to put the constructor functionality into a
library. The concern here is the complete failure of isolation between
the application code and the library code/data.

> If the parent [or any application] has a stray pointer,
> then I'd say it's failing.

Of course it is failing. That isn't the question. The question is:

  You have a million line application with a stray pointer error
  somewhere that manifests only when the moon is full, it is raining
  in shanghai, and you tap the power cord with your left foot at just
  the right frequency.

  How the @)_&% do you find it?

>   It's just as likely that it messes up itself, or
> gets a segmentation fault.

The empirical evidence is that this really isn't true. A very large
number of stray pointers result from uninitialized data, and there is a
substantial likelihood that the old value at that location is a pointer
to something valid. In consequence, many stray pointer errors do NOT
result in SEGV.

Meta-comment: as designers, we often assume randomness whenever we
encounter a problem that is too hard to understand. The process goes
something like "Well, I don't know what it will be, but if I can cope
with a completely random error here I am okay." The problem with this is
that "coping with random errors" often turns into a statistical matter:
"I can handle 99% of the random errors, and that's good enough in
practice".

Unfortunately, if the original problem wasn't random, or if it was
mostly random but with internal correlations, this entire model of
problem simplification is inappropriate.

Meta-comment 2: Computers are deterministic systems. We actually have to
work EXTREMELY hard to get randomness. In my experience, "random" errors
almost universally result from deterministic causes.

So I am very suspicious of arguments of the form "X is just as likely as
Y" without hard data and a good causal story for why the outcomes are
indeed just as likely.

> > We are operating from fundamentally different assumptions. I do not
> > accept the assumption that a viable system can be built in which no
> > child program can every guard its runtime memory image from its parent.
> 
> But with my definitions for child and parent, there really is no practical
> difference.  Look at it this way: The meta-constructor does not have a parent.
> Every constructor is a child of the meta-constructor.  Every other program is
> the child of a constructor, and a grand-child of the meta-constructor.

Yes, but who supplied the storage for the new yield? I do not believe
that all programs that guard against their parents will be able to live
in the upper part of the process hierarchy.

> If program A wants to spawn an instance of program B, it invokes a capability
> to the B-constructor.  The B-constructor then builds B.  When all this has
> happened, A and B are both children of their own constructor, which are
> children of the meta-constructor.  In particular, B is not a child of A.  So B
> has successfully guarded its runtime memory image from A (which with these
> definitions isn't the parent, but I think it would be in the situation you
> were envisioning).

No, because B was built using storage provided by A, so B is transparent
to A, so B cannot guard its content against A.

> > > > If opaqueness is the default, and you leave out the "make it visible"
> > > > option, your program tends to break very quickly and you notice it.
> > > 
> > > Not at all.  In my use case above, it took 3 years (which obviously was an
> > > arbitrary "long time").
> > 
> > If you really managed to go 3 years without noticing that you weren't
> > transparent, then you didn't need to be transparent.
> 
> So how do you want me to debug this program?  By just looking at the code,
> trying to find a place which might be corrupted over time?  That just isn't
> going to work.  Agreed, with a debugger it might not work either.  But chances
> are much better.

Two answers:

In production, if the program must guard it's content, the whole point
is that you MUST NOT be able to debug it without special privilege.

In development, or for programs that are not sensitive, you add a bit of
protocol to the program:

        object->GetDebuggingCapability()

or you modify the constructor protocol to return this in the first
place.

> > We can let the user provide any storage we need, except for parts which
> > > must not be disclosed to them.  Obviously a buffer for an IPC transfer may
> > > be disclosed to the user.
> > 
> > No, you cannot. Any storage supplied by the user is destroyable by the
> > user, and the service must then guard itself against the possibility of
> > memory faults. From a strictly pragmatic perspective this is not really
> > feasible in the general case. There are a few restricted idioms where
> > this is survivable, but they are *very* restricted for reasons of
> > pragmatics.
> 
> And how is this different if the user would be able to give out (revocable)
> opaque memory?

In the opaque memory case the entire design pattern is different.
Instead of providing storage to a pre-existing process, the user
instantiates an entire sub-process using storage provided by the user.
The user can destroy the storage, but this destroys the entire
sub-process.

The problem here isn't the destruction of storage per se. It is the fact
that the destruction of storage used by the child wasn't "all or
nothing".

> > I have an enormous number of examples where private storage is
> > essential. All of them have been rejected here (without, so far as I can
> > tell, any semblance of serious consideration) on grounds of ideology.
> 
> I don't remember it as such, but will look back in the archives.

If you go back and look at my use cases for confinement, you will find
that Marcus read all of them with excessive haste, misunderstood them
completely, and then dismissed them as "not Hurd objectives" without
(apparently) understanding them.

> > It is legitimate for the Hurd project to introduce and entire new area
> > (inter-party interactions on a single machine) of "socially beneficial
> > restriction" as an experiment, and to lobby that this is the right thing
> > to do. However, it is important to recognize that this has HUGE
> > implications, and that FSF may or may not wish to be associated with
> > such a major change. At the very least, FSF input must be obtained
> > before adopting a policy of this level of intrusiveness in an
> > FSF-sponsored project.
> 
> I think you're making this bigger than it is.  All we propose is not to
> implement something that most current systems don't have either.  I don't
> think we should be forced to implement such a thing.

I disagree with this technical description.

In UNIX, it is possible for me to execute a setuid program that I cannot
debug or inspect.

So it is apparent that analogous mechanisms exist. This is not as new as
you want to make it.


shap