cache consistency

[Marcus Brinkmann]

At Mon, 25 Oct 2004 22:07:46 +0100,
Neal H. Walfield wrote:
> What about cache consistency?  The server has the device driver read
> data into a container.  When the device driver returns, the container
> has the data and the file system takes a COW copy into its cache as
> part of its extra page allocation.  If the client is able to
> manipulate the data between the DMA operation and the placement of the
> data into the cache, there is an opportunity for corruption.  (Hence
> the exclusive lock.)

You are right.  I did not properly address that (in fact, I think my
earlier reply was quite bogus).

So, you want to do the following:

User creates container, gives access to filesystem.
The filesystem gives access to the same container to the driver.
The driver does its thing.
The filesystem copies the frames into its cache.
The filesystem returns.

An alternative would be to have the filesystem share a container it
owns with the driver, and copy frames from there to the user
container.  This would mean that the user container is not used at the
device driver level at all, but it would also mean that the filesystem
has to temporarily pay for the physical memory used in the
transaction.  Still, sharing a container with a driver may have its
own advantage, as it cuts down the number of RPCs.

For the original plan, we need some form of locking operation.  This
is not easy.

I think it has to be something like the following:

The client creates a limited-access proxy container object P from the
container C.  It passes this to a server S.  The server S can lock the
object P, which removes all mappings related to C (made through C or
P), prevents future use of C _and_ P for any dangerous operations, and
returns a new proxy object Q, through which now the server has
exclusive access to the dangerous operations.

The Q is passed to the next lower layer, like the device driver, but
it could be some intermediate server T, too.  If that intermediate
server T wants to protect against changes to the container from S, it
can lock the object Q, which will again destroy all mappings made from
C, P or Q, forbid all dangerous operations on C, P or Q, and returns a
new proxy object R.

At any time, you can break the lock from an upper layer.  For example,
by destroying P you can invalidate Q and all proxy objects made from
it.  Breaking a lock will also destroy all mappings (only lower level
mappings can exist because of the lock).

The call to create P can in fact be the same as the one to create Q or
R.  Ie, "locking" a real container produces the first proxy object.

This sounds excessive, but seems to be required to handle this while
still supporting multiple layers which don't trust each other.  Ie,
this scheme supports some sort of locking hierarchy, or locking
recursively.

This needs more thought, IMO.

Thanks,
Marcus