Re: proxy memory objects patch

[Marcus Brinkmann]

On Wed, Nov 20, 2002 at 09:16:12PM -0500, Roland McGrath wrote:
> If you can implement the interface that takes many objects and ranges from
> each, please do that.  The reason to want this is so that we can use it to
> compose proxy memory objects that span adjacent memory objects of logically
> contiguous underlying objects like files and disks.  That lets us use the
> new io_map interface that I prefer, and keep the mmap atomic to the user,
> for 64-bit files/devices.  (Remember the RPCs were all updated except for
> io_map, which is still constrained to the first 4G of the file.)

So we want the interface to be right to anticipate such a nifty version of
proxy memory objects.  What interface do you want?  Resolving this question
is the last step toward plugging that hole.

We have: memory objects, protection, linear ranges.  Thomas suggested we
need an array of linear ranges and you say we need an array of objects.  I
can only consider an array of arbitrarily long arrays a nuisance, so the
only thing I can come up with to make both reasonably possible is the
following syntax (for both variants, the memory object and the mach hack
interface):

create_proxy (..., memory_object_t object[], vm_prot_t maxprot[],
              vm_size_t start[], vm_size_t end[]);

with the following meaning:  All arrays should have the same size (with one
exception, see below).  The values in the arrays with the same index are
related to each other:

object[0], maxprot[0], start[0], end[0]
object[1], maxprot[1], start[1], end[1]
object[2], maxprot[2], start[2], end[2]
...

The proxy memory object will have the protection maxprot[0] in the range
[0;end[0]-start[0]], and the range [0;end[0]-start[0]] is mapped to
[start[0];end[0]] in the first memory object.  The range
[end[0]-start[0]+1;end[1]-start[1]] of the proxy object is mapped to
[start[1];end[1] in the second object, and has the protection maxprot[1].
And so on for the other indices.

Note that you can specify MACH_PORT_NULL as a memobj if you want a hole.

Now, we only need one more feature, and that is to allow to omit the size
for the last memory object listed, to make it open ended.  So, the arrays of
start and end are allowed to be one entry shorter than the others
(alternatively, specifying END to be smaller than START would have the same
effect), and in this case the whole last memory object is pasted at the end
of the proxy object with the given maximum protection.  If only one memory
object is given, then you can simply omit those arrays entirely.  This makes
the currently needed syntax convenient:
create_proxy (..., &memobj, 1, &maxprot, 1, NULL, 0, NULL, 0);
while making all other variants possible.  It's a bit silly that you have to
specify the protection via a real variable, but well, that's the price.

Variants:

There are a couple of variants thinkable.  All these variants have in common
that they can not express all possible proxy memory object variations.  To
construct any arbitrary memory object, you would have to create several
objects and then paste them together via another call.  The implementation
would then recurse over the proxy objects and resolve the nesting.

One variant that seems feasible is to simply specify the protection that
should apply to all ranges in all objects:

create_proxy (..., memory_object_t object[], vm_prot_t maxprot,
              vm_size_t start[], vm_size_t end[]);

This makes the simple case very convenient:
create_proxy (..., &memobj, 1, VM_PROT_READ, NULL, 0, NULL, 0);

To get a memory object proxy that combines several ranges in objects with
different protection, you would need to create one proxy object for each
interval which has the same protection, and then paste them together.

Another variant that is quite specific but should work well in the cases we
want to apply is the following:
create_proxy (..., memory_object_t memobj, vm_prot_t maxprot,
              vm_size_t start, vm_size_t end, memory_object_t memobj2);

This one has the following meaning: Create a proxy memory object from
memobj with maxprot prot and, if start <= end, from the range start to end
(ie, a remapping).  If memobj2 is not MACH_PORT_NULL, paste the memobj2 to
the end of that memory object.  (memobj2 is only allowed to be not
MACH_PORT_NULL if start <= end).  This is certainly inconvenient if you have
to paste many object or ranges.  But for the two cases we might need it, it's
perfect:

create_proxy (..., memobj, VM_PROT_READ, 1, 0, MACH_PORT_NULL);

and to glue together the end of the first 4gb of a large file to the rest of
it:

create_proxy (..., memobj[0], VM_PROT_READ|VM_PROT_WRITE, 3.9gb, 4gb, 
memobj[1]);

Well, I don't fancy this one very much, but I wanted to mention it to spur
creativity.  I don't really care as I won't implement anything but the most
simple case for now.

All other variations would be much more inconvenient in the generic case and
don't buy you much in the simple case, so I don't think I will list any of
them here.

But maybe you have a different, much better idea.

Thanks,
Marcus


-- 
`Rhubarb is no Egyptian god.' GNU      http://www.gnu.org    marcus@gnu.org
Marcus Brinkmann              The Hurd http://www.gnu.org/software/hurd/
Marcus.Brinkmann@ruhr-uni-bochum.de
http://www.marcus-brinkmann.de/