Re: I/O permission control in OSKit-Mach

[Roland McGrath]

> On Sat, Oct 20, 2001 at 10:42:35PM -0400, Roland McGrath wrote:
> > Indeed.  I think the best plan is to scrutinize the pmap locking strategy
> > and do something similar.
> 
> I am looking into this, but it looks quite hard.  In fact, it requires an
> interrupt to signal the other processors (SMP_IPI_VECTOR == 13 is used for
> pmap changes).  Is there a free interrupt that can be used?  Can we
> piggyback on the pmap interrupt without increasing the cost for it?

There is only one interprocessor interrupt supported in the oskit smp
library interface.  The interrupt handler function was
pmap_update_interrupt because that was the only thing that needed to be
done.  It will probably be worthwhile to rewrite the oskit smp library
using the newest Linux smp support code, and generalize the interface to
support multiple interrupts.  The hardware supports multiple interrupts and
the Linux 2.4.x code uses different ones for different things.  But as an
interim solution, it's not a big deal to have one interrupt handler that
has to check flags to see what it needs to do.

> I have only a vague idea how the pmap interrupt stuff works, the comments in
> i386/intel/pmap.c helped.  However, they also discouraged me a bit from this
> complex solution.

I too know only what I read.  But the comments and code in pmap.c are
pretty easy to understand.  It has to synchronize all processors using the
pmap before it can modify it, and then all those processors have to process
the pmap updates after the CPU doing the change unlocks.  

For the io bitmap you don't really need quite as much synchronization,
because there is no harm in diddling the task's data structures while other
CPUs are still running.  You just need to wake up the other CPUs before the
ioperm call returns to the user.  I don't think you even really need to
synchronize with them, because anything they might still be doing while you
are returning is not distinguishable to the user from them having done it
before the ioperm call finished.

But because pmap maps 1-to-1 with task and that's what you need, you can
use the pmap state to know which CPUs are running in your task and need to
be updated (pmap->cpus_using).  

> Makes me wonder how Linux behaves, I think that I/O bit mangling is task
> based here, but as threads are implemented as tasks, I don't think it works
> out there the way we want it to behave for us.

Linux doesn't really have threads per se, it has "tasks" that might share
an address space.  So essentially ioperm is thread-based, not task-based.

> I am inclined to postpone this, and just have a simple lock for the other
> races and a #warning for SMP machines, that other cpus will only pick it up
> after they are rescheduled.

Ok, fine.  There are many unfinished things in SMP, and none of them should
delay any work on features for the uniprocessor case.

But note that this is not really "until rescheduled" (switching to a
different thread), it's until the CPU switches to a different task and back.