Re: Drivers

[Peter 'p2' De Schrijver]

> For one, with drivers at POSIX level, there is no need for a sophisticated,
> full-featured special driver framework: Library functions, loading and
> unloading drivers, communication amongst drivers and to the outside world,
> configuration management/parameter passing, resource management -- it's all
> already there, using the ordinary POSIX/Hurd interfaces. All we need are a few
> extensions to the standard POSIX/Hurd mechanisms, to allow for driver-specific
> stuff like IRQ handling and I/O port access.
> 

That depends on what you mean by 'POSIX level'. Drivers should be 'user
processes', but they don't need the full POSIX functionality. Actually
most of the posix functions are useless for driver programming. At most
you need basic C functions like memcpy etc. The standard posix
read/write interface is not suited for communication between drivers as
it's stream based. This means the driver has to figure out where the
message boundaries are which is a waste of CPU time and error prone. 

> Having no extra framework for drivers also means driver developement becomes
> much easier: No need to cope with some limited environment. No need to learn
> special APIs for a driver-specific library; driver registering/startup and
> shutdown; memory management; threading and locking; configuration management;
> communication to other drivers and the outside world; permission handling.
> Drivers are written just like any other translator, using all your normal
> progamming experience. All one has to know are a few fairly simple extensions
> to the ordinary POSIX/Hurd interfaces. The only specific APIs a driver 
> handles,
> are access to lower level drivers via their filesystem interfaces, and
> exporting an own filesystem to the world.
> 

I never found the 'limited environment' a problem for doing drivers.
Lack of clear specifications and correct understanding of the hardware 
operation is what generally causes mistakes. The differences in the way
hardware and software people look at a system obviously don't help here.

> Furthermore, drivers being ordinary programs removes the need for any
> (imperfect) magic making drivers more accessible by giving them some semantics
> of ordinary processes -- they just *are* ordinary processes, with all the nice
> things that come with that. Perfect transparency.
> 

But they are not ordinary programs. You can't just kill -9 a driver and
hope the system survives for example. If you're lucky not much happens,
but likely some other programs will stop working and if you're unlucky
you seriously damaged your filesystem for example. This is because
hardware does things completely asynchronous from the main CPUs. Only
the driver knows how to properly shut down the hardware so resources can
be freed. 

> Transparency is also achieved by the fact that, in a hurdish manner, all
> connection/communication between the drivers happens through the filesystem.
> 
> There is one more advantage of drivers being ordinary programs, contributing 
> to
> the ease of driver developement: All the ordinary debugging tools, like GDB or
> rpctrace, can be used in the usual manner. The fact that we use standard
> filesystem operations for all the interfaces of a driver, also considerably
> helps debugging.
> 

You don't need to have full POSIX functionality to be able to debug an
L4 process. Also you should be very careful as stopping drivers can
cause the system to crash (eg. if you would stop the HD driver, other
programs will likely fail when they need to page in code for example.)
In general GDB is not very helpful for debugging drivers either. Some
sort of trace functionality makes much more sense in my experience.

> application program modules -- is possible without any anomalies caused by
> having to work at several different layers to manage a single piece of
> hardware. If some application needs access at a lower level than usual, there
> is no need for creating special interfaces circumventing the higher level
> driver parts. Just plug in at the desired level in the hierarchy -- the driver
> components not being any special, every program can take over their function.
> 

Most hardware has no provisions to allow access by multiple processes
safely. Probably the most important task of the driver is to provide
this multiplexing. 

> Hot-plugging isn't special anymore: From a fully dynamic system, with drivers
> always being launched explicitely (automatically on system startup and hotplug
> events, or manually by the user), to a totally static system, where drivers
> are set up only once and remain there (using passive translators), everything
> is possible. You can handle non-removable devices dynamically, to 
> automatically
> adapt to changed system configuration, or you can set up hot-pluggable devices
> statically, like a mouse that is always connected to the same USB port for
> example. You can even combine them in kind of an inverse manner, e.g. handle
> connecting to a docking station by dynamically inserting a tree that has
> drivers for all the devices in the docking station set up statically.
> 

The special thing about hotplugging is not the loading of the drivers
itself, but deciding which driver to load, having the driver cope with
unexpected removal of the device, handle power management properly, cope
with a (theoretically) unlimited number of device instances, ...

> Another thing that becomes trivial is starting drivers on demand: Just set a
> passive translator, and the system will take care of the rest.
> 
> The drivers themself can also be designed very flexibly: Maybe using a
> simplistic approach with a single process handling everything. Or separate the
> critical lowest-level parts (register access) from the more complex but
> uncritical higher-level parts into two separate components (like KGI/GGI 
> does).

I don't see how POSIX helps here. You can always divide functionality in
multiple processes and have them communicate in some way. (eg using L4
messages)

>    _Dependencies_
> 
> One interesting problem is handling dependencies between the drivers: What
> happens if a driver tries to access some functionality that is not available
> due to some other driver missing?
> 
> Note that this problem is not really specific to POSIX level drivers; it's 
> only
> somewhat more tricky, because the use of libc can make such dependencies less
> obvious. Nonetheless, after considering it for a while, it doesn't seem to be
> such a big problem after all: If we call a libc function that relies on some
> specific device, it will just generate an error, like it does in many other
> situations. The calling driver has to decide whether a failure in this call is
> non-critical and can be ignored, or it's better to bail out. Nothing special
> about it. The user (or driver manager) has to fix the order in which drivers
> are loaded, if such a prblem occurs. I don't think there is anything the 
> system
> can or needs to do about it.
> 
> A special case of dependencies are drivers referencing themselfs. A console
> driver for example obviously shouldn't try to print an error message on the
> screen. Note that such self-reference loops could go over several drivers, so
> they are not always obvious. (The error could happen in some lower-level 
> driver
> the console driver depends on, for example.) Still, this can be easily fixed:
> The driver just needs to set some kind of lock preventing it from reentering
> itself. Again, an issue to be handled by the individual drivers.
> 

It's much more difficult then that. Eg the VM generally relies on the
backingstore being available. So having the drivers for disk or network
rely on libraries being paged in on demand is obvious a recipe for a deadlock.

> 
>    _I/O_
> 
> The fundametal distinction of hardware drivers against normal programs is of
> course that they have to access the actual hardware devices. This means access
> to regions in the I/O address space, as well as special regions in the normal
> address space (for memory mapped I/O).
> 
> It is important to understand that drivers do *not* get any special privileges
> to do the hardware access, not available to other programs. All hardware 
> access
> 

It's important to have as few overhead as possible for accessing the
registers. Some devices (eg UART) are latency sensitive.

>    _DMA_
> 
> There are basically two kinds of DMA: The old ISA DMA uses a central DMA
> controller in the chipset, with a number of DMA channels statically assigned 
> to
> individual devices. This one is easy to handle: Simply use a driver for the 
> DMA
> controller, which handles requests from the drivers of devices using DMA. To
> make sure a client actually has the permission to read/write the requested
> memory region, it is required to map that region to the DMA driver.
> 

Some Non PC architectures have seperate DMA controllers which can
autonomously do memory - memory, memory - bus and bus - memory
transactions. They are very useful as DMA controllers can use the busses
more efficiently as they tend to have better capabilities in generating
burst transactions. Also the CPUs can continue processing while the DMA
is in progress. I believe it's sometimes more efficient to have the DMA
controller on the host side then on the PCI side, as read transactions
over a PCI bus are rather slow due to the bus turnaround times etc. So
in practice we need to support both efficiently.

> The other kind of DMA is more problematic: Modern systems (PCI) use a builtin
> DMA facility in the individual device, allowing the device to access RAM
> completely on its own. This means however that only the specific device driver
> knows how to setup DMA -- there is no way for the parent driver to prevent the
> child from doing something harmful, by writing wrong values into the device's
> address registers.
> 
> While this efficiently prevents really optimal robustness, the robustness can
> be improved by a few orders of magnitude using modular drivers, as explained
> above: Use an extra process (sub-driver) that is responsible exclusively for
> setting up DMA for the specific device. (Of course checking for a valid memory
> mapping from the requesting process beforehand.)
> 

The driver process needs to know the physical address of the pages
involved. There is no way around that in systems without an IO-MMU.

>    _IRQ_
> 
> I'm not sure how IRQs should be handled. If possible, I'd go for a solution
> using a central IRQ driver receiving all interrupts (L4 allows setting a
> receiver thread for each interrupt slot, which in trun gets the interrupts
> through special IPCs), and passing them on to the individual drivers through
> IRQ ports exported as a filesystem. Connecting to those ports would be managed
> by the bus drivers.
> 
> There are two problems with this approach: For one, the interrupt driver doing
> an RPC to the actual driver on each interrupt introduces an overhead, which 
> can
> be considerable in some extreme cases. (Fast serial ports and gigabit Ethernet
> can generate up to several hundred thousand interrupts per second.)
> 
> Also, I'm not sure whether a central IRQ driver could handle PCI shared IRQs.
> (Did I mention already that these should be outlawed? ;-) )
> 

The solution here is to have the driver directly listen for L4
interrupts. For shared IRQs the drivers have to cooperate. Basically a
driver forwards the IRQ message to the next driver unless it's the last
driver in the chain.


Cheers,

Peter (p2).

signature.asc
Description: Digital signature