Hi -
I've been wondering about using Hurd on a cluster computer; i.e, a configuration where each node has multiple identical cores and its own memory. For example, an eight node cluster where each node has 8 GB of RAM and eight cores. I stress that the cores are identical, so that processes can run on any core and even migrate between them.
I'd like to see the whole thing running an integrated POSIX operating system. So, when I run 'top', I see 64 processors and 64 GB of RAM.
Code tuned for this architecture might behave like this: A program forks eight processes, and each process spawns eight threads. Our basic programming paradigm is that threads share memory and run on the same node, while processes do not share memory and likely run on different nodes. We can setup shared memory between processes (System V IPC), but we know that this is expensive because it has to be emulated, so we try to avoid it. Process migration is expensive, too, so we try to avoid it as well. In the example, we migrate seven of our eight processes, right at the beginning, during program initialization, take the performance hit once, then leave them to run on their separate nodes.
All we really have between the nodes is a fast LAN, so we can do message passing. Yet that's exactly what Mach/Hurd is designed for - a kernel built around message passing.
Can Hurd work, well, in such an environment?
Since I haven't dug into a single line of Hurd, I don't know - that's why I'm asking. I've done some homework, though, and there are some things that I am aware of.
First, it's basically Mach that would have to be modified, right? Changes to Hurd servers might be required for performance reasons, but so long as Mach works on the cluster, Hurd should work.
Next, Mach/Hurd's memory limitations and 32-bit pointers. My first through was to ignore it for right now, since these are well known problems. If we could get Hurd running at all on a cluster computer, then we've have to come back and make sure it can actually use the entire 8 GB of RAM on a single node. Yet I'm not sure. There might be situations where we have to address the entire cluster's RAM, even though accessing a non-local part of it will be slow.
Sending large blocks of data in Mach messages becomes problematic, since we can't play shared memory games. It would have to be emulated, and avoided whenever possible. These are the kinds of changes that would be needed to the Hurd servers themselves - they can no longer assume that firing virtual memory across a port is fast.
In-order and guaranteed delivery. For the moment, let's assume that our LAN can do this natively. Since we're not going through routers, only a single Ethernet switch, maybe virtualized, this might work.
Can a Hurd network driver be built to pass kernel messages, or is this a huge problem? Something like, you load an Ethernet driver, and it has some kind of interface that allows Mach messages to be passed through it?