Status report on the conversion of the Hurd to pthreads

[Richard Braun]

Hello,

With the help of Barry deFreese and Thomas DiModica, a few machines are
currently running Hurd systems with no code dependency on the previous
cthreads library. Concerning upstream code, the appropriate branches can
be found there :
http://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/hurd_with_pthreads
http://git.savannah.gnu.org/cgit/hurd/libpthread.git/log/?h=rbraun/hurd_with_pthreads
http://git.savannah.gnu.org/cgit/hurd/hurd.git/log/?h=rbraun/hurd_with_pthreads

The Debian-specific branches are available at :
http://git.sceen.net/rbraun/debian_hurd.git/
http://git.sceen.net/rbraun/debian_netdde.git/

Debian packages are available for testing using this repository :
deb http://ftp.sceen.net/debian-hurd experimental/
deb-src http://ftp.sceen.net/debian-hurd experimental/

Upgrade glibc packages first, then the hurd and netdde ones.


The tests run so far show that almost everything behaves as it used to
with cthreads. Very few problems have been seen and most of them already
exist in the original implementation as well. The only major issue faced
concerns the "aggressive multithreading" of most Hurd servers. Some
slight differences between cthreads and libpthread cause servers to
suffer from starvation a lot more than they did. Although various
workarounds have been tried, this problem appears to have no real
solution without a design change. It happens when messages are received
faster than they can be processed, which cause servers to spawn new
threads to handle those messages. Limiting the number of threads isn't
an appropriate solution since some servers block until an expected
message arrives. Throttling thread creation (either based on time or
with thread priority as it is done in the current implementation) only
reduces the severity of the problem without solving it. The libports
ports_manage_port_operations_multithread function accepts timeouts to
regulate the lifetime of worker threads, but it only partially helps
since threads are detached, which means their stack remains allocated
(although this could be limited with some form of garbage collection in
libpthread).

On the other hand, one major source of "message flood" is the select
call. When select returns, either because of an event or a timeout,
reply ports are destroyed, which in turn cause as many dead name
notifications to be received by the server. It's a bit surprising to
discover that the reply port for such a call is passed as a normal
send right (using MACH_MSG_TYPE_MAKE_SEND). A send-once right would
probably be more appropriate. Future work will attempt to reduce the
message floods at their source so that libpthread based Hurd servers
can be used until a redesign allows really fixing this issue.

Thanks.

-- 
Richard Braun