Re: Future Direction of GNU Hurd?

[Paul Boddie]

On Saturday, 12 September 2020 01:53:10 CEST Richard Braun wrote:
> On Fri, Sep 11, 2020 at 11:52:02PM +0200, Paul Boddie wrote:
> > Social factors: you will see plenty of inertia or what people used to call
> > "stop energy" in the mailing lists. Just review the previous messages on
> > this list or look at recent messages on bug-hurd for examples. One gets
> > the impression that unless someone is either a top-level operating system
> > architect and/or willing to burn themselves out fixing up huge amounts of
> > existing code or writing new code in precisely the way envisaged by
> > others who nevertheless aren't going to be writing any themselves, then
> > new contributions are not really welcome.
> 
> Well it doesn't need to be "precisely" the way those who aren't active
> at the time like me imagine it. Good surprises are welcome too, but those
> are rare.

Let's be honest with ourselves here: no top-level OS architect is likely to 
roll up and work on the Hurd at this point, partly because they will have 
their own interests which don't intersect with the visions of the project, or 
perhaps they have already realised the visions of the project that interest 
them.

After all, despite the almost singular emphasis in advocacy around the project 
of translators being the Hurd's most compelling feature, which seems like a 
hard sell to most people these days, there are other systems that have 
implemented configurable namespaces and other related features over the years. 
Bell Labs managed to do at least two of them in all this time, but such 
systems tend to get dismissed because they weren't "done right" or whatever 
excuse tends to be offered to pursue the usual turf wars and factionalism.

> You seem to imply one doesn't need to be "a top-level operating system
> and/or willing to burn themselves out fixing up huge amounts of existing
> code" to work on something like the Hurd. While I'm not personally requiring
> "top-level" anything, I very, very, very strongly believe that a project
> that combines concurrency, virtual memory, and distribution, requires
> developers who care deeply about both the big picture and small details,
> and to me, that's usually what makes a top-level developer. Besides,
> considering how badly designed some critical parts of the code are, yes, it
> would require a lot of work to get that fixed. So yes, we need people with
> both great technical skills and very hard-working. That too is rare.

It certainly is challenging to work with concurrent systems, and I can say 
from personal experience that it can be very frustrating. Generally, though, 
people can only get to the point of being a "top-level developer" by learning 
from others and collaborating with them. And on a particular code base, they 
need opportunities to become familiar with that code, which will necessarily 
involve actually writing, testing and debugging that code.

If you don't provide opportunities for people to get to the right level, and 
in practice precisely no-one will be - because even an expert will not be 
familiar with the code, particularly with the documentation available - then 
it is like you are waiting for some very generous experts to show up and get 
their hands dirty for no other reward than the hassle involved and the vague 
possibility of a somewhat better system down the road. Or to use a sports 
metaphor, the talent development strategy of this particular team would appear 
to be to neglect the development of players already with the team, instead 
expecting that the league's big-name players will show up and offer to play 
for free.

And that brings me nicely to a point about expecting volunteers to show up and 
burn out on doing what would really be paid work in other contexts. Quite why 
people should emphasise their ethics around Free Software, only to advocate 
the abandonment of ethics around treating people decently and making sure that 
paid work is actually paid, remains one of the biggest contradictions of the 
Free Software movement.

> > I recently read an article about Linux kernel development where someone
> > actually wrote that if something or other made kernel development easier
> > or more accessible then it would be attracting the "wrong kind" of people.
> > Well, that would seem to be the view of some people in the Hurd
> > development community, too, judging from recently expressed sentiments.
> 
> That's absolutely not correct about the Hurd, however. I guess you're
> referring to me saying "I don't want to work with people who give up
> because it takes them more than a few seconds to find information".
> Note that I'm absolutely not implying to artificially make finding
> things hard.

So why are you so against people improving the Web site and documentation?

> They just are, and I'm not aware of any tool that grabs a big, old,
> fragmented code base like that of the Hurd, and makes it easy and quick to
> understand.

Some projects might generate reference documentation using automated tools. 
That would only be a start, however: where projects try and use reference 
documentation as the primary means of communicating the details of a system, 
they tend to fail to communicate architectural details, and other kinds of 
documentation just tend to get lost or obscured. Documentation is chronically 
undervalued in the software development profession, although I feel that I 
might be wasting my time making that point in this discussion.

> Serious projects need the right kind of lazy people, those who try to do
> things well from the start to avoid wasting time fixing bugs and rewriting
> large code blocks, instead of the wrong kind of lazy people, who just give
> up quickly and are negligent.

I can understand that no-one wants to spend time interacting with people who 
like the idea of contributing to a project but who don't have the skills or 
patience to perform certain kinds of tasks. But a project like the Hurd - a 
large-scale software project - has requirements beyond people working on 
fixing bugs and rewriting large code blocks.

> > Of course, many other people in the wider world realise that successful
> > projects require many different types of contributors in order to actually
> > be successful, and judging by the increasing stagnation of what was a
> > fairly well-resourced and well-publicised project, I think we can guess
> > whose viewpoint is the more realistic of the two.
> 
> Again, not correct. I said I didn't want "someone not fit for the job"
> and you're intepreting I don't want "different types of contributors".
> I also would like to know when the Hurd was well-resourced these last
> twenty years.

Well, to respond to your last remark, there was a considerably larger amount 
of interest and list traffic in previous years, meaning that some people must 
have had time and energy to pursue matters related to the Hurd. Furthermore, 
the Free Software Foundation was still directing people towards the project 
and must have offered some kind of assistance. If you are saying that they 
didn't pay anyone then I can only refer back to my point about volunteer 
culture in Free Software.

> Overall, you seem to claim that the Hurd would be better off with
> many mediocre developers than very few excellent ones. In addition,
> your argument makes a vague comparison between the Hurd, which is at
> its core an amibitious and technically difficult project, and other
> "successful projects" which are statistically far from being that
> difficult, not to mention the survival bias.

Actually, I am claiming that a more diverse range of contributors would be 
beneficial for the Hurd, taking into consideration more than just developers 
fixing up the code as it is today. And even within the realm of development, I 
cannot really see why there cannot be a range of different levels of 
developer. After all, much of the software development going on in the wider 
world encompasses a range of tasks from those which are technically and 
intellectually challenging through to mundane tooling and maintenance.

I actually doubt that all of the development tasks related to the Hurd are the 
most challenging kind, and if they do happen to be so, then it says quite a 
bit about the viability of the project and the architecture of the system. One 
benefit of a microkernel-based system should be that certain components should 
actually be more approachable because you can write them as "normal" programs 
and not necessarily be subject to particularly onerous constraints (although 
there will always be challenges of certain kinds). This particular benefit has 
never seemed to be worth communicating with regard to the Hurd, suggesting 
that the project has either undervalued it or the software architecture fails 
to leverage or deliver such a benefit.

As for the "survival bias" remark, what you are saying that the bulk of 
software development isn't challenging and so recommended practices somehow 
don't apply here. What I was saying is that established practices for the 
development of systems tend to emphasise the involvement of many different 
roles. Naturally, small-scale systems can have a few people doing everything 
and even neglecting various roles if they have everything they need to know in 
their own heads. This doesn't scale up to large and/or complicated systems, 
necessitating the involvement of people who take on distinct roles that may 
have little to do with the most challenging development tasks.

> Well I simply disagree, I'd rather see the Hurd stagnate in a form that
> gives it a chance to take off again than evolve into a hopeless abomination.

Given that people can do whatever they like with the code and the original 
non-hopeless code will remain available, what would appear to be missing, 
then, is a constructively formulated plan that would take the system in what 
would supposedly be the right direction.

> Productivity can be very low, even negative, i.e. doing more work fixing
> contributions than the work put in the contributions themselves.
> Fine-grained multithreading and parallelism/distribution in general are
> particularly prone to create such situations.

Yes, certain development tasks and situations are challenging. The other 
issues are simply project management issues. If someone improving the Web site 
and documentation is somehow driving down productivity of the "top-level 
developers" then, well, you are doing it wrong.

> In the end, those statements are morally charged misinformation,
> which is simply annoying.

All I noted were the sentiments expressed about the kind of contributors that 
would appear to be welcome in the project, and you have pretty much confirmed 
that this is your position. So, I think that "morally charged misinformation" 
must be another way of saying that you don't agree with me, which is fine, and 
that you don't appreciate me pointing things out.

But since I merely aimed to give the original enquirer some idea of the state 
of the project and to temper his expectations, I think I have been fairly 
accurate and hopefully of some assistance.

Paul