Re: Concurrency via isolated process/thread

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: luangruo@yahoo.com, emacs-devel@gnu.org
> Date: Sun, 09 Jul 2023 09:36:15 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > Which variables can safely and usefully be made thread-local?
> >> 
> >> PT, ZV, BEGV
> >
> > Even that is not enough: you forgot the gap.
> 
> Am I missing something, does the gap remain intact when the just move
> point? AFAIU, the gap only needs to move when we do edits.

That's not the issue.  The issue is that without the gap being
accurate Emacs cannot convert buffer positions to pointers to buffer
text.  So not saving the gap is asking for trouble.

Once again: please do NOT try designing Emacs features based on what
you happen to see in the current code.  First, it is easy to miss
important stuff that invalidates your design; and second, code does
change over time, and if you introduce a feature that assumes some
changes will never happen, you are introducing ticking time bombs into
Emacs.  So instead, you need to understand the assumptions and the
invariants on which the code relies, and either keep them or go over
everything and adapt the code to assumptions that are no longer true.
In this case, the assumption is that the gap is always accurate except
for short periods of time, during which buffer text cannot be accessed
via buffer positions.

> I was thinking about thread-local variables just to move around and read
> buffer.

You will see in xml.c that we move the gap even though we do not
"edit" (i.e. do not modify the buffer).  We do this so that a pointer
to a portion of buffer text could be passed to an external library as
a simple C char array -- a legitimate technique that must be
available.  So even reading the buffer sometimes might require moving
the gap.

> Asynchronous writing is probably a poor idea anyway - the very
> idea of a gap does not work well when we need to write in multiple
> far-away places in buffer.

What if the main thread modifies buffer text, while one of the other
threads wants to read from it?

> >> and the buffer-local variables that are represented by the
> >> global C variables.
> >
> > That's a lot!
> 
> Do you mean that "a lot" is bad?

Yes, because it will require a huge thread-local storage.

> >> > I don't see how this could be practically useful.
> >> 
> >> For example, `org-element-interpret-data' converts Org mode AST to
> >> string. Just now, I tried it using AST of one of my large Org buffers.
> >> It took 150seconds to complete, while blocking Emacs.
> >
> > It isn't side-effect-free, though.
> 
> It is, just not declared so.

No, it isn't.  For starters, it changes obarray.

> >> Yes. I mean... look at Haskell. There is no shortage of pure functional
> >> libraries there.
> >
> > I cannot follow you there: I don't know Haskell.
> 
> In short, pure functions in Haskell can utilize multiple CPUs
> automatically, without programmers explicitly writing code for
> multi-threading support.
> https://wiki.haskell.org/Parallelism

Thanks, but I'm afraid this all is a bit academic.  Haskell is a
language, whereas Emacs is a text-processing program.  So Emacs
doesn't only define a programming language, it also implements gobs of
APIs and low-level subroutines whose purpose is to facilitate a
specific class of applications.  The huge global state that we have is
due to this latter aspect of Emacs, not to the design of the language.

> > ... More importantly, when you call some function from
> > simple.el, how do you know whether all of its subroutines and
> > primitives are 'pure'?
> 
> We do not, but it may be possible to add assertions that will ensure
> purity in whatever sense we need.

Those assertions will fire in any useful program with 100% certainty.
Imagine the plight of an Emacs Lisp programmer who has to write and
debug such programs.

We have in Emacs gazillion lines of Lisp code, written, debugged, and
tested during 4 decades.  We use those, almost without thinking, every
day for writing Lisp programs.  What you suggest means throwing away
most of that and starting from scratch.

I mean, take the simplest thing, like save-buffer-excursion or
with-selected-window, something that we use all the time, and look how
much of the global state they access and change.  Then imagine that
you don't have these and need to write programs that switch buffers
and windows temporarily in thread-safe way.  Then reflect on what this
means for all the other useful APIs and subroutines we have.