Re: Concurrency via isolated process/thread

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: luangruo@yahoo.com, emacs-devel@gnu.org
> Date: Fri, 07 Jul 2023 15:17:23 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> maybe need to have multiple heaps.
> >
> > All modern implementation of malloc already do use several different
> > heaps internally.
> 
> I was talking about Elisp heaps.

I don't understand what you mean by "Elisp heaps".  Emacs allocates
memory by using the system's memory-allocation routines.  We don't
have our own heaps.

> AFAIU, Elisp memory management is conceptually single-threaded.

Again, I don't understand why you say that and what you mean by that.
We just call malloc, and malloc on modern systems is thread-safe.

> > There are much more:
> >
> >   buffer-alist
> 
> I do not see how this is a problem to lock/unlock this variable.

So your solution to each such problem is to lock variables?  If so,
you will end up locking a lot of them, and how is this different from
using the global lock we do today with Lisp threads?

> >   buffer's base-buffer
> 
> I do not see it. May you point me to where this is changed?

See set_buffer_internal_2.

How do you investigate this stuff?  I type M-. on every macro and
function call I see, recursively, and look what they do.  If you do
the same, how come you overlook all these details?  And if you do not
use M-., how do you expect to learn what the code does internally?

> >   buffer's undo-list
> 
> That's just a synchronization between old_buffer and
> old_buffer->base_buffer.
> I am not 100% sure why it is necessary to be done this way and manually
> instead of making undo-list values in indirect buffers point to base
> buffer.

So you are now saying that code which worked in Emacs for decades does
unnecessary stuff, and should be removed or ignored?

How is it useful, in the context of this discussion, to take such a
stance?  IMO, we should assume that whatever the current code does it
does for a reason, and look at the effects of concurrency on the code
as it is.

> >   buffer's point and begv/zv markers
> 
> AFAIU, these store the last point position and narrowing state.
> I do not see much problem here, except a need to lock these variables
> while writing them. They will not affect PT, BEGZ, and ZV in other
> threads, even if those operate on the same buffer now.

Oh, yes, they will: see fetch_buffer_markers, called by
set_buffer_internal_2.

> >   buffer's marker list
> 
> May you point me where it is?

In fetch_buffer_markers.  Again, I don't understand how you missed
that.

> >   buffer's local variables
> 
> I admit that I do not understand what the following comment is talking
> about:
> 
>   /* Look down buffer's list of local Lisp variables
>      to find and update any that forward into C variables.  */

The C code accesses some buffer-local variables via Vfoo_bar C
variables.  Those need to be updated when the current buffer changes.

> > Yes, but we have just looked at a single primitive: set-buffer.  Once
> > in the buffer, any useful Lisp program will do gobs of stuff, and each
> > one of those accesses more and more globals.  How do you protect all
> > that in a 100% reliable way? by identifying the variables and
> > structures one by one? what if tomorrow some change in Emacs adds one
> > more?
> 
> This sounds like a problem that is already solved by any program that
> uses async threads. Maybe Po Lu can provide good insights.

Programs that use async threads avoid global variables like the
plague.  Emacs is full of them.

> > Searching for "setq case-fold-search" finds more than 30 hits in Emacs
> > alone.  And this variable is just an example.
> 
> These are mostly major/minor modes and constructs like
> 
> (setq case-fold-search ...)
> (do staff)
> (setq case-fold-search old-value)
> 
> The last one is legitimate problem with logic. Although not much
> different from Elisp threads doing the same and changing buffer-local
> variables in the midst of other thread running in the same buffer.
> So, I do not see how this should prevent async threads. We just need to
> take care about setq not attempting to write into buffer-local variables
> at the same time, which is solved by locking.

That "we just need to" is the problem, because it is multiplied by the
number of such variables, and they are a lot in Emacs.