Re: Concurrency, again

[Eli Zaretskii]

> From: Ken Raeburn <address@hidden>
> Date: Wed, 19 Oct 2016 06:18:18 -0400
> Cc: Emacs-Devel devel <address@hidden>
> 
> > This could be solved by moving parts of thread.h into lisp.h, with the
> > appropriate #ifdef guards.  I think Lisp objects should be in lisp.h
> > anyway, even if they are optional; anything else is confusing.
> 
> For big, specialized objects like buffers, I think the modularity can help 
> keep things organized, but where we’ve got a structure with all the 
> per-thread state from random parts of the program, like condition handlers 
> and regex state, we don’t have the luxury of separating things.  But even if 
> we move most of thread.h into lisp.h, I think there may still be mutual 
> recursion between it and the other headers.

We already have that, and deal with it.

> >> Stefan’s message: 
> >> http://lists.gnu.org/archive/html/emacs-devel/2013-08/msg00755.html
> > 
> > I don't see that as a critical problem, perhaps because we don't yet
> > realize how serious it can be.  The whole purpose of trying to merge
> > the concurrency branch is to collect practical experience as to what
> > should and shouldn't be in this kind of Emacs feature.  So I'd tend to
> > let this be, until we find out we can't, and why.
> > 
> >>>> * interaction of SIGCHLD handling and threads?
> >>> 
> >>> Details?
> >> 
> >> Your message 
> >> http://lists.gnu.org/archive/html/emacs-devel/2013-08/msg00738.html raised 
> >> questions.  If they were ever satisfactorily answered, I overlooked it 
> >> when putting together my notes….
> > 
> > That should be easy: since a subprocess is locked to a single thread,
> 
> by default, but if that thread exits, that lock disappears

And the process gets locked to some other thread.

> 
> > SIGCHLD should be delivered to that thread.  If we don't have that
> > already, we should add that, it doesn't sound hard, given the
> > infrastructure we already have (deliver_thread_signal etc.).
> 
> It’s not completely trivial.  Under POSIX, we get no control over which 
> thread receives SIGCHLD due to a subprocess exiting, except if we want 
> certain threads to block receiving the signal completely.  We can use 
> pthread_kill to explicitly send SIGCHLD to a specific thread, if it’s not 
> blocking the signal, but we don’t know which thread until we’ve fetched the 
> child’s pid and looked it up in some data structure; we do that by calling 
> waitpid() but then the kernel discards the child process’s status info, so 
> the “correct” thread can no longer respond to the signal by making another 
> waitpid() call to collect the status info.  We’d have to save the info the 
> first time we call waitpid().  But doing it from within the signal handler 
> could be tricky, because in that context we’re limited to async-signal-safe 
> functions, and helpful routines like malloc() and pthread_mutex_lock() aren’t 
> on the list.

So you are saying this problem was never encountered before in any
other program out there, and doesn't already have a solution?  I find
that hard to believe.

> On the other hand, perhaps we can create one special thread to do all the 
> waitpid() calls and pass info to the Lisp-running threads.

Sounds like an unnecessary complication, but if that's how others solve
this problem, so shall we.

> >> It’s easy enough to disable stack overflow checking when enabling thread 
> >> support.
> > 
> > Or add some simple code in the stack overflow handler to check if we
> > are in the main thread, and if not, punt (i.e. crash).
> > 
> >> If only one thread is allowed into the image processing code at a time 
> >> (i.e., don’t release the global lock for that code) then that’s probably 
> >> fine for now, and there’s probably other state there that different 
> >> threads shouldn’t be mucking around with in parallel.
> > 
> > Redisplay runs in the main thread anyway, right?  If so, there's no
> > problem.
> 
> If some random thread calls (redisplay) or (sit-for …)?  I think it’ll run in 
> whichever Lisp-running thread triggers it.  But, it’ll be the one holding the 
> lock.

No, sit-for causes a thread switch.

> >> The keyboard.c one is the only one I’m a bit concerned about, in part 
> >> because I haven’t looked at it.
> > 
> > What part(s) of keyboard.c, exactly?
> 
> Anything looking at getcjmp; that means read_event_from_main_queue and 
> read_char.  Like I said, I haven’t looked very closely; if the static storage 
> isn’t ever used across a point where the global lock could be released to 
> allow a thread switch, it may be fine.

That should already be solved, or else threads cannot receive keyboard
input safely.