Re: Dynamic modules: MODULE_HANDLE_SIGNALS etc.

[Paul Eggert]

Philipp Stephani wrote:
> > > We could merely require that any module needing recursion must call a
> > new stack-size-checking function at each level of recursion, in order to
> > detect stack overflow.
> >
> > That's performance penalty.  I don't think we should incur that, on
> > behalf of a problem that really should "never" happen.
>
> It's also impossible. Modules will be written in arbitrary languages and
> make use of arbitrary libraries, some of which might not even know about
> Emacs's existence. We cannot impose any constraints on module code.

There's also the issue that other languages may have their own assumptions about 
how stack overflow is detected, assumptions that may disagree with Emacs's 
method or even prevent Emacs's method with working.  My own preference also is 
to rely on the usual VM techniques for ordinary C-code modules, and hope that 
other languages don't get in the way. Still, it may not be possible to do that, 
and we may be forced to impose a bit of software overflow-checking on modules 
implemented via recursion, for module authors who care about reliability.

Does anybody know how C++ and/or Java and/or Python modules detect and recover 
from stack overflow on GNU/Linux? That would be a reasonable sanity check here. 
(And if nobody knows, that's not a good sign....)

> > > Also, any module with an unbounded amount of computation should call the
> > equivalent of QUIT every now and then. If the module API doesn't let (or
> > ideally, require) modules to do that now, it should. Otherwise it's an
> > Emacs freeze waiting to happen.
> >
> > I agree, but I think it's unrelated to stack overflow.  (Though I
> > think we will need to have a module-specific version of QUIT, as long
> > as we keep the constraint of not directly longjmping from module
> > code.)
>
> There's no harm in providing such a functionality (but of course we can't
> enforce its use).

We *could* enforce the use by requiring at least one call to QUIT every (say) 
100 ms, and by doing the equivalent of 3x C-g when the time limit is exceeded. 
That'd be useful even without modules.

> Even if we
> assume "benign UB" (which is dangerous because UB tends to become more
> malign as compilers evolve), it will cause internal state to become
> inconsistent, which is truly disastrous.

Emacs has been relying on this sort of "benign UB" for years, in areas just as 
disaster-prone as C++ cleanup, and it has worked well enough. I don't see why 
C++ would be any different. Typically C++ cleanup is just recovering memory, so 
we'll have a memory leak; big deal.

> handle_sigsegv already has several cases where the longjmp is skipped: when
> in a GC, when the signal is received from the wrong thread, and when it's
> not guaranteed to be a SO. In those cases Emacs aborts (still autosaving).

1. You're right about the GC, but that aspect of GC is considered to be a bug 
(it's called "hard GC error" in the handle_sigsegv comment) and should not be 
precedent for us installing similar bugs elsewhere.

2. The wrong-thread check is because we are attempting to detect stack overflow 
only within Emacs (which can occur due to a user program error, which should not 
crash Emacs); stack-overflow errors in other threads are considered to be 
internal errors in Emacs and indeed never should happen, so it's OK to treat 
them as fatal errors (I expect this should be true even if the other threads 
were created by a module).

> Why can't we do the same if SO is detected while a module function is
> active?

It might be reasonable to do that if the stack overflow were entirely due to the 
module -- that'd be like stack overflow in some other thread created by the 
module. But it would not work when Lisp code almost overflows the stack, then 
calls the module code, which overflows; in this case Emacs needs to recover 
rather than aborting.