Re: Fontification error backtrace [Was: Why is it so difficult to get a Lisp backtrace?]

[Alan Mackenzie]

Hello, Eli.

On Tue, Jun 28, 2022 at 21:17:04 +0300, Eli Zaretskii wrote:
> > Date: Tue, 28 Jun 2022 17:28:38 +0000
> > Cc: larsi@gnus.org, emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > > > > >   . emit the backtrace int *Messages*
> > > > > > >   . add the backtrace to delayed-warnings-list

> > > > > > Why can't the backtrace be generated and stored, and only displayed 
> > > > > > at a
> > > > > > subsequent redisplay (probably the next one)?

> > > > > That's what the second alternative above already does.

> > > > OK, I've got some code.  :-)

> > > Sigh.  I suggested two alternatives, but you effectively
> > > ignored/rejected both of them.  Why?

> > *Messages* just doesn't feel the right place to put a backtrace.

> That's where we currently log all the errors during redisplay, since
> Emacs 21.1.  So why this one is different?

It's not a single line, or a small number of lines.  It could potentially
be a stack overflow error, generating a large backtrace, which might have
more lines than message-log-max.

> But you _really_ dislike *Messages*, then put the stuff into another
> buffer, like *backtrace-in-redisplay* or somesuch.

I'm currently using the buffer *Backtrace*.  Is there anything wrong with
that?

> > delayed-warnings-list consists of a list of structured lists, rather than
> > the vanilla text output by debug-early.

> Why is it a problem?  You could again stash the backtrace somewhere
> and only show the name of that buffer in the warning text.  Using
> delayed-warnings-list makes sure the problem is not missed.

> > Also these delayed warnings get displayed between post-command-hook
> > and redisplay, which is not an optimal time to display a backtrace
> > generated in redisplay.

> The buffer with warnings pops up very soon after redisplay, which is
> more-or-less exactly what you wanted?

OK, I've never used it.  On reading the documentation, I expected it to
wait until after the next command.

> > > Third, why are you singling out the fontifications? they are not the
> > > only source of Lisp errors in code called by the display engine.
> > > Moreover, handle_fontified_prop is not the only way of calling
> > > expensive Lisp code that deals with faces.

> > All that's true, yet fontification errors are an important source of
> > difficult to debug errors.  We could certainly extend the new backtrace
> > mechanism to handle these other types of bug, too.

> It makes very little sense to me to create infrastructure that caters
> only to your current and immediate problem.  If we introduce such a
> feature, it should work with any Lisp called from redisplay.

OK.  It shouldn't actually be much more difficult to keep the new
facility generic.

> > > > As already noted, the *Backtrace* buffer doesn't appear on the screen,
> > > > yet.  Any hints on how to achieve this would be most welcome.

> > > Any reasons you'd want that?  Once the buffer exists, why do you need
> > > to actually pop it up (and thus potentially disrupt the very
> > > window/frame configuration which you are trying to debug)?

> > A debug-on-error backtrace gets displayed immediately, so it would help
> > consistency.  If the window where the error occurred isn't occupying its
> > entire frame, there should be room to put the backtrace somewhere else, I
> > think.  I'm not at all experienced with things like pop-to-buffer, but
> > they're supposed to be very flexible.

> I think showing a delayed-warning with the text that mentions the
> buffer with the backtrace is completely adequate.  It stops short of
> popping the window with a backtrace itself, but that could be added
> later, if necessary, perhaps as a feature of delayed-warning.

I think that would be a good compromise.

> > > > It seems I could really do with after-redisplay-hook, in which I
> > > > could call pop-to-buffer, then cause redisplay to be restarted.

> > > I think you misunderstand what such a hook could do, and why it could
> > > be useful, if at all, in the first place.  If all you want is to cause
> > > the main loop perform some command after redisplay cycle ends, we
> > > already have several mechanisms in place for doing that.

> > That's what I want, yes, but couldn't find any such mechanism anywhere
> > when I looked.

> One thing that comes to mind is a timer.  Another thing is pushing
> something onto unread-command-events.  I'm sure there are more.

OK, yes, maybe a timer would be the thing.

> > Yes.  How about what you suggested above, namely creating the backtrace
> > in save_eval_handler rather than signal_or_quit, at least as a second
> > attempt.  Though I don't think this would handle errors in other Lisp
> > called from redisplay.

Now I remember why I created the backtrace in signal_or_quit - it needs
to be done before the stack gets unwound, which happens later in
signal_or_quit.  On return to save_eval_handler is just too late for
this.

> It will, because any Lisp we call goes though safe_eval.  The only
> reason why your original proposal didn't was because it bound the
> variable which enables this only in handle_fontified_prop.  But we
> don't need this flag if the logic to decide what to do with the error
> will be in save_eval_handler.

Then the mechanism I've implemented, namely to set redisplay_deep_handler
to the handler which would handle an error, could be made to serve for
other sorts of Lisp code.  Probably I'll need to rename some variables
whose name contains "fontification" and suchlike.

> And you need to consider one more factor: code from display engine,
> including fontification via handle_fontified_prop, is called also from
> user commands, not via redisplay_internal.  For example, commands like
> C-n and C-v invoke display code.  Errors signaled there _can_ be left
> to go to top-level and probably could call the debugger.  To test
> whether any given code was called from redisplay_internal, test the
> value of the variable redisplaying_p.

OK, this is a further complication.  But it would be good to get to the
debugger if possible, rather than just having a "historical" backtrace.

-- 
Alan Mackenzie (Nuremberg, Germany).