bug#41321: 27.0.91; Emacs aborts due to invalid pseudovector objects

[Eli Zaretskii]

> Date: Sun, 17 May 2020 18:57:53 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 41321@debbugs.gnu.org
> 
> > Date: Sun, 17 May 2020 18:28:04 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: 41321@debbugs.gnu.org
> > 
> > I'm already running with such a breakpoint, let's how it will catch
> > something.                                        ^^^
> 
> Should have been "hope".  Sorry.

It happened again, and now insert-file-contents wasn't involved, so I
guess it's off the hook.  The command which triggered the problem was
self-insert-command, as shown in the backtrace below.  The problem
seems to be with handling overlays when buffer text changes.

The backtrace below, as well as some tinkering with values of relevant
variables, indicate that the buffer has two overlays, both of which
point to invalid memory.  The crash happens here:

  /* Now run the before-change-functions if any.  */
  if (!NILP (Vbefore_change_functions))
    {
      rvoe_arg.location = &Vbefore_change_functions;
      rvoe_arg.errorp = 1;

      PRESERVE_VALUE;
      PRESERVE_START_END;

      /* Mark before-change-functions to be reset to nil in case of error.  */
      record_unwind_protect_ptr (reset_var_on_error, &rvoe_arg);

      /* Actually run the hook functions.  */
      CALLN (Frun_hook_with_args, Qbefore_change_functions,
             FETCH_START, FETCH_END);

      /* There was no error: unarm the reset_on_error.  */
      rvoe_arg.errorp = 0;
    }

  if (buffer_has_overlays ())
    {
      PRESERVE_VALUE;
      report_overlay_modification (FETCH_START, FETCH_END, 0,  <<<<<<<<<<<<
                                   FETCH_START, FETCH_END, Qnil);
    }

FETCH_END calls marker-position, and that segfaults because the marker
points to invalid memory, which was probably unmapped from the process
address space (so I guess this is w32-specific, as GNU systems don't
really return memory to the system).  The start_marker is also
invalid, it's just that FETCH_END is called first.

Since the previous call to before-change-functions already used the
same overlay markers, I suspect that the call to
before-change-functions caused the memory to be unmapped (perhaps due
to GC).  As you see below, the value of before-change-functions is

  (t syntax-ppss-flush-cache)

So the prime suspect is what happens when syntax-ppss-flush-cache
runs, and thus I CC Stefan.  The main question to answer now from my
POV is how come a marker on buffer position 3116 which was valid
before before-change-functions was called became invalid as result of
some Lisp, in particular as result of calling before-change-functions.

Here's the backtrace; ideas for further debugging are welcome.

  Thread 1 received signal SIGSEGV, Segmentation fault.
  PSEUDOVECTORP (code=<optimized out>, a=<optimized out>) at lisp.h:1720
  1720          return PSEUDOVECTOR_TYPEP (XUNTAG (a, Lisp_Vectorlike,
  (gdb) bt
  #0  PSEUDOVECTORP (code=<optimized out>, a=<optimized out>) at lisp.h:1720
  #1  MARKERP (x=<optimized out>) at lisp.h:2618
  #2  CHECK_MARKER (x=XIL(0xa000000018ac0518)) at marker.c:133
  #3  0x010f073c in Fmarker_position (marker=XIL(0xa000000018ac0518))
      at marker.c:452
  #4  0x010edd34 in signal_before_change (preserve_ptr=0x0, end_int=3116,
      start_int=3116) at insdel.c:2179
  #5  prepare_to_modify_buffer_1 (start=start@entry=3116, end=end@entry=3116,
      preserve_ptr=preserve_ptr@entry=0x0) at insdel.c:2007
  #6  0x010ee27d in prepare_to_modify_buffer (start=3116, end=3116,
      preserve_ptr=preserve_ptr@entry=0x0) at insdel.c:2018
  #7  0x010ee54d in insert_1_both (string=string@entry=0x82ef1b "r",
      nchars=nchars@entry=1, nbytes=nbytes@entry=1, inherit=inherit@entry=true,
      prepare=prepare@entry=true, before_markers=before_markers@entry=false)
      at insdel.c:896
  #8  0x010ef005 in insert_1_both (before_markers=false, prepare=true,
      inherit=true, nbytes=1, nchars=1, string=0x82ef1b "r") at insdel.c:697
  #9  insert_and_inherit (string=string@entry=0x82ef1b "r",
      nbytes=nbytes@entry=1) at insdel.c:692
  #10 0x01107160 in internal_self_insert (c=114, n=<optimized out>)
      at cmds.c:477
  #11 0x01107804 in Fself_insert_command (n=make_fixnum(1), c=<optimized out>)
      at cmds.c:302
  #12 0x0114fb6c in funcall_subr (subr=<optimized out>,
      numargs=<optimized out>, numargs@entry=2, args=<optimized out>,
      args@entry=0x82f120) at eval.c:2869
  #13 0x0114d9fd in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x82f118)
      at eval.c:2794
  #14 0x01148f7d in Ffuncall_interactively (nargs=3, args=0x82f118)
      at callint.c:254
  #15 0x0114d9fd in Ffuncall (nargs=4, args=0x82f110) at eval.c:2794
  #16 0x0114dca3 in Fapply (nargs=<optimized out>, nargs@entry=3,
      args=<optimized out>, args@entry=0x82f288) at eval.c:2424
  #17 0x0114aecb in Fcall_interactively (function=XIL(0x43b3350),
      record_flag=<optimized out>, keys=XIL(0xa000000016a31530))
      at callint.c:342
  #18 0x0114fb99 in funcall_subr (subr=<optimized out>,
      numargs=<optimized out>, numargs@entry=3, args=<optimized out>,
      args@entry=0x82f430) at eval.c:2872
  #19 0x0114d9fd in Ffuncall (nargs=4, args=args@entry=0x82f428) at eval.c:2794
  #20 0x0118eaf7 in exec_byte_code (bytestr=<optimized out>,
      vector=<optimized out>, maxdepth=<optimized out>,
      args_template=<optimized out>, nargs=<optimized out>, nargs@entry=1,
      args=<optimized out>, args@entry=0x82f7b8) at bytecode.c:633
  #21 0x0115125f in funcall_lambda (fun=<optimized out>, nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x82f7b8) at eval.c:2989
  #22 0x0114d953 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x82f7b0)
      at eval.c:2808
  #23 0x0114db2c in call1 (fn=XIL(0x3f30), arg1=XIL(0x43b3350)) at eval.c:2654
  #24 0x010d0efe in command_loop_1 () at keyboard.c:1463
  #25 0x0114c91f in internal_condition_case (
      bfun=bfun@entry=0x10d0a0e <command_loop_1>, handlers=XIL(0x90),
      hfun=hfun@entry=0x10c5049 <cmd_error>) at eval.c:1355
  #26 0x010bdbda in command_loop_2 (ignore=XIL(0)) at keyboard.c:1091
  #27 0x0114c8a6 in internal_catch (tag=XIL(0xdfb0),
      func=func@entry=0x10bdbb3 <command_loop_2>, arg=XIL(0)) at eval.c:1116
  #28 0x010bdb5d in command_loop () at keyboard.c:1070
  #29 0x010c4bf3 in recursive_edit_1 () at keyboard.c:714
  #30 0x010c4f0c in Frecursive_edit () at keyboard.c:786
  #31 0x0124a4a4 in main (argc=<optimized out>, argv=<optimized out>)
      at emacs.c:2054

  Lisp Backtrace:
  "self-insert-command" (0x82f120)
  "funcall-interactively" (0x82f118)
  "call-interactively" (0x82f430)
  "command-execute" (0x82f7b8)
  (gdb) fr 3
  #3  CHECK_MARKER (x=XIL(0xa000000018ac0518)) at marker.c:133
  133       CHECK_TYPE (MARKERP (x), Qmarkerp, x);
  (gdb) up
  #4  0x010f073c in Fmarker_position (marker=XIL(0xa000000018ac0518))
      at marker.c:452
  452       CHECK_MARKER (marker);
  (gdb) p marker
  $3 = XIL(0xa000000018ac0518)
  (gdb) xtype
  Lisp_Vectorlike
  Cannot access memory at address 0x18ac0518
  (gdb) p marker+0
  $4 = -6917529027227155176
  (gdb) p/x marker+0
  $5 = 0xa000000018ac0518
  (gdb) up
  #5  0x010edd34 in signal_before_change (preserve_ptr=0x0, end_int=3116,
      start_int=3116) at insdel.c:2179
  2179          report_overlay_modification (FETCH_START, FETCH_END, 0,
  (gdb) p Vbefore_change_functions
  $6 = XIL(0xc000000018dbef20)
  (gdb) xtype
  Lisp_Cons
  (gdb) xcar
  $7 = 0x30
  (gdb) xtype
  Lisp_Symbol
  (gdb) xsymbol
  $8 = (struct Lisp_Symbol *) 0x15ca210 <lispsym+48>
  "t"
  (gdb) p Vbefore_change_functions
  $9 = XIL(0xc000000018dbef20)
  (gdb) xcdr
  $10 = 0xc000000018dbf410
  (gdb) xcar
  $11 = 0xd5c0
  (gdb) xtype
  Lisp_Symbol
  (gdb) xsym
  xsymbol   xsymname
  (gdb) xsymbol
  $12 = (struct Lisp_Symbol *) 0x15d77a0 <lispsym+54720>
  "syntax-ppss-flush-cache"
  (gdb) p Vbefore_change_functions
  $13 = XIL(0xc000000018dbef20)
  (gdb) xcdr
  $14 = 0xc000000018dbf410
  (gdb) xcdr
  $15 = 0x0
  (gdb) p start
  $16 = <optimized out>
  (gdb) p start_int
  $17 = 3116
  (gdb) p end_int
  $18 = 3116
  (gdb) p start_marker
  $19 = XIL(0xa000000018ac04f8)
  (gdb) p end_marker
  $20 = XIL(0xa000000018ac0518)
  (gdb) p start_marker
  $21 = XIL(0xa000000018ac04f8)
  (gdb) xtype
  Lisp_Vectorlike
  Cannot access memory at address 0x18ac04f8
  (gdb) p preserve_ptr
  $22 = (ptrdiff_t *) 0x0
  (gdb) p *(current_buffer->text->beg+3000)
  $23 = 115 's'
  (gdb) p *(current_buffer->text->beg+3000)@200
  $24 = "sense would then\nsuggest us that the feature should be extended to 
other means of\ndkispaying messages in the echo a", '\000' <repeats 84 times>
  (gdb) p *(current_buffer->text->beg+3116)
  $25 = 0 '\000'
  (gdb) p GPT
  $26 = 3116
  (gdb) p GPT_ADDR
  $27 = (unsigned char *) 0x7d80c2b ""
  (gdb) p current_buffer->overlays_before
  $28 = (struct Lisp_Overlay *) 0x170cb080
  (gdb) p $28->start
  $29 = XIL(0xa0000000170cb040)
  (gdb) xtype
  Lisp_Vectorlike
  Cannot access memory at address 0x18ac04f8
  (gdb) p $28->next
  $30 = (struct Lisp_Overlay *) 0x13050320
  (gdb) p $28->next->start
  $31 = XIL(0xa000000016172310)
  (gdb) xtype
  Lisp_Vectorlike
  Cannot access memory at address 0x18ac04f8
  (gdb) p current_buffer->overlays_after
  $32 = (struct Lisp_Overlay *) 0x0
  (gdb) p $28->next->next
  $33 = (struct Lisp_Overlay *) 0x0
  (gdb) p rvoe_arg.location
  $35 = (Lisp_Object *) 0x15c9298 <globals+120>
  (gdb) xtype
  Lisp_Vectorlike
  Cannot access memory at address 0x18ac04f8
  (gdb) p rvoe_arg.errorp
  $36 = false