bug#41321: 27.0.91; Emacs aborts due to invalid pseudovector objects

[Pip Cet]

On Wed, May 27, 2020 at 6:39 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 5/27/20 10:57 AM, Pip Cet wrote:
>
> > Do you know of anything like this happening on 64-bit systems?
>
> I think it's unlikely on 64-bit systems; it'd happen only on platforms where
> alignof (void *) < 8, such as x86.
>
> > Emacs GC does rely, and has always relied since
> > GCPRO was removed, on compilers being sensible about what they put on
> > the stack.
>
> This isn't merely an issue about what compilers put into the stack; it's an 
> also
> an issue of what's in registers. There may not be any pointer in the stack 
> that
> points into the Lisp object. And compilers are not always "sensible" about
> temps; they may cache &P->x into a temp with no copy of P anywhere.

Or they may cache &P->x + 1, and use negative offsets to access it.
That used to be the most efficient way of accessing arrays on some
machines. We simply can't cater to that.

Think about code like:

Lisp_Object reverse(Lisp_Object vector)
{
  ptrdiff_t count = ASIZE (vector);
  Lisp_Object new_vector = make_nil_vector (count);
  Lisp_Object *p = aref_addr (vector, count);
  Lisp_Object *q = new_vector->contents;
  while (count--)
    {
      garbage_collect ();
      *q++ = *--p;
    }
}

(which is what many compilers would generate from more sensible code).
On the first iteration, p points to a totally different vector, or
some random other object, but it still needs to keep its vector alive.

So, at the very least, we need to always keep the immediately
preceding object alive if we go that way.

> > I'm pretty sure we figured out the crash that Eli observed. It's not
> > anything that involved, just a Lisp_Object being stored
> > non-consecutively and simultaneously being misaligned for the purposes
> > of maybe_lisp_pointer.
>
> Not sure what the point is here. None of this is "that involved". We can have
> pointers into Lisp objects, pointers that are not aligned for the purposes of
> maybe_lisp_pointer. Emacs should follow all of them, not just the one that Eli
> happened to observe.

Or pointers past them, and that's a significant overhead because it
usually means two objects are being kept alive by one reference.

> > I don't see how you plan to solve it without treating any pointer that
> > points even in the vicinity of a valid lisp object as keeping that
> > object alive.

> Yes, of course.

I didn't mean just "within the object", I did mean "in the vicinity".
With prefetch instructions, it's quite likely the compiler concludes
it's easiest to prefetch something 256 bytes ahead of where it
actually makes the access, then make the actual access relative to
that address...

> Any pointer that points somewhere within a Lisp object (in the C
> sense) should count as pointing to the object.

The C standard explicitly allows pointers (and that's C pointers) to
point one past the end of an allocated array, I believe.

> If memory serves, we already
> treat pointers that way in some places; unfortunately we're not doing it
> consistently.

Yes, we do.

> But I take your point; I'll post the change here before committing to master.

I'm sorry, I misunderstood. If you want to fix only pointers within
objects, that is quite a small change, but I believe it is incomplete.