Re: Comparing symbol-with-position using eq

[Alan Mackenzie]

Hello, Daniel.

On Fri, Apr 05, 2019 at 13:18:55 -0700, Daniel Colascione wrote:
> > Hello, Alex.

> > On Fri, Apr 05, 2019 at 11:05:59 -0600, Alex wrote:
> >> Hello, Alan.

> >> Alan Mackenzie <address@hidden> writes:

> >> > On Thu, Apr 04, 2019 at 22:49:22 -0600, Alex wrote:

> >> >> I apologize if this topic already reached its conclusion, but IMO
> >> >> having eq return true for two different object types is quite
> >> >> surprising behaviour.

> >> > We are comparing two symbols, both of which are 'foo, but one of which
> >> is
> >> > annotated with its position in a source file.  The two symbols are the
> >> > same symbol.

> >> Is it not comparing a symbol with a pseudovector containing that symbol
> >> and a position?

> > At the machine code level, that is what it's doing, yes.

> >> > I understand the reaction to the idea, though.  Even though the
> >> > representation of these two objects is different, conceptually they
> >> are
> >> > the same object.

> >> Similar objects, but I don't believe that's enough for eq. Consider that
> >> it's regarded non-portable in Lisp to compare integers with eq since the
> >> same number may be represented by different objects, or (eq 3 3.0), or
> >> (eq (list 1 2) (list 1 2)).

> > The point is that comparing 'foo with (Symbol "foo" at 339) with `eq',
> > and returning t doesn't do any harm.  On the contrary, it enables correct
> > source positions to be output in byte compiler warning messages.  That it
> > does no harm is verified by the fact that a make bootstrap with such
> > annotated symbols works.

> > However, there is a slight slowdown in this Emacs, compared with the
> > master branch.  The powers that be have intimated that this slowdown is
> > unacceptable, so I'm having to make more far reaching changes in the C
> > code to confine this slowdown to byte compilation.

> I'm also concerned that by overloading eq this way we'll make it easy to
> "lose" information about positions. In general, when (eq a b), we can
> substitute a for b and vice versa.

You could still do that (not that you'd want to), and your code would
still work up to the point where the byte compiler warning output
wouldn't have a position to output, and would degrade to a less accurate
position, in the limit not outputting a position at all.  But this isn't
going to happen in practice.

A symbol with position is merely an annotated version of an ordinary
symbol.  It behaves identically to that ordinary symbol, provided only
that the enabling flag, symbols-with-pos-enabled, is bound to non-nil.
The normal way these annotated symbols come into existence is via the
reader when a form is read with read-positioning-symbols (as contrasted
with the standard read).

All the details are in the code in branch scratch/accurate-warning-pos.

> The objects are equivalent in the strongest sense. Now, they're not
> equivalent, and choosing a instead of b can lead to subtle bugs,
> especially since we're talking about error-path and warning-path code
> that might not be frequently exercised.

In the byte compiler, the warning path code is all too frequently
exercised.  ;-(  But I've just found (and fixed) a subtle bug, which was
what this thread was about.  The fact that make bootstrap works with
these annotated symbols is a very strong test.

> You mention that we'd need to change the use of EQ throughout the byte
> compiler in order to work with positional symbols properly. Can we just do
> that, in one big renaming patch?

I'm not quite sure what you mean here, but I think the answer's no.  The
byte compiler calls C primitives which use EQ.  

> In cases where we don't want positions, we can just define a macro
> making the new eq-for-position function equivalent to eq.

> But yes, it's kind of unfortunate that we haven't been using an explicit
> AST representation.

I've been thinking that for the time (nearly 3 years) that I've been
trying to fix this bug.  Is this how compilers for other Lisp systems are
written?  It seems horribly easy to compile as Emacs does, by taking the
(read) starting form and gradually transforming it as a Lisp form.  It is
difficult to keep track of (text) source positions when one does this.

-- 
Alan Mackenzie (Nuremberg, Germany).