bug#54079: 29.0.50; Method dispatching eratically fails

[Stefan Monnier]

>> I'm not sure what "such a case" you're thinking of.
>
> One of the "99% other cases that *do* occur" that you referred to in your
> previous paragraph.  You say that these wouldn't "work right".  I'm
> asking you for an example of such a "not working right".

(defalias 'FOO BAR)

>> But in general, evaluation of code doesn't expect symbols to have
>> positions:
>
> The evaluation is completely indifferent to the SWPs, when
> symbols-with-pos-enabled is t.

While evaluating (defalias 'FOO BAR) everything will be dandy.
But when you later call FOO while `symbols-with-pos-enabled` is not
t anymore, then things go haywire.

Same with

    (put FOO BAR (lambda ...))

or

    (add-hook FOO (lambda ...))

or

    (setcdr FOO (lambda ...)))

or

    ...

Of course, similar thing will go wrong without `lambda` because for example

    (setq FOO 'bar)

will set FOO to `bar`-with-pos, so when you later compare FOO to
`bar` you will find that `eq` will say nil.
Etc... the list goes on and on.

>> ... it may test `eq` between symbols and it may be run without having
>> the magical `symbols-with-pos-enabled`.
> I've lost the thread here.  What scenario are you considering?  I thought
> we were talking only about the `eval' within eval-{when,and}-compile.

Whatever appears in the code passed through `eval-{when,and}-compile`
can survive that code, so the fact that `symbols-with-pos-enabled` is
non-nil while evaluating it doesn't make it correct.

>> So as a general rule we should strip symbols before we pass it to `eval`.
> I don't see this, due to all the confusion we're experiencing.  As a
> general rule, within byte compilation, symbol stripping should be
> postponed as long as possible whilst a compilation of the form is still
> possible.

`eval` is exactly this "as long as possible" point where we know the
code won't be compiled because it's about to be interpreted.

> But, looking at the code, I don't think byte-compile binds
> symbols-with-pos-enabled to t.  This could be a bug.

Maybe not: it has no reason to presume that its argument has positions.
If it does, then it must be because the caller explicitly arranged for
it to happen and so the caller could be in charge of binding
that variable.

>>     And why bother stripping the result of `byte-compile-eval`?
>
> (This is in eval-when-compile only.)  That result may contain SWPs.  For
> example:
>
>     (eval-when-compile (cons 'foo 'bar))

But the result is then passed to the compiler anyway, so this is the
same as writing

      '(foo bar)

which will come "with pos", so I still don't see why we'd need to strip
pos from the result here.  Even more so because the code passed to
`byte-compile-eval` is compiled so it presumably doesn't have any sympos
anyway, so there shouldn't be any sympos in its output either anyway.

>>     Fundamentally, `eval` should always strip before doing its job.
> You mean, by "always", you meant ALWAYS???

Yes.

> I understood you to mean "always, when in the context of
> eval-{when,and}-compile".  If we're not inside a compilation, and thus
> there're no SWPs hanging around, stripping symbols from an expression
> will just mean a loss of time for a null operation.

That's right: the only cases where not stripping the arg of `eval` is OK
is when we know that stripping would do nothing.
IOW it's an optimization.

> At the moment, I disagree with you.  I don't think you have given an
> example of a form FOO which will only work if stripping is done before
> evaluation in
>
>     (eval-when-compile FOO)
>
> or
>
>     (eval-and-compile FOO)
>
> ..  At the moment, I still think it is better to strip the positions after
> the evaluation.

And I still haven't heard any good reason why stripping them after the
evaluation would be of any use since that result is passed to the
compiler which is perfectly happy to accept code-with-pos.

>> The misbehavior I'm referring to is what happens when you call the
>> function before you byte-compile it (or, more likely, when you never end
>> up byte-compiling it), because the presence of sympos in the function
>> will mess up its semantics (because `symbols-with-pos-enabled` won't be
>> set any more when it's called).
>
> I'm puzzled.  Are we still talking about eval-{when,and}-compile, here?

No.  We're talking about a hypothetical case where `(eval '(defun foo ...))`
is executed somehow and where the `(defun foo ...)` part somehow
contains symposes.

I used it as an example of why `eval` should conceptually always strip
its argument (or if `eval` doesn't do it, then its caller should make
sure that the code passed to it doesn't contains symposes).

>From what I understand you suggested that in that cases `eval` should
preserve the symposes in the hope that the user later calls
`(byte-compile 'foo)` which can then benefit from those symposes to give
better warnings.  And I pointed out that this still means that (until
that `byte-compile` call comes) the code of `foo` will likely be broken
so any uses of `foo` will likely misbehave.

> If so, how can a form with SWPs get into a symbol's function cell?
> The positions are stripped inside the e-w/a-compile.

Hmm... I agree that the positions *should* be stripped inside
e-w/a-compile before passing the code to `eval`, but the code I see
right now in bytecomp.el says:

    (eval-and-compile . ,(lambda (&rest body)
                           (byte-compile-recurse-toplevel
                            (macroexp-progn body)
                            (lambda (form)
                              ;; Don't compile here, since we don't know
                              ;; whether to compile as byte-compile-form
                              ;; or byte-compile-file-form.
                              (let* ((print-symbols-bare t) ; Possibly 
redundant binding.
                                     (expanded
                                      (macroexpand--all-toplevel
                                       form
                                       macroexpand-all-environment)))
                                (eval expanded lexical-binding)
                                expanded)))))

I don't see where the symbols are stripped before passing the code to `eval`.
So if `form` is of the form `(defun foo ...)` I think we have a problem.

>     (eval-when-compile
>       (fset 'foo '(defun bar ....)))
>
> , where SWPs will escape into foo.

Why do you need the `put` in there?  Doesn't the same problem show up with

     (eval-and-compile
       (defun bar ....))

which does not seem "vanishingly unlikely" at all.


        Stefan