bug#41531: 27.0.91; Better handle asynchronous eldoc backends

[Stefan Monnier]

[ Sorry I can't really take part in the discussion, 'cause it's
  a rather busy time here.  ]

[ The discussion is becoming too tense for my taste.  Please take
  a break and think whether all this is important enough to warrant
  criticizing each other this way.  I think you're both great hackers
  whose opinion and expertise I value very much.  ]

I think getting async support for eldoc is more important than *how* we
get it.

I suggest we install the current eldoc-async code into `master` and in
parallel work quickly on a futures/promises/aio/... package for
`master`.

As long as it's done before we cut the `emacs-28` branch, I can't see
any reason why we couldn't change the new (and hence not yet released,
except for the GNU ELPA package) API in a backward-incompatible way
(just like the eldoc-async already does).


        Stefan


João Távora [2020-07-07 11:56:01] wrote:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> Please don't make it sound as if I'm the only one here having a strong
>> opinion about proposed code. You disregarded the simple solution in 
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41531#8, and then went
>
> Are you in any way trying to say that you presented a "simple solution"
> that somehow magically solves the problems of Eldoc?  If you do, you're
> living in conflict with reality.
>
> There is the problem to be solved.  And then there are techniques for
> solving the problem.  These are two entirely separate things.  To be
> absolutely clear: you have not, ever, proposed an alternate solution to
> the Eldoc async _problem_.
>
> You talked and talked and presented your favourite async handling
> techniques, like having functions return functions that return
> functions.  You contented that I -- not you -- should work with it to
> solve the problem.
>
> Callbacks, futures, promises, ad-hoc or proper, are just techniques.  I
> tried a couple techniques, including a basic futures-based one proposed
> by Stefan.  I didn't find any as compelling as the simplest and most
> widely used in Emacs: callbacks.
>
> For the quintillionth time: I AM NOT AGAINST FUTURES IN PRINCIPLE: I
> just don't use these Eldoc fixes to shoehorn something rushedly into
> Emacs.  Make a new thread, or join the existing one:
>
>     https://lists.gnu.org/archive/html/emacs-devel/2020-03/msg00935.html
>
> Afterwards, propose a change to the technique, not only in Eldoc but
> elsewhere, too.  This idea is so simple that it boggles the mind that
> you don't grasp it.
>
>> urgent endeavor. E.g., Flymake is stable, and I don't have any
>> particular bugs in mind that need solving.
>
> Great.  I'll just note here that it uses exactly the same technique I'm
> proposing in Eldoc to communicate with multiple backends: it's curious
> how that doesn't bother you.  I would reasonably expect "futures" or
> something else its implementation much simpler too.
>
>> Aside: given that this discussion has user interface in mind, it needs
>> more consideration of actual user experiences we'd want to allow. Ones 
>> not provided by eldoc itself as well. For instance, I think we sooner
>> or later should have a callsig floating popup (in the vein of MS's
>> editors) as an alternative to showing the signature in the echo area
>> only.
>
> That is addressed in a comment in eldoc-docs
>
>       ;; Finally, output to the echo area.  We handle the
>       ;; `truncate-sym-name-if-fit' special case first, by selecting a
>       ;; top-section of the `*eldoc' buffer.  I'm pretty sure nicer
>       ;; strategies can be used here, probably by splitting this
>       ;; function into some `eldoc-display-functions' special hook.
>
> I agree there is ample space for improvement, including a 
> "floating popup" (which I wouldn't limit to callsigs though).  Make
> another bug report to study this.
>
>> The new API is incompatible with the previous one, it requires
>> changing a lot of functions (though admittedly in a minor way).
>
> This is demonstrably false. As I've explained to Eli there is no
> incompatibility in practice.  0 (zero) third-party extensions make use
> of the protocol being changed from Eldoc 1.0.0 to Eldoc 1.1.0, unless
> you're stalling here to secretly work on one.
>
> But if we really, really wanted to, it's easy to get rid of the
> arguments, too, with a variation to the callback technique.  I just
> don't think it's worth it: a technique is a technique.
>
>> is easy to miss, as evidenced by describe-char-eldoc which still
>> doesn't accept any arguments.
>
> Oh, an actual useful comment! Easily solved, thanks.  And it was only
> "missed" because it wasn't used anywhere.
>
>> Whereas we could limit ourselves to the change which would only make
>> the clients use the different hook (or keep using the old one, for
>> compatibility with older Flymake/Emacs versions).
>
> Compatibility to the old `eldoc-documentation-function' protocol is
> fully guaranteed.
>
>> The choice to require a return of a non-nil value if the callback is
>> going to be used is also kinda odd (+1 control flow to test). What's
>> the problem with using the callback synchronously if the doc is
>> available right away?
>
> Nothing, you can do it.  As long as you return non-nil, non-string.  But
> if you are are going to synchronously call the callback on a string, you
> might as well return that string, just seems simpler.
>
>> The decision to double down on having different
>> eldoc-documentation-strategy values seems odd to me in several
>> respects.
>>
>> First of all, if I understand the main motivation behind it, it's to
>> delegate the implementation of asynchronous primitives to Eldoc.
>
> It's rather to make clients don't worry about the synchronization.  Not
> specifically to make Eldoc worry about it.  As soon as you and others
> come up with the uber-async library, I think Eldoc and Flymake and many
> others will be very pleased to delegate work to it.
>
>> We don't want to bother with that in Eglot, etc. But you mentioned "a
>> type of docstring" to be returned in the discussion (and the callbacks
>> have the plist argument as a future proofing). If the type is a
>> buffer, its contents might as well be created from several async calls
>
> If you want to support "buffers" as a "type of docstring", just do that:
> make buffers a type of docstring.  The obvious way is to have multiple
> sources produce multiple buffers.
>
> Thing: why would you ever want to put buffer-joining complexity in the
> client?  They're not in the business of doing async gymnastics, they're
> in the business of serving things coming from servers as efficiently and
> effectively as possible.
>
>> , which would require the same async primitives (though rather in
>> general purpose form) available on the client anyway. Though it
>> doesn't seem to be necessary for LSP in common operations, it's
>> unlikely to be the only such protocol we'd ever want to support.
>
> I don't know about that, but if we did, the current mechanism work
> nicely for the example you presented.
>
>> The strategies themselves:
>>
>> - eldoc-documentation-enthusiast: What's the difference compared to
>> the 'default' one? Sacrificing CPU load for lower latency? It's an odd
>> choice to force the user to make. The only people who can make an
>> ideal decision regarding this are probably the authors of
>> eldoc-documentation-functions, but it wouldn't help if
>> eldoc-documentation-functions has functions coming from different
>> authors, facilities, etc.
>
> Has nothing to do with CPU.  This is the way Eglot works now, or at
> least tries to: there are two delayed doc-producing backends, and
> neither is guaranteed to complete.  One has priority over the other (and
> special hooks are a decent, standard Emacs way to manage priority)
>
> Eglot shows the lower-priority one if it shows it can survive for more
> than x seconds (currently x = 0.3, uncustomizable).  No more doc
> blinking.
>
>> - eldoc-documentation-compose: Okay, this is kinda interesting (though
>>   not essential),
>
>> I think the only reasonably predictable behavior would be to truncate
>> each of them to one line, always.
>
> That's a display problem, not a composition problem For now it works OK
> for one liners and also multiple multi-liners.  Displaying doc is not
> the main goal of these patches, there is certainly room for improvement,
> as I said above.
>
>> - eldoc-documentation-compose-eagerly: Ooh, now I think I see why
>> Returning futures instead (when async is needed) would provide this
>> kind of guarantee without the seeming duplication of signals.
>
> Can't let go of that futures drum, can you?  It'd be a pleasure to see
> you walk the walk.
>
>> On a related note, I believe some facilities might want to use only
>> certain "kinds" of documentation functions (as indicated by the plist 
>> arguments). If the plist is only provided together with the response,
>> there is no way to avoid unnecessary computations (e.g. making HTTP 
>> requests that return some other kinds of values). If the plist was
>> returned together with a future value, however, it would be easy to
>> do, as long as we document that the futures should start the
>> computation until a callback is provided (if possible, at least).
>
> Save it for your future futures implementation.
>
>> And in a different high-level argument: you stated that you intend to
>> have Eglot use a non-default value of eldoc-documentation-strategy. 
>
> OK, but this has nothing to do with Eldoc, does it?  Make a bug report
> for Eglot, I'll explain why it does this, and maybe I'll change it..
>
>> Next, eldoc-print-current-symbol-info is a mess, very hard to
>> follow. I am frankly offended that you argued that both ad-hoc futures
>
> I meant no effense.
>
>> idea). This should very well be possible to untangle in the future,
>> but I'd rather not have code like this in Emacs if we can help it.
>
> You're such an ace programmer that you code alternatives that are so
> brief that they occupy no code at all!
>
>> Further, having all strategies basically delegate to hardcoded options
>> inside eldoc-print-current-symbol-info seems to confirm that the set
>> of available strategies is a closed one. Which is not what I would
>> expect from a variable called eldoc-documentation-strategy.
>
> There are four strategies to choose from but you can make more.  What
> did you have in mind?  
>
>> These are my objections, for now. I'll have to study
>> eldoc--handle-docs a bit later, but any problems it has are probably
>> orthogonal to the rest of the list.
>
> Thanks.
>
> João