bug#61350: Eglot over Tramp freezes with large project

[Stefan Monnier]

>> I think we understand some of the problems well enough to think
>> about solutions.
> Does that "some" include _this_ problem?

Not on my side, sadly, no, but maybe on Michael's?

> We could also ask other questions, of course, like: could Tramp use a
> process filter?  IMO these are very powerful tools and bring a
> substantial degree of freedom of implementation.  Currently Tramp relies
> on accept-p-o and then searches the process buffer for regexps.  To me,
> that just seems more difficult and error-prone than process filters.

FWIW, I believe the crux of the matter here is that process filters
receive an arbitrary chunk at a time, so we almost always need to make
them accumulate the received data into some kind of buffer-like data
structure until we have "enough" data.

IOW, most process filters need to do something like

    (lambda (proc string)
      (let* ((buffer (process-get proc 'my-buffer))
             (new (concat buffer string)))
        (if (not (had-enough-p new))
            (process-put proc 'my-buffer new)
          (process-put proc 'my-buffer nil)
          (do-the-actual-processing new))))

Another issue for Tramp is that basically all of the API it has to
implement (for `file-name-handler-alist`) is synchronous, so that forces
it to resort to `accept-p-o` :-(

If we want Tramp to work better, we'll need to provide asynchronous
versions of most of those operations (presumably using some kind of
promise/future abstraction).


        Stefan