[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Concurrency via isolated process/thread

From: Hugo Thunnissen
Subject: Re: Concurrency via isolated process/thread
Date: Mon, 17 Jul 2023 22:43:26 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0

On 7/5/23 04:31, Eli Zaretskii wrote:
No, because we already handle sub-process output in a way that doesn't
require true concurrency.

Not to hijack this thread and this is only tangentially related to the the message I'm responding to, but asynchronous IO like Emacs has for sub processes, network connections and the like are, in my opinion, more important features than true concurrency. Asynchronous IO is also what can make current lisp threads useful. I think that most of the time when people want to reach for concurrency in an Emacs lisp program it is to retrieve, process, store or emit data in the background. Correct me if you think I'm wrong, but I suspect that execution time, parallelism or performance isn't usually much of an issue: I think what people usually want is a way of processing things or synchronizing things without having the user wait on an unresponsive Emacs. Combining lisp threads, asynchronous IO (not async execution) facilities and a little bit of polling of `input-pending-p' can in my opinion already get this job done most of the time.

Ihor Radchenko wrote:

Yes, most of Elisp is about text processing. But when we really need to
utilize asynchronous code, it is usually not about reading/writing text
- it is about CPU-heavy analysis of text. This analysis is what truly
needs async threads. Writing back, if it is necessary, may be separated
from the analysis code or even done in separate buffer followed by
`buffer-swap-text' or `replace-buffer-contents'.

I am skeptical that there are all that many use cases for the asynchronous analysis of text in buffers that are currently being edited. Correct me if I'm wrong, AFAIU programs that analyze text as it is being edited will usually need to re-parse after every edit anyways, making a parse during the edit itself not all that useful: After all, the result is no longer accurate for the buffer contents by the time it becomes available. A synchronous parser can probably do just as good of a job in such a scenario, as long as it is interruptible by user input.

I'm not familiar with the C core at all, do you think it might be more realistic to add a little more facilities for asynchronous data streams than to rework the execution model of Emacs to add multi threading? I'm mainly thinking of something like "channels" or "queues" for lisp objects, with an easy to use scheduling mechanism that makes it straightforward for people to not stall the main thread for too long.

And another (very) nice to have feature: asynchronous filesystem IO. Consider the code below which I use in a package I'm working on. My package reads and parses a large amount of files in the background within a short time frame. Parsing/processing the files in a timely manner is never really an issue, but the blocking IO of `insert-file-contents' does often take so long that it is  impossible to not have the user notice, even if polling `input-pending-p' and yielding in between operations. The workaround below makes the thread virtually unnoticeable in the majority of cases, but it would have been nice to have a native elisp facility for this as opposed to using `cat` in an asynchronous process.


(defconst phpinspect--cat-executable (executable-find "cat")
  "The executable used to read files asynchronously from the filesystem.")

(defsubst phpinspect--insert-file-contents-asynchronously (file)
  "Inserts FILE contents into the current buffer asynchronously, while yielding the current thread.

Errors when executed in main thread, as it should be used to make
background operations less invasive. Usage in the main thread can
only be the result of a logic error."
  (let* ((thread (current-thread))
         (mx (make-mutex))
         (condition (make-condition-variable mx))
          (lambda (process event)
            (with-mutex mx
              (if (string-match-p "^\\(deleted\\|exited\\|failed\\|connection\\)" event)
                    (setq err (format "cat process %s failed with event: %s" process event))
                    (condition-notify condition))
                (when (string-match-p "^finished" event)
                  (condition-notify condition)))))))
    (when (not phpinspect--cat-executable)
       "ERROR: phpinspect--insert-file-contents-asynchronously called when cat-executable is not set"))

    (when (eq thread main-thread)
      (error "ERROR: phpinspect--insert-file-contents-asynchronously called from main-thread"))

    (with-mutex mx
      (make-process :name "phpinspect--insert-file-contents-asynchronously"
                    :command `(,phpinspect--cat-executable ,file)
                    :buffer (current-buffer)
                    :sentinel sentinel)

      (condition-wait condition)
      (when err (error err)))))

(cl-defmethod phpinspect-fs-insert-file-contents ((fs phpinspect-fs) file &optional prefer-async)
  "Insert file contents from FILE. "
  (if (and prefer-async (not (eq (current-thread) main-thread))
      (phpinspect--insert-file-contents-asynchronously file)
    (insert-file-contents-literally file)))

reply via email to

[Prev in Thread] Current Thread [Next in Thread]