Re: We have asynchronous sessions, why have anything else?

[John Kitchin]

The Jupyter project is one approach to this. It currently has dozens of kernels for different languages, and new kernels can certainly be made. The emacs-jupyter package provides one implementation of an interface. It is complex, and relies on a compiled module for the zeromq message passing library.  

I am not advocating for this as the solution for org-babel, but it is an interesting case study. You can even connect to remote kernels. 

I use it a lot. 

On Mon, Jun 27, 2022 at 8:56 PM Tim Cross <theophilusx@gmail.com> wrote:

Tom Gillespie <tgbugs@gmail.com> writes:

>> I am not even sure if all the babel backends support try-except.
>> Think about ob-gnuplot or, say, ob-latex.
>
> Indeed many do not. Defining some standard "features"
> for org babel language implementations is something that
> is definitely of interest so that we can provide clear interfaces
> for things like stdio, error handling, return values, async,
> file output, remote execution, sessions, return value caching,
> module discovery/tangling, execution from file vs stdin, execution
> without a file system path, runtime environment specification,
> and much more. However, at the moment there is only a preliminary
> survey of a subset of these that was put together by Ian Martins.
>
> https://orgmode.org/worg/org-contrib/babel/languages/lang-compat.html
>
>> the two could be unified if we expand the functionality of the async filter
>
> While this might be possible, I would definitely hold off on this because
> the changes in semantics absolutely will break many users' blocks. We
> barely knew what the impact of changing the default return value for shell
> blocks would be.
>
> I absolutely look forward to the day when this can be done safely and
> with confidence, but I think we need a much stronger handle on babel
> interfaces in general before such a change could even be considered.
>
> At the moment each ob lang impl pretty much has to be considered
> to be completely unique, even if the text looks like bash for e.g.
> shell, comint, and screen. Users are known to rely on undocumented
> quirks of the ob lang impls that can differ wildly in their semantics.
>

Well said Tom.

As you point out, there are numerous deficiencies with the current
implementation, despite the fact it all sort of works. To get the sort
of improvements and consistency users want, I suspect this needs more
than just small tweaks around the edges.

To some extent, I see the current babel implementation as similar to a
prototype. It has worked well and we have learnt a lot about what people
want to use it for and the type of functionality they are wanting and
what some of the core challenges are. Now comes the next step, which is
definitely non-trivial. We need to take all that knowledge and
consolidate it into a single model from which we can define the
interfaces and associated APIs. A big job which will take considerable
time.

--

John

-----------------------------------
Professor John Kitchin (he/him/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803

@johnkitchin
https://kitchingroup.cheme.cmu.edu
https://pointbreezepubs.gumroad.com/ pycse bookstore