Re: Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets)

[Eli Zaretskii]

> From: Akira Kyle <akira@akirakyle.com>
> Cc: larsi@gnus.org, emacs-devel@gnu.org
> Date: Sat, 28 Nov 2020 18:29:08 -0700
> 
> One such fundamental incompatibility stems from the fact that 
> Emacs may have multiple views on one buffer split across various 
> windows and/or frames. Gtk assumes that each widget will only have 
> one view. The 'webkit xwidget type currently works around this by 
> drawing to an offscreen window then for every window that's 
> supposed to display the widget, creating a gtk drawing area widget 
> and copying the offscreen window's surface to the drawing 
> surface. I don't think gtk really intends for ofscreen widgets to 
> be used this way and as far as I can tell offscreen window widgets 
> have been removed in gtk4.
>
> Another issue is that emacs uses a GtkFixed (or a custom 
> subclassed EmacsFixed for gtk3) as the container widget for the 
> main frame area. There isn't a mechanism to control z-ordering of 
> child widgets other than removing and re-adding them in the 
> desired z-order, which is what emacs-webkit has to do in order for 
> child frames to not render below the webkit widget.

The same problems plague your experiment with a module, don't they?

Anyway, this just tells me that GTK is not a good starting point for
embedding widgets.  Do other projects build embedded widgets on such
shaky grounds?

> > As long as the widget can be told to use a given rectangular
> > portion of the screen, we should be fine: the current Emacs
> > display engine is perfectly capable of handling display elements
> > that occupy an area of certain dimensions.
> 
> This is more an issue on the gtk side of things. With the webkit 
> widget it isn't an issue since the widget doesn't define a minimum 
> size so any size you request it to render at, it will 
> obey. However other widgets, such as buttons, sliders, etc, have a 
> minimum size they will render at. If you tell it to render itself 
> smaller, it will refuse and so it's impossible to simply tell the 
> widget "here's the space you have to go in". This becomes a 
> problem when the widget needs to be clipped at a window boundary 
> as gtk will happily draw the widgets across them as gtk has no 
> internal knowledge of Emacs window boundaries being a place that 
> widget must be clipped at.

This is solvable.  We already have similar situation in the display
engine, where some display element cannot be usefully "clipped".  We
either don't display it at all or display it on the next screen line,
depending on the wrap mode.

> The direction I'm thinking of going in the future since I feel 
> like thus far the xwidget route has been fairly troublesome is to 
> expose interfaces to the cairo surfaces used in image.c. This 
> could be through modifying image.c or through defining a new 
> display element type. I would try to expose the cairo API to lisp 
> and allow dynamic modules to directly draw to a cairo surface 
> given to it through the lisp interface. Gtk along with many other 
> libraries (such as librsvg and poppler) support drawing to a cairo 
> surface and then Emacs gets to have full control over how that 
> cario surface ends up on screen. I would see this as a stepping 
> stone to attempting to do something similar with a opengl surface.

My suggestion is to think how to integrate this idea into Emacs
proper, without introducing modules into the equation.  Modules add
another layer of complexity, so I would suggest to design the solution
inside Emacs, and only after that see how to do that from a module.
One step at a time.