Re: Implementing image support for kitty terminal

[Eli Zaretskii]

> From: Jose A Ortega Ruiz <jao@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Wed, 07 Sep 2022 19:49:34 +0100
> 
> > It's hard to tell, because you haven't described your ideas of
> > implementing that.  In particular, the current model of image display
> > in the Emacs display engine is that an image is basically considered a
> > single very large character, from the screen layout POV.
> 
> Would it be possible to keep that model and only alter it when the
> display actually draws the character in the underlying terminal?  
> 
> For an image-enhanced tty that'd mean that when we are going to write
> this "single very large character" (whose data can be either an array of
> bytes or simply a path in the filesystem, stored in an associated text
> property perhaps) what we do is write a ESC character sequence starting
> with _G, as the one shown above, and specifying not only the data to
> draw, but also its size in rows and columns.  Then, according to the
> kitty API promises, the image will be rendered in that rectangle for us.

This is a different level: the level of actually delivering the stuff
to the glass.  I think it's much easier than what was bothering me.  I
was talking about the level of layout: the one that knows how much
space each "display element" (character, image, stretch of white
space, etc.) takes on display.  On TTY frames, there's a built-in
assumption that every display element takes just one pixel in the
vertical direction, and each screen line takes one pixel on display.
That will have to change for image support, I presume.

> I am assuming here that the display engine (for a tty) knows somehow
> that it's printing a very lage "character" that is going to spawn
> several rows and columns, and also that we can always tell how wide and
> tall the window around point is when we display this "character".

That's exactly the problem: it currently assumes that no display
element takes more than one row, and that each row is exactly one
pixel (= one character height) tall.

> We
> could then decide to tell kitty to either scale the image if it doesn't
> fit or display only part of it.  That's done by passing x,y,w,h in the
> escape sequence instead or r,c... something like
> 
>       <ESC>_Gf=100,t=f,x=0,y=0,w=100,h=20;<encoded /path/to/file.png><ESC>\

I think that's the easy part.