Re: Supporting stylistic sets

[Nicolas Ouellet-payeur]

> From: Nicolas Ouellet-payeur <nicolaso@google.com>
> Date: Fri, 23 Sep 2022 13:31:30 -0400
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org

> This is not a font property, this is a face property.  It will be in
> effect for every font used for the default face, including fonts used
> for non-ASCII characters, like CJK and Emoji.  Are you sure this is
> what you want?

Thanks for clearing up the confusion RE: font vs. face properites.

> I'm not against adding a face property, I just think it isn't enough,
> even if that's what people will want.
>
> Also, AFAIU this feature is meant for special styling of select text
> segments, not for the entire buffer.

Ah, that's a good point. I hadn't considered what happens to different
scripts if, say, the 'ss03' variant means a different thing in each
font.

There's no reason it *can't* be for the entire buffer though.
See below.

> > This uses my chosen stylistic sets everywhere with a single line of
> > Lisp. If I don't like having it *everywhere* (e.g. minibuffer,
> > mode-line), I can still set it separately for each face.
>
> That'd be very tedious, since Emacs uses so many different faces.

Well yes, but what's the alternative? And in the specific use-case of

> > With a text property, I'd have to hook into everything that displays
> > text, somehow, and add the text property there.
>
> Yes.
>
> Maybe we should talk about higher-level use cases: when and why would
> one want to use stylistic-sets in Emacs?

In my case specific: because the 'cv31' variant of my font changes the
shape of parentheses, and I like that shape better. I would rather set
it everywhere, even the minibuffer, mode-line, header-line.

This is all just ASCII though. And ligatures, which might be weirder.
IIUC, Sameer's suggestion with set-fontset-font would address this
use-case.

  (set-fontset-font t 'ascii
   (font-spec
    :family "Fira Code"
    :stylistic-set '("cv31")))

> > Also, while we're on this topic: I'm working on a patch to pass *all*
> > strings/buffer contents to hbfont_shape() during redisplay, and making
> > all text a composition. That way we could have stylistic sets for Latin
> > scripts as well (and most other scripts), not just "composed" scripts
> > like Bengali and Arabic. It'd also achieve "Support ligatures out of the
> > box" from etc/TODO, by giving HarfBuzz the means to shape text properly.
>
> I think you will find out that this makes Emacs redisplay unbearably
> slow.
>
> IMO, it is impractical to shape everything via a shaping engine
> without completely redesigning how we handle character compositions in
> the display engine, because what we have now was not designed to be
> used for all the text we display.

That's one of my big worries, as well. That's why I'm anxious to share
a proof-of-concept ASAP, and see if it's worth pursuing.

Initially it *was* super janky, because I did the naïve thing and passed
strings through hbfont_shape() without caching the result. Each
redisplay would pass the strings to HarfBuzz again, and create a bunch
of new Lisp vectors... Then the GC would go crazy, and cause a *lot* of
jank.

I then tried to put everything into `gstring_hash_table' in composite.c.
That made things better, but the hashtable would keep growing in size
all the time. Basically leaking memory on purpose...

My latest attempt is to tweak `region_cache' so it can store arbitrary
Lisp_Objects, rather than just 1s or 0s for known-ness of regions. Then
I stick the lgstrings into that cache. Different frames have different
face properties, so there's one region_cache per (frame,buffer) pair.
This works, but right now it's incomplete and crashes all the time like
I said.

It's not *noticeably* jankier so far, and I even used the
`gstring_hash_table' version as my main editor for a while. But maybe my
computer is just fast enough, or my use-cases are too simple. It's
*definitely* way too slow for very long lines. For example, a JSON file
that's a single line and multiple megabytes. We'd need to fall back to
the existing code in cases like that.

I haven't tried running it through `gprof' yet, and this is all
subjective. So I'm still worried about performance.

> You can have ligatures today without passing everything through the
> shaping engine.

Yes, but you need to configure those manually. Here, the advantage of
passing them to HarfBuzz is that it can read those ligatures from the
font file without any extra configuration. i.e., "Support ligatures out
of the box" as described in etc/TODO. That's not going to change the
world, but if we can make it work it'll be neat.

> And the main difficulty with such a redesign is to do it in a way that
> still allows easy customization of composition rules from Lisp.

A quick test shows that `prettify-symbols-mode' still works, but I'm
sure there's subtle bugs hiding underneath.