bug#41005: problem with rendering Persian text in Emacs 27

[Pip Cet]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pip Cet <pipcet@gmail.com>
>> Cc: valizadeh.ho@gmail.com,  41005@debbugs.gnu.org,  nicholasdrozd@gmail.com
>> Date: Sat, 06 Jun 2020 08:38:39 +0000
>> 
>> >> Given these two bugs, I wonder whether it wouldn't be more reasonable
>> >> always to let HarfBuzz guess the direction, at least for Emacs-27:
                                                  ~~~~~~~~~~~~~~~~~~~~~

I should have been clearer here and said I was only concerned with Emacs 27.

>> >> scripts which change direction, if they are supported by HarfBuzz, won't
>> >> work anyway.
>> >
>> > Please explain "scripts that change direction" and "won't work
>> > anyway", I don't think I understand that part.
>> 
>> I think your example (RLO..PDF in RTL text) is better: that won't work
>> anyway, right now, because if, for example, you type
>> 
>> <HEBREW LETTER SHIN> <RIGHT-TO-LEFT OVERRIDE> f i
>> 
>> and have set the char table to treat "fi" as a ligature, the result will
>> (at least sometimes) be an "fi" ligature, but it should look like the
>> word "if".
>
> That's not how shaping engines work, at least not how HarfBuzz does
> AFAIU.  It gets the characters in the logical order, so it always
> wants to see "fi", even if the directionality of the characters was
> overridden, and it also wants to know the local text directionality.
> What is produced from that depends on the font: if it has different
> ligatures for "fi" in different directions, then HarfBuzz should give
> us back the ligature appropriate for the direction it was passed.
>
> (Personally, I think that when some text uses a directional override,
> they don't intend to see ligatures, because the override is mostly for
> treating characters as independent of the surrounding context.  But
> this is eventually up to the font to specify.  AFAIU, Arabic shaping
> works differently in different directional contexts, for example.)
>
>> > The reason we don't let HarfBuzz guess in all cases is because the
>> > resolved bidi level, when we have it, is a more accurate indication of
>> > the required direction.
>> 
>> Yes, but we'll still cache the wrong direction.
>
> Why "wrong"?  We will cache the same direction as we passed to
> HarfBuzz, and thus the produced glyphs will be consistent with the
> cached direction.  And if we ever need to display the same sequence of
> characters with a different direction, the cached sequence will fail
> to match, and we will call HarfBuzz again to produce glyphs for this
> other direction.  That sounds TRT to me.

You're absolutely correct, sorry for wasting so much of your time with
this: caching directions is the right thing, I was just concerned about
what to do in Emacs 27 where AIUI we don't want to cache directions...

> Once the caching of direction is
> implemented, my point is that passing the direction to HarfBuzz and
> caching it will produce better results for text in a directional
> override than if we let HarfBuzz guess the direction.

Again, I agree. Sorry for the misunderstanding.