Re: improving bidi documents display

[Eli Zaretskii]

> From: Miles Bader <address@hidden>
> Cc: address@hidden, address@hidden, address@hidden,
>  address@hidden
> Date: Fri, 04 Mar 2011 13:25:17 +0900
> 
> Eli Zaretskii <address@hidden> writes:
> > This one's different, believe me: no other text property changes the
> > _order_ of characters on display in creative ways.  It could easily
> > render the text illegible, under just the right circumstances.  Other
> > text properties are either non-intrusive, or are almost immediately
> > fixed by JIT Lock, or are simply rare enough to not get in our way.
> > This one, if used to implement partial reordering of buffer text, will
> > be ubiquitous in any buffer with program sources that include bidi
> > comments and strings, i.e. we will see it _a_lot_.
> 
> I'm a bit confused by how an ordering that's "good" in (e.g.) a TeX
> document, would be "illegible" in another context...
> 
> Can you give an example of such a case?

Not really, since we are talking about an idea, not about a feature
that has a detailed design which we could reason about.  Any example I
give can be countermanded by saying something like "but if we
implement the idea such and such, then this particular problem won't
happen".

I can, however, show you why I'm afraid of leaving the "reordering"
properties on the text.  The reason is that the visual results of
reordering are highly context-dependent.  Changing a single character
very far away from the locus of reordering can completely change the
visual appearance of an unrelated portion of text.

You can try the below in Emacs 24, after setting
bidi-display-reordering non-nil (but be sure to do that in a buffer
other than *scratch* or any other buffer whose major mode is
programming-language oriented, because these are forced to have a
left-to-right paragraph direction).

Here's an example.  Type this text, left to right:

 abcdef \foo{ABCDEF} xyz

(As usual, lower-case letters are Latin, upper-case are in some R2L
script, such as Arabic or Hebrew.)

The above will be displayed like this:

 abcdef \foo{FEDCBA} xyz

So far so good, right?  And looks "good" in a TeX document, right?

Now go to the beginning of the line (C-a) and type a single R2L
letter (W).  The resulting display will change dramatically:

                      xyz {FEDCBA}abcdef \fooW

The reason, in this case, for such a significant change is that
directionality of the first character of a paragraph determines the
"base direction" of that paragraph.  (Actually, only "strong
directional characters" count, those which have "L" or "R" bidi
category in the Unicode data base.)

There are other examples, but the underlying reason is the same: bidi
reordering is highly dependent on the surrounding context.  Copy a
region of text to another context, and you might have undesirable
results.  If these results are due to characters that are part of the
buffer text (as in the above example), one can cope with that by
editing after the yank.  But if the results are due to some invisible
factor like text properties, this is tougher on users.

That is why I prefer that these invisible factors not be copied with
the text.  If needed, they will be re-applied according to the rules
of the target buffer.