Re: bidi-display-reordering is now non-nil by default

[Mohsen BANAN]

>>>>> On Mon, 01 Aug 2011 05:54:24 +0300, Eli Zaretskii <address@hidden> said:

  Mohsen> For (1) paragraph direction detection, there may
  Mohsen> be things that can be done in the core to improve
  Mohsen> auto direction detection.

  Eli> The detection algorithm follows the UAX#9.  What improvements do you
  Eli> have in mind, and what are the use cases where these improvements will
  Eli> do a better job than what we have now?

The particular case I have in mind is where a
number of paragraphs get prefixed and then are
treated as one paragraph.

I am suggesting that the detection algorithm looks
for the likelyhood of a prefix being in the works
and then ignores the prefix and does detection as
if there were no prefixes.

The obvious example of such scenario is citations
in MUAs. However this is common enough that I
think considerations of doing it in the core is
reasonable.

Here is an example:

---

This is paragraph 1 all in english.

اين پاراگراف ۲ تماما به فارسى است.

This is paragraph 3 which is mixed with فارسى and english.

اين پاراگراف ۴ مخلوت English و فارسى است.



  >>>
  >>> This is paragraph 1 all in english.
  >>>
  >>> اين پاراگراف ۲ تماما به فارسى است.
  >>>
  >>> This is paragraph 3 which is mixed with فارسى and english.
  >>>
  >>> اين پاراگراف ۴ مخلوت English و فارسى است.
  >>>

---

Note all 4 paragraphs' directionality are detected
fine based on UAX#9 prior to citation.

Once cited/prefixed they all become 1 paragraph.

The communication ramifications on paragraph 2 are
minor. The reader would get it. 

But communication ramifications on paragraph 4 are
very problematic. The sentence becomes
incomprehensible/mangled.

So, here is what I am suggesting:

 - We add an additional value of 'fancy to
   bidi-paragraph-direction making it:
    nil, left-to-right, right-to-left, fancy

 - If bidi-paragraph-direction is 'fancy,
    
     - do prefix guessing
     - ignore prefix
     - do UAX#9
     - re-insert prefix

This will likely help with simple tabularization
in addition to mail citations.

What do you think?

...Mohsen