Re: Locale-independent paragraph formatting

[Gavin Smith]

(Reply accidentally not sent to the list.)

----- Forwarded message from Gavin Smith <GavinSmith0123@gmail.com> -----

Date: Thu, 9 Nov 2023 21:25:28 +0000
From: Gavin Smith <GavinSmith0123@gmail.com>
To: Bruno Haible <bruno@clisp.org>
Subject: Re: Locale-independent paragraph formatting [was Re: Texinfo 7.0.93
        pretest available]

On Thu, Nov 09, 2023 at 08:11:59PM +0000, Gavin Smith wrote:
> On Tue, Oct 10, 2023 at 07:29:15PM +0200, Bruno Haible wrote:
> > Given that the only encoding you want to deal with is UTF-8, Eli's 
> > suggestion
> > to use GNU libunistring is better than my iconv() suggestion. It has 
> > functions
> > for width determination:
> > https://www.gnu.org/software/libunistring/manual/html_node/uniwidth_002eh.html
> > 
> > > but I doubt it is urgent to do before the release, as the current 
> > > approach,
> > > however flawed, has been in place and worked fairly well for a long time
> > > (since the XS paragraph module was written).
> > 
> > Well, it does not work on Windows.
> > 
> > I agree with you that it's not urgent to do before the 7.1 release, since
> > the Windows port is work-in-progress.
> 
> I have just pushed a commit (e3a28cc9bf) to use gnulib/libunistring
> functions instead of the locale-dependent functions mbrtowc and wcwidth.
> This allows for a significant simplification as we do not have to try
> to switch to a UTF-8 encoded locale.

I have realised this changes the test results under tp/tests.  I haven't
updated those test results yet, but am happy for this commit to be reverted
until I investigate the changes.  Some of the changes look incorrect, e.g.

--- a/tp/tests/coverage/res_parser/formatting_info/formatting.info
+++ b/tp/tests/coverage/res_parser/formatting_info/formatting.info
@@ -55,16 +55,14 @@ Now !  !@ but , ,
 
    @point ⋆ (⋆) @print ⊣ (⊣) @result ⇒ (⇒) @today a sunny day
 
-   @aa å (å) @AA Å (Å) @ae æ (æ) @oe œ (œ) @AE Æ (Æ) @OE Œ (Œ) @o ø (ø)
-@O Ø (Ø) @ss ß (ß) @l ł (ł) @L Ł (Ł) @DH Ð (Ð) @TH Þ (Þ) @dh ð (ð) @th þ
-(þ)
+   @aa å (å) @AA Å (Å) @ae æ (æ) @oe œ (œ) @AE Æ (Æ) @OE Œ (Œ) @o ø (ø) @O Ø (Ø
) @ss ß (ß)
+@l ł (ł) @L Ł (Ł) @DH Ð (Ð) @TH Þ (Þ) @dh ð (ð) @th þ (þ)
 

where the line in the new output is too long.

----- End forwarded message -----