macOS test failures - LC_ALL / LC_CTYPE / LC_MESSAGES

[Gavin Smith]

On Sun, Oct 23, 2022 at 12:49:26AM +0200, Bruno Haible wrote:
> On macOS 10.13, with perl v5.28.3, two tests fail:
> 
> FAIL: test_scripts/layout_formatting_info.sh
> FAIL: test_scripts/layout_formatting_plaintext.sh
> 
> Find attached the logs. It looks like the line breaking is different than
> expected.
> 
> In case it matters: The Gnulib replacement function 'rpl_wcwidth' is being
> compiled:
> $ nm gnulib/lib/libgnu_a-wcwidth.o
>                  U _locale_charset
> 0000000000000000 T _rpl_wcwidth
>                  U _uc_width
>                  U _wcwidth
> 

I had found these same two tests failed on some FreeBSD systems as well
as DragonFlyBSD 5.9.

I was able to fix this on DragonFlyBSD 5.9 and hopefully this is the same
fix needed on these other OS's too.  The erroneous line breaking in your
log file (from macOS 10.13) is exactly the same.

wcwidth has been the problem in the past with line breaking
decisions, but this time, I traced it down to a locale change in
tp/Texinfo/Translations.pm (gdt), to LC_ALL.  When LC_ALL was restored
(for the tests, this was to "C") this overrode a change to LC_CTYPE
that had taken place elsewhere in the code, in the XS paragraph
formatting module.  This broke iterating over UTF-8 strings, leading
to the wrong values being passed to wcwidth.

I fixed this by changing LC_ALL to LC_MESSAGES in gdt, as that was
the only locale category we needed to change.

(Avoiding restoring LC_ALL at the end of gdt also fixed the problem,
but this seemed riskier.)

However, it had previously been reported that LC_MESSAGES didn't
work on MS-Windows.  Hence, I placed this code inside a conditional:

  # We need to set LC_MESSAGES to a valid locale other than "C" or "POSIX"
  # for translation via LANGUAGE to work.  (The locale is "C" if the
  # tests are being run.)
  #   LC_MESSAGES was reported not to exit for Perl on MS-Windows.  We
  # could use LC_ALL instead, but (a) it's not clear if this would help,
  # and (b) this could interfere with the LC_CTYPE setting in XSParagraph.

  if ($^O ne 'MSWin32') {
    $saved_LC_MESSAGES = POSIX::setlocale(LC_MESSAGES);
    _switch_messages_locale();
  }
  $saved_LANGUAGE = $ENV{'LANGUAGE'};