Re: gnulib does not always detect need for iconv() hack on musl

[Bruno Haible]

Hello Sergei,

Sergei Trofimovich wrote:
> The following fails bison-3.8.2 tests:
>     $ ./configure && make && make check
> The following succeeds:
>     $ ./configure --host=x86_64-unknown-linux-musl && make && make check
> 
> The failure happens due to unexpected '*' output in report logs instead
> of '%empty' on 'ASCII' locales.
> 
> These unexpected '*' pop back again because gnulib relies on '--host='
> parameter for './configure' to detect musl target (for lack of better
> signal?):
> 
>   https://git.savannah.gnu.org/cgit/gnulib.git/tree/m4/musl.m4#n16
> 
>     case "$host_os" in
>       *-musl*) AC_DEFINE([MUSL_LIBC], [1], [Define to 1 on musl libc.]) ;;
> 
>   https://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/unicodeio.c#n151
> 
>     /* FreeBSD iconv(), NetBSD iconv(), and Solaris 11 iconv() insert
>        a '?' if they cannot convert.  */
>     # if !defined _LIBICONV_VERSION
>               || (res > 0 && outptr - outbuf == 1 && *outbuf == '?')
>     # endif
>       /* musl libc iconv() inserts a '*' if it cannot convert.  */
>     # if !defined _LIBICONV_VERSION && MUSL_LIBC
>               || (res > 0 && outptr - outbuf == 1 && *outbuf == '*')
>     # endif
>          )
>         return failure (code, NULL, callback_arg);
> 
> What do you think of enabling the workaround regardless of MUSL_LIBC
> define?

The MUSL_LIBC symbol is supposed to be set on musl platforms; this is
what musl.m4 is for. The difference between your two invocations is that
in the first case, it used a $host triple inferred by config.guess,
while in the second case, it used the $host that you specified on the
command line.

When I try your two commands (just the configure step), the first one
prints
  checking for host system type... x86_64-pc-linux-musl
while the second one prints
  checking for host system type... x86_64-unknown-linux-musl

The next steps of the investigation are: In the first case,
  - What did the "checking for host system type..." line look like?
  - Which of the environment variables CC_FOR_BUILD, HOST_CC, CC,
    CONFIG_SITE did you have defined, and to which values?

> Or perhaps gnulib should perform runtime testing to detect the need for
> a hack? Here is how musl mangles symbols:
> 
>   https://git.musl-libc.org/cgit/musl/tree/src/locale/iconv.c#n545
> 
>     case US_ASCII:
>         if (c > 0x7f) subst: x++, c='*';
> 
> Below implements unconditional workaround.

Thanks for the suggestion. But we try to limit the performance implications
of hacks/workarounds needed for one platform (here: musl) on other platforms
(especially glibc platforms).

Bruno