new module 'setlocale-null'

[Bruno Haible]

It is the usual expectation that use of setlocale() to change the global
locale is not multithread-safe (of course), but that setlocale(...,NULL)
to query the name of the global locale is.

However, POSIX does not guarantee it:
<https://pubs.opengroup.org/onlinepubs/9699919799/functions/setlocale.html>
says
  "The returned string pointer might be invalidated or the string content
   might be overwritten by a subsequent call to setlocale()."

We need to distinguish two cases:

* querying the name of one category of the global locale
  A unit test shows that this is not MT-safe on
  OpenBSD, AIX.

* querying the name of the global locale (LC_ALL case)
  A unit test shows that this is not MT-safe on
  musl libc, macOS, FreeBSD, NetBSD, OpenBSD, AIX, Haiku, Cygwin.

This info, together with a test program that runs setlocale(...,NULL)
in different threads at different times, provides the following answer
to the question "Where is the return value of setlocale (..., NULL)?":


                        simple category                LC_ALL

glibc                in the category of the locale    in the locale
musl libc            in the category of the locale    in a global   NOT MT-SAFE!
macOS                in the category (global)         in a global   NOT MT-SAFE!
FreeBSD              in the category (global)         in a global   NOT MT-SAFE!
NetBSD               in the category of the locale    in a global   NOT MT-SAFE!
OpenBSD              in a global      NOT MT-SAFE!    in a global   NOT MT-SAFE!
AIX                  in a global      NOT MT-SAFE!    in a global   NOT MT-SAFE!
HP-UX                in the thread                    in the thread
IRIX                 in the category (global)         in a global   actually 
MT-SAFE
Solaris 10           in the locale / thread           in the locale
Solaris 11.0         in the category of the locale    in the locale
Solaris 11.4         in the category of the locale    in the locale
Solaris OpenIndiana  in the category of the locale    in the locale
Haiku                in the category (global)         in a global   NOT MT-SAFE!
Cygwin               in the category (global)         in a global   NOT MT-SAFE!
mingw                in the category of the locale    in the thread
MSVC                 in the category of the locale?   in the thread


The only way to fix this is to introduce a lock around the calls to
setlocale(...,NULL), since in particular
  - in OpenBSD, code inspection shows that there is no way to fetch the infos
    directly in a multithread-safe way,
  - in AIX, there is no documented API for fetching the infos either.

How can the API look like?
  (a) We could override setlocale(), so that setlocale (..., NULL) returns
      a string in a buffer in thread-local storage.
  (b) We could introduce a new API, that takes a caller-provided buffer as
      argument. Like the functions getlogin_r, ttyname_r, ptsname_r do.

I decided to go with approach (b), for the following reasons:
  * Returning a string in thread-local storage is technically complex
    (1. get a pointer to thread-local storage, 2. store a malloc()ed
    buffer in it, resize it when needed, 3. make sure the buffer gets
    freed when the thread exits [easy with POSIX and ISO C threads, but
    hard with Windows threads]).
    Really, it is better to store ALL thread-local data in the thread's
    stack. Isn't that what a stack is for?
  * We already have a setlocale() override, and having the ability to
    activate one override or the other or both together could lead to
    complex code. In approach (b), the setlocale() override will use
    the new API - simple.

The attached patches implement this.


2019-12-15  Bruno Haible  <address@hidden>

        setlocale-null: New module.
        * lib/locale.in.h (SETLOCALE_NULL_MAX, SETLOCALE_NULL_ALL_MAX,
        setlocale_null): New declarations.
        * lib/setlocale_null.c: New file.
        * lib/setlocale-lock.c: New file.
        * m4/threadlib.m4 (gl_PTHREADLIB_BODY): Define C macro HAVE_PTHREAD_API.
        * m4/setlocale_null.m4: New file.
        * m4/locale_h.m4 (gl_LOCALE_H_DEFAULTS): Initialize
        GNULIB_SETLOCALE_NULL.
        * modules/locale (Makefile.am): Substitute GNULIB_SETLOCALE_NULL.
        * modules/setlocale-null: New file.
        * doc/posix-functions/setlocale.texi: Mention the new module.

        setlocale-null: Add tests.
        * tests/test-setlocale_null.c: New file.
        * tests/test-setlocale_null-one.c: New file.
        * tests/test-setlocale_null-all.c: New file.
        * modules/setlocale-null-tests: New file.

0001-setlocale-null-New-module.patch
Description: Text Data

0002-setlocale-null-Add-tests.patch
Description: Text Data