Re: [PATCH] freopen SEGFAULT on win32 with filename == NULL

[Bruno Haible]

Claudio Bley wrote:
> Furthermore, using NULL as filename does not work at all using the MS
> C runtime library (debug assertion violation).
> 
> --------------------
> --- freopen.c.1       2011-08-25 21:05:34 +0200
> +++ freopen.c 2011-08-25 21:08:52 +0200
> @@ -38,6 +38,9 @@
>  rpl_freopen (const char *filename, const char *mode, FILE *stream)
>  {
>  #if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__
> +#  ifdef _MSC_VER
> +  if (!filename) return NULL; /* would trigger a runtime error on MSVC */
> +#  endif
>    if (filename && strcmp (filename, "/dev/null") == 0)
>      filename = "NUL";
>  #endif
> 
> --------------------

Unfortunately, freopen (NULL, ...) is not something gnulib can support
portably. POSIX:2008 specifies the effects of this call in
<http://pubs.opengroup.org/onlinepubs/9699919799/functions/freopen.html>:

  If filename is a null pointer, the freopen() function shall attempt to
  change the mode of the stream to that specified by mode, as if the name
  of the file currently associated with the stream had been used. In this
  case, the file descriptor associated with the stream need not be closed
  if the call to freopen() succeeds. It is implementation-defined which
  changes of mode are permitted (if any), and under what circumstances.

For testing, I added this line to the freopen test:

*** tests/test-freopen.c.orig   Wed Aug 31 10:30:38 2011
--- tests/test-freopen.c        Wed Aug 31 10:25:43 2011
***************
*** 29,34 ****
--- 29,35 ----
  main ()
  {
    ASSERT (freopen ("/dev/null", "r", stdin) != NULL);
+   ASSERT (freopen (NULL, "r", freopen ("/dev/null", "rw", stdin)) != NULL);
  
    return 0;
  }

The result is that it fails on several platforms
(OpenBSD 4.9, AIX 7.1, HP-UX 11.23, IRIX 6.5, OSF/1 5.1, Solaris 10, mingw):
test-freopen.c:32: assertion failed
FAIL: test-freopen

And implementing this requires a conversion from file descriptor to filename
(at least, that's what the glibc implementation of freopen (NULL, ...) uses),
which is only available on very few operating systems.

> Using MinGW and trying to change stdout's mode to binary simply fails:
> 
> if (!isatty(fileno(stdout))
>    freopen(NULL, "wb", stdout); /* <- returns NULL */

You can use the gnulib module 'binary-io' instead of freopen here.

> Would it be feasible to use the _setmode function in rpl_freopen
> instead of freopen if the filename == NULL on win32 (non-cygwin)?

As explained above, this would not scale. There is no point in trying
to use a POSIX API to solve a problem that exists only on Win32 systems.

> What about the fsetbinary, xfchangemode functions discussed here
> [http://lists.gnu.org/archive/html/bug-gnulib/2010-03/msg00111.html]?

Eric?

Bruno


2011-08-31  Bruno Haible  <address@hidden>

        freopen: Documentation.
        * doc/posix-functions/freopen.texi: Document the bug with the NULL file
        name.
        Reported by Claudio Bley <address@hidden>.

--- doc/posix-functions/freopen.texi.orig       Wed Aug 31 10:46:55 2011
+++ doc/posix-functions/freopen.texi    Wed Aug 31 10:46:44 2011
@@ -22,6 +22,10 @@
 On Windows platforms (excluding Cygwin), this function does not set 
@code{errno}
 upon failure.
 @item
+This function does not support a @code{NULL} file name argument on some
+platforms:
+OpenBSD 4.9, AIX 7.1, HP-UX 11.23, IRIX 6.5, OSF/1 5.1, Solaris 10, mingw.
address@hidden
 This function does not fail when the file name argument ends in a slash
 and (without the slash) names a nonexistent file or a file that is not a
 directory, on some platforms:
-- 
In memoriam Magomed Yevloyev <http://en.wikipedia.org/wiki/Magomed_Yevloyev>