[PATCH 3/2] open_memstream: port to more systems

[Eric Blake]

Provide brute-force implementation that works on all remaining
systems.  Some day, it would be nice to override all APIs that
take FILE*, so that we can guarantee efficient in-memory access,
but in the meantime, this works even if it is slow on deficient
platforms.  The idea here is that eventually those platforms will
comply with POSIX 2008 and provide an efficient open_memstream
themselves, or they will die off as reasonable porting targets.

test-open_memstream now passes on cygwin 1.5, mingw, and Solaris
10.  However, this exposed some additional bugs: mingw had
C++ compilation failures with the replacement <time.h>, and this
patch exposed a Solaris 10 bug in __fpurge().

Several licenses need to be relaxed before this can be LGPLv2+.

* m4/open_memstream.m4(gl_PREREQ_OPEN_MEMSTREAM): Update
prerequisites for alternate implementation.
* modules/open_memstream (Depends-on): Likewise.
* lib/stdio.in.h (gnulib_open_memstream_hook): Declare.
* lib/fflush.c (rpl_fflush): Call into hook.
* lib/open_memstream.c (open_memstream) [!HAVE_FUNOPEN]: Provide
alternate implementation.
* doc/posix-functions/open_memstream.texi (open_memstream):
Update.

Signed-off-by: Eric Blake <address@hidden>
---

This is a proof-of-concept patch; it solves the problem at hand,
but has some dependencies on incompatible modules that would
need to be relaxed to LGPLv2+ before this module could be used
in libvirt.

fflush - GPL
fpurge - LGPLv3
freading - LGPLv3
ftello - LGPLv3
full-read - LGPLv3
full-write - LGPLv3
safe-read - LGPLv3
safe-write - LGPLv3
tmpdir - LGPLv3
tmpfile - GPL

And in reviewing this email, the ChangeLog doesn't quite match the
code (I ended up adding a rpl_close hook call after writing the
initial ChangeLog entry).

Do we want a tmpfile-gnu module that guarantees that tmpfile() is
not constrained by silly platform limits, and does not put garbage
on stderr on failure to create a temporary file?

 ChangeLog                               |   12 +++
 doc/posix-functions/open_memstream.texi |   11 +-
 lib/fclose.c                            |    4 +
 lib/fflush.c                            |   14 +++-
 lib/open_memstream.c                    |  160 ++++++++++++++++++++++++++++++-
 lib/stdio.in.h                          |    5 +
 m4/open_memstream.m4                    |    9 ++-
 modules/open_memstream                  |    6 +
 modules/open_memstream-tests            |    3 -
 9 files changed, 209 insertions(+), 15 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index a7ec8b2..4f354a5 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,17 @@
 2010-04-26  Eric Blake  <address@hidden>

+       open_memstream: port to more systems
+       * m4/open_memstream.m4(gl_PREREQ_OPEN_MEMSTREAM): Update
+       prerequisites for alternate implementation.
+       * modules/open_memstream (Depends-on): Likewise.
+       * lib/stdio.in.h (gnulib_open_memstream_hook): Declare.
+       * lib/fflush.c (rpl_fflush): Call into hook.
+       * lib/open_memstream.c (open_memstream) [!HAVE_FUNOPEN]: Provide
+       alternate implementation.
+       * modules/open_memstream-tests (Status): Test is now portable.
+       * doc/posix-functions/open_memstream.texi (open_memstream):
+       Update.
+
        open_memstream: new module
        * modules/open_memstream: New file.
        * m4/open_memstream.m4 (gl_FUNC_OPEN_MEMSTREAM): Likewise.
diff --git a/doc/posix-functions/open_memstream.texi 
b/doc/posix-functions/open_memstream.texi
index 8388f56..7a580f3 100644
--- a/doc/posix-functions/open_memstream.texi
+++ b/doc/posix-functions/open_memstream.texi
@@ -10,13 +10,14 @@ open_memstream
 @itemize
 @item
 This function is missing on some platforms:
-MacOS X 10.3, FreeBSD 6.0, NetBSD 3.0, OpenBSD 3.8, Interix 3.5.
+MacOS X 10.3, FreeBSD 6.0, NetBSD 3.0, OpenBSD 3.8, AIX 5.1, HP-UX 11,
+IRIX 6.5, OSF/1 5.1, Solaris 10, Cygwin 1.5.x, mingw, Interix 3.5, BeOS.
+However, the gnulib replacement involves the file system on platforms
+that lack stdio hooks, and may leave behind a file if the process
+is killed while a string stream is open.  Also, the gnulib replacement
+may write to stderr on failure to create a string stream.
 @end itemize

 Portability problems not fixed by Gnulib:
 @itemize
address@hidden
-This function is missing on some platforms:
-AIX 5.1, HP-UX 11, IRIX 6.5, OSF/1 5.1, Solaris 10, Cygwin 1.5.x,
-mingw, BeOS.
 @end itemize
diff --git a/lib/fclose.c b/lib/fclose.c
index 9f43889..cffca5c 100644
--- a/lib/fclose.c
+++ b/lib/fclose.c
@@ -33,6 +33,10 @@ rpl_fclose (FILE *fp)
   if (fflush (fp))
     saved_errno = errno;

+#if GNULIB_OPEN_MEMSTREAM_HOOK
+  gnulib_open_memstream_hook_close (fp);
+#endif
+
   if (close (fileno (fp)) < 0 && saved_errno == 0)
     saved_errno = errno;

diff --git a/lib/fflush.c b/lib/fflush.c
index ead4875..c2e822b 100644
--- a/lib/fflush.c
+++ b/lib/fflush.c
@@ -136,7 +136,19 @@ rpl_fflush (FILE *stream)
      what we need to know is whether the stream holds a "read buffer", and on
      mingw this is indicated by _IOREAD, regardless of _IOWRT.  */
   if (stream == NULL || ! freading (stream))
-    return fflush (stream);
+    {
+      int result = fflush (stream);
+#if GNULIB_OPEN_MEMSTREAM_HOOK
+      if (gnulib_open_memstream_hook_flush (stream))
+        result = EOF;
+#endif
+      return result;
+    }
+
+#if GNULIB_OPEN_MEMSTREAM_HOOK
+  if (gnulib_open_memstream_hook_flush (stream))
+    return EOF;
+#endif

 #if defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, 
Linux libc5 */

diff --git a/lib/open_memstream.c b/lib/open_memstream.c
index 4947b02..51ec2a4 100644
--- a/lib/open_memstream.c
+++ b/lib/open_memstream.c
@@ -25,12 +25,14 @@
 #include <errno.h>
 #include <stdlib.h>
 #include <string.h>
+#include <unistd.h>

+#include "full-read.h"
 #include "verify.h"

-#if !HAVE_FUNOPEN
-# error Sorry, not ported to your platform yet
-#else
+#if HAVE_FUNOPEN
+
+/* Light-weight implementation that relies on stdio hooking.  */

 # define INITIAL_ALLOC 64

@@ -188,4 +190,154 @@ open_memstream (char **buf, size_t *len)
     }
   return f;
 }
-#endif /* HAVE_FUNOPEN */
+
+#else /* !HAVE_FUNOPEN */
+
+struct memstream {
+  FILE *f; /* Backing file.  */
+  char **buf; /* User's buffer.  */
+  size_t *len; /* User's length.  */
+};
+typedef struct memstream memstream;
+
+/* Array of open memstreams, indexed by fileno() of the backing file.  */
+static memstream *memstreams;
+
+/* Size of memstreams array; always >= number of open memstreams.  */
+static int num_memstreams;
+
+/* Sync the memstream with index FD, updating the user's pointers.
+   Return EOF on error, errno may be modified even on success.  */
+static int
+memstream_sync (int fd)
+{
+  off_t offset;
+  char *string;
+  assert (0 <= fd && fd < num_memstreams);
+  assert (memstreams[fd].f && fileno (memstreams[fd].f) == fd);
+  if ((offset = ftello (memstreams[fd].f)) < 0
+      || lseek (fd, 0, SEEK_SET) != 0
+      || !(string = malloc (offset + 1)))
+    return EOF;
+  if ((size_t) offset != offset)
+    {
+      errno = E2BIG;
+      return EOF;
+    }
+  if (offset && full_read (fd, string, offset) != offset)
+    {
+      int saved_errno = errno;
+      lseek (fd, offset, SEEK_SET);
+      free (string);
+      errno = saved_errno;
+      return EOF;
+    }
+  string[offset] = '\0';
+  free (*memstreams[fd].buf);
+  *memstreams[fd].buf = string;
+  *memstreams[fd].len = offset;
+  return 0;
+}
+
+/* If F is visiting a memstream, then update the user's buffer and
+   size variables to contain malloc'd contents from the temporary
+   file.  If F is NULL, update all such memstreams.  Return 0 on
+   success with no change to errno, or EOF with errno set on failure.  */
+int
+gnulib_open_memstream_hook_flush (FILE *f)
+{
+  int saved_errno = errno;
+  int fd;
+  int result = 0;
+  if (!f)
+    {
+      for (fd = 0; fd < num_memstreams; fd++)
+        {
+          if (memstreams[fd].f)
+            {
+              /* Remember first failure, but visit all streams.  */
+              if (result)
+                memstream_sync (fd);
+              else if (!(result = memstream_sync (fd)))
+                saved_errno = errno;
+            }
+        }
+      errno = saved_errno;
+      return result;
+    }
+  fd = fileno (f);
+  if (fd < 0 || num_memstreams <= fd || !memstreams[fd].f)
+    {
+      errno = saved_errno;
+      return 0;
+    }
+  if (!(result = memstream_sync (fd)))
+    errno = saved_errno;
+  return result;
+}
+
+/* If F is visiting a memstream, then update bookkeeping to note that
+   the memstream is no longer active, without affecting errno.
+   Assumes gnulib_open_memstream_hook_flush was already called.  */
+void
+gnulib_open_memstream_hook_close (FILE *f)
+{
+  int saved_errno = errno;
+  int fd = fileno (f);
+  if (0 <= fd && fd < num_memstreams && memstreams[fd].f)
+    {
+      assert (memstreams[fd].f == f);
+      memset (&memstreams[fd], 0, sizeof memstreams[fd]);
+    }
+  errno = saved_errno;
+}
+
+/* Heavyweight implementation that requires a temporary file as
+   backing store.  Relies on an override to fflush() to keep things in
+   sync, and an override to fclose() to clean up memory.  */
+FILE *
+open_memstream (char **buf, size_t *len)
+{
+  /* FIXME: tmpfile is allowed to write to stderr on failure - are we
+     happy with that?  */
+  FILE *f = tmpfile ();
+  int fd;
+
+  if (!f)
+    return NULL;
+  if (!(*buf = strdup ("")))
+    {
+      fclose (f);
+      errno = ENOMEM;
+      return NULL;
+    }
+  fd = fileno (f);
+  assert (0 <= fd);
+  if (num_memstreams <= fd)
+    {
+      memstream *temp;
+      int new_size = fd + 1;
+
+      if (new_size < num_memstreams + num_memstreams / 2)
+        new_size = num_memstreams + num_memstreams / 2;
+      temp = realloc (memstreams, new_size * sizeof *temp);
+      if (!temp)
+        {
+          fclose (f);
+          free (*buf);
+          errno = ENOMEM;
+          return NULL;
+        }
+      memset (temp + num_memstreams, 0,
+              (new_size - num_memstreams) * sizeof *temp);
+      num_memstreams = new_size;
+      memstreams = temp;
+    }
+  assert (!memstreams[fd].f);
+  memstreams[fd].f = f;
+  memstreams[fd].buf = buf;
+  memstreams[fd].len = len;
+  return f;
+}
+
+#endif /* ! HAVE_FUNOPEN */
diff --git a/lib/stdio.in.h b/lib/stdio.in.h
index 8b49774..cd75961 100644
--- a/lib/stdio.in.h
+++ b/lib/stdio.in.h
@@ -602,6 +602,11 @@ _GL_CXXALIASWARN (obstack_vprintf);
    the current stream size.  After closing the stream, call
    free(*BUF).  */
 # if address@hidden@
+#  if GNULIB_OPEN_MEMSTREAM_HOOK
+/* Internal functions only used by gnulib.  */
+extern int gnulib_open_memstream_hook_flush (FILE *);
+extern void gnulib_open_memstream_hook_close (FILE *);
+#  endif
 _GL_FUNCDECL_SYS (open_memstream, FILE *, (char **, size_t *));
 # endif
 _GL_CXXALIAS_SYS (open_memstream, FILE *, (char **, size_t *));
diff --git a/m4/open_memstream.m4 b/m4/open_memstream.m4
index 148fe11..4213001 100644
--- a/m4/open_memstream.m4
+++ b/m4/open_memstream.m4
@@ -19,8 +19,13 @@ AC_DEFUN([gl_FUNC_OPEN_MEMSTREAM],
 # Prerequisites of lib/open_memstream.c.
 AC_DEFUN([gl_PREREQ_OPEN_MEMSTREAM],
 [
+  dnl Presence of stdio hooking, as on BSD, allows a much more
+  dnl efficient implementation.
   AC_CHECK_FUNCS([funopen])
-  if test "$ac_cv_func_funopen" = no; then
-    AC_MSG_ERROR([Sorry, open_memstream not yet ported to your platform.])
+  if test "$ac_cv_func_funopen" != yes; then
+    gl_REPLACE_FCLOSE
+    gl_REPLACE_FFLUSH
+    AC_DEFINE([GNULIB_OPEN_MEMSTREAM_HOOK], [1], [Define to 1 if the only
+      way to implement open_memstream is with overrides of fflush and fclose])
   fi
 ])
diff --git a/modules/open_memstream b/modules/open_memstream
index d789c1d..92a9afe 100644
--- a/modules/open_memstream
+++ b/modules/open_memstream
@@ -7,9 +7,15 @@ m4/open_memstream.m4

 Depends-on:
 extensions
+fclose
+fflush
+ftello
+full-read
 malloc-posix
 realloc-posix
 stdio
+strdup-posix
+tmpfile
 unistd
 verify

diff --git a/modules/open_memstream-tests b/modules/open_memstream-tests
index 431ca6e..b3bf049 100644
--- a/modules/open_memstream-tests
+++ b/modules/open_memstream-tests
@@ -1,6 +1,3 @@
-Status:
-unportable-test
-
 Files:
 tests/test-open_memstream.c
 tests/signature.h
-- 
1.6.6.1