From 1ca8ecafc9066c9c0fb4b1030088969a11dce3af Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Sat, 17 Apr 2021 18:44:25 -0700 Subject: [PATCH 1/2] malloc, etc.: check for ptrdiff_t overflow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In glibc 2.30 and later, malloc, realloc and calloc reject attempts to create objects larger than PTRDIFF_MAX bytes. This patch changes malloc-gnu etc. to support this behavior on non-GNU hosts. It also makes this change for malloc-posix etc. since it’s a safety measure that ought to be in POSIX (perhaps we can talk them into that...). In writing this patch I found a complicated set of code that had accumulated over the years, some written by yours truly. I got rid of the code I couldn’t see the need for nowadays. Among other things, the GNU realloc behavior is no longer incompatible with the C standard, because in C17 the latter was relaxed to allow the former. If I went too far in cleaning up, the old stuff can be resurrected. This change is mostly for 32-bit platforms, since practical 64-bit platforms cannot create objects larger than PTRDIFF_MAX bytes anyway. * doc/posix-functions/calloc.texi: * doc/posix-functions/malloc.texi: * doc/posix-functions/realloc.texi: Mention ptrdiff_t issues, and go into more detail about what the gnu extension module does. * doc/posix-functions/realloc.texi: Fix now-obsolete commentary about C99 vs glibc, as C17 allows the glibc behavior and POSIX will follow suit when it gets around to it. * lib/calloc.c, lib/malloc.c, lib/realloc.c: Simplify by always supplying a GNU-compatible version, as that suffices for correctness and is good enough for performance. Include xalloc-oversized.h, and use xalloc_oversized to check for ptrdiff_t overflow. (NEED_CALLOC_GNU, NEED_MALLOC_GNU, NEED_REALLOC_GNU): Remove. * m4/calloc.m4 (_AC_FUNC_CALLOC_IF): * m4/malloc.m4 (_AC_FUNC_MALLOC_IF): * m4/realloc.m4 (_AC_FUNC_REALLOC_IF): Don’t start with a newline. Fix message to match behavior. * m4/calloc.m4 (_AC_FUNC_CALLOC_IF): Don’t test for size_t overflow, as the ptrdiff_t test is good enough. * m4/calloc.m4 (gl_FUNC_CALLOC_GNU): * m4/malloc.m4 (gl_FUNC_MALLOC_GNU): * m4/realloc.m4 (gl_FUNC_REALLOC_GNU): Do not define HAVE_CALLOC_GNU, HAVE_MALLOC_GNU, HAVE_REALLOC_GNU. It’s not worth the aggravation of maintaining these, as they are confusing (they don’t really mean GNU-compatible anyway). Don’t bother testing for GNU behavior if we have already decided to replace the function, since the replacement is always GNUish. * m4/calloc.m4 (gl_FUNC_CALLOC_POSIX): * m4/realloc.m4 (gl_FUNC_REALLOC_POSIX): Defer to gl_FUNC_MALLOC_POSIX. * m4/malloc.m4 (gl_FUNC_MALLOC_PTRDIFF, gl_CHECK_MALLOC_PTRDIFF): New macros. (gl_FUNC_MALLOC_POSIX): Use them to check for ptrdiff_t overflow. * modules/calloc-gnu, modules/malloc-gnu, modules/realloc-gnu: Remove no-longer-needed module indicators. * modules/calloc-posix, modules/malloc-posix, modules/realloc-posix: Depend on xalloc-oversized. * modules/malloc-posix: Require gl_FUNC_MALLOC_POSIX instead of calling it directly, so that other code can require it. * modules/realloc-posix: Depend on free-posix and malloc-posix. --- ChangeLog | 62 +++++++++++++++++++++++++ doc/posix-functions/calloc.texi | 17 +++++-- doc/posix-functions/malloc.texi | 17 ++++--- doc/posix-functions/realloc.texi | 37 ++++++++++----- lib/calloc.c | 37 +++++---------- lib/malloc.c | 32 +++++-------- lib/realloc.c | 55 +++++++--------------- m4/calloc.m4 | 67 +++++++-------------------- m4/malloc.m4 | 79 ++++++++++++++++++++++++-------- m4/realloc.m4 | 32 ++++--------- modules/calloc-gnu | 1 - modules/calloc-posix | 1 + modules/malloc-gnu | 1 - modules/malloc-posix | 4 +- modules/realloc-gnu | 1 - modules/realloc-posix | 4 +- 16 files changed, 243 insertions(+), 204 deletions(-) diff --git a/ChangeLog b/ChangeLog index 1e4c95524..825201fe2 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,65 @@ +2021-04-17 Paul Eggert + + malloc, etc.: check for ptrdiff_t overflow + In glibc 2.30 and later, malloc, realloc and calloc reject + attempts to create objects larger than PTRDIFF_MAX bytes. + This patch changes malloc-gnu etc. to support this behavior + on non-GNU hosts. It also makes this change for malloc-posix etc. + since it’s a safety measure that ought to be in POSIX (perhaps + we can talk them into that...). + + In writing this patch I found a complicated set of code that had + accumulated over the years, some written by yours truly. I got + rid of the code I couldn’t see the need for nowadays. Among other + things, the GNU realloc behavior is no longer incompatible with + the C standard, because in C17 the latter was relaxed to allow the + former. If I went too far in cleaning up, the old stuff can be + resurrected. + + This change is mostly for 32-bit platforms, since practical 64-bit + platforms cannot create objects larger than PTRDIFF_MAX bytes anyway. + * doc/posix-functions/calloc.texi: + * doc/posix-functions/malloc.texi: + * doc/posix-functions/realloc.texi: + Mention ptrdiff_t issues, and go into more detail about what + the gnu extension module does. + * doc/posix-functions/realloc.texi: Fix now-obsolete commentary + about C99 vs glibc, as C17 allows the glibc behavior and POSIX + will follow suit when it gets around to it. + * lib/calloc.c, lib/malloc.c, lib/realloc.c: + Simplify by always supplying a GNU-compatible version, + as that suffices for correctness and is good enough for performance. + Include xalloc-oversized.h, and use xalloc_oversized to + check for ptrdiff_t overflow. + (NEED_CALLOC_GNU, NEED_MALLOC_GNU, NEED_REALLOC_GNU): Remove. + * m4/calloc.m4 (_AC_FUNC_CALLOC_IF): + * m4/malloc.m4 (_AC_FUNC_MALLOC_IF): + * m4/realloc.m4 (_AC_FUNC_REALLOC_IF): + Don’t start with a newline. Fix message to match behavior. + * m4/calloc.m4 (_AC_FUNC_CALLOC_IF): Don’t test for size_t overflow, + as the ptrdiff_t test is good enough. + * m4/calloc.m4 (gl_FUNC_CALLOC_GNU): + * m4/malloc.m4 (gl_FUNC_MALLOC_GNU): + * m4/realloc.m4 (gl_FUNC_REALLOC_GNU): + Do not define HAVE_CALLOC_GNU, HAVE_MALLOC_GNU, HAVE_REALLOC_GNU. + It’s not worth the aggravation of maintaining these, as they + are confusing (they don’t really mean GNU-compatible anyway). + Don’t bother testing for GNU behavior if we have already decided + to replace the function, since the replacement is always GNUish. + * m4/calloc.m4 (gl_FUNC_CALLOC_POSIX): + * m4/realloc.m4 (gl_FUNC_REALLOC_POSIX): + Defer to gl_FUNC_MALLOC_POSIX. + * m4/malloc.m4 (gl_FUNC_MALLOC_PTRDIFF, gl_CHECK_MALLOC_PTRDIFF): + New macros. + (gl_FUNC_MALLOC_POSIX): Use them to check for ptrdiff_t overflow. + * modules/calloc-gnu, modules/malloc-gnu, modules/realloc-gnu: + Remove no-longer-needed module indicators. + * modules/calloc-posix, modules/malloc-posix, modules/realloc-posix: + Depend on xalloc-oversized. + * modules/malloc-posix: Require gl_FUNC_MALLOC_POSIX instead of + calling it directly, so that other code can require it. + * modules/realloc-posix: Depend on free-posix and malloc-posix. + 2021-04-17 Bruno Haible stdio: Fix build error in some configurations (regression 2021-04-11). diff --git a/doc/posix-functions/calloc.texi b/doc/posix-functions/calloc.texi index 22c51be02..9ba40c0d4 100644 --- a/doc/posix-functions/calloc.texi +++ b/doc/posix-functions/calloc.texi @@ -12,11 +12,22 @@ Portability problems fixed by Gnulib: Upon failure, the function does not set @code{errno} to @code{ENOMEM} on some platforms: mingw, MSVC 14. -@end itemize -Portability problems not fixed by Gnulib: -@itemize +@item +On some platforms, @code{calloc (n, s)} can succeed even if +multiplying @code{n} by @code{s} would exceed @code{PTRDIFF_MAX} or +@code{SIZE_MAX}. Although failing to check for exceeding +@code{PTRDIFF_MAX} is arguably allowed by POSIX it can lead to +undefined behavior later, so @code{calloc-posix} does not allow +going over the limit. @end itemize Extension: Gnulib provides a module @samp{calloc-gnu} that substitutes a @code{calloc} implementation that behaves more like the glibc implementation. +It fixes this portability problem: + +@itemize +@item +On some platforms, @code{calloc (0, s)} and @code{calloc (n, 0)} +return @code{NULL} on success. +@end itemize diff --git a/doc/posix-functions/malloc.texi b/doc/posix-functions/malloc.texi index 6d5995078..829517311 100644 --- a/doc/posix-functions/malloc.texi +++ b/doc/posix-functions/malloc.texi @@ -12,14 +12,19 @@ Portability problems fixed by Gnulib: Upon failure, the function does not set @code{errno} to @code{ENOMEM} on some platforms: mingw, MSVC 14. -@end itemize -Portability problems not fixed by Gnulib: -@itemize -@item @code{malloc (0)} always returns a NULL pointer on some platforms: -AIX 5.1. +@item +On some platforms, @code{malloc (n)} can succeed even if @code{n} +exceeds @code{PTRDIFF_MAX}. Although this behavior is arguably +allowed by POSIX it can lead to behavior not defined by POSIX later, +so @code{malloc-posix} does not allow going over the limit. @end itemize Extension: Gnulib provides a module @samp{malloc-gnu} that substitutes a @code{malloc} implementation that behaves more like the glibc implementation, -regarding the result of @code{malloc (0)}. +by fixing this portability problem: + +@itemize +@item +On some platforms, @code{malloc (0)} returns @code{NULL} on success. +@end itemize diff --git a/doc/posix-functions/realloc.texi b/doc/posix-functions/realloc.texi index 1f0b18e95..282e36051 100644 --- a/doc/posix-functions/realloc.texi +++ b/doc/posix-functions/realloc.texi @@ -12,24 +12,37 @@ Portability problems fixed by Gnulib: Upon failure, the function does not set @code{errno} to @code{ENOMEM} on some platforms: mingw, MSVC 14. -@end itemize -Portability problems not fixed by Gnulib: -@itemize @item -It is not portable to call @code{realloc} with a size of 0. With a +On some platforms, @code{realloc (p, n)} can succeed even if @code{n} +exceeds @code{PTRDIFF_MAX}. Although this behavior is arguably +allowed by POSIX it can lead to behavior not defined by POSIX later, +so @code{realloc-posix} does not allow going over the limit. +@end itemize + +Without the @samp{realloc-gnu} module described below, it is not portable +to call @code{realloc} with a size of 0. With a NULL pointer argument, this is the same ambiguity as @code{malloc (0)} on whether a unique zero-size object is created. With a non-NULL -pointer argument, C99 requires that if @code{realloc (p, 0)} returns -@code{NULL} then @code{p} is still valid. Among implementations that -obey C99, behavior varies on whether @code{realloc (p, 0)} always +pointer argument @code{p}, C17 says that it is implementation-defined +whether @code{realloc (p, 0)} frees @code{p}. +Behavior varies on whether @code{realloc (p, 0)} always frees @code{p} +and successfully returns a null pointer, or always fails and leaves @code{p} valid, or usually succeeds and returns a -unique zero-size object; either way, a program not suspecting these +unique zero-size object; a program not suspecting these variations in semantics will leak memory (either the still-valid @code{p}, or the -non-NULL return value). Meanwhile, several implementations violate -C99, by always calling @code{free (p)} but returning NULL: -glibc, Cygwin -@end itemize +non-NULL return value). Extension: Gnulib provides a module @samp{realloc-gnu} that substitutes a @code{realloc} implementation that behaves more like the glibc implementation. +It fixes these portability problems: + +@itemize +@item +On some platforms, @code{realloc (NULL, 0)} returns @code{NULL} on success. + +@item +On some platforms, @code{realloc (p, 0)} with non-null @code{p} +might not free @code{p}, or might clobber @code{errno}, +or might not return @code{NULL}. +@end itemize diff --git a/lib/calloc.c b/lib/calloc.c index 5ab1975cf..c9da08047 100644 --- a/lib/calloc.c +++ b/lib/calloc.c @@ -19,49 +19,34 @@ #include -/* The gnulib module 'calloc-gnu' defines HAVE_CALLOC_GNU. */ -#if GNULIB_CALLOC_GNU && !HAVE_CALLOC_GNU -# define NEED_CALLOC_GNU 1 -#endif - /* Specification. */ #include #include +#include "xalloc-oversized.h" + /* Call the system's calloc below. */ #undef calloc -/* Allocate and zero-fill an NxS-byte block of memory from the heap. - If N or S is zero, allocate and zero-fill a 1-byte block. */ +/* Allocate and zero-fill an NxS-byte block of memory from the heap, + even if N or S is zero. */ void * rpl_calloc (size_t n, size_t s) { - void *result; - -#if NEED_CALLOC_GNU if (n == 0 || s == 0) + n = s = 1; + + if (xalloc_oversized (n, s)) { - n = 1; - s = 1; - } - else - { - /* Defend against buggy calloc implementations that mishandle - size_t overflow. */ - size_t bytes = n * s; - if (bytes / s != n) - { - errno = ENOMEM; - return NULL; - } + errno = ENOMEM; + return NULL; } -#endif - result = calloc (n, s); + void *result = calloc (n, s); -#if !HAVE_CALLOC_POSIX +#if !HAVE_MALLOC_POSIX if (result == NULL) errno = ENOMEM; #endif diff --git a/lib/malloc.c b/lib/malloc.c index 272b8b9f4..a900046ec 100644 --- a/lib/malloc.c +++ b/lib/malloc.c @@ -20,43 +20,35 @@ #define _GL_USE_STDLIB_ALLOC 1 #include -/* The gnulib module 'malloc-gnu' defines HAVE_MALLOC_GNU. */ -#if GNULIB_MALLOC_GNU && !HAVE_MALLOC_GNU -# define NEED_MALLOC_GNU 1 -#endif - #include -/* A function definition is only needed if NEED_MALLOC_GNU is defined above - or if the module 'malloc-posix' requests it. */ -#if NEED_MALLOC_GNU || (GNULIB_MALLOC_POSIX && !HAVE_MALLOC_POSIX) +#include -# include +#include "xalloc-oversized.h" /* Call the system's malloc below. */ # undef malloc -/* Allocate an N-byte block of memory from the heap. - If N is zero, allocate a 1-byte block. */ +/* Allocate an N-byte block of memory from the heap, even if N is 0. */ void * rpl_malloc (size_t n) { - void *result; - -# if NEED_MALLOC_GNU if (n == 0) n = 1; -# endif - result = malloc (n); + if (xalloc_oversized (n, 1)) + { + errno = ENOMEM; + return NULL; + } + + void *result = malloc (n); -# if !HAVE_MALLOC_POSIX +#if !HAVE_MALLOC_POSIX if (result == NULL) errno = ENOMEM; -# endif +#endif return result; } - -#endif diff --git a/lib/realloc.c b/lib/realloc.c index c3e3cdfc5..c0d94b439 100644 --- a/lib/realloc.c +++ b/lib/realloc.c @@ -21,66 +21,43 @@ #define _GL_USE_STDLIB_ALLOC 1 #include -/* The gnulib module 'realloc-gnu' defines HAVE_REALLOC_GNU. */ -#if GNULIB_REALLOC_GNU && !HAVE_REALLOC_GNU -# define NEED_REALLOC_GNU 1 -#endif - -/* Infer the properties of the system's malloc function. - The gnulib module 'malloc-gnu' defines HAVE_MALLOC_GNU. */ -#if GNULIB_MALLOC_GNU && HAVE_MALLOC_GNU -# define SYSTEM_MALLOC_GLIBC_COMPATIBLE 1 -#endif - #include -/* A function definition is only needed if NEED_REALLOC_GNU is defined above - or if the module 'realloc-posix' requests it. */ -#if NEED_REALLOC_GNU || (GNULIB_REALLOC_POSIX && !HAVE_REALLOC_POSIX) +#include -# include +#include "xalloc-oversized.h" -/* Call the system's malloc and realloc below. */ -# undef malloc -# undef realloc +/* Call the system's realloc below. */ +#undef realloc /* Change the size of an allocated block of memory P to N bytes, - with error checking. If N is zero, change it to 1. If P is NULL, - use malloc. */ + with error checking. If P is NULL, use malloc. Otherwise if N is zero, + free P and return NULL. */ void * rpl_realloc (void *p, size_t n) { - void *result; + if (p == NULL) + return malloc (n); -# if NEED_REALLOC_GNU if (n == 0) { - n = 1; - - /* In theory realloc might fail, so don't rely on it to free. */ free (p); - p = NULL; + return NULL; } -# endif - if (p == NULL) + if (xalloc_oversized (n, 1)) { -# if GNULIB_REALLOC_GNU && !NEED_REALLOC_GNU && !SYSTEM_MALLOC_GLIBC_COMPATIBLE - if (n == 0) - n = 1; -# endif - result = malloc (n); + errno = ENOMEM; + return NULL; } - else - result = realloc (p, n); -# if !HAVE_REALLOC_POSIX + void *result = realloc (p, n); + +#if !HAVE_MALLOC_POSIX if (result == NULL) errno = ENOMEM; -# endif +#endif return result; } - -#endif diff --git a/m4/calloc.m4 b/m4/calloc.m4 index eeeb033d8..143f3bc3f 100644 --- a/m4/calloc.m4 +++ b/m4/calloc.m4 @@ -1,4 +1,4 @@ -# calloc.m4 serial 23 +# calloc.m4 serial 24 # Copyright (C) 2004-2021 Free Software Foundation, Inc. # This file is free software; the Free Software Foundation @@ -14,12 +14,10 @@ # _AC_FUNC_CALLOC_IF([IF-WORKS], [IF-NOT]) # ------------------------------------- -# If 'calloc (0, 0)' is properly handled, run IF-WORKS, otherwise, IF-NOT. +# If calloc is compatible with GNU calloc, run IF-WORKS, otherwise, IF-NOT. AC_DEFUN([_AC_FUNC_CALLOC_IF], -[ - AC_REQUIRE([AC_TYPE_SIZE_T])dnl - AC_REQUIRE([AC_CANONICAL_HOST]) dnl for cross-compiles - AC_CACHE_CHECK([for GNU libc compatible calloc], +[ AC_REQUIRE([AC_CANONICAL_HOST]) dnl for cross-compiles + AC_CACHE_CHECK([whether calloc (0, n) and calloc (n, 0) return nonnull], [ac_cv_func_calloc_0_nonnull], [if test $cross_compiling != yes; then ac_cv_func_calloc_0_nonnull=yes @@ -35,32 +33,6 @@ AC_DEFUN([_AC_FUNC_CALLOC_IF], ]])], [], [ac_cv_func_calloc_0_nonnull=no]) - AC_RUN_IFELSE( - [AC_LANG_PROGRAM( - [AC_INCLUDES_DEFAULT], - [[int result; - typedef struct { char c[8]; } S8; - size_t n = (size_t) -1 / sizeof (S8) + 2; - S8 * volatile s = calloc (n, sizeof (S8)); - if (s) - { - s[0].c[0] = 1; - if (s[n - 1].c[0]) - result = 0; - else - result = 2; - } - else - result = 3; - free (s); - return result; - ]])], - dnl The exit code of this program is 0 if calloc() succeeded with a - dnl wrap-around bug (which it shouldn't), 2 if calloc() succeeded in - dnl a non-flat address space, 3 if calloc() failed, or 1 if some leak - dnl sanitizer terminated the program as a result of the calloc() call. - [ac_cv_func_calloc_0_nonnull=no], - []) else case "$host_os" in # Guess yes on glibc systems. @@ -82,38 +54,31 @@ AC_DEFUN([_AC_FUNC_CALLOC_IF], $2 ;; esac -])# AC_FUNC_CALLOC +]) # gl_FUNC_CALLOC_GNU # ------------------ -# Report whether 'calloc (0, 0)' is properly handled, and replace calloc if -# needed. +# Replace calloc if it is not compatible with GNU libc. AC_DEFUN([gl_FUNC_CALLOC_GNU], [ AC_REQUIRE([gl_STDLIB_H_DEFAULTS]) - _AC_FUNC_CALLOC_IF( - [AC_DEFINE([HAVE_CALLOC_GNU], [1], - [Define to 1 if your system has a GNU libc compatible 'calloc' - function, and to 0 otherwise.])], - [AC_DEFINE([HAVE_CALLOC_GNU], [0]) - REPLACE_CALLOC=1 - ]) + AC_REQUIRE([gl_FUNC_CALLOC_POSIX]) + test $REPLACE_CALLOC = 1 || _AC_FUNC_CALLOC_IF([], [REPLACE_CALLOC=1]) ])# gl_FUNC_CALLOC_GNU - # gl_FUNC_CALLOC_POSIX # -------------------- # Test whether 'calloc' is POSIX compliant (sets errno to ENOMEM when it -# fails), and replace calloc if it is not. +# fails, and doesn't mess up with ptrdiff_t or size_t overflow), +# and replace calloc if it is not. AC_DEFUN([gl_FUNC_CALLOC_POSIX], [ AC_REQUIRE([gl_STDLIB_H_DEFAULTS]) - AC_REQUIRE([gl_CHECK_MALLOC_POSIX]) - if test $gl_cv_func_malloc_posix = yes; then - AC_DEFINE([HAVE_CALLOC_POSIX], [1], - [Define if the 'calloc' function is POSIX compliant.]) - else - REPLACE_CALLOC=1 - fi + AC_REQUIRE([gl_FUNC_MALLOC_POSIX]) + REPLACE_CALLOC=$REPLACE_MALLOC + dnl Although in theory we should also test for size_t overflow, + dnl in practice testing for ptrdiff_t overflow suffices + dnl since PTRDIFF_MAX <= SIZE_MAX on all known Gnulib porting targets. + dnl A separate size_t test would slow down 'configure'. ]) diff --git a/m4/malloc.m4 b/m4/malloc.m4 index 32ab42ec0..503da2cf8 100644 --- a/m4/malloc.m4 +++ b/m4/malloc.m4 @@ -1,4 +1,4 @@ -# malloc.m4 serial 22 +# malloc.m4 serial 23 dnl Copyright (C) 2007, 2009-2021 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -7,9 +7,8 @@ dnl with or without modifications, as long as this notice is preserved. # This is adapted with modifications from upstream Autoconf here: # https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=04be2b7a29d65d9a08e64e8e56e594c91749598c AC_DEFUN([_AC_FUNC_MALLOC_IF], -[ - AC_REQUIRE([AC_CANONICAL_HOST])dnl for cross-compiles - AC_CACHE_CHECK([for GNU libc compatible malloc], +[ AC_REQUIRE([AC_CANONICAL_HOST])dnl for cross-compiles + AC_CACHE_CHECK([whether malloc (0) returns nonnull], [ac_cv_func_malloc_0_nonnull], [AC_RUN_IFELSE( [AC_LANG_PROGRAM( @@ -44,47 +43,89 @@ AC_DEFUN([_AC_FUNC_MALLOC_IF], # gl_FUNC_MALLOC_GNU # ------------------ -# Test whether 'malloc (0)' is handled like in GNU libc, and replace malloc if -# it is not. +# Replace malloc if it is not compatible with GNU libc. AC_DEFUN([gl_FUNC_MALLOC_GNU], [ AC_REQUIRE([gl_STDLIB_H_DEFAULTS]) - dnl _AC_FUNC_MALLOC_IF is defined in Autoconf. - _AC_FUNC_MALLOC_IF( - [AC_DEFINE([HAVE_MALLOC_GNU], [1], - [Define to 1 if your system has a GNU libc compatible 'malloc' - function, and to 0 otherwise.])], - [AC_DEFINE([HAVE_MALLOC_GNU], [0]) - REPLACE_MALLOC=1 + AC_REQUIRE([gl_FUNC_MALLOC_POSIX]) + test $REPLACE_MALLOC = 1 || _AC_FUNC_MALLOC_IF([], [REPLACE_MALLOC=1]) +]) + +# gl_FUNC_MALLOC_PTRDIFF +# ---------------------- +# Test whether malloc (N) reliably fails when N exceeds PTRDIFF_MAX, +# and replace malloc otherwise. +AC_DEFUN([gl_FUNC_MALLOC_PTRDIFF], +[ + AC_REQUIRE([gl_STDLIB_H_DEFAULTS]) + AC_REQUIRE([gl_CHECK_MALLOC_PTRDIFF]) + test "$gl_cv_malloc_ptrdiff" = yes || REPLACE_MALLOC=1 +]) + +# Test whether malloc, realloc, calloc refuse to create objects +# larger than what can be expressed in ptrdiff_t. +# Set gl_cv_func_malloc_gnu to yes or no accordingly. +AC_DEFUN([gl_CHECK_MALLOC_PTRDIFF], +[ + AC_CACHE_CHECK([whether malloc is ptrdiff_t safe], + [gl_cv_malloc_ptrdiff], + [AC_COMPILE_IFELSE( + [AC_LANG_PROGRAM( + [[#include + ]], + [[/* 64-bit ptrdiff_t is so wide that no practical platform + can exceed it. */ + #define WIDE_PTRDIFF (PTRDIFF_MAX >> 31 >> 31 != 0) + + /* On rare machines where size_t fits in ptrdiff_t there + is no problem. */ + #define NARROW_SIZE (SIZE_MAX <= PTRDIFF_MAX) + + /* glibc 2.30 and later malloc refuses to exceed ptrdiff_t + bounds even on 32-bit platforms. We don't know which + non-glibc systems are safe. */ + #define KNOWN_SAFE (2 < __GLIBC__ + (30 <= __GLIBC_MINOR__)) + + #if WIDE_PTRDIFF || NARROW_SIZE || KNOWN_SAFE + return 0; + #else + #error "malloc might not be ptrdiff_t safe" + syntax error + #endif + ]])], + [gl_cv_malloc_ptrdiff=yes], + [gl_cv_malloc_ptrdiff=no]) ]) ]) # gl_FUNC_MALLOC_POSIX # -------------------- # Test whether 'malloc' is POSIX compliant (sets errno to ENOMEM when it -# fails), and replace malloc if it is not. +# fails, and doesn't mess up with ptrdiff_t overflow), and replace +# malloc if it is not. AC_DEFUN([gl_FUNC_MALLOC_POSIX], [ AC_REQUIRE([gl_STDLIB_H_DEFAULTS]) + AC_REQUIRE([gl_FUNC_MALLOC_PTRDIFF]) AC_REQUIRE([gl_CHECK_MALLOC_POSIX]) - if test $gl_cv_func_malloc_posix = yes; then + if test "$gl_cv_func_malloc_posix" = yes; then AC_DEFINE([HAVE_MALLOC_POSIX], [1], - [Define if the 'malloc' function is POSIX compliant.]) + [Define if malloc, realloc, and calloc set errno on allocation failure.]) else REPLACE_MALLOC=1 fi ]) -# Test whether malloc, realloc, calloc are POSIX compliant, +# Test whether malloc, realloc, calloc set errno on failure. # Set gl_cv_func_malloc_posix to yes or no accordingly. AC_DEFUN([gl_CHECK_MALLOC_POSIX], [ - AC_CACHE_CHECK([whether malloc, realloc, calloc are POSIX compliant], + AC_CACHE_CHECK([whether malloc, realloc, calloc set errno on failure], [gl_cv_func_malloc_posix], [ dnl It is too dangerous to try to allocate a large amount of memory: dnl some systems go to their knees when you do that. So assume that - dnl all Unix implementations of the function are POSIX compliant. + dnl all Unix implementations of the function set errno on failure. AC_COMPILE_IFELSE( [AC_LANG_PROGRAM( [[]], diff --git a/m4/realloc.m4 b/m4/realloc.m4 index a80a02a6b..4939516a0 100644 --- a/m4/realloc.m4 +++ b/m4/realloc.m4 @@ -1,4 +1,4 @@ -# realloc.m4 serial 20 +# realloc.m4 serial 21 dnl Copyright (C) 2007, 2009-2021 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -7,9 +7,8 @@ dnl with or without modifications, as long as this notice is preserved. # This is adapted with modifications from upstream Autoconf here: # https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=04be2b7a29d65d9a08e64e8e56e594c91749598c AC_DEFUN([_AC_FUNC_REALLOC_IF], -[ - AC_REQUIRE([AC_CANONICAL_HOST])dnl for cross-compiles - AC_CACHE_CHECK([for GNU libc compatible realloc], +[ AC_REQUIRE([AC_CANONICAL_HOST])dnl for cross-compiles + AC_CACHE_CHECK([whether realloc (0, 0) returns nonnull], [ac_cv_func_realloc_0_nonnull], [AC_RUN_IFELSE( [AC_LANG_PROGRAM( @@ -44,33 +43,22 @@ AC_DEFUN([_AC_FUNC_REALLOC_IF], # gl_FUNC_REALLOC_GNU # ------------------- -# Test whether 'realloc (0, 0)' is handled like in GNU libc, and replace -# realloc if it is not. +# Replace realloc if it is not compatible with GNU libc. AC_DEFUN([gl_FUNC_REALLOC_GNU], [ AC_REQUIRE([gl_STDLIB_H_DEFAULTS]) - dnl _AC_FUNC_REALLOC_IF is defined in Autoconf. - _AC_FUNC_REALLOC_IF( - [AC_DEFINE([HAVE_REALLOC_GNU], [1], - [Define to 1 if your system has a GNU libc compatible 'realloc' - function, and to 0 otherwise.])], - [AC_DEFINE([HAVE_REALLOC_GNU], [0]) - REPLACE_REALLOC=1 - ]) + AC_REQUIRE([gl_FUNC_REALLOC_POSIX]) + test $REPLACE_REALLOC = 1 || _AC_FUNC_REALLOC_IF([], [REPLACE_REALLOC=1]) ])# gl_FUNC_REALLOC_GNU # gl_FUNC_REALLOC_POSIX # --------------------- # Test whether 'realloc' is POSIX compliant (sets errno to ENOMEM when it -# fails), and replace realloc if it is not. +# fails, and doesn't mess up with ptrdiff_t overflow), +# and replace realloc if it is not. AC_DEFUN([gl_FUNC_REALLOC_POSIX], [ AC_REQUIRE([gl_STDLIB_H_DEFAULTS]) - AC_REQUIRE([gl_CHECK_MALLOC_POSIX]) - if test $gl_cv_func_malloc_posix = yes; then - AC_DEFINE([HAVE_REALLOC_POSIX], [1], - [Define if the 'realloc' function is POSIX compliant.]) - else - REPLACE_REALLOC=1 - fi + AC_REQUIRE([gl_FUNC_MALLOC_POSIX]) + REPLACE_REALLOC=$REPLACE_MALLOC ]) diff --git a/modules/calloc-gnu b/modules/calloc-gnu index ffc8b50ef..2aa2dd1c0 100644 --- a/modules/calloc-gnu +++ b/modules/calloc-gnu @@ -13,7 +13,6 @@ gl_FUNC_CALLOC_GNU if test $REPLACE_CALLOC = 1; then AC_LIBOBJ([calloc]) fi -gl_MODULE_INDICATOR([calloc-gnu]) Makefile.am: diff --git a/modules/calloc-posix b/modules/calloc-posix index cead843ae..dc9cadd5c 100644 --- a/modules/calloc-posix +++ b/modules/calloc-posix @@ -8,6 +8,7 @@ m4/malloc.m4 Depends-on: stdlib +xalloc-oversized [test $REPLACE_REALLOC = 1] configure.ac: gl_FUNC_CALLOC_POSIX diff --git a/modules/malloc-gnu b/modules/malloc-gnu index 0bfa92d75..f8bcdae55 100644 --- a/modules/malloc-gnu +++ b/modules/malloc-gnu @@ -17,7 +17,6 @@ gl_FUNC_MALLOC_GNU if test $REPLACE_MALLOC = 1; then AC_LIBOBJ([malloc]) fi -gl_MODULE_INDICATOR([malloc-gnu]) Makefile.am: diff --git a/modules/malloc-posix b/modules/malloc-posix index 1a2d72c5a..bafcf3801 100644 --- a/modules/malloc-posix +++ b/modules/malloc-posix @@ -7,14 +7,14 @@ m4/malloc.m4 Depends-on: stdlib +xalloc-oversized [test $REPLACE_MALLOC = 1] configure.ac: -gl_FUNC_MALLOC_POSIX +AC_REQUIRE([gl_FUNC_MALLOC_POSIX]) if test $REPLACE_MALLOC = 1; then AC_LIBOBJ([malloc]) fi gl_STDLIB_MODULE_INDICATOR([malloc-posix]) -gl_MODULE_INDICATOR([malloc-posix]) Makefile.am: diff --git a/modules/realloc-gnu b/modules/realloc-gnu index 760edcda1..7752d8c6a 100644 --- a/modules/realloc-gnu +++ b/modules/realloc-gnu @@ -17,7 +17,6 @@ gl_FUNC_REALLOC_GNU if test $REPLACE_REALLOC = 1; then AC_LIBOBJ([realloc]) fi -gl_MODULE_INDICATOR([realloc-gnu]) Makefile.am: diff --git a/modules/realloc-posix b/modules/realloc-posix index 7f481102b..a30356f31 100644 --- a/modules/realloc-posix +++ b/modules/realloc-posix @@ -8,6 +8,9 @@ m4/malloc.m4 Depends-on: stdlib +free-posix [test $REPLACE_REALLOC = 1] +malloc-posix [test $REPLACE_REALLOC = 1] +xalloc-oversized [test $REPLACE_REALLOC = 1] configure.ac: gl_FUNC_REALLOC_POSIX @@ -15,7 +18,6 @@ if test $REPLACE_REALLOC = 1; then AC_LIBOBJ([realloc]) fi gl_STDLIB_MODULE_INDICATOR([realloc-posix]) -gl_MODULE_INDICATOR([realloc-posix]) Makefile.am: -- 2.27.0