[PATCH] alignalloc: work around AddressSanitizer bug

[Paul Eggert]

* doc/posix-functions/aligned_alloc.texi (aligned_alloc):
Mention AddressSanitizer bug.
* lib/alignalloc.h (ALIGNALLOC_VIA_ALIGNED_ALLOC):
Define to 0 if AddressSanitizer is in use.
* tests/test-alignalloc.c (test_alignalloc): New function,
which tests for non-aligned sizes too.
(main): Use it.  Don’t bother checking for alignments
greater than 16 MiB, as this flummoxes AddressSanitizer
and there seems little point to testing them.
---
 ChangeLog                              | 11 +++++++
 doc/posix-functions/aligned_alloc.texi | 20 ++++++++++++-
 lib/alignalloc.h                       | 15 ++++++++++
 tests/test-alignalloc.c                | 40 +++++++++++++++-----------
 4 files changed, 68 insertions(+), 18 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 075f81124d..1c036a5697 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,16 @@
 2022-01-27  Paul Eggert  <eggert@cs.ucla.edu>
 
+       alignalloc: work around AddressSanitizer bug
+       * doc/posix-functions/aligned_alloc.texi (aligned_alloc):
+       Mention AddressSanitizer bug.
+       * lib/alignalloc.h (ALIGNALLOC_VIA_ALIGNED_ALLOC):
+       Define to 0 if AddressSanitizer is in use.
+       * tests/test-alignalloc.c (test_alignalloc): New function,
+       which tests for non-aligned sizes too.
+       (main): Use it.  Don’t bother checking for alignments
+       greater than 16 MiB, as this flummoxes AddressSanitizer
+       and there seems little point to testing them.
+
        doc: use UTF-8 encoding
        * doc/gnulib.texi: Use ‘@documentencoding UTF-8’.
        Partly this is because there are a few UTF-8 characters
diff --git a/doc/posix-functions/aligned_alloc.texi 
b/doc/posix-functions/aligned_alloc.texi
index 1f33ea3d37..e78ca16f9e 100644
--- a/doc/posix-functions/aligned_alloc.texi
+++ b/doc/posix-functions/aligned_alloc.texi
@@ -17,12 +17,30 @@ macOS 11.1, AIX 7.2.
 Portability problems not fixed by Gnulib:
 @itemize
 @item
-This function is missing on all non-glibc platforms:
+On some platforms, @code{aligned_alloc} crashes if the requested size is
+not a multiple of the alignment:
+AddressSanitizer (gcc 11.2 or clang 13).
+
+@item
+This function is missing on many older platforms:
 glibc 2.15, macOS 10.13, FreeBSD 6.4, NetBSD 7.1, OpenBSD 6.0, Minix 3.1.8, 
AIX 7.1, HP-UX 11.31, IRIX 6.5, Solaris 11.3, Cygwin 1.7.x, mingw, MSVC 14, 
Android 8.1.
 @end itemize
 
+Gnulib has partial substitutes for @code{aligned_alloc}
+that do not crash even if the AddressSanitizer bug is present:
+
+@itemize
+@item
+The Gnulib module @code{alignalloc} provides a portable function
+@code{alignalloc} that is a near-substitute for for glibc
+@code{aligned_alloc}, except that the result must be freed
+with @code{alignfree} rather than plain @code{free}.
+
+@item
 The Gnulib module @code{aligned-malloc} provides functions for
 allocating and freeing blocks of suitably aligned memory.
 
+@item
 The Gnulib module @code{pagealign_alloc} provides a similar API for
 allocating and freeing blocks of memory aligned on a system page boundary.
+@end itemize
diff --git a/lib/alignalloc.h b/lib/alignalloc.h
index 7e4c4743e9..f47aa86dcd 100644
--- a/lib/alignalloc.h
+++ b/lib/alignalloc.h
@@ -41,6 +41,21 @@ _GL_INLINE_HEADER_BEGIN
 # define ALIGNALLOC_VIA_ALIGNED_ALLOC 0
 #endif
 
+/* Work around AddressSanitizer bug.
+   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104262
+   
https://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20220124/1001910.html
+   */
+#ifdef __SANITIZE_ADDRESS__
+# undef ALIGNALLOC_VIA_ALIGNED_ALLOC
+# define ALIGNALLOC_VIA_ALIGNED_ALLOC 0
+#endif
+#ifdef __has_feature
+# if __has_feature (address_sanitizer)
+#  undef ALIGNALLOC_VIA_ALIGNED_ALLOC
+#  define ALIGNALLOC_VIA_ALIGNED_ALLOC 0
+# endif
+#endif
+
 #if ALIGNALLOC_VIA_ALIGNED_ALLOC || HAVE_POSIX_MEMALIGN
 
 /* Free storage allocated via alignalloc.  Do nothing if PTR is null.  */
diff --git a/tests/test-alignalloc.c b/tests/test-alignalloc.c
index 161ab384db..eccaea0e68 100644
--- a/tests/test-alignalloc.c
+++ b/tests/test-alignalloc.c
@@ -29,26 +29,32 @@ SIGNATURE_CHECK (alignfree, void, (void *));
 
 #include "macros.h"
 
-int
-main ()
+static void
+test_alignalloc (idx_t alignment, idx_t size)
 {
-  /* Check that alignalloc returns properly aligned storage,
-     when it succeeds.  */
-  for (idx_t alignment = 1; ; )
+  void *p = alignalloc (alignment, size);
+  if (p)
     {
-      for (idx_t size = 0; size <= 1024; size = size ? 2 * size : 1)
-        {
-          void *p = alignalloc (alignment, size);
-          if (p)
-            {
-              memset (p, 0, size);
-              ASSERT ((uintptr_t) p % alignment == 0);
-            }
-          alignfree (p);
-        }
-      if (INT_MULTIPLY_WRAPV (alignment, 2, &alignment))
-        break;
+      memset (p, 0, size);
+      ASSERT ((uintptr_t) p % alignment == 0);
     }
+  alignfree (p);
+}
+
+int
+main ()
+{
+  /* Check that alignalloc returns properly aligned storage when it succeeds.
+     Stop at 16 MiB alignments because circa-2022 AddressSanitizer goes
+     catatonic with large alignments in posix_memalign,
+     and there seems to be little point to testing them.  */
+  for (idx_t alignment = 1; alignment <= 16 * 1024 * 1024; alignment *= 2)
+    for (idx_t size = 1; size <= 1024; size *= 2)
+      {
+        test_alignalloc (alignment, size - 1);
+        test_alignalloc (alignment, size);
+        test_alignalloc (alignment, size + 1);
+      }
 
   /* Check that alignfree is a no-op on null pointers.  */
   alignfree (NULL);
-- 
2.32.0