Fix calloc.m4 test

[Bruno Haible]

Tim Rühsen wrote:
> - gcc-10 with unset CFLAGS (*.gcc-10)
> - clang-10 with unset CFLAGS (*.clang-10)

Comparing these two results, there is a difference:

-ac_cv_func_calloc_0_nonnull=${ac_cv_func_calloc_0_nonnull=yes}
+ac_cv_func_calloc_0_nonnull=${ac_cv_func_calloc_0_nonnull=no}

Why this test result? Let's see:

$ cat foo.c
#include <stdlib.h>
int main ()
{
  int result = 0;
  char *p = calloc (0, 0);
  if (!p)
    result |= 1;
  free (p);
  p = calloc ((size_t) -1 / 8 + 1, 8);
  if (p)
    result |= 2;
  free (p);
  return result;
}

$ clang -O2 -S foo.c
$ cat foo.s
        .text
        .file   "foo.c"
        .globl  main                    # -- Begin function main
        .p2align        4, 0x90
        .type   main,@function
main:                                   # @main
        .cfi_startproc
# %bb.0:
        movl    $2, %eax
        retq
.Lfunc_end0:
        .size   main, .Lfunc_end0-main
        .cfi_endproc
                                        # -- End function
        .ident  "clang version 10.0.0 "
        .section        ".note.GNU-stack","",@progbits
        .addrsig

As you can see:

  1) clang has eliminated the calloc() and free() calls from the program.
     Apparently it "knows" how to optimize matching calloc - free pairs.

  2) It has estimated that the second call would return a non-NULL pointer,
     although the address space does not allow this.
     Reported at <https://bugs.llvm.org/show_bug.cgi?id=37304>. But some
     people claim it is not a bug. Paul, can you please help with ISO C
     citations?

This patch provides a workaround.


2020-05-23  Bruno Haible  <address@hidden>

        calloc-gnu: Avoid wrong configure results with clang.
        * m4/calloc.m4 (_AC_FUNC_CALLOC_IF): Mark the pointer variable as
        'volatile', to defeat compiler optimizations.

diff --git a/m4/calloc.m4 b/m4/calloc.m4
index 2e0d8ff..3361cba 100644
--- a/m4/calloc.m4
+++ b/m4/calloc.m4
@@ -1,4 +1,4 @@
-# calloc.m4 serial 20
+# calloc.m4 serial 21
 
 # Copyright (C) 2004-2020 Free Software Foundation, Inc.
 # This file is free software; the Free Software Foundation
@@ -25,7 +25,7 @@ AC_DEFUN([_AC_FUNC_CALLOC_IF],
        [AC_LANG_PROGRAM(
           [AC_INCLUDES_DEFAULT],
           [[int result = 0;
-            char *p = calloc (0, 0);
+            char * volatile p = calloc (0, 0);
             if (!p)
               result |= 1;
             free (p);