Re: Fix calloc.m4 test

[Paul Eggert]

On 5/23/20 11:48 AM, Bruno Haible wrote:
>   2) It has estimated that the second call would return a non-NULL pointer,
>      although the address space does not allow this.
>      Reported at <https://bugs.llvm.org/show_bug.cgi?id=37304>. But some
>      people claim it is not a bug. Paul, can you please help with ISO C
>      citations?

Unfortunately Vincent Lefevre is correct that the C Standard allows calloc
(SIZE_MAX, 2) to succeed on a hypothetical machine that actually can allocate
that amount of memory. This could in theory occur on a machine that doesn't have
a flat address space.

A program like the following should defeat Clang's optimization, though (at
least, if Clang is not buggy). And it would also detect the hypothetical machine
that Vincent alluded to, which would make it more-accurate than the 'volatile'
workaround. How about if we switch calloc.m4 to use this test instead?

  #include <stdlib.h>
  int main ()
  {
    struct { char c[8]; } *s;
    size_t sn = (size_t) -1 / sizeof *s + 2;
    int result = 0;
    char *p = calloc (0, 0);
    if (!p)
      result |= 1;
    free (p);
    s = calloc (sn, sizeof *s);
    if (s)
      {
        s[0].c[0] = 1;
        if (s[sn - 1].c[0])
          result |= 2;
      }
    free (s);
    return result;
  }