Re: ptrdiff_t overflow checks for malloc-posix etc.

[Paul Eggert]

On 4/18/21 5:11 AM, Bruno Haible wrote:
> Paul Eggert wrote:
> > Come to think of it, why do we have both malloc-gnu and malloc-posix
> > modules (and similarly for calloc and realloc)?
>
> Package owners had two decisions to make:
>    "Does my package ever call malloc (0)? - If yes, I need 'malloc-gnu'."
>    "Does my package attempt portability to native Windows? - If yes, I
>     need 'malloc-posix'."

Sure, but that sort of approach doesn't scale well. Now that we know the 
issues better, that approach would require package owners to answer 11 
questions:

A (for malloc, realloc, calloc). Does my package need errno to be set 
when malloc/realloc/calloc fails?

B (for malloc, realloc, calloc). Does my package require 
malloc/realloc/calloc to fail if it would create an object with more 
than PTRDIFF_MAX bytes?

C (for malloc, realloc, calloc). Does my package require malloc/calloc 
to return NULL if and only if it fails, and similarly for realloc (P, N) 
when !P || N?

D (for realloc). Does my package require realloc (P, 0) with nonnull P 
to free P and return NULL?

E (for calloc). Does my package require calloc (N, S) to fail when N * S 
would exceed SIZE_MAX?

And I've probably forgotten one or more questions.

Should we have 11 different modules, one for each question? Surely not. 
We shouldn't expect other packages to carefully select exactly which of 
these 11 questions they need GNU compatibility with. Such a selection 
will be error-prone and won't be well-tested if we get the selections wrong.

It's also not reasonable to test all 2**11 combinations of answers to 
these questions. Some combinations will never occur, and others will be 
rare and hard to be tested.

Instead, I suggest that we simply implement malloc, realloc and calloc 
the GNU way on all platforms. Packages that rely on any GNU malloc 
feature can simply require the malloc-gnu module without worrying about 
individual questions.

Come to think of it, it might be better to have a single module (say, 
alloc-gnu) check and fix all three functions (malloc, realloc, calloc) 
as that'd be simpler yet.

> For example, if my package never calls malloc (0) but desires portability
> to native Windows, would I want that *every* malloc call on *all*
> non-glibc platforms goes through hoops before it reaches the malloc()
> function in libc?

I would want that, yes, because that'd be more reliable than what we 
were doing. Any performance penalty on non-GNU platforms is surely 
insignificant in real applications, and so is outweighed by the 
reliability improvement. Admittedly it's hard to measure the reliability 
improvement, but I've already fixed an unlikely bug or two as part of 
this cleanup, and I won't be surprised to fix more.