Re: checking against signed integer overflow

[Paul Eggert]

On 12/6/20 8:30 AM, Bruno Haible wrote:

> Thanks. This could even be moved to the Autoconf manual, if there is
> sufficient agreement among GNU developers.

Could be, once we figure out what should be in it. :-) Though this stuff really 
should be documented better in the GCC manual, I suppose.

> for '-fsanitize=signed-integer-overflow' there is no reason for an
> ABI change. It's only the code inside functions which behaves differently.

The main issue here is which of these options are intended to be used in 
production code. For those options there will be a lot of user pressure for 
backward-compatibility support, due to the dusty-binary problem. For debugging 
options there won't be. -fsanitize=undefined is intended to be for debugging; if 
we start using it in production code there may be a clash down the line (RHEL 
changes some way that the error message gets printed, say).

> So, '-fsanitize=signed-integer-overflow -fsanitize-undefined-trap-on-error' and
> '-ftrapv' both work. The former generates better code, whereas the latter has
> less surprising behaviour (an abort() is a better response than an illegal
> instruction, IMO).

Yes, we've discussed this before. I'd rather not have call 'abort' here, since 
arithmetic overflow failures are in the same category as dividing by zero or 
(INT_MIN / -1) and 'abort' isn't called there either.  Calling 'abort' slightly 
complicates the way I debug, and it's just one more thing that can go wrong at 
least in theory (stack space exhaustion, PLT corruption, etc.). Plus it bloats 
the code a bit.

It's not a big deal either way, for typical debugging. For production code, 
though, traps look better to me.