GSL master and UBsan findings

[Jeffrey Walton]

Hi Everyone/Patrick,

A quick testing pass with CFLAGS that include '-fsanitize=undefined
-fno-sanitize-recover=all' shows some undefined behavior in the
library or test suite. Several of them are listed below.

Some of them, like "runtime error: left shift of 1 by 31 places cannot
be represented in type 'int'...", are easy to fix. Just use "1U << 31"
since unsigned types do not suffer overflow. Unsigned types can wrap,
but that's well defined behavior in C/C++.

Jeff

====================

$ cat gsl-2.6/sys/test-suite.log
...
pow_int.c:34:10: runtime error: negation of -2147483648 cannot be
represented in type 'int'; cast to an unsigned type to negate this
value to itself
FAIL test (exit status: 1)

====================

$ cat gsl-2.6/specfunc/test-suite.log
...
FAIL: res[60]+0.0 [2351]
  expected: 6.7753780053831863e+71
  obtained: 6.7753780053832010e+71
  fracdiff: 1.0856927230202910e-15
  value not within tolerance of expected value
  6.775378005383201023e+71
FAIL: res[100]+0.0 [2353]
  expected: 2.9579660004912027e+118
  obtained: 2.9579660004911946e+118
  fracdiff: 1.3629435007875587e-15
  value not within tolerance of expected value
  2.957966000491194605e+118
FAIL: gsl_sf_hermite_array_deriv(23, 100, 0.75) [2354]
hermite.c:470:34: runtime error: signed integer overflow: 100028 *
100028 cannot be represented in type 'int'

====================

$ cat gsl-2.6/rng/test-suite.log
...
random.c:126:9: runtime error: signed integer overflow:
1508052306957066566 + 7775806152332120189 cannot be represented in
type 'long int'
FAIL test (exit status: 1)

====================

$ cat gsl-2.6/randist/test-suite.log
...
test.c:861:45: runtime error: left shift of 1 by 31 places cannot be
represented in type 'int'
FAIL test (exit status: 1)

====================

$ cat gsl-2.6/multifit_nlinear/test-suite.log
...
trs = levenberg-marquardt
solver = cholesky
solver = mcholesky
solver = qr
solver = svd
trs = levenberg-marquardt+accel
solver = cholesky
solver = mcholesky
solver = qr
solver = svd
trs = dogleg
solver = mcholesky
nielsen.c:98:7: runtime error: left shift of 4611686018427387904 by 1
places cannot be represented in type 'long int'
FAIL test (exit status: 1)