[Chicken-hackers] [PATCH] Do not assume all fixnum literals will be fixnums at runtime [was: Re: [PATCH][5] types.db entries for the overflow-aware ops]

[Peter Bex]

On Sun, May 21, 2017 at 06:25:09PM +0200, Peter Bex wrote:
> Consider the case where you're compiling on a
> 64-bit machine.  The compiler folds the fixnum operations at
> compile-time, resulting in a fixnum.  However, if the fixnum is not
> representable as a fixnum on the target computer, it will be represented
> as a bignum (or on CHICKEN 4, as a flonum).
> 
> This not only produces a different answer than when the same code would
> be compiled on the 32-bit machine and in the interpreter, but it also
> may trigger other unexpected behaviour resulting from a mismatch of the
> expectation that you're dealing with fixnums only.  For example, a
> specialised unsafe version of a generic procedure might end up being
> specialised for fixnums, but on a 32-bit platform it might still receive
> a bignum (or a flonum on master).
> 
> Hm, I wonder if this stuff causes other problems as well...

Yep, it does.

Try compiling this program to C on a 64-bit machine:

(use (chicken bitwise) (chicken io))
(print (bitwise-and (vector-length (read)) #xfffffffffffffff))

Then compile the resulting C on a 32-bit machine.  I expected bogus
output, but the program actually hung on the tests that I did.
You'll notice that the compiler decided to specialise bitwise-and for
fixnums because both #xfffffffffffffff and the result of vector-length
are fixnums.  This is completely bogus, of course.

Luckily, the fix is quite simple: just don't commit to calling the
literal a fixnum (or bignum!) if you know on another platform it
might not be the same type.  Instead, use 'integer.

This issue also affects master, so I went ahead and added a patch
for that as well.  This is slightly different because we don't have
the integer type, of course.

I also removed some dead code: the (number?) check would never be
reached because the preceding checks are exhaustive.

Cheers,
Peter

0001-In-the-scrutinizer-do-not-assume-big-fixnums-will-fi.chicken-5.patch
Description: Text Data

0001-In-the-scrutinizer-do-not-assume-big-fixnums-will-fi.master.patch
Description: Text Data

signature.asc
Description: Digital signature