Hello,
eq? and eqv? are sort of a funny pair. eqv? actually has sensible semantics - if two things are eqv?, then a normal scheme program should never notice the difference between them (they are operationally equivalent). eq? is defined not in terms of Scheme, but in terms of Scheme's implementation - two things are eq? if they are represented with the same bit of memory.
The reason for eq? is that eq? can be implemented very efficiently, especially on old hardware that was current when that part of the Scheme standard was written. For some types (i.e. booleans and symbols), eq? is the same as eqv?, so eq? is used like a higher-performing shortcut to eqv?. Nowadays, it's probably best to just use eqv? and spend your time worrying about cache misses if you care about performance.
The particular case you mention is not a bug, but it's also not guaranteed to work for all numbers. Guile represents small numbers (less than 2^62 on 64-bit systems, I believe) without a pointer, which means that the obvious eq? implementation treats them as the same thing. This is allowed by the standard, but it won't hold true for big numbers, which are represented as blocks of memory allocated in the heap.
Best,
Noah