In fiddling about with mpc_eta_fund() I noticed that, for an mpc_t that has real part NaN and imaginary part Inf, mpc_abs() sets the absolute value to Inf.
I would have expected the absolute value to instead be Nan - because sqrt((NaN^2) + (Inf^2)) is NaN.
As it stands, with that return value of Inf, and the current documentation of mpc_eta_fund(), that mpc_t lies "in the fundamental domain".
Similarly, if the real and imaginary parts of this mpc_t are interchanged, then mpc_abs() still sets an absolute value of Inf.
I'm feeling a bit lazy and at this stage I've only tested this using the perl interface ... I don't think that's where the problem lies ... apologies if it turns out I'm wrong about that.
>perl -MMath::MPC=":mpc" -le "print MPC_VERSION_STRING; $rop = Math::MPFR->new(); $op = Math::MPC->new('NaN' + 0, 'Inf' + 0); print $op; Rmpc_abs($rop, $op, 0); print $rop;"
1.3.0rc2
(@NaN@ @Inf@)
Inf
On looking a bit further, I see mpc_abs() just calls on mpfr_hypot().
Again, using the perli interface only, it seems that mpfr_hypot() is the culprit:
perl -MMath::MPFR=":mpfr" -le "$rop = Math::MPFR->new(); $op1 = Math::MPFR->new(); $op2 = Math::MPFR->new('Inf' + 0); print $op1; print $op2; Rmpfr_hypot($rop, $op1, $op2, 0); print $rop;"
NaN
Inf
Inf
Cheers,
Rob