[Octave-bug-tracker] [bug #62319] Another nchoosek bug for integer inputs

[Arun Giridhar]

URL:
  <https://savannah.gnu.org/bugs/?62319>

                 Summary: Another nchoosek bug for integer inputs
                 Project: GNU Octave
            Submitted by: arungiridhar
            Submitted on: Fri 15 Apr 2022 11:20:21 AM EDT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Regression
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
                 Release: 7.1.0
         Discussion Lock: Any
        Operating System: Any

    _______________________________________________________

Details:

This is to get a bug ID for an infinite loop with nchoosek for certain integer
inputs.

Test: nchoosek (int8 (32), 5)

Numerically correct answer: 201376

Acceptable answer: saturate at intmax == 255 and give a warning so the user
can switch to a different data type

Actual behavior: it freezes and needs control-C.

The problem is to do with the way the gcd calculation is set up, which causes
an infinite loop for even modestly sized inputs when the integer bitwidth is
small.

A fix will be attached with this bug ID.




    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?62319>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/