bug#55212: GNU Linux "sort -g" can hang indefinitely when run on standard input if NaNs are involved

[Pádraig Brady]

On 02/05/2022 07:03, Paul Eggert wrote:
> Thanks for the bug report. This bug is entertaining, as it comes from
> GCC now being so smart that it optimizes away a memset that cleared
> padding bits. We added the memset in coreutils 8.14 (2011) to try to fix
> the sort -g infinite loop bug (introduced in 1999), but the memset isn't
> guaranteed to fix the bug because the memset can be optimized away.
>
> If the padding bits happen to be clear already sort is OK, but if not
> the results can be inconsistent when you compare two NaNs to each other,
> and inconsistent results can make sort infloop.
>
> The C standard allows this level of intelligence in the compiler, so
> it's a bug in GNU 'sort'.
>
> I installed the attached patch; please give it a try. For now I'll
> boldly close the bug report; we can easily reopen it if this patch
> doesn't actually fix the problem.

This is a bit slower of course, but since an edge case not a big concern:

  $ time yes nan | head -n128095 | timeout 10 sort -g >/dev/null
  real  0m0.693s

  $ time yes nan | head -n128095 | timeout 10 src/sort -g >/dev/null
  real  0m0.924s

I'll add the test case I think (attached)
since the existing one didn't trigger this.

thanks!
Pádraig

sort-nan-inf-adjustments.patch
Description: Text Data