Re: grep-3.10.12-0f2c test results

[Bruno Haible]

Paul Eggert wrote:
> >    I don't understand this test result: On Manjaro 17 and AIX 7.2
> >    it reports PASS. What is going on here? How can the 'grep' program work 
> > with
> >    a file that has a timestamp > 2038, when the time_t type is only 32-bit??
> >    For analysis, I attach the two log files of this test.
> 
> In 32-bit AIX 7.2, fstat delivers the low-order 32 bits of the file's 
> timestamp, instead of failing with errno == EOVERFLOW as it should. 
> Since 'grep' doesn't use the timestamp it goes ahead and does the right 
> thing.

The same thing happens on Manjaro 17, indeed:

grep does three relevant system calls:
  - openat,
  - fstat
  - read.

'strace -v -k' on a 64-bit Linux system gives the stack traces:

openat(AT_FDCWD, "in", O_RDONLY|O_NOCTTY) = 3
 > /usr/lib/x86_64-linux-gnu/libc.so.6(openat64+0x46) [0x114866]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(openat_safer+0x11e)
 >  [0x24700]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grepfile+0x67)
 >  [0x981b]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grep_command_line_arg+0x89)
 >  [0x9fd8]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(main+0x19de)
 >  [0xc896]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_init_first+0x90) [0x29d90]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x29e40]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(_start+0x25)
 >  [0x5075]
newfstatat(3, "", {st_dev=makedev(0x8, 0x1), st_ino=34575669, 
st_mode=S_IFREG|0664, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, 
st_blocks=8, st_size=5, st_atime=1682125409 /* 
2023-04-22T03:03:29.273120776+0200 */, st_atime_nsec=273120776, 
st_mtime=2177449200 /* 2039-01-01T00:00:00+0100 */, st_mtime_nsec=0, 
st_ctime=1682121724 /* 2023-04-22T02:02:04.752971489+0200 */, 
st_ctime_nsec=752971489}, AT_EMPTY_PATH) = 0
 > /usr/lib/x86_64-linux-gnu/libc.so.6(fstatat+0xe) [0x113eee]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grepdesc+0x52)
 >  [0x9aa0]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grepfile+0xb0)
 >  [0x9864]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grep_command_line_arg+0x89)
 >  [0x9fd8]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(main+0x19de)
 >  [0xc896]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_init_first+0x90) [0x29d90]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x29e40]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(_start+0x25)
 >  [0x5075]
read(3, "text\n", 98304)                = 5
 > /usr/lib/x86_64-linux-gnu/libc.so.6(read+0x12) [0x114992]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(safe_read+0x2c)
 >  [0x26775]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(fillbuf+0x329)
 >  [0x7cf4]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grep+0x107) 
 > [0x8e23]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grepdesc+0x3bb)
 >  [0x9e09]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grepfile+0xb0)
 >  [0x9864]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(grep_command_line_arg+0x89)
 >  [0x9fd8]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(main+0x19de)
 >  [0xc896]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_init_first+0x90) [0x29d90]
 > /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x29e40]
 > /media/develdata/devel/build/grep-3.10.12-0f2c/build-64/src/grep(_start+0x25)
 >  [0x5075]

* On Manjaro 17 (Linux 4.14.65, glibc 2.28), the details of these system calls
look like this:

openat(AT_FDCWD, "in", O_RDONLY|O_NOCTTY|O_LARGEFILE) = 3
fstat64(3, {st_dev=makedev(0x8, 0x1), st_ino=617913, st_mode=S_IFREG|0644, 
st_nlink=1, st_uid=1000, st_gid=1001, st_blksize=4096, st_blocks=8, st_size=5, 
st_atime=-2117518096 /* 1902-11-25T17:31:44+0100 */, st_atime_nsec=0, 
st_mtime=-2117518096 /* 1902-11-25T17:31:44+0100 */, st_mtime_nsec=0, 
st_ctime=1682123115 /* 2023-04-22T02:25:15.313519201+0200 */, 
st_ctime_nsec=313519201}) = 0
read(3, "text\n", 98304)                = 5

As you can see, the st_mtime value is negative, but fstat() does not fail
due to this, and 'grep' doesn't care.

* On Ubuntu 22.04 (Linux 5.15.0, glibc 2.35), in a 32-bit build with
--disable-year2038, things are different. grep fails
  grep: in: Value too large for defined data type
and the system calls look like this:

openat(AT_FDCWD, "in", O_RDONLY|O_NOCTTY|O_LARGEFILE) = 3
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH, 
STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_blksize=4096, 
stx_attributes=0, stx_nlink=1, stx_uid=1000, stx_gid=1000, 
stx_mode=S_IFREG|0664, stx_ino=34575669, stx_size=5, stx_blocks=8, 
stx_attributes_mask=STATX_ATTR_COMPRESSED|STATX_ATTR_IMMUTABLE|STATX_ATTR_APPEND|STATX_ATTR_NODUMP|STATX_ATTR_ENCRYPTED|STATX_ATTR_AUTOMOUNT|STATX_ATTR_MOUNT_ROOT|STATX_ATTR_VERITY|STATX_ATTR_DAX,
 stx_atime={tv_sec=1682125091, tv_nsec=319165606} /* 
2023-04-22T02:58:11.319165606+0200 */, stx_btime={tv_sec=0, tv_nsec=0}, 
stx_ctime={tv_sec=1682121724, tv_nsec=752971489} /* 
2023-04-22T02:02:04.752971489+0200 */, stx_mtime={tv_sec=2177449200, tv_nsec=0} 
/* 2039-01-01T00:00:00+0100 */, stx_rdev_major=0, stx_rdev_minor=0, 
stx_dev_major=8, stx_dev_minor=1}) = 0

Here you can see:
  - It uses a different system call, statx, instead of fstat64.
  - The st_mtime value is positive and > INT_MAX.

The change in behaviour comes from glibc, files
  glibc/sysdeps/unix/sysv/linux/fstatat64.c (for the different system call),
  glibc/sysdeps/unix/sysv/linux/fstatat.c (for the st_mtime check and EOVERFLOW
    return).
These changes went into glibc 2.33.

So, on x86 builds, grep fails if and only if glibc ≥ 2.33 is used, even though
it does not use st_mtime.

* On the same Ubuntu 22.04 (Linux 5.15.0, glibc 2.35), in a 32-bit build with
--enable-year2038, the system calls are the same, but the glibc fstat() does
not give an EOVERFLOW since it can pass back the 2039 timestamp to the
application.

Bruno