Re: isatty() and errno

[Eric Blake]

On 11/07/2011 04:18 PM, Eric Blake wrote:
> Our assumption in ptsname.c is that isatty() will set errno on invalid
> fds. But this is not required by POSIX, and indeed not enforced by
> Solaris 10; that is, our ptsname.c implementation was copied from glibc,
> and was making an assumption only guaranteed by glibc's implementation
> choice for isatty(), which ends up leaving errno unchanged when that
> code is compiled on Solaris.
>
> But that means it is likely that other GNU code will assume isatty()
> sets errno. Should we provide an isatty() module that guarantees that
> errno will be set when returning 0? Or should I just patch ptsname.c to
> guarantee that errno is set without relying on isatty() to do the job?

Yuck.  Making ptsname_r wrap ptsname() makes it possible to avoid the 
isatty() non-errno on Solaris, but exposes a FreeBSD 8.2 bug that 
ptsname(-1) doesn't set errno.

Also, glibc ptsname_r(-1, buf, len) sets errno to ENOTTY, even though 
EBADF seems like it would be the more canonical choice (and given the 
comments in ptsname.c, that seems like EBADF should be the behavior, 
since isatty() is supposed to distinguish between open but 
non-terminals, and altogether bad fds).

--
Eric Blake   address@hidden    +1-801-349-2682
Libvirt virtualization library http://libvirt.org