Re: Design question(s), re: why use of tmp-files or named-pipes(/dev/fd/N) instead of plain pipes?

[Bob Proulx]

Linda Walsh wrote:
> Bob Proulx wrote:
> >Where does the OS claim it is a file descriptor?
> 1) in the kernel documentation:
> file 'devices.txt:
> /dev/fd               /proc/self/fd   symbolic        File descriptors

That shows the file descriptor map for the process.  It isn't the
program's actual internal file descriptor.  Obviously it can't be.
They have abbreviated it trying to make it be concise and fit there.
If you disagree with the description then take that up with the
devices.txt authors.

> 2) ls -Ll shows the type of what the symlink points to:
> >ls -Ll /dev/fd/
> total 0
> crw--w---- 1 law tty      136, 0 Oct 21 19:49 0
> crw--w---- 1 law tty      136, 0 Oct 21 19:49 1
> crw--w---- 1 law tty      136, 0 Oct 21 19:49 2

The -L hides the symlink.  Try it without the -L.

  $ ls -log /dev/fd/
  total 0
  lrwx------ 1 64 Oct 23 15:45 0 -> /dev/pts/13
  lrwx------ 1 64 Oct 23 15:45 1 -> /dev/pts/13
  lrwx------ 1 64 Oct 23 15:45 2 -> /dev/pts/13
  lr-x------ 1 64 Oct 23 15:45 3 -> /proc/31883/fd/

It is showing where the program's file descriptors are redirected to
and from.  But those are still file path names.  Paths.  Names.  This
is an email from me to you.  I am not defining this exactly for a
standard.  It is a string of characters.  It isn't the integer value
index offset into the data structure holding the file data inside the
program.  It's not.  Don't confuse the two.

> and 3)
> 
> YOU claim it is a descriptor further down in this note:
> >  tmpname=<(sort /dev/null); ls -l "$tmpname"
> >    ls: cannot access /dev/fd/63: No such file or directory
> >
> >But as soon as the first commend is finished at the ';' then the
> >descriptor will be closed and the path will evaporate.  Therefore the
>  ^^^^^^^^^^^^^^^^^^^^^^^^^
> >path won't exist for the second command.

At that point I am talking about the program's internal behavior.  I
also mispelled command too.  I am talking conversationally here.

   if (close(fd) < 0) { ...; exit(1); }
   exit(0);

Don't confuse the integer value of the program's file descriptor,
which is an offset into the program's data structure array, with a
path to a file.  In this case they are confusingly similar because the
/proc one is actively trying to represent the other!  But life is like
an analogy and being similar does not make it the actual thing.
Actively trying to represent the other here means one is really a
mapping show how one is mapped to the other.  That doesn't make it
actually the other.

>       If you close a file-descriptor to a normal file
> it doesn't disappear.  If it was deleted via 'rm' the name disappears
> immediately, and the only access to it is through the descriptor.
>
> In this case, closing the descriptor deletes the temporary
> name in the process's 'file descriptor' "/fd/".  Normal filenames
> are not "auto-deleted" when you close an I/O handle to them.

Confusion such as yours is precisely why I cringe whenever I see
someone using /dev/stdin, /dev/stdout, /dev/stderr or any of the
/dev/fd/* symlinks in a program instead of using the normal
redirections.  It causes people to think they are something that they
absolutely are not.  That and the unportant nature of them.

The /proc/self/fd/ directory is a kernel construct.  It does not have
normal file system semantics.  I am sorry to be the bearer of bad news
to you.  But it doesn't.  If you don't like this then that would be a
discussion with the kernel folks responsible for it.

And really what would be a sane purpose in being able to remove files
from it?  What would that do?  Would that make any sense at all?  I
think it wouldn't make any sense.

> >But if you use it for the ls command itself then it exists for that
> >command.
> >
> >  ls -l <(sort /dev/null)
> >    lr-x------ 1 rwp rwp 64 Oct 19 15:56 /dev/fd/63 -> pipe:[102059434]
> ----
>       But only as a pointer to something one can do I/O on.
> You can't set any file attributes or metadata on "pipe:[xxxx]" It's not a
> real file somewhere.

Exactly!  Stick with that thought.  You are on the right track with
it.  It is only a symlink in a /proc kernel space representation.  You
can do I/O on it but you can't actually change permissions or remove
it or other things.

Bob