Re: non-executable files in $PATH cause errors

[Robert Elz]

    Date:        Wed, 13 Jan 2021 21:45:08 -0500
    From:        worley@alum.mit.edu (Dale R. Worley)
    Message-ID:  <87im806xu3.fsf@hobgoblin.ariadne.com>

  | Of course, as described in the manual page, Bash first searches for an
  | executable with the right name in the PATH, and then if that fails, it
  | searches for a non-executable file in the PATH.

Where exactly does the bash man page say that?   What I see is:

       If the name is neither a shell function nor a builtin, and contains no
       slashes, bash searches each element of the PATH for a directory
       containing an executable file by that name.  Bash uses a hash table to
       remember the full pathnames of executable files (see hash under SHELL
       BUILTIN COMMANDS below).  A full search of the directories in PATH is
       performed only if the command is not found in the hash table.  If the
       search is unsuccessful, the shell searches for a defined shell function
       named command_not_found_handle.  If that function exists, it is invoked
       [... otherwise] the shell prints an error message and returns an exit
       status of 127.

Nothing about looking for non-executable files at all that I can see.

I suspect what you're being confused by, is that the "searches for" is
typically done (in shells, I have no idea what the code inside bash is
like) by simply taking each element of PATH, appending "/command_name"
and attempting an exec.   If that succeeds, the command is running, and
everyone is happy (exec*(2) only work for executable files - ones with
the appropriate 'x' bit (or any if the caller is root) set).

If the exec fails, then the shell does some saving of the error number
returned - more or less ignoring "file not found" type errors, but saving
the error number from the first attempt that failed for some different
reason.

Once all directories in PATH have been attempted, and failed, if an error
number was saved, the shell issues that one (that's where the "not executable"
message comes from if you only have a non-executable version of "command"
in PATH) - or if no message was saved, then all the failures were "not found"
and the shell just says "command not found" in whatever format it prefers.

There's no second search of PATH (I assume), that would be dumb and wasteful.

  | My belief is that the reason is compatibility with historical usage.

No, that's not it, as it doesn't happen as you believe, and never did,
there is nothing to be compatible with.

  | I have dim memories that there were days before shell scripts had the
  | executable bit set

There was never such a time, to be interpreted as a script, the file
has always needed 'x' permission.

  | and the first line started with "#!".

But that time certainly existed.

  | Instead, they weren't marked as executable but the first line started
  | with ": "

That one (that marker) I have some dim recollection of as well, but I have
no idea who or what introduced that.

The historical behaviour was that any executable file (ie: the appropriate
'x' bit set, but not a directory) which failed exec() (there's actually no
libc function called exec(), I mean whichever of the exec*() family the
shell chooses to use, often execve() but not necessarily), then the shell
would treat it as a script, and attempt to interpret its contents that way.

That still persists in many shells, though modern shells mostly check for
various kinds of binary files, and ignore those, attempting to run only
text files.   I believe a few shells have abandoned this (since these days
it is more often a mistake that intentional, people use #! for sh scripts)
and if exec() fails, simply give up.

But when supported, now and historically, such a script requires/required
'x' permission.

  | (either mandatory or by convention).

There may have been some convention to do that (the : line), if that's
all it was, it was just wasting time...

  | And the scripting facility was
  | implemented entirely in the shell -- if the shell's call to the kernel
  | to execute the script failed, it would decide that the file was a
  | script, then spawn and initialize a subshell to execute it.  Of course,
  | there was no check that your file actually was a script, so if you had
  | named a data file, the subshell would spew a stream of errors.

All that is correct.

  | But the consequence to this day is that scripts without the executable
  | bit can be executed if they are given as command names to bash, and that
  | executable scripts take precedence over similarly-named non-executable
  | scripts earlier in PATH.

But none of that is.   That is, unless bash is even weirder than I believe,
but in this case, I think not.

kre

ps: searches in PATH for files which can be non-executable occur to implement
the '.' command (when given a file name with no '/' chars), but that's an
entirely different thing.