Re: posix command search and execution

[Mike Jonkmans]

Thanks for the answers, Chet.

On Mon, Nov 06, 2023 at 02:28:24PM -0500, Chet Ramey wrote:
> On 11/6/23 10:48 AM, Mike Jonkmans wrote:
> 
> >   Q: Why check for regular builtins? That was already done in 1d.
> Implementations can provide other builtins. The check in 1d is only for
> those specific ones.

It was my earlier understanding that POSIX partitions the builtins into:
- special builtins
- regular builtins (listed in 1d).
- ordinary builtins (i.c. not the two others)
Because the first two are specific lists.
XBD definition of builtins:
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_83
(Note that this definition says that 'regular builtins are defined
in detail in XCU Command Search and Excecution'
which makes one think that the regulars are the list of 20 in 1d).
And the table of regular builtins:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tagtcjh_18
(which is mostly the same as the list in 1d - except for the XSI stuff).

Upon rereading those, I think it is more like:
- special builtins
- regular builtins listed in 1d
- regular builtins


> > 1eIb) Run the utility.
> >       (This is where ordinary builtins should run).
> >       (It seems logical that a builtin takes precedence over PATH).
> You'd be surprised. Note that this seems to require that you can only run
> a builtin if it exists (or something with that name exists) in $PATH.

Yes. that is weird.

> Look at https://www.austingroupbugs.net/view.php?id=854 for a discussion
> of this issue.

Thanks for the link, I find that very hard to read though.

> > 1eII) Nothing in PATH, exit with 127
> So if you have a builtin that doesn't exist in $PATH and isn't listed as
> one of the regular builtins, what do you do? Even the unspecified list
> doesn't give much help.

Interpretation depends on the definition of regular builtins.
Which IMHO is not well specified.

> > I hope I have understood POSIX correctly on these points.
False hope ;)

> > In posix mode, it seems that bash:
... 
> >        A function defined before `set -o posix' will mask a
> >       special builtin. (This seems to be ok).
> It will not, at least not while posix mode is enabled. If you mean that a
> function will be executed before a special builtin when not in posix mode,
> you are correct, because there are no special builtins when not in posix
> mode.

I was mistaken. Guess I toggled the posix option too much.

...
> > Thus in posix mode, bash does not follow this part of the standard.
> Exactly which part of the standard are you saying bash is not following?

That would be PATH search failing and executing a builtin.
Though i think that standard utilities *must* be in PATH,
otherwise there is no conformance a priori.

> The requirements concerning PATH search and builtins are different in the
> next version of POSIX, the result of interp 854. The standard already
> says this about functions with the same name as a special builtin:
> 
> "The function is named fname; the application shall ensure that it is a name
> (see XBD Name) and that it is not the name of a special built-in utility."

Nice.

> > But should it?
> > I would rather have POSIX modified to *also* accept the, more logical,
> > bash way (i.c. first matching functions, then builtins, then PATH).
> > Would that be a feasible modification to suggest to the Austingroup?
> I think the resolution to interpretation 854 addresses this. Shells
> who want this odering just declare all the builtins they implement as
> `intrinsic' so they're not subject to a PATH search. That way there's no
> difference between the regular builtins and the ones an implementation
> chooses to provide. It still leaves posix special builtins, but I think
> those are with us forever.

They should be named 'forever builtins', like 'forever chemicals'.
It's a real shame. There certainly are use cases for overriding the
special builtins (e.g. logging around `.' a.k.a. source).

> > - Regarding 1eIb.
> >    The shells posh, dash, ksh and zsh
> >    also run builtins, even when not found in PATH.
> >    Checked with the `test' builtin (mv /usr/bin/test{,.sav})
> >    on the versions found on Ubuntu 22.04.
> Yes, this is part of the discussion of interp 854. The business of running
> builtins other than the ones listed only after a PATH search was always
> ahistorical.

Hmm, my check maybe incorrect.
Removing the standard utility `test' is not conformant.
Then again, is there a requirement for the standard utilities to be
found in the current PATH? Or do they just need to be present somewhere.

> > - The 'newgrp' utility (mentioned in 1d) is not a builtin in bash.
> >    This is ok. The regular builtins from 1d need not be provided. See:
> >    https://lists.gnu.org/archive/html/bug-bash/2005-02/msg00129.html
> >    Builtins are defined in:
> >    
> > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_83
> >    Q: Isn't that incorrect in stating where regular builtins are defined?
> 
> Kind of; regular builtins aren't really defined anywhere. They're builtins
> that aren't special builtins, and they're mostly defined in terms of how
> they differ from special builtins.

Regualar builtins are defined but pourly (two links above).

...
> https://www.austingroupbugs.net/view.php?id=705 is where the discussion
> happened.

That is an easy read.

> > - Utilities:
> >    https://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html
> >    Q: Where is `standard utilities' defined - as used in 1d.
> These are the standard utilities.

Some of these utilities are marked with optional `codes'.
Are these also considered standard utilities - even when the option is 
not true?
If not explicitely somewhere defined in POSIX, I think it should be.

--
Regards, Mike Jonkmans