Re: man bash does not list 'in' as a builtin command

[Peter Benjamin]

Clint,

My thoughts on man page changes are near identical to yours.  And I
thought wiser minds might first try their hand, as I do not know the
conventions the authors have used.  I like how it's done now.  And
something might be done a little better, I agree.  So, here is my
thought.  Pardon the long email, a train of thought, as I type, and is
typical to seek a technical solution, only to find a management
solution is most cost effective.  My logic is exposed for your
collective review, discussion, and decision on implementation degree.

Any RESERVED WORDS when used as the first word on the CLI, should be in
the list of BUILTIN COMMANDS, as so:

  in - See RESERVED WORDS.  Do not use as a custom script name.

Only those RESERVED WORDS when tried at the CLI, alone, that generate a
'confusing' error message, would qualify for going into BUILTIN
COMMANDS list.  Adding all reserved words would be wrong, especially if
an error message is not generated.

Another way of handling this, without modifying the man page, is modify
the code, which strikes me as a worse solution, but informs the end
user so much faster, immediately, of what the problem is.

Just change the error message, to include the words "See 'man bash'
RESERVED WORDS".

And perhaps in the RESERVED WORDS section, include a sentence like:

   These words should not be used as a custom script name as bash will
output an error message that is not easily deciphered.

Which I have seen such error messages before, sort of humorous.  But in
my code I have always put a very exact error message.  Maybe:

   Bash found the custom script name 'in' in the PATH, but this word is
a RESERVED WORD, not to be used as a script name.

Of course, that means during the syntax error processing, adding an if
statement, checking for the reserved word being the first word on the
command line, and if the word is found in the PATH, then use this error
message.  (See better solution below.)

That is what I would do, combined with the generic change to the man
page, that does not specify the exact error message, as that would list
too many error messages, eventually.

Some combination is good.  I like the idea of your's, what would have
clued me in the fastest.  A custom error message, telling me to read
the bash man page, and which section, where I would find a brief
description, would have done the job nicely.  And could be used for any
such error condition involving RESERVED WORDS when used "out of
context", beyond the bash programmer's expectations.  

And that last phrase, beyond expectations, is key to resolving future
issues, with a consistent policy, as determined by the maintainers. 
However, unknown surprises like this are not predictable, by
definition.  Thus, my confusion, and your excellent question of what I
would do.

1) New error trapping involving checking the PATH for a matching
script.

2) Two edits in the man page:

 a) Into the RESERVED WORD section add this paragraph:

RESERVED WORDS used as a command, first word on command line, may
produce an error message, that may or may not aid you.  If a PATH
script matches a RESERVED WORD, then the RESERVED WORD is invoked.

 b) BUILTIN COMMANDS additions ONLY for those RESERVED WORDS that do a)
above.

I am trying to define a policy, that can be easily followed in the
future.  Please, as maintainers of bash, modify or rejected my ideas,
as you see best, as you are the experts here.  Who I do admire.

Hmm, I reread again the RESERVED WORD list, to see if 'in' is the only
one that can not be 'first.'

Here is the list I read:

! case  coproc  do done elif else esac fi for function if in select
then until while { } time [[ ]]

It appears that all these words, but 'in', occur in the first position.
If there is agree between the experts, whose accumulative expertise in
scripting is greater than mine, for sure, then it occurs to me, the
only addition to the BUILTIN COMMANDS would be one new command, for the
word, 'in'.

And that would have solved my time waste issue, as I did first check
for BUILTIN COMMANDS.  

It did not occur to me, that bash would be programmed to generate an
error message, when 'in' was first, as it can never, ever be first.  My
thought the bash command line parser would be coded such to only
generate syntax errors based upon the first word's required syntax.  

Leading to the thought, if the above change is made, then the word "in"
needs to be removed from the RESERVED WORD list, and allowed in PATH as
a script name.  This thought is only that, not my recommendation.

And so, to conclude this overly long email, I conclude this issue has
been before the bash experts for a long time.  And if I went looking,
in the bash forums, I would find a set of posts, dealing with the issue
of this special word "in", and it's unique role in bash.  And policy
consistency would be a focus of those forum threads.

Given, "in" is a unique case, there likely will never be a "consistent
policy" for it, until there is an addition to bash, of a second such
RESERVED WORD, that can never be used as the first word.  Only then,
will any progress be made in determining, in discussing, in setting a
consistent policy.

So, I forgo thinking that 'now' is good time to discuss policy.  It
would waste everyone's time, as compared to making this a single
management decision, to make a one time edit to the man page.

I see the issue as one of: Is there a volunteer who can step up to make
code changes and man page edits, for this issue, that the other
involved configuration control staff can 'quickly' agree to, so as to
not waste the volunteer's time and effort?

That said, any improvement in the man page, would be faster than coding
and testing.  Managing this issue as solely a documentation change is
easy at this time.  Very easy, compared to doing a change to the syntax
parsing error processing algorithm, for this one "unique" RESERVED
WORD.  A man page edit is my recommendation.  No code changes.

And when a second such RESERVED WORD is invented, only then would code
changes be justified, the volunteer time be worth it, in addition to
the man page edit to support the behavior coded into bash for this
second word.  And even then, perhaps the third reserved word is the
decision point, not the second, as again, for the second word, easy,
lowest cost, by far, is to the edit the man page to cover the
condition.  

Only the expert coder of the 2nd word would truly know the cost benefit
of changing the syntax error message, to reflect the true condition.  

I have not looked at the code, so ... I defer to the experts, and the
druthers of the coder of the 2nd reserved word syntax parsing.  And to
those who talk to him, and ask him, what is the effort level, to
support a truly useful error message, for the 2nd word, and if they
would code the same for the word "in".

That's one man's opinion.  He is not changing it.

Peter

P.S.  Volunteer time is the most precious commodity on this planet.  It
created "Ecology."  Which then cleaned up the air, water and food, and
greatly increases the quality of living for every individual on this
planet.  Volunteer time generates over 100 trillion dollars in economic
benefit via the concept of "open source" - code that rejects the past
failures, since 1957, on how not to implement an algorithm, and used
only the best algorithm known at the time.  Open source will enable us
to travel to the rest of Solar System, and see the Milky Way.  That
Pale Blue Dot is the single nest of humanity.  We must get off this
planet, for long term survival.  Only Open Source can do this, as it's
less expensive.  You heard it here first.



On Mon, 2019-11-25 at 20:15 -0500, Clint Hepner wrote:
> > On 2019 Nov 25 , at 4:43 p, Peter Benjamin <pete@peterbenjamin.com>
> > wrote:
> > 
> > 
> > Description:
> > 'in' is a builtin command and is not listed in the man page as
> > such.
> > 
> > Repeat-By:
> > 
> > type at the bash command line:
> > 
> > $ in
> > bash: syntax error near unexpected token `in'
> 
> This should be a big hint that it is *not* a built-in command. If it
> were, you would have run the command, not gotten a syntax error.
> 
> > 
> > Why is this bug report important?  Why change the man page?  I
> > wasted
> > 20 minutes of my time, to prove to my satisfaction that 'in' was
> > not
> > invoking my script at all.  Search engines did not find a match to
> > the
> > error message.  I can not imagine this report is the first time
> > this
> > bug was found.
> 
> Given that ``in'' *is* documented as a reserved word
> 
>     RESERVED WORDS
>            Reserved words are words that have a special meaning to
> the shell.  The
>            following words are recognized as reserved when unquoted
> and either the
>            first word of a simple command (see SHELL GRAMMAR below)
> or  the  third
>            word of a case or for command:
> 
>            !  case   coproc   do  done elif else esac fi for function
> if in select
>            then until while { } time [[ ]]
> 
> the question becomes, what change to the man page do you think would
> have helped you 
> find this more quickly?
> 
> One thing that comes to mind would be a short (though off-topic)
> entry in
> the SHELL BUILTIN COMMANDS section, like
> 
>     in
>         See the Compound Commands section under SHELL GRAMMMAR. Used
>         to form the ``case'', ``select'', and ``for'' commands.
> 
> I don't really like that idea, as it starts down a slippery slope for
> adding
> all sorts of non-command entries, turning the section into a general
> glossary rather than
> a listing of built-in commands.
> 
> Alternatively, maybe a short list of sections at the top of the man
> page would provide
> a hint.
> 
> For example, imagine the man page started out like this:
> 
>     NAME
>             bash - GNU Bourne-Again Shell
> 
>     SYNOPSIS
>             bash [options] [command_string | file]
> 
>     DESCRIPTION
> 
>             Bash is [...]
> 
>     TABLE OF CONTENTS
> 
>         OPTIONS
>         ARGUMENTS
>         INVOCATION
>         DEFINITIONS
>         RESERVED WORDS
>         SHELL GRAMMAR
>         COMMENTS
>         QUOTING
>         PARAMETERS
>         EXPANSION
>         REDIRECTION
>         ALIASES
>         FUNCTIONS
>         ARITHMETIC EVALUATION
>         CONDITIONAL EXPRESSIONS
>         SIMPLE COMMAND EXPANSION
>         COMMAND EXECUTION
>         COMMAND EXECUTION ENVIRONMENT
>         ENVIRONMENT
>         EXIT STATUS
>         SIGNALS
>         JOB CONTROL
>         PROMPTING
>         READLINE
>         HISTORY
>         HISTORY EXPANSION
>         SHELL BUILTIN COMMANDS
>         RESTRICTED SHELL
>         SEE ALSO
>         FILES
>         AUTHORS
>         BUG REPORTS
>         BUGS
> 
>     OPTIONS
>            All of the single-character shell options ...
> 
>     [etc]
> 
> The table of contents inserted between the existing DESCRIPTION and
> OPTIONS sections is relatively brief,
> and both RESERVED WORDS and SHELL GRAMMAR seem like they would be
> good starting places to look for
> information about the ``in'' token mentioned in the syntax error.
> 
> Given how short most of the section names are, it could easily be
> formatted as two columns, making it
> easier to scan.
> 
> 
> 
> --
> Clint