Re: !(.pattern) can match . and .. if dotglob is enabled

[Nora Platiel]

Thank you for your effort in understanding my problem.

> The actual change, captured in the `devel' branch that tracks bash
> development, happened sometime in 2011.

I see.

> How would you improve the wording? What do you think is most important to
> cover?

Here is the full paragraph for reference:
> When a pattern is used for filename expansion, the character `.' at the
> start of a filename or immediately following a slash must be matched
> explicitly, unless the shell option dotglob is set. The filenames `.'
> and `..' must always be matched explicitly, even if dotglob is set. In
> other cases, the `.' character is not treated specially.

First:
> The filenames `.' and `..' must always be matched explicitly, even if
> dotglob is set.

I agree with gregrwm here.
(https://lists.gnu.org/archive/html/bug-bash/2021-01/msg00251.html)
The sentence seems to imply that you need 2 literal dots to match `..'.

I read your answer and I understand: if that was the case, then the only 
(non-extended) pattern capable of matching `..' would be `..*'.
But my understanding of the expression "matched explicitly" is: matched in its 
entirety via characters that stand for themselves (i.e. not via special pattern 
characters).

It would be much clearer if we prepend this part:
"[The character `.' at the start of] the filenames `.' and `..' must always be 
matched explicitly, even if dotglob is set."

Next, there's nothing in the docs about dot treatment in the specific context 
of extended globbing. The only behavior that I would consider self-explanatory 
enough not to require a description is the following:

I expect @(P1|P2) to expand to the union of the matches of the separate 
subpatterns P1 and P2.
(I.e. if I take the expansion of P1 P2, sort it, remove duplicates and 
non-existing files, I expect to get the same result as @(P1|P2). )

And:
x@(P1|P2)y ->    xP1y xP2y
x?(P1|P2)y -> xy xP1y xP2y
x*(P1|P2)y -> xy xP1y xP2y xP1P1y xP1P2y xP2P1y xP2P2y ...
x+(P1|P2)y ->    xP1y xP2y xP1P1y xP1P2y xP2P1y xP2P2y ...
x!(P1|P2)y -> x*y excluding results where @(P1|P2) matches the part matched by 
the `*'

Example of expected results:
$ touch .foo bar
$ shopt -s dotglob
$ echo @(.foo|*)
.foo bar
$ echo !(.foo)
bar

I think this would be the least surprising and least error prone behavior. I'm 
wondering if the actual behavior is different by design, or if it was just 
easier to implement that way, or if the consequences were not considered.

> > So why it does here?
> > $ shopt -s dotglob; echo @(.foo|*)
> > . .. .a b
> > $ shopt -u dotglob; echo @(.foo|*)
> > b
>
> It's not intuitive. The dotglob causes all files starting with `.' to be
> in the list, the .foo pattern keeps `.' and `..' from being discarded,
> and the `*' matches it (since dotglob disables the requirement that an
> initial `.' be matched explicitly).

Ok, if things happen in the way and order you described here, then I can 
understand it.
But yes, it's not intuitive. It seems totally arbitrary to me that alternative 
subpatterns in a pattern-list influence each other's behavior concerning `.' 
and `..', but not concerning dot-files in general.

Not being familiar with the actual implementation, I've never considered the 
exclusion of dot-files and the exclusion of `.' and `..' as two different 
mechanisms. To me it was just that dotglob-unset has a stricter "filter" than 
dotglob-set has (filter activated by the absence of a literal dot at start of 
pattern).
I.e. `.' and `..' are to dotglob-set, as all dot-files (including `.' and `..') 
are to dotglob-unset.

I would still prefer the behavior I was expecting from the start. I'm having a 
hard time finding good words to document the current behavior, which is 
probably an indication that it is too complex (or that my English sucks :D).

Here is an attempt:
"If dotglob is set, and an extended pattern contains at least one subpattern 
beginning with a literal dot, then in the context of such extended pattern, 
there is no requirement for the first character of the filenames `.' or `..' to 
be matched explicitly."

Another minor observation:
it is not documented that the dot in the pattern must be also at the beginning 
to be able to "match explicitly".
$ shopt -u dotglob; echo *.foo  # doesn't match `.foo'
$ shopt -s dotglob; echo *..    # doesn't match `..'
Even though you could say that the dot is "matched explicitly" in both cases.

I find it also interesting that:
$ shopt -u dotglob; echo !(bar).foo  # doesn't match `.foo'
$ shopt -u dotglob; echo ?(bar).foo  # matches `.foo'
$ shopt -u dotglob; echo *(bar).foo  # matches `.foo'

Ok, this is not so likely to confuse people, so I'm not sure it's worth 
documenting if it makes the explanation too cumbersome.
Here is my (bad) attempt:
"If a path component in the pattern doesn't start with a literal dot, than such 
path component can never expand to a name beginning with dot, unless the shell 
option dotglob is set. If a path component in the pattern doesn't start with a 
literal dot, than such path component can never expand to `.' or `..', even if 
dotglob is set."

Regards,
NP