Re: Revisiting Error handling (errexit)

[Yair Lenga]

Thanks for sharing your thoughts. I admit that my goals are
significantly less ambitious compared with what you described (lexical
scope, etc.). I do not think that it's possible to stretch my proposal to
meet all the use cases you describe. For me, the 'errfail' is similar to
'pipefail' option - practical solution for real problems. The suggested
'errfail' in opt-in - anyone that want the old way (errexit) can use it,
without saying anything. As you said, errfail was not 'good' solution when
conceived, no point in trying to match it (IHMO).

Yair

On Tue, Jul 12, 2022 at 6:08 PM Martin D Kealey <martin@kurahaupo.gen.nz>
wrote:

>
>
> On Sun, 10 Jul 2022 at 05:39, Yair Lenga <yair.lenga@gmail.com> wrote:
>
>> Re: command prefaced by ! which is important:
>> * The '!' operator 'normal' behavior is to reverse the exit status of a
>> command ('if ! check-something ; then ...').
>>
>
> Unless that status is ignored, in which case, well, it's still ignored.
>
>
>> * I do not think it's a good idea to change the meaning of '!' when
>> running with 'error checking'.
>> * I think that the existing structures ('|| true', or '|| :') to force
>> success status are good enough and well understood by beginner and advanced
>> developers.
>>
>
> I'm not suggesting a change; rather I'm suggesting that your new errfail
> should honour the existing rule for "!" (as per POSIX.1-2008 [
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html],
> under the description of the "set" built-in):
>
> 2. The *-e* setting shall be ignored when executing the compound list
>> following the *while*, *until*, *if*, or *elif* reserved word, *a
>> pipeline beginning with the ! reserved word*, or any command of an
>> AND-OR list other than the last.
>>
>
> So the exit status of a command starting with "!" (being the inverse of
> the command it prefaces) is *not* considered by errexit, regardless of
> whether it in turn is "tested".
>
> It follows that
>
>>  ! (( expression ))
>>
> and
>
>>  (( expression )) || true
>>
> are equivalent under errexit; the former is the preferred idiom in some
> places precisely because it is expressly called out in that clause of the
> standard.
>
> If you propose to make the former unsafe under errfail, then I suggest
> that the onus is on you to explain why you would break code where the
> author has clearly indicated that its exit status should not be taken as
> indicating success or failure.
>
> Question: What is the penalty for "|| true" ? true is bash builtin, and in
>> bash it's equivalent to ':'. (see:
>> https://unix.stackexchange.com/questions/34677/what-is-the-difference-between-and-true
>> )
>>
>
> Even if there were no performance difference (and I'll admit, the
> performance difference is very small), there's the visual clutter and the
> cognitive load this places on subsequent maintainers. (One can adopt a
> strategy of pushing the "|| true" off to the right with lots of whitespace,
> but then there is the converse problem that any change to the expression is
> just that bit harder to read in "git diff".)
>
> Re: yet another global setting is perpetuating the "wrong direction".
>> Most other scripting solutions that I'm familiar with are using dynamic
>> (rather than lexical) scoping for 'try ... catch.'.
>>
>
> You're quite right that throw+catch is dynamically scoped, though
> try+catch is of course a lexically scoped block.
>
> The problem here is that you're retrofitting; in effect, you're making a
> global declaration that *removes* an implicit try+catch+ignore around
> every existing statement; *that's* what I want to have under
> lexically-scoped control.
>
> Considering that bash is stable, I do not think that it is realistic to
>> try to expect major changes to 'bash'.
>>
>
> Expect, maybe not. Hope for? certainly. Fork it and do it myself? I'll
> think about it.
>
>
>> For a more sophisticated environment, python, groovy, javascript or (your
>> favorite choice) might be a better solution.
>>
>
> Agreed that there are better languages for most complex tasks.
>
> However they're not as pervasively available as Bash; the only language
> that comes close to the same availability is Perl, and much as I like Perl,
> it's abjectly detested by many folk. (The Shell would be *as* abjectly
> detested if people actually understood it, but they're under the delusion
> that it's "simple", and so they don't bother to hate it until it bites
> them, and by the time they understand why it bit them, they're hooked and
> can't leave even if they do hate it.)
>
> Question: Out of curiosity, can you share your idea for a better solution ?
>>
>
> The direction I personally would take the shell is towards a "sane"
> language where the current syntax is but a subset, and the oddball
> semantics are "opt in".
>
> Obviously your intent is to amend the behaviour specified by the first
> part of that POSIX rule, so that only the last pipeline before "then" or
> "do" is considered exempt, and not any earlier pipelines within the
> compound list that appears there.
>
> I see some merit in that approach, but on the whole I would not opt for
> that approach. It's not as if long lists of commands between if/then is the
> norm, and where they do occur it's for good reason.
>
> Consider:
>
> if  some_cmd
>>     (( $? == 0 || $? == 43 ))
>> then
>>     echo "some_cmd succeeded or reported 'no action necessary'"
>> fi
>>
>
> Where clearly the exit status of some_cmd is being given due
> consideration, and carefully made exempt from errexit.
>
> Instead I would focus on two things:
>
>    1. abolishing "weird effects at a distance", which are a much bigger
>    problem for code maintenance: the fact that calling a function within
>    if/then or while/do turns off errexit with dynamic scope is the key pain
>    point.
>    2. making it possible to "catch" an error, do some local clean-up, and
>    re-throw it
>
> To this end, I would introduce some way of performing most "shopt", "set
> -o", and "compat" settings with lexical scope; « local -o errfail » comes
> to mind, but the exact syntax is not important and can be thrashed out
> later. And yes, this would be allowed at file level, not just within
> functions. Indeed it would be *preferred*  at file level, effectively as
> a declaration of which language variant applies to *this file*.
>
> When a function is defined with such an option in effect, it would attach
> to the function itself, so that when the function is called, those settings
> are performed automatically at function entry, even if it's called from
> somewhere that the setting is not in effect. Conversely, if they're *not*
> in effect when the function is defined, they're turned off even if they
> were on where it's called from.
>
> When "sourcing"/"dotting" a file, the default settings apply (until it
> reaches a « local -o » statement, or whatever syntax is chosen for that),
> even if called from somewhere with other settings in effect.
>
> This makes it possible to have different settings in different files,
> which is important for managing large projects.
>
> Then I would encourage *every* bash file to start with the "compat"
> option for the *current* version when it's written, so that it doesn't
> get broken by subsequent updates to Bash. (Yes, that means we would need to
> actually define the compat option when the version is created, instead of
> when the *next* version replaces it.)
>
> Side note: this means that tab completion functions for interactive Bash
> wouldn't have to be some huge monolith that needs to be immediately
> re-checked and updated when Bash is updated in the future. Instead the
> maintenance of individual completion functions could be safely devolved to
> the projects that they apply to.
>
> I would also take a declarative approach to whether each function
> participates in "errfail" or "errexit" handling:
>
>    1. Does its return status indicate success or failure? If not, then
>    all calls to it would automatically ignore errexit and errfail, whether
>    "tested" or not.
>    2. Does it "catch" errors that occur inside it? That is, despite «
>    errexit » being in effect when the function was called, block unwinding due
>    to inner errors as if « errfail » were in effect instead.
>
> I expect both of these could be set in three ways: as an outer-scope
> setting (« local -o ») when the function is defined; or as an option flag
> after « function » when defining the function; or as an extra option to «
> declare -f » before or after the function is actually defined. Of course,
> any time you have a setting, it also applies to subsequent ordinary
> commands within that lexical scope.
>
> A key effect of this would be to allow interoperation between "modules",
> being files containing sets of functions, where some use set -e (and need
> it as a failsafe), and some don't (and can't operate with it enabled). Or
> with set -f, or compat40, or whatever.
>
> I would define a lexical scope as extending from the current statement
> until the end of the inner-most enclosing compound statement, or the end of
> an explicit subshell, or the end of the file, or the end of the string
> that's being read by "eval"; whichever comes first.
>
> I'm in two minds whether this should be some variation on "local" or
> "declare", or a new keyword such as "decl" or "using" or "with". (I find it
> ironic that the effects of "local" are not, in fact, actually local.)
>
> Lastly, all of this needs to be done in a way that won't blow up on older
> versions of Bash. That's a work in progress.
>
> Yes I can see that this is a much more substantial change than you're
> proposing, but it's not insanely huge.
>
> -Martin
>
> PS: I would also make it possible to define a function within a function
> such that it too has local scope.
>
> PPS: my longer-term goal would be complete lexically scoped symbol tables
> for variables and function names, but that's not a prerequisite for
> implementing this change.
>