Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0001879 [1003.1(2024)/Issue8] Shell and Utilities Editorial Clarification Requested 2024-11-26 04:30 2024-11-30 05:14
Reporter calestyo View Status public  
Assigned To
Priority normal Resolution Open  
Status New  
Name Christoph Anton Mitterer
Organization
User Reference
Section Shell Command Language
Page Number various
Line Number various
Interp Status ---
Final Accepted Text
Summary 0001879: claifications/improvements around command/exec and special built-in redirection errors
Description Hey.


1)

The 2018 edition said for the exit status of the exec special built-in:
> If a redirection error occurs (see Consequences of Shell
> Errors), the shell shall exit with a value in the range 1-125.

with 2.8.1 Consequences of Shell Errors of the 2018 edition also saying that:
"Redirection error with special built-in utilities" … "shall exit" in "Non-Interactive
Shell[s]" and "shall not exit" in "Interactive Shells"

IMO that was already ambiguous, as the former *generally* said "shell shall exit" but the latter only for the non-interactive case.


With the 2024 that got IMO even more ambiguous:
2.8.1 Consequences of Shell Errors stayed the same (for Redirection error with
special built-in utilities), but the exit status for exec is now described as:
> If a redirection error occurs (see 2.8.1 Consequences of Shell
> Errors ), the exit status shall be a value in the range 1-125.

In particular it no longer says "the shell shall exit", but rather talks about an exit status (using the same wording it uses in other places for regular exit statuses where the shell doesn't abort per se).




2)
The `command` utility says:
> If the command_name is the same as the name of
> one of the special built-in utilities, the special
> properties in the enumerated list at the beginning
> of 2.15 Special Built-In Utilities shall not occur.

These properties in turn are:
> 1. An error in a special built-in utility may cause
> a shell executing that utility to abort, while an
> error in a regular built-in utility shall not cause
> a shell executing that utility to abort. (See 2.8.1
> Consequences of Shell Errors for the consequences of
> errors on interactive and non-interactive shells.)
> If a special built-in utility encountering an error
> does not abort the shell, its exit value shall be non-zero.
> 2. <not important here>

The above says, that (1) and (2) shall NOT occur when `command`-utility is used.
But what exactly does that mean for (1)?
(1) says "error in a special built-in utility may cause a shell executing that utility to abort"... but are these errors ONLY the "Special built-in utility error" mentioned in the table in 2.8.1. or are they ALSO the "Redirection error with special built-in utilities" from the same table.

The former, I'd interpret this as:
While (non-interactively) `set -Q` (-Q does not exist) causes the shell to abort, but `command set -Q` doesn't do so, as in e.g.:
  $ dash -c 'set -Q; echo foo'
  dash: 1: set: Illegal option -Q
  $ dash -c 'command set -Q; echo foo'
  dash: 1: set: Illegal option -Q
  foo
  $

The latter, I'd interpret as (the above plus), I can NOT do:
  exec 1>/does-not-exist/file || es=$?
but CAN do:
  command exec 1>/does-not-exist/file || es=$?
to capture the exit status of the exec, and not have the shell abort.

Again, this would be the behaviour of at least dash:
  $ dash -c 'exec >/does-not-exist/file || es=$?; echo es: $es>&2'
  dash: 1: cannot create /does-not-exist/file: Directory nonexistent
  $ dash -c 'command exec >/does-not-exist/file || es=$?; echo es: $es>&2'
  dash: 1: cannot create /does-not-exist/file: Directory nonexistent
  es: 2
  $

This goes along with the changes from issue #1427, which added the following note #1 to 2.8.1:
> The shell shall exit only if the special built-in utility
> is executed directly. If it is executed via the command
> utility, the shell shall not exit.

But ONLY, to the "Special built-in utility error" row. So either it was forgotten for the "Redirection error with special built-in utilities" or it's indeed NOT meant for redirection errors (but then see (3))




3) Why I think, redirection errors in at least `exec` should be ignored with `command`?!
I've stumbled over this whole issue when I saw a question on help-bash, which effectively was:
  How can one differentiate the exit status from the
  utility from an error exit status that resulted from
  redirection errors?

A simple example:
In
  grep foo </does-not-exist/file
it's unclear whether the pattern was not matched (grep exit status 1) or whether a redirection error occurred, in which case bash uses e.g. also exit status 1.

The original poster of the question proposed a solution like:
  ( exec </does-not-exist/file || exit 100; grep foo )
where the application would need to make sure that 100 is an exit status not used by grep.

This seems to work in bash, but only because that seems to not abort on redirection errors (per default):
  $ bash -c '( exec </does-not-exist/file || exit 100; grep foo )' ; echo $?
  bash: line 1: /does-not-exist/file: No such file or directory
  100
  $ bash --posix -c '( exec </does-not-exist/file || exit 100; grep foo )' ; echo $?
  bash: line 1: /does-not-exist/file: No such file or directory
  1
  $ dash -c '( exec </does-not-exist/file || exit 100; grep foo )' ; echo $?
  dash: 1: cannot open /does-not-exist/file: No such file
  2
  $

AFAICS, the only[0] portable solution would be using command, i.e.:
  $ bash -c '( command exec </does-not-exist/file || exit 100; grep foo )' ; echo $?
  bash: line 1: /does-not-exist/file: No such file or directory
  100
  $ bash --posix -c '( command exec </does-not-exist/file || exit 100; grep foo )' ; echo $?
  bash: line 1: /does-not-exist/file: No such file or directory
  100
  $ dash -c '( command exec </does-not-exist/file || exit 100; grep foo )' ; echo $?
  dash: 1: cannot open /does-not-exist/file: No such file
  100

But honestly, I'm not even sure whether this is intended now by POSIX (given that it's not clear whether the no-abort property of `command` applies to special built-in redirection errors or not),... and even if it would, I'm still not sure whether the above is really functioning in any possible context (like in subshells/command substitutions, with set -e, set -e ignoring contexts, ugly PATHs, or whatever).
Though I'd guess so; maybe some expert can confirm or deny.


[0] It might be possibly to do some very ugly hacks without `command exec` but just `exec` in subshells... since then only those should abort on a redirection error. But that wouldn't be really practical IMO.
Desired Action 1) I assume a redirection error in plain exec in a non-interactive shell is still intended to cause the shell to abort/exit with an error (IIRC, bash doesn't do this).
If so, the "Exit Status" section should be somehow improved to better reflect that.
Perhaps simply like:
> If a redirection error occurs, the exit status
> shall be a value in the range 1-125 with the
> consequences given in 2.8.1 Consequences of Shell Errors.

But perhaps this should even be expanded to mention the note #1 in 2.8.1., i.e. that the consequences don't apply if `exec` is called as `command exec`.




2) Depending on whether or not ,`command <special built-int>` should also cause special built-in redirection errors to not cause the shell to exit:

a) clarify in the description of the `command`-utility and/or the list item #1 in 2.15 Special Built-In Utilities, that redirection errors with special built-ins are included or explicitly mention that this is not the case.
Beware, that if doing so in the 2.15, it might affect far more things than just `command`.

b) Add (or don't add) the ¹ note superscript to the "shall exit" for "Redirection error with special built-in utilities".




3) IMO having a way to differentiate between a redirection error and a utility error is quite important.

a) This would be my case for requesting the behaviour to be specified as such (unless that's anyway intended), barring of course any shell implementations would already behave different.

b) I know, the standard is no teaching document, but it nevertheless gives many valuable examples. So I'd ask to consider whether it would make sense to find some place in the rationale of perhaps `exec` or `command` or maybe even in the Redirections chapter, where something like:
  ( command exec </does-not-exist/file || exit 100; grep foo )

could be shown as a way how to differentiate between utility and redirection error, or better said: how one can alter the exit status the shell uses in case of an error of a redirection that gets applied to some command.

btw:
Not really sure whether there's a way to also filter the error message from the redirection error.
  ( { command exec </does-not-exist/file || exit 100; } 2>/dev/null; grep foo )
but this might of course then fail+abort (at least in principle) when redirection to /dev/null.
While:
  ( command exec </does-not-exist/file 2>/dev/null || exit 100; grep foo )
does not work and:
  ( command exec 2>/dev/null </does-not-exist/file || exit 100; grep foo )
does seem to work (and I guess the latter should work portably), they'd have of course the downside of also swallowing any stderr from grep.

Interestingly, this is also caused by the upper "solution":
$ dash -c '( { command exec </does-not-exist/file || exit 100; } 2>/dev/null; grep --sdsd foo )' ; echo $?
100

Not really sure why, TBH.


Thanks.
Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0006970)
larryv (reporter)
2024-11-26 08:23
edited on: 2024-11-26 08:27
1) I assume a redirection error in plain exec in a non-interactive shell is still intended to cause the shell to abort/exit with an error (IIRC, bash doesn't do this).
It does in POSIX mode.
Perhaps simply like:
> If a redirection error occurs, the exit status
> shall be a value in the range 1-125 with the
> consequences given in 2.8.1 Consequences of Shell Errors.
How is this any better than the current text, which already has that xref? You're just moving the xref around.
( command exec 2>/dev/null </does-not-exist/file || exit 100; grep foo )
does seem to work (and I guess the latter should work portably), they'd have of course the downside of also swallowing any stderr from grep.

Interestingly, this is also caused by the upper "solution":
$ dash -c '( { command exec </does-not-exist/file || exit 100; } 2>/dev/null; grep --sdsd foo )' ; echo $?
100
What do you mean? The grep command never runs here. If it does, the error message is printed fine.
% dash -c '( { command exec </dev/null || exit 100; } 2>/dev/null; grep --sdsd foo )'
grep: unrecognized option `--sdsd'
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
    [-e pattern] [-f file] [--binary-files=value] [--color=when]
    [--context[=num]] [--directories=action] [--label] [--line-buffered]
    [--null] [pattern] [file ...]
% echo $?
2
(0006971)
calestyo (reporter)
2024-11-26 18:48

 > It does in POSIX mode.

Yes. I've mentioned that via an example a bit further down in my original text. But at least per default it doesn't do so.


> How is this any better than the current text,
> which already has that xref? You're just moving
> the xref around.
.

Admittedly, it's still no perfect. But I thought the "WITH the consequences" would make it a bit clearer, that these actually follow from the 1-125 exit status.
The original text just says "see..." but that is IMO less strong.
Anyway... I'm open for better wordings, perhaps one should use one that makes also clear that the consequences happen *immediately* after the `exec` "returns"?

> What do you mean? The grep command never runs here.
> If it does, the error message is printed fine.

Ah, of course. Thanks for pointing that out.
So then I'd guess - if it was agreed that such an example was a worthy addition to the standard - one should use:
( { command exec <any-redirections> || exit <some-exit-status-not-used-by-utility>; } 2>/dev/null; <utility> <argument>… )
and perhaps mention that instead of ( ... ) it would of course also work with other subshells ( `$(...)`).
Not sure whether one should also mention that it can be done without subshells as well.
(0006972)
larryv (reporter)
2024-11-27 07:19

 Note: 0006971:
Admittedly, it's still no perfect. But I thought the "WITH the consequences" would make it a bit clearer, that these actually follow from the 1-125 exit status.
The original text just says "see..." but that is IMO less strong.
Anyway... I'm open for better wordings, perhaps one should use one that makes also clear that the consequences happen *immediately* after the `exec` "returns"?
Given that 0001157 added the following text to the DESCRIPTION section...
If the exec command fails, a non-interactive shell shall exit from the current shell execution environment; [UP] [Option Start] an interactive shell may exit from a subshell environment but shall not exit if the current shell environment is not a subshell environment.

If the exec command fails and the shell does not exit, any redirections associated with the exec command that were successfully made shall take effect in the current shell execution environment. [Option End]
...it seems to me that the lack of descriptions of consequences of errors in the EXIT STATUS section is intentional.
(0006973)
calestyo (reporter)
2024-11-27 20:47

 > ...it seems to me that the lack of descriptions of
> consequences of errors in the EXIT STATUS section is intentional.

May be. But as it's written now, I would not necessarily interpret it as the shell must abort when a redirection error occurs on exec (without any other command in it).
IOW, bash's (default) behaviour would now be POSIX compliant.

But even that, is IMO not really said by the text, because it still loosely links to 2.8.1, and there are still numerous other parts in the standard, where it says that errors (or explicitly redirection errors) on special built-ins shall cause the shell to exit (and AFAIU, "immediately", that is in exec <does-not-exist || echo foo ... foo should NOT be printed,
(0006974)
larryv (reporter)
2024-11-27 22:57

 Looks like Note: 0006973 got truncated.
(0006975)
geoffclare (manager)
2024-11-28 16:21
edited on: 2024-11-28 16:23

 > With the 2024 that got IMO even more ambiguous:
> [...]
> In particular it no longer says "the shell shall exit", but rather talks about an exit status (using the same wording it uses in other places for regular exit statuses where the shell doesn't abort per se).

Your parenthetical comment is the whole point of this change. Unless explicitly stated otherwise, when the EXIT STATUS section for exec talks about "the exit status", it is talking about the exit status of the exec utility. If the shell itself exits, then the exec utility has no exit status. Perhaps to make this intent clearer, that sentence should be reworded as:
If a redirection error occurs (see [xref to 2.8.1]) and the shell does not exit, the exit status (of exec) shall be a value in the range 1−125.
(0006976)
geoffclare (manager)
2024-11-28 16:45

 Suggested changes ...

On page 2498 line 81135 section 2.8.1 Consequences of Shell Errors, change:
shall exit
to:
shall exit¹

On page 2526 line 82346 section 2.15 Special Built-In Utilities, change:
An error in a special built-in utility may cause a shell executing that utility to abort, while an error in a regular built-in utility shall not cause a shell executing that utility to abort.
to:
An error in a special built-in utility or redirection error with a special built-in utility may cause a shell executing that utility to abort, while for a regular built-in utility such errors shall not cause a shell executing that utility to abort.

On page 2539 line 82804 section exec, change:
If a redirection error occurs (see [xref to 2.8.1]), the exit status shall be a value in the range 1−125.
to:
If a redirection error occurs (see [xref to 2.8.1]) and the shell does not exit, the exit status (of exec) shall be a value in the range 1−125.
(0006977)
geoffclare (manager)
2024-11-28 16:49
edited on: 2024-11-28 16:51

 Note that I did not include anything for part 3 in my suggested changes because
( command exec </does-not-exist/file || exit 100; grep foo )
is not guaranteed to distinguish between a redirection error and utility error, at least not portably. This is because grep can have exit status 100 on error.
(0006978)
calestyo (reporter)
2024-11-30 05:14
edited on: 2024-11-30 05:17

 > Looks like Note: 0006973 got truncated.

Indeed. Strange. But IIRC, in the missing part I merely brought up again, that even in that case it wouldn't be clear to me whether or not exec <does-not-exist || echo foo, would cause foo to be printed.
Edit: Something's weird with the tracker... it just took again a minute or so to load when I submitted this reply, then aborted without finishing, but after reloading the reply was actually recorded.


@geoffclare

As for my original point (1), with which your proposed changed on page 2539 line 82804 deals with:

I can live with that, though would personally think that the parentheses should rather go behind the "the shell does not exit" because 2.8.1 is where it's explained when it exits, isn't it?

Also is it still clear by that, that in the case of the shell exiting (because of 2.8.1), the exit status is *also* in 1-125?

Cause that used to be the definition in the 2018 edition. But with the proposed wording, one could argue that if it has an exit status (and the shell does not exit), it's between 1-125, while when the exits because of 2.8.1 the exit status (of the shell) is, AFAIU, defined to be... well actually only for the case of a subshell it says "exit from the subshell environment with a non-zero status"... which would be more than just the 1-125.



As for my point (2b) and your proposed changes on page 2498 line 81135:

So command special-built-in is indeed expected to also prevent the shell from aborting because of any redirection errors, right?
And thus command exec <redirection> *is* the only portable way to differ the utility exit status from "redirection exit statuses".


As for my point (2a) and your proposed changes on page 2526 line 82346 section 2.15 Special Built-In Utilities:

Fine with that.

The only thing I'd ask to consider:

Is it necessary to explicitly mention for `command`, that in `command exec <redirections>` the redirections are on `exec` not on command.
Or rather, because of:
> In every other respect, if command_name is not the name
> of a function, the effect of command (with no options)
> shall be the same as omitting command,...

I'd say that this is even the case for any command_name of the `command`-utility, which has "Stdin Not used." but does have a "Stdout" defined, though only for the -v -V cases.
IMO, I'd say because of the sentence I've quoted above, no special explanation is needed... but maybe I'm wrong.


As for my point (3) and your comment in #0006977:
But, for `grep` in specific, this would only mean that one cannot differentiate between possible different error statuses, while one can differentiate between the "success statuses" 0 and 1 and an error condition.

Also, `grep` was just an example of mine, which I chose because for it the exit status 1 is not really an error.
There may be plenty of (non-POSIX) programs, for which it's known that e.g. all errors are exit status 2 or whatever.

So I'd still say that such example would be useful (perhaps not with `grep` but `utility`, of course with the information, that the application must ensure that that whichever chosen exit status that indicates the redirection error is not used y the utility.

Anyway, was just a proposal.

- Issue History
Date Modified Username Field Change
2024-11-26 04:30 calestyo New Issue
2024-11-26 04:30 calestyo Name => Christoph Anton Mitterer
2024-11-26 04:30 calestyo Section => Shell Command Language
2024-11-26 04:30 calestyo Page Number => various
2024-11-26 04:30 calestyo Line Number => various
2024-11-26 08:23 larryv Note Added: 0006970
2024-11-26 08:26 larryv Note Edited: 0006970
2024-11-26 08:27 larryv Note Edited: 0006970
2024-11-26 18:48 calestyo Note Added: 0006971
2024-11-27 07:19 larryv Note Added: 0006972
2024-11-27 20:47 calestyo Note Added: 0006973
2024-11-27 22:57 larryv Note Added: 0006974
2024-11-28 16:21 geoffclare Note Added: 0006975
2024-11-28 16:23 geoffclare Note Edited: 0006975
2024-11-28 16:45 geoffclare Note Added: 0006976
2024-11-28 16:49 geoffclare Note Added: 0006977
2024-11-28 16:51 geoffclare Note Edited: 0006977
2024-11-30 05:14 calestyo Note Added: 0006978
2024-11-30 05:17 calestyo Note Edited: 0006978

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker