Anonymous | Login | 2024-09-16 22:14 UTC |
Main | My View | View Issues | Change Log | Docs |
Viewing Issue Simple Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||
ID | Category | Severity | Type | Date Submitted | Last Update | ||
0001393 | [Issue 8 drafts] Shell and Utilities | Objection | Error | 2020-08-24 19:15 | 2024-06-11 09:12 | ||
Reporter | McDutchie | View Status | public | ||||
Assigned To | |||||||
Priority | normal | Resolution | Accepted As Marked | ||||
Status | Closed | Product Version | Draft 1 | ||||
Name | Martijn Dekker | ||||||
Organization | https://github.com/ksh93 [^] | ||||||
User Reference | |||||||
Section | XCU 3 | ||||||
Page Number | 2510 | ||||||
Line Number | 82523-82525 | ||||||
Final Accepted Text | Note: 0005003 | ||||||
Summary | 0001393: 'command' should not be treated as a declaration utility | ||||||
Description |
The current POSIX draft adds three lines to the specification of the 'command' utility, lines 82523-82525: | The command utility shall be treated as a declaration utility if the | first argument passed to the utility is recognized as a declaration | utility. In this case, subsequent words of the form name=word shall | be expanded in an assignment context. See Section 2.9.1.1 (on page | 2292). (As an aside, if I were to be pedantic, I could argue that the phrasing "if the first argument passed to the utility is recognized as a declaration utility" effectively makes this behaviour unspecified; a 'command' utility implementation could be written not to recognise any utility as such and this phrasing would allow that. I am assuming that this is not the intended meaning.) This addition effectively makes the 'command' utility into a kind of "Schrödinger's declaration utility" that is simultaneously a declaration utility and not a declaration utility. I think that is a bad idea, for the following three reasons: ***1.*** In the currently published release version of POSIX, 'command' is a regular utility (grammatically no different from, say, 'ls' or 'printf'), and there is nothing that allows regular utilities to treat their arguments specially in a shell grammar sense. So it follows that declaration utility behaviour *must* be disabled along with all other special behaviour by the 'command' utility as specified in the release version. Consequently, the behaviour specified in the draft is incompatible with the behaviour specified in the current release version, as it is not possible to be simultaneously compliant with both. (A similar argument could be made for 'export' and 'readonly' themselves, but at least they are already special built-in utilities, so the inconsistently is not so serious.) ***2.*** It does not specify sufficiently widespread current practice. For the following test: $shell -c 'v="a y=b"; command export x=$v; echo "[$x] [$y]"' bash, bosh (Schily), dash < 0.5.11, NetBSD sh, pdksh (including OpenBSD (k)sh), yash, and 'zsh --emulate sh' output: [a] [b] meaning they do *not* treat 'command export' as a declaration utility. Dash >= 0.5.11, its derivative gwsh, FreeBSD sh, ksh93, and mksh output [a y=b] [] meaning they do treat 'command export' as a declaration utility. But together they are not enough to establish a de facto standard behaviour. ***3.*** Since 'command' is specified as a regular utility, both of the following *must* work: c='command'; "$c" export foo=bar set -- command export; "$@" foo=bar So, in the draft version, this effectively means that the shell must determine at runtime whether to treat 'command' as a declaration utility, and not at parse time. And that requirement is both innovative and unreasonable. For the following test: $shell -c 'v="a y=b"; set -- command export; "$@" x=$v; echo "[$x] [$y]"' only dash 0.5.11 and its derivative gwsh currently manage to output '[a y=b] []' meaning they recognise 'command export' as a declaration utility at runtime, and all other shells (except ksh93 which breaks completely[*]) output '[a] [b]' meaning they don't. [*] ksh93 (which I currently maintain the only actively and openly developed fork of) has a bug that breaks this test case altogether -- the 'export' utility is not run at all; instead, the equivalent of 'command -v export' is run. It would be good if I could implement a clean fix, which would disassociate the 'command' utility from the parser altogether and treat it as a regular built-in utility as currently specified in the release version. |
||||||
Desired Action |
Delete lines 82523-82525. Alternatively, change these lines to the following: "If the first argument passed to the command utility is the name of a declaration utility (see Section 2.9.1.1 on page 2292), it is unspecified whether subsequent words of the form name=word are expanded in an assignment context." |
||||||
Tags | issue8 | ||||||
Attached Files | |||||||
|
Relationships | |||||||||||||
|
Notes | |
(0004943) McDutchie (reporter) 2020-08-24 19:16 |
Argh. Could someone please edit the summary to say "'command' should NOT be treated as a declaration utility"? Thanks and apologies. |
(0004944) Don Cragun (manager) 2020-08-24 23:09 |
Re Note: 0004943: Done, |
(0004945) geoffclare (manager) 2020-08-25 08:25 |
I have moved this to the Issue 8 drafts project and set the product version to Draft 1. |
(0004971) McDutchie (reporter) 2020-09-04 03:53 |
I had written: "So, in the draft version, this effectively means that the shell must determine at runtime whether to treat 'command' as a declaration utility, and not at parse time. And that requirement is both innovative and unreasonable." I'd just like to expound this point a little. On shells where shell assignments allow special grammar like "a=(foo bar)" (which are bash, ksh93, mksh and zsh), the 'command' utility must be integrated into the parser if it is to preserve a declaration command's assignment behaviour including grammatical properties. ksh93 does this; bash, mksh, and zsh do not. So command typeset a=(1 2 3) works on ksh93, but throws a syntax error on the others (zsh attempts and fails to interpret it as some kind of zsh-style pattern). Of course this can only work on ksh93 because its command builtin is integrated into the parser (in ways that also cause multiple bugs including one serious bug, see footnote above). The assignments are recognised at parse time, not at runtime. If assignments can use special grammar such as parentheses, then it is not feasible or even possible to do this at runtime. Dash manages it because it doesn't support assignments with special syntax. So, I think ksh93 is a case study on the reasons why it is a bad idea for 'command' to be anything else than a plain old regular builtin utility. |
(0004972) geoffclare (manager) 2020-09-04 09:34 |
Re: Note: 0004971 There is no requirement for this to be determined at runtime. The APPLICATION USAGE on page 2513 line 82660 says that the shell need only perform lexical analysis of the next argument when deciding if command should be treated as a declaration utility. (There is an editorial error here: "command" should be in italics. We should fix that if the text here is otherwise unchanged by this bug.) |
(0004973) joerg (reporter) 2020-09-04 11:43 |
Re: Note: 0004971 bosh only looks at the flags in it's builtin command data structures if a builtin is a special builtin and in this case does not make variable assignments a push operation that is popped after the builtin command terminated. My impression is that this best fits to the previous shell behavior. BTW: is there a list with all ksh93 related problems? |
(0005002) joerg (reporter) 2020-09-24 14:55 |
It seems that before 0000351 no shell had a special treatment for quoting for the arguments of "export". So a requirement for this behavior looks like invention. |
(0005003) geoffclare (manager) 2020-09-24 15:55 |
On page 2332 line 75618 section 2.14 export (APPLICATION USAGE), add a new paragraph:In shells that support extended assignment syntax, for example to allow an array to be populated with a single assignment, such extensions can typically only be used in assignments specified as arguments to export if the command word is literally export, and not if it is some other word that expands to export. For example:# Shells that support array assignment as an extension generally support this: export x=(1 2 3); echo ${x[0]} # outputs 1 # But generally do not support this: e=export; $e x=(1 2 3); echo ${x[0]} # syntax error On page 2332 line 75637 section 2.14 export (RATIONALE), add a new paragraph: Some implementations extend the shell's assignment syntax, for example to allow an array to be populated with a single assignment, and in order for such an extension to be usable in assignments specified as arguments to export these shells have export as a separate token in their grammar. This standard only permits an extension of this nature when the input to the shell would contain a syntax error according to the standard grammar. Note that although export can be a separate token in the shell's grammar, it cannot be a reserved word since export is a candidate for alias substitution whereas reserved words are not (see [xref to 2.3.1]). On page 2335 line 75721 section 2.14 readonly (APPLICATION USAGE), change: None.to: In shells that support extended assignment syntax, for example to allow an array to be populated with a single assignment, such extensions can typically only be used in assignments specified as arguments to readonly if the command word is literally readonly, and not if it is some other word that expands to readonly. For example:# Shells that support array assignment as an extension generally support this: readonly x=(1 2 3); echo ${x[0]} # outputs 1 # But generally do not support this: r=readonly; $r x=(1 2 3); echo ${x[0]} # syntax error On page 2335 line 75739 section 2.14 readonly (RATIONALE), add a new paragraph: Some implementations extend the shell's assignment syntax, for example to allow an array to be populated with a single assignment, and in order for such an extension to be usable in assignments specified as arguments to readonly these shells have readonly as a separate token in their grammar. This standard only permits an extension of this nature when the input to the shell would contain a syntax error according to the standard grammar. Note that although readonly can be a separate token in the shell's grammar, it cannot be a reserved word since readonly is a candidate for alias substitution whereas reserved words are not (see [xref to 2.3.1]). On page 2514 line 82700 section command (RATIONALE), change: ... or that prevents function lookup on b or c.to: ... or that prevents function lookup on b or c. However, some implementations extend the shell's assignment syntax, for example to allow an array to be populated with a single assignment, and in order for such an extension to be usable in assignments specified as arguments to export and readonly these shells have those utility names as separate tokens in their grammar. When command is used to execute these utilities it also needs to be a separate token in the grammar so that the same extended assignment syntax can still be recognized in this case. This standard only permits an extension of this nature when the input to the shell would contain a syntax error according to the standard grammar, and therefore it cannot affect how '|' and ';' are parsed in the example above. Note that although command can be a separate token in the shell's grammar, it cannot be a reserved word since command is a candidate for alias substitution whereas reserved words are not (see [xref to 2.3.1]). |
Issue History | |||
Date Modified | Username | Field | Change |
2020-08-24 19:15 | McDutchie | New Issue | |
2020-08-24 19:15 | McDutchie | Name | => Martijn Dekker |
2020-08-24 19:15 | McDutchie | Organization | => https://github.com/ksh93 [^] |
2020-08-24 19:15 | McDutchie | Section | => XCU 3 |
2020-08-24 19:15 | McDutchie | Page Number | => 2510 |
2020-08-24 19:15 | McDutchie | Line Number | => 82523-82525 |
2020-08-24 19:16 | McDutchie | Note Added: 0004943 | |
2020-08-24 23:09 | Don Cragun | Interp Status | => --- |
2020-08-24 23:09 | Don Cragun | Note Added: 0004944 | |
2020-08-24 23:09 | Don Cragun | Summary | 'command' should be treated as a declaration utility => 'command' should not be treated as a declaration utility |
2020-08-25 08:24 | geoffclare | Project | 1003.1(2016/18)/Issue7+TC2 => Issue 8 drafts |
2020-08-25 08:25 | geoffclare | Note Added: 0004945 | |
2020-08-25 08:25 | geoffclare | version | => Draft 1 |
2020-09-04 03:53 | McDutchie | Note Added: 0004971 | |
2020-09-04 09:34 | geoffclare | Note Added: 0004972 | |
2020-09-04 11:43 | joerg | Note Added: 0004973 | |
2020-09-21 15:48 | eblake | Relationship added | child of 0000351 |
2020-09-24 14:55 | joerg | Note Added: 0005002 | |
2020-09-24 15:55 | geoffclare | Note Added: 0005003 | |
2020-09-24 15:56 | geoffclare | Final Accepted Text | => Note: 0005003 |
2020-09-24 15:56 | geoffclare | Status | New => Resolved |
2020-09-24 15:56 | geoffclare | Resolution | Open => Accepted As Marked |
2020-09-24 15:56 | geoffclare | Tag Attached: issue8 | |
2020-10-08 10:37 | geoffclare | Status | Resolved => Applied |
2023-10-28 05:10 | Don Cragun | Relationship added | related to 0001784 |
2023-10-28 05:20 | Don Cragun | Relationship replaced | related to 0000351 |
2023-10-28 06:28 | Don Cragun | Relationship added | related to 0001785 |
2023-10-28 06:28 | Don Cragun | Relationship deleted | related to 0001784 |
2024-06-11 09:12 | agadmin | Status | Applied => Closed |
Mantis 1.1.6[^] Copyright © 2000 - 2008 Mantis Group |