 Relationships |
|||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||
| Anonymous | Login | 2024-12-12 14:53 UTC |
| Main | My View | View Issues | Change Log | Docs |
| Viewing Issue Simple Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||
| ID | Category | Severity | Type | Date Submitted | Last Update | ||
| 0000351 | [1003.1(2008)/Issue 7] Shell and Utilities | Objection | Error | 2010-11-09 18:42 | 2024-06-11 08:53 | ||
| Reporter | eblake | View Status | public | ||||
| Assigned To | ajosey | ||||||
| Priority | normal | Resolution | Accepted As Marked | ||||
| Status | Closed | ||||||
| Name | Eric Blake | ||||||
| Organization | Red Hat | ||||||
| User Reference | ebb.export | ||||||
| Section | 2.9.1 Simple Commands | ||||||
| Page Number | 2316 | ||||||
| Line Number | 73089 | ||||||
| Interp Status | Approved | ||||||
| Final Accepted Text | See Note: 0000865 | ||||||
| Summary | 0000351: certain shell special built-ins should expand arguments in assignment context | ||||||
| Description |
Historical behavior of ksh88 was that 'export' and 'readonly' can treat arguments in an assignment context. From the ksh88 man page: | Words that are in the format of a parameter assignment, | are expanded with the same rules as a parameter assignment. | This means that | tilde substitution is performed after the | .B = | sign and word splitting and file name generation are not | performed. The standard even assumes this is the case, in the EXAMPLES section of export at line 74320: | Set and export the PATH variable: | export PATH=/local/bin:$PATH However, the standard fails to document this, so a current strict reading would require that if $PATH contains any whitespace or globbing characters, then the result might be the exporting of more than just PATH, or a syntax error if the expansion results in a non-name argument, and that the above example should have been written export PATH=/local/bin:"$PATH" to avoid word splitting and file name expansion. Additionally, the ksh behavior is documented as treating arguments as assignment context only if they could be recognized as assignment context in isolation. In isolation, the use of quoting rejects a word from being recognized as an assignment, so for words after export, the same logic applies where the use of quoting can make a word subject to traditional field splitting and no special tilde expansion. Current dash behavior is strictly compliant, but it is inconsistent with historical behavior; for example: $ ksh -c 'set "1 b=2" "3 d=4"; export a=$1 \c=$2; echo "$a.$b.$c.$d"' 1 b=2..3.4 $ dash -c 'set "1 b=2" "3 d=4"; export a=$1 \c=$2; echo "$a.$b.$c.$d"' 1.2.3.4 This proposal mandates the historical ksh behavior, even though it renders some existing implementations non-compliant; so it must be sent down the interpretations track. However, it is worth inclusion in the TC1 revision, rather than deferring to a future revision, even though it will require some implementations to make fixes. For example: $ ksh -c 'HOME=/tmp; export \a=~; echo "$a"' ~ $ mksh -c 'HOME=/tmp; export \a=~; echo "$a"' /tmp Here, ksh correctly recognizes that the quoting of the name a means that the argument to export is not in the form of an assignment; therefore, no tilde expansion takes place. mksh mistakenly performs the tilde expansion. $ bash --posix -c 'HOME=/tmp; command export a=~; echo "$a"' ~ $ ksh -c 'HOME=/tmp; command export a=~; echo "$a"' /tmp Since the 'command' utility is not a special built-in, and this proposal only assignment context if the command word is recognized as a special built-in, the bash behavior is correct and ksh is in error for treating a=~ in an assignment context. There is one thing that this proposal intentionally leaves unspecified, in order to allow two styles of existing behavior: $ mksh -c 'HOME=/tmp b=export; $b a=~; echo "$a"' /tmp $ ksh -c 'HOME=/tmp b=export; $b a=~; echo "$a"' ~ Here, mksh recognizes the first command word as 'export', and performs assignment-style tilde expansion; but ksh performs textual recognition (see also 0000053 for an example of ksh recognizing trap via textual recognition), so none of the subsequent arguments are recognized as an assignment context, and no tilde expansion is performed. Finally, the proposal intentionally allows shells to implement other special built-ins as extensions, such as 'typeset' or 'local', that can also treat arguments as being in an assignment context. |
||||||
| Desired Action |
At line 73093 (XCU 2.9.1 step 2), add: If the command name is recognized as a special built-in utility that can affect variable assignments, then any remaining words that would be recognized as a variable assignment in isolation shall be expanded as a variable assignment (tilde expansion after the first <equals-sign> and after any unquoted <colon>, parameter expansion, command substitution, arithmetic expansion, and quote removal, but no field splitting or pathname expansion); while words that would not be a variable assignment in isolation are subject to regular expansion. For all other command names, all subsequent words are subject to regular expansion (tilde expansion for only a leading <tilde>, parameter expansion, command substitution, arithmetic expansion, field splitting, pathname expansion, and quote removal). After line 73100, add a paragraph: When determining whether a command name is present and represents a special built-in that would require subsequent words to be considered for assignment expansion, an implementation may use only lexical analysis. For example, it is unspecified whether 'var=export; $var a=~' sets the variable a to a literal <tilde> or to the value of $HOME, since lexical analysis sees '$var' rather than 'export' as the command name. At line 73820 (XCU 2.14), change "two" to "three". At line 73827, add a paragraph: 3. When a special built-in that modifies variables is recognized as the command word, subsequent words shall be considered for expansion in an assignment context (allowing multiple tilde expansion, and inhibiting field splitting and pathname expansion). At line 74276 (XCU export DESCRIPTION), add a sentence: During parsing of the simple command, if export was recognized as the command word, then subsequent words of the form name=word shall be expanded in an assignment context. At line 74360 (XCU readonly DESCRIPTION), add a sentence: During parsing of the simple command, if readonly was recognized as the command word, then subsequent words of the form name=word shall be expanded in an assignment context. At line 81561 (XCU command APPLICATION USAGE), add: Similarly: command export a=~ suppresses the parsing of a=~ in an assignment context, such that the variable a will contain a literal <tilde> rather than the contents of $HOME. At line 124658 (XRAT C.2.9.1), add: Expansion of words in an assignment context following the command word can only occur for special built-ins, and only when the word can be used as a variable assignment in isolation. For example, this code sequence exports the single variable a with the value '1 b=2', but invokes make with the macro a set to '1' and b set to '2', since make is not a special built-in: $ set '1 b=2' $ export a=$1 $ make a=$1 Conversely, this code sequence exports two variables, a set to '1' and b set to '2', because the use of quoting means that the word could not be recognized as a variable assignment, and regular expansion rules require that field splitting occurs on the unquoted expansion of $1: $ set '1 b=2' $ export \a=$1 Implementations are permitted to provide extensions of other special built-ins that can cause assignment context expansion, such as 'typeset' or 'local'. |
||||||
| Tags | issue8 | ||||||
| Attached Files | |||||||
|
|
|||||||
|
Relationships |
|||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||
|
Notes |
|
|
(0000613) eblake (manager) 2010-11-09 18:46 |
Given the behavior of ksh 'command export a=~', and the on-list comments from David Korn: ksh93 has the concept of declaration commands and these are recognized while the script is tokenized. Changing this would complicate extensions in ksh93 such as user defined types which are declaration commands that contain declaration commands. It may be worth an alternative proposal that defines the notion of a declaration command, states that parsing arguments in an assignment context shall happen for all simple commands that are recognized as a declaration command whether or not they are special built-ins, then mandates that 'export', 'readonly' are declaration commands, that it is implementation-defined whether 'command' is a declaration command, and that implementations may define extensions that allow the user to introduce other declaration commands. |
|
(0000618) eblake (manager) 2010-11-18 16:43 |
Mail sequence 14988 proposed this in response to note 613: New definition in XBD 3: Declaration Command A utility which can take arguments that cause variable assignments (of the form varname=value) which will persist in the current shell environment. During parsing, when the shell recognizes a declaration command as the command name, then subsequent arguments that would be a valid variable assignment in isolation are subject to different expansion rules (field splitting and pathname expansion are suppressed, and tilde expansion occurs after the <equals-sign> and any unquoted <colon>). Arguments which are not a valid variable assignment in isolation are processed according to normal argument expansion rules. The following standard utilities are declaration commands: export, readonly, and certain instances of command. An implementation may provide other declaration commands. At line 3276, XBD 4.22 Variable Assignment, add: Assignment context occurs in the cmd_prefix portion of a shell simple command, as well as in arguments of a recognized declaration command. At line 71964, XCU 1.4 Utility Description Defaults DESCRIPTION, add: A standard utility shall not be treated as a declaration command unless explicitly stated in this section. At line 73093 (XCU 2.9.1 step 2), add: If the command name is recognized as a declaration command, then any remaining words that would be recognized as a variable assignment in isolation shall be expanded as a variable assignment (tilde expansion after the first <equals-sign> and after any unquoted <colon>, parameter expansion, command substitution, arithmetic expansion, and quote removal, but no field splitting or pathname expansion); while words that would not be a variable assignment in isolation are subject to regular expansion. For all other command names, all subsequent words are subject to regular expansion (tilde expansion for only a leading <tilde>, parameter expansion, command substitution, arithmetic expansion, field splitting, pathname expansion, and quote removal). After line 73100, add a paragraph: When determining whether a command name is declaration command, an implementation may use only lexical analysis. It is unspecified whether assignment context will be used if the command name is not known prior to the expansion of the first field which is not a variable assignment or redirection. At line 74276 (XCU export DESCRIPTION), add: The export special built-in shall be a declaration command. Therefore, during parsing of a simple command, if export is recognized as the command word, then subsequent words of the form name=word shall be expanded in an assignment context. At line 74360 (XCU readonly DESCRIPTION), add: The readonly special built-in shall be a declaration command. Therefore, during parsing of a simple command, if readonly is recognized as the command word, then subsequent words of the form name=word shall be expanded in an assignment context. At line 81462 (XCU command DESCRIPTION), add: The command utility shall be a declaration command if the command_name argument is recognized as a declaration command. Therefore, during parsing of a simple command, if command is recognized as the command word and a declaration command is recognized as the next argument, then subsequent words of the form name=word shall be expanded in an assignment context. At line 81588 (XCU command APPLICATION USAGE), add: The command utility takes on the parsing behavior of the command that it is wrapping. Therefore, command command export a=~ is recognized as a declaration command, and sets the variable a to the value of $HOME because it performs tilde-expansion of an assignment context; while command echo a=~ outputs the literal string "a=~" because regular expansion can only perform tilde-expansion at the beginning of the word. However, the shell need only perform lexical analysis of the command_name argument when deciding if command should be treated as a declaration command; therefore, with: var=export; command $var a=~ it is unspecified whether the word 'a=~' is handled in an assignment context or as a regular expansion. At line 124658 (XRAT C.2.9.1), add: Expansion of words in an assignment context following the command word can only occur for declaration commands, and only when the word can be used as a variable assignment in isolation. For example, this code sequence exports the single variable a with the value '1 b=2', but invokes make with the macro a set to '1' and b set to '2', since make is not a declaration command: $ set '1 b=2' $ export a=$1 $ make a=$1 Conversely, this code sequence exports two variables, a set to '1' and b set to '2', because the use of quoting means that the word could not be recognized as a variable assignment, and regular expansion rules require that field splitting occurs on the unquoted expansion of $1: $ set '1 b=2' $ export \a=$1 Likewise, this code sequence will not be parsed in assignment context, but is still required to export the variable named foo with the value 1: $ var=foo $ export $var=1 Implementations are permitted to provide extensions that serve as declaration commands, such as 'typeset' or 'local', or even a way to define a function that can behave as a declaration command. Declaration commands are only required to be recognized via lexical analysis; if any expansions are required before the command name is known, then it is unspecified whether subsequent arguments will be treated with an assignment context during expansion. For example, it is unspecified whether 'var=export; $var a=~' sets the variable a to a literal <tilde> or to the value of $HOME, since lexical analysis sees '$var' rather than 'export' as the command name. |
|
(0000865) Don Cragun (manager) 2011-06-23 15:28 edited on: 2011-08-25 15:25 |
Interpretation response ------------------------ The standard states that the current behavior of ksh93 does not conform, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor. Rationale: ------------- Some shells behave as specified; some others match ksh93 behavior; and others have gone different routes. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- Make the changes suggested in Note: 0000943. |
|
(0000933) ajosey (manager) 2011-08-10 15:24 edited on: 2011-08-26 08:55 |
Interpretation proposed 10 August 2011 for final 30 day review -- see note 946, which has the review clock being restarted. |
|
(0000942) geoffclare (manager) 2011-08-25 15:23 |
From mailing list item 16402 ... Jilles Tjoelker raised some issues a while ago with the resolution of bug 351 which should be addressed during the 30 day review ... Jilles Tjoelker <jilles@stack.nl> wrote, on 25 Jun 2011: > > > New definition in XBD 3: > > Declaration Command > > A utility which can take arguments that cause variable assignments (of > > the form varname=value) which will persist in the current shell > > environment. During parsing, when the shell recognizes a declaration > > command as the command name, then subsequent arguments that would be a > > valid variable assignment in isolation are subject to different > > expansion rules (field splitting and pathname expansion are suppressed, > > and tilde expansion occurs after the <equals-sign> and any unquoted > > <colon>). Arguments which are not a valid variable assignment in > > isolation are processed according to normal argument expansion rules. > > The "During parsing" bit is only really needed for kshisms like > readonly constants=(1 2 4) > and conflicts with the later "It is unspecified whether assignment > context will be used if the command name is not known..." if the > implementation chooses to use assignment context in that case. Yes, specifying "during parsing" forbids recognition at a later stage, which we want to allow. > > The following standard utilities are declaration commands: export, > > readonly, and certain instances of command. An implementation may > > provide other declaration commands. > > The wording blurs the distinction between "utility" and "command". > Perhaps it helps to use "declaration utility" instead of "declaration > command"; "certain instances of command" needs to be replaced with > something like "command, under certain conditions". I agree. > > At line 81462 (XCU command DESCRIPTION), add: > > The command utility shall be a declaration command if the command_name > > argument is recognized as a declaration command. Therefore, during > > parsing of a simple command, if command is recognized as the command > > word and a declaration command is recognized as the next argument, then > > subsequent words of the form name=word shall be expanded in an > > assignment context. > > I read the first sentence to mean that also things like > command -p export a=~ > command -- export a=~ > command -p -- export a=~ > get the special treatment. However, the second sentence denies it again > and that's also what ksh93 does. > > Allowing the options would be more orthogonal but not particularly > useful and the implementations could become uglier. I can see that there is no need to handle "command -p" since declaration utilities are always builtins, but I'm a little surprised that "command -- export ..." doesn't work in ksh93. Nevertheless, since it is existing practice we should allow the ksh93 behaviour. I have attempted to make the necessary changes to address all these points in the following note (an updated version of note 618). |
|
(0000943) geoffclare (manager) 2011-08-25 15:24 |
New definition in XBD 3: Declaration Utility A utility which can take arguments that cause variable assignments (of the form varname=value) which will persist in the current shell environment. When the shell recognizes a declaration utility as the command name, subsequent arguments that would be a valid variable assignment in isolation are subject to different expansion rules (field splitting and pathname expansion are suppressed, and tilde expansion occurs after the <equals-sign> and any unquoted <colon>). Arguments which are not a valid variable assignment in isolation are processed according to normal argument expansion rules. The following standard utilities are declaration utilities: export, readonly, and, under certain conditions, command. An implementation may provide other declaration utilities. At line 3276, XBD 4.22 Variable Assignment, add: Assignment context occurs in the cmd_prefix portion of a shell simple command, as well as in arguments of a recognized declaration utility. At line 71964, XCU 1.4 Utility Description Defaults DESCRIPTION, add: A standard utility shall not be treated as a declaration utility unless explicitly stated in this section. At line 73093 (XCU 2.9.1 step 2), add: If the command name is recognized as a declaration utility, then any remaining words that would be recognized as a variable assignment in isolation shall be expanded as a variable assignment (tilde expansion after the first <equals-sign> and after any unquoted <colon>, parameter expansion, command substitution, arithmetic expansion, and quote removal, but no field splitting or pathname expansion); while words that would not be a variable assignment in isolation shall be subject to regular expansion. For all other command names, all subsequent words shall be subject to regular expansion (tilde expansion for only a leading <tilde>, parameter expansion, command substitution, arithmetic expansion, field splitting, pathname expansion, and quote removal). After line 73100, add a paragraph: When determining whether a command name is a declaration utility, an implementation may use only lexical analysis. It is unspecified whether assignment context will be used if the command name would only become recognized as a declaration utility after word expansions. At line 74276 (XCU export DESCRIPTION), add: The export special built-in shall be a declaration utility. Therefore, if export is recognized as the command word of a simple command, then subsequent words of the form name=word shall be expanded in an assignment context. See Section 2.9.1. At line 74360 (XCU readonly DESCRIPTION), add: The readonly special built-in shall be a declaration utility. Therefore, if readonly is recognized as the command word of a simple command, then subsequent words of the form name=word shall be expanded in an assignment context. See Section 2.9.1. At line 81462 (XCU command DESCRIPTION), add: The command utility shall be treated as a declaration utility if the first argument passed to the utility is recognized as a declaration utility. In this case, subsequent words of the form name=word shall be expanded in an assignment context. See Section 2.9.1. At line 81588 (XCU command APPLICATION USAGE), add: The command utility takes on the expansion behavior of the command that it is wrapping. Therefore, in command command export a=~ command is recognized as a declaration utility, and the command sets the variable a to the value of $HOME because it performs tilde-expansion of an assignment context; while command echo a=~ outputs the literal string "a=~" because regular expansion can only perform tilde-expansion at the beginning of the word. However, the shell need only perform lexical analysis of the next argument when deciding if command should be treated as a declaration utility; therefore, with: var=export; command $var a=~ and command -- export a=~ it is unspecified whether the word 'a=~' is handled in an assignment context or as a regular expansion. At line 124658 (XRAT C.2.9.1), add: Expansion of words in an assignment context following the command word can only occur for declaration utilities, and only when the word can be used as a variable assignment in isolation. For example, this code sequence exports the single variable a with the value '1 b=2', but invokes make with the macro a set to '1' and b set to '2', since make is not a declaration utility: $ set '1 b=2' $ export a=$1 $ make a=$1 Conversely, this code sequence exports two variables, a set to '1' and b set to '2', because the use of quoting means that the word could not be recognized as a variable assignment, and regular expansion rules require that field splitting occurs on the unquoted expansion of $1: $ set '1 b=2' $ export \a=$1 Likewise, this code sequence will not be parsed in assignment context, but is still required to export the variable named foo with the value 1: $ var=foo $ export $var=1 Implementations are permitted to provide extensions that serve as declaration utilities, such as 'typeset' or 'local', or even a way to define a function that can behave as a declaration utility. Declaration utilities are only required to be recognized via lexical analysis; if any expansions are required before the command name is known, or before the first argument to the command utility is known, then it is unspecified whether subsequent arguments will be treated with an assignment context during expansion. For example, it is unspecified whether 'var=export; $var a=~' sets the variable a to a literal <tilde> or to the value of $HOME, since lexical analysis sees '$var' rather than 'export' as the command name. |
|
(0000946) ajosey (manager) 2011-08-26 08:54 |
After comments were raised on the reflector in mail seq 16402, the bug was discussed at the 25 August 2011 teleconference, and the proposed interpretation was revised. The review time for the interpretation is being restarted. This revised Interpretation was proposed 26 August 2011 for final 30 day review |
|
(0000980) ajosey (manager) 2011-10-04 07:36 |
The interpretation has now been approved. |
|
(0003026) mirabilos (reporter) 2016-01-19 23:15 |
| Here, mksh recognizes the first command word as 'export', and performs | assignment-style tilde expansion; but ksh performs textual recognition This is actually wrong… mksh in default mode always expands tildes after equals signs. What you want is mksh -o posix -c … |
|
Issue History |
|||
| Date Modified | Username | Field | Change |
| 2010-11-09 18:42 | eblake | New Issue | |
| 2010-11-09 18:42 | eblake | Status | New => Under Review |
| 2010-11-09 18:42 | eblake | Assigned To | => ajosey |
| 2010-11-09 18:42 | eblake | Name | => Eric Blake |
| 2010-11-09 18:42 | eblake | Organization | => Red Hat |
| 2010-11-09 18:42 | eblake | User Reference | => ebb.export |
| 2010-11-09 18:42 | eblake | Section | => 2.9.1 Simple Commands |
| 2010-11-09 18:42 | eblake | Page Number | => 2316 |
| 2010-11-09 18:42 | eblake | Line Number | => 73089 |
| 2010-11-09 18:42 | eblake | Interp Status | => --- |
| 2010-11-09 18:46 | eblake | Note Added: 0000613 | |
| 2010-11-18 16:43 | eblake | Note Added: 0000618 | |
| 2011-06-23 15:28 | Don Cragun | Interp Status | --- => Pending |
| 2011-06-23 15:28 | Don Cragun | Note Added: 0000865 | |
| 2011-06-23 15:28 | Don Cragun | Status | Under Review => Interpretation Required |
| 2011-06-23 15:28 | Don Cragun | Resolution | Open => Accepted As Marked |
| 2011-06-23 15:29 | Don Cragun | Final Accepted Text | => See Note: 0000865 |
| 2011-06-23 15:30 | Don Cragun | Tag Attached: issue8 | |
| 2011-08-10 15:24 | ajosey | Interp Status | Pending => Proposed |
| 2011-08-10 15:24 | ajosey | Note Added: 0000933 | |
| 2011-08-25 15:23 | geoffclare | Note Added: 0000942 | |
| 2011-08-25 15:24 | geoffclare | Note Added: 0000943 | |
| 2011-08-25 15:25 | Don Cragun | Note Edited: 0000865 | |
| 2011-08-26 08:54 | ajosey | Note Added: 0000946 | |
| 2011-08-26 08:55 | ajosey | Note Edited: 0000933 | |
| 2011-10-04 07:36 | ajosey | Interp Status | Proposed => Approved |
| 2011-10-04 07:36 | ajosey | Note Added: 0000980 | |
| 2014-04-10 14:44 | eblake | Relationship added | related to 0000352 |
| 2016-01-19 23:15 | mirabilos | Note Added: 0003026 | |
| 2019-02-14 16:43 | eblake | Relationship added | related to 0001172 |
| 2020-02-10 15:16 | geoffclare | Status | Interpretation Required => Applied |
| 2020-09-21 15:48 | eblake | Relationship added | parent of 0001393 |
| 2021-11-19 09:45 | geoffclare | Relationship added | related to 0001535 |
| 2023-10-28 05:10 | Don Cragun | Relationship added | child of 0001784 |
| 2023-10-28 05:20 | Don Cragun | Relationship replaced | related to 0001393 |
| 2023-10-28 06:30 | Don Cragun | Relationship added | related to 0001785 |
| 2023-10-28 06:34 | Don Cragun | Relationship deleted | child of 0001784 |
| 2024-06-11 08:53 | agadmin | Status | Applied => Closed |
| Mantis 1.1.6[^] Copyright © 2000 - 2008 Mantis Group |  |
