View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000457 | 1003.1(2008)/Issue 7 | Shell and Utilities | public | 2011-06-03 18:23 | 2013-04-16 13:06 |
| Reporter | eblake | Assigned To | ajosey | ||
| Priority | normal | Severity | Objection | Type | Omission |
| Status | Closed | Resolution | Accepted | ||
| Name | Eric Blake | ||||
| Organization | Red Hat | ||||
| User Reference | ebb.expansion | ||||
| Section | 2.6.2 Parameter Expansion | ||||
| Page Number | 2306 | ||||
| Line Number | 72719 | ||||
| Interp Status | Approved | ||||
| Final Accepted Text | See 0000457:0000837 | ||||
| Summary | 0000457: omitted word in parameter expansions | ||||
| Description | On all shells that I tried (ksh88, ksh93, bash, dash, zsh, mksh, all of which are trying to comply with POSIX; as well as Solaris /bin/sh for historical background), the ${parameter:-word} and ${parameter:=word} forms with an empty word were parsed without problems, and with useful effects. I tested with: $ $candidate -c 'echo ${foo-unset}; : ${foo=}; echo ${foo+set}' unset set $ $candidate -c 'echo ${foo-unset}; : ${foo:=}; echo ${foo+set}' unset set $ $candidate -c 'echo .${foo-}.; foo=a; echo ${foo-}' .. a $ $candidate -c 'echo .${foo:-}.; foo=a; echo ${foo:-}' .. a Additionally, all shells understood an omitted word in the ${parameter:+word} form, although that always results in an empty string. However, it is worth standardizing if for no other reason than to guarantee this result: $candidate -cn '${foo+}' && echo $? 0 when contrasted with other syntax failures like ${+}. Furthermore, among all shells that understand the forms ${parameterOPword} for OP in %, %%, #, and ##, an omitted word resulted in the use of an empty string as the pattern, which ends up not removing any characters from the parameter expansion, or as a way of detecting unset variables when 'set -u' is enabled (actually, bash 4.1 and zsh 4.3.10 were buggy in this respect, but that bug was fixed in bash 4.2 and zsh 4.3.12). In fact, the standard requires that ${foo##} be parsed as ${parameter#word} with word of #, which results in the output of foo with the shortest match to the pattern # removed, but all shells tested instead treated it like ${parameter##} with an optional word, and output ${foo} instead (likewise for %%): $ $candidate -c 'foo=##; echo ${foo#"#"} ${foo##} ${foo###}' # ## # $ $candidate -c 'foo=%%; echo ${foo%"%"} ${foo%%} ${foo%%%}' % %% % Therefore, since the standard disagrees with existing practice, this is worth fixing. Conversely, allowing an omitted word introduces an ambiguity in ${##} (length of $# vs. shortest-prefix removal of empty string pattern from $#); however, behavior varies wildly here (generally in the form of implementations trying to parse ${##""} as ${#parameter}, but #"" is not a parameter), so the best we can do is leave prefix- and suffix-removal forms unspecified on $#. $ bash -c 'set a b; echo ${##} ${##""} ${###} ${###""}' 1 2 2 2 $ dash -c 'set a b; echo ${##} ${##""} ${###} ${###""}' 1 2 2 2 $ zsh -c 'set a b; echo ${##} ${##""} ${###} ${###""}' 2 2 2 2 $ mksh -c 'set a b; echo ${##} ${##""} ${###} ${###""}' mksh: ${##""}: bad substitution $ mksh -c 'set a b; echo ${##} ${###} ${###""}' mksh: ${###}: bad substitution $ mksh -c 'set a b; echo ${##} ${###""}' mksh: ${###""}: bad substitution $ mksh -c 'set a b; echo ${##}' 1 $ ksh -c 'set a b; echo ${##} ${##""} ${###} ${###""}' ksh: syntax error at line 1: `"' unexpected $ ksh -c 'set a b; echo ${##} ${###} ${###""}' 1 2 2 If this issue is accepted in isolation, it would introduce an ambiguity in ${#-} (length of $- vs. empty string in (impossible) case that $# is unset); all shells I tested favored length of $-. However, when coupled with the solution in 0000417, there is no problem, that bug was intentionally worded to require ${#-} to be the length of $-, and to provide a word when probing whether $# is unset. | ||||
| Desired Action | At line 72719, change: ${parameter:−word} Use Default Values. If parameter is unset or null, the expansion of word shall be substituted; otherwise, the value of parameter shall be substituted. ${parameter:=word} Assign Default Values. If parameter is unset or null, the expansion of word shall be assigned to parameter. to: ${parameter:−[word]} Use Default Values. If parameter is unset or null, the expansion of word (or an empty string if word is omitted) shall be substituted; otherwise, the value of parameter shall be substituted. ${parameter:=[word]} Assign Default Values. If parameter is unset or null, the expansion of word (or an empty string if word is omitted) shall be assigned to parameter. At line 72730, change: ${parameter:+word} Use Alternative Value. If parameter is unset or null, null shall be substituted; otherwise, the expansion of word shall be substituted. to: ${parameter:+[word]} Use Alternative Value. If parameter is unset or null, null shall be substituted; otherwise, the expansion of word (or an empty string if word is omitted) shall be substituted. At line 72749, add a sentence to ${#parameter}: If parameter is unset and set -u is in effect, the expansion shall fail. At line 75752, change: If parameter is ’*’ or ’@’, the result of the expansion is unspecified. to: If parameter is ’#’, ’*’, or ’@’, the result of the expansion is unspecified. If parameter is unset and set -u is in effect, the expansion shall fail. At line 72755, add a sentence: In each variety, if word is omitted, the empty pattern shall be used. At line 72756, change "${parameter%word}" to "${parameter%[word]}". At line 72758, add a sentence: If present, word shall not begin with an unquoted '%'. At line 72759, change "${parameter%%word}" to "${parameter%%[word]}". At line 72762, change "${parameter#word}" to "${parameter#[word]}". At line 72764, add a sentence: If present, word shall not begin with an unquoted '#'. At line 72765, change "${parameter##word}" to "${parameter##[word]}". | ||||
| Tags | tc1-2008 | ||||
|
|
Interpretation response ------------------------ The standard states that "word" must be present in these cases, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor. Rationale: ------------- The standard's requirements for ${parameter##} expansion do not match existing practice, which is to treat it as ${parameter##word} with word as an empty string. The other requested changes also match existing practice and should be done at the same time for consistency. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- Make the changes suggested by the submitter. |
|
|
Interpretation proposed 16 June 2011 for final 30 day review |
|
|
The interpretation is now approved. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2011-06-03 18:23 | eblake | New Issue | |
| 2011-06-03 18:23 | eblake | Status | New => Under Review |
| 2011-06-03 18:23 | eblake | Assigned To | => ajosey |
| 2011-06-03 18:23 | eblake | Name | => Eric Blake |
| 2011-06-03 18:23 | eblake | Organization | => Red Hat |
| 2011-06-03 18:23 | eblake | User Reference | => ebb.expansion |
| 2011-06-03 18:23 | eblake | Section | => 2.6.2 Parameter Expansion |
| 2011-06-03 18:23 | eblake | Page Number | => 2306 |
| 2011-06-03 18:23 | eblake | Line Number | => 72719 |
| 2011-06-03 18:23 | eblake | Interp Status | => --- |
| 2011-06-03 18:24 | eblake | Relationship added | related to 0000417 |
| 2011-06-03 18:24 | eblake | Relationship added | related to 0000155 |
| 2011-06-03 18:29 | eblake | Desired Action Updated | |
| 2011-06-15 15:12 | msbrown | Tag Attached: issue8 | |
| 2011-06-15 15:12 | msbrown | Resolution | Open => Future Enhancement |
| 2011-06-15 15:19 | msbrown | Tag Detached: issue8 | |
| 2011-06-15 15:19 | msbrown | Tag Attached: tc1-2008 | |
| 2011-06-15 15:23 | Don Cragun | Interp Status | --- => Pending |
| 2011-06-15 15:23 | Don Cragun | Note Added: 0000837 | |
| 2011-06-15 15:23 | Don Cragun | Status | Under Review => Interpretation Required |
| 2011-06-15 15:23 | Don Cragun | Resolution | Future Enhancement => Accepted |
| 2011-06-15 15:24 | Don Cragun | Final Accepted Text | => See 0000457:0000837 |
| 2011-06-15 15:29 | Don Cragun | Note Edited: 0000837 | |
| 2011-06-15 15:40 | Don Cragun | Note Edited: 0000837 | |
| 2011-06-15 15:47 | eblake | Desired Action Updated | |
| 2011-06-16 10:19 | ajosey | Interp Status | Pending => Proposed |
| 2011-06-16 10:19 | ajosey | Note Added: 0000856 | |
| 2011-07-29 06:11 | ajosey | Interp Status | Proposed => Approved |
| 2011-07-29 06:11 | ajosey | Note Added: 0000896 | |
| 2013-04-16 13:06 | ajosey | Status | Interpretation Required => Closed |
