Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000457 [1003.1(2008)/Issue 7] Shell and Utilities Objection Omission 2011-06-03 18:23 2013-04-16 13:06
Reporter eblake View Status public  
Assigned To ajosey
Priority normal Resolution Accepted  
Status Closed  
Name Eric Blake
Organization Red Hat
User Reference ebb.expansion
Section 2.6.2 Parameter Expansion
Page Number 2306
Line Number 72719
Interp Status Approved
Final Accepted Text See Note: 0000837
Summary 0000457: omitted word in parameter expansions
Description On all shells that I tried (ksh88, ksh93, bash, dash, zsh, mksh, all of
which are trying to comply with POSIX; as well as Solaris /bin/sh for
historical background), the ${parameter:-word} and ${parameter:=word}
forms with an empty word were parsed without problems, and with useful
effects. I tested with:

$ $candidate -c 'echo ${foo-unset}; : ${foo=}; echo ${foo+set}'
unset
set
$ $candidate -c 'echo ${foo-unset}; : ${foo:=}; echo ${foo+set}'
unset
set
$ $candidate -c 'echo .${foo-}.; foo=a; echo ${foo-}'
..
a
$ $candidate -c 'echo .${foo:-}.; foo=a; echo ${foo:-}'
..
a

Additionally, all shells understood an omitted word in the
${parameter:+word} form, although that always results in an
empty string. However, it is worth standardizing if for no
other reason than to guarantee this result:

$candidate -cn '${foo+}' && echo $?
0

when contrasted with other syntax failures like ${+}.

Furthermore, among all shells that understand the forms
${parameterOPword} for OP in %, %%, #, and ##, an omitted word
resulted in the use of an empty string as the pattern, which
ends up not removing any characters from the parameter expansion,
or as a way of detecting unset variables when 'set -u' is enabled
(actually, bash 4.1 and zsh 4.3.10 were buggy in this respect,
but that bug was fixed in bash 4.2 and zsh 4.3.12).

In fact, the standard requires that ${foo##} be parsed as
${parameter#word} with word of #, which results in the output of
foo with the shortest match to the pattern # removed, but all
shells tested instead treated it like ${parameter##} with an
optional word, and output ${foo} instead (likewise for %%):

$ $candidate -c 'foo=##; echo ${foo#"#"} ${foo##} ${foo###}'
# ## #
$ $candidate -c 'foo=%%; echo ${foo%"%"} ${foo%%} ${foo%%%}'
% %% %

Therefore, since the standard disagrees with existing practice,
this is worth fixing.

Conversely, allowing an omitted word introduces an ambiguity in
${##} (length of $# vs. shortest-prefix removal of empty string
pattern from $#); however, behavior varies wildly here (generally
in the form of implementations trying to parse ${##""} as
${#parameter}, but #"" is not a parameter), so the best we can do
is leave prefix- and suffix-removal forms unspecified on $#.

$ bash -c 'set a b; echo ${##} ${##""} ${###} ${###""}'
1 2 2 2
$ dash -c 'set a b; echo ${##} ${##""} ${###} ${###""}'
1 2 2 2
$ zsh -c 'set a b; echo ${##} ${##""} ${###} ${###""}'
2 2 2 2
$ mksh -c 'set a b; echo ${##} ${##""} ${###} ${###""}'
mksh: ${##""}: bad substitution
$ mksh -c 'set a b; echo ${##} ${###} ${###""}'
mksh: ${###}: bad substitution
$ mksh -c 'set a b; echo ${##} ${###""}'
mksh: ${###""}: bad substitution
$ mksh -c 'set a b; echo ${##}'
1
$ ksh -c 'set a b; echo ${##} ${##""} ${###} ${###""}'
ksh: syntax error at line 1: `"' unexpected
$ ksh -c 'set a b; echo ${##} ${###} ${###""}'
1 2 2

If this issue is accepted in isolation, it would introduce an ambiguity
in ${#-} (length of $- vs. empty string in (impossible) case that $#
is unset); all shells I tested favored length of $-. However, when
coupled with the solution in 0000417, there is no problem, that bug
was intentionally worded to require ${#-} to be the length of $-, and
to provide a word when probing whether $# is unset.
Desired Action At line 72719, change:

${parameter:−word} Use Default Values. If parameter is unset or null, the
  expansion of word shall be substituted; otherwise, the value of
  parameter shall be substituted.
${parameter:=word} Assign Default Values. If parameter is unset or null,
  the expansion of word shall be assigned to parameter.

to:

${parameter:−[word]} Use Default Values. If parameter is unset or null,
  the expansion of word (or an empty string if word is omitted) shall be
  substituted; otherwise, the value of parameter shall be substituted.
${parameter:=[word]} Assign Default Values. If parameter is unset or
  null, the expansion of word (or an empty string if word is omitted)
  shall be assigned to parameter.

At line 72730, change:

${parameter:+word} Use Alternative Value. If parameter is unset or null,
  null shall be substituted; otherwise, the expansion of word shall be
  substituted.

to:

${parameter:+[word]} Use Alternative Value. If parameter is unset or
  null, null shall be substituted; otherwise, the expansion of word (or
  an empty string if word is omitted) shall be substituted.

At line 72749, add a sentence to ${#parameter}:

If parameter is unset and set -u is in effect, the expansion shall fail.

At line 75752, change:

If parameter is ’*’ or ’@’, the result of the expansion is unspecified.

to:

If parameter is ’#’, ’*’, or ’@’, the result of the expansion is
unspecified. If parameter is unset and set -u is in effect, the
expansion shall fail.

At line 72755, add a sentence:

In each variety, if word is omitted, the empty pattern shall be used.

At line 72756, change "${parameter%word}" to "${parameter%[word]}".

At line 72758, add a sentence:

If present, word shall not begin with an unquoted '%'.

At line 72759, change "${parameter%%word}" to "${parameter%%[word]}".

At line 72762, change "${parameter#word}" to "${parameter#[word]}".

At line 72764, add a sentence:

If present, word shall not begin with an unquoted '#'.

At line 72765, change "${parameter##word}" to "${parameter##[word]}".
Tags tc1-2008
Attached Files

- Relationships
related to 0000417Closedajosey ${#?} is ambiguous 
related to 0000155Closedajosey set -u should require an error when $1 etc. are unset 

-  Notes
(0000837)
Don Cragun (manager)
2011-06-15 15:23
edited on: 2011-06-15 15:40

Interpretation response
------------------------
The standard states that "word" must be present in these cases, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
The standard's requirements for ${parameter##} expansion do not match
existing practice, which is to treat it as ${parameter##word} with word
as an empty string. The other requested changes also match existing
practice and should be done at the same time for consistency.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Make the changes suggested by the submitter.

(0000856)
ajosey (manager)
2011-06-16 10:19

Interpretation proposed 16 June 2011 for final 30 day review
(0000896)
ajosey (manager)
2011-07-29 06:11

The interpretation is now approved.

- Issue History
Date Modified Username Field Change
2011-06-03 18:23 eblake New Issue
2011-06-03 18:23 eblake Status New => Under Review
2011-06-03 18:23 eblake Assigned To => ajosey
2011-06-03 18:23 eblake Name => Eric Blake
2011-06-03 18:23 eblake Organization => Red Hat
2011-06-03 18:23 eblake User Reference => ebb.expansion
2011-06-03 18:23 eblake Section => 2.6.2 Parameter Expansion
2011-06-03 18:23 eblake Page Number => 2306
2011-06-03 18:23 eblake Line Number => 72719
2011-06-03 18:23 eblake Interp Status => ---
2011-06-03 18:24 eblake Relationship added related to 0000417
2011-06-03 18:24 eblake Relationship added related to 0000155
2011-06-03 18:29 eblake Desired Action Updated
2011-06-15 15:12 msbrown Tag Attached: issue8
2011-06-15 15:12 msbrown Resolution Open => Future Enhancement
2011-06-15 15:19 msbrown Tag Detached: issue8
2011-06-15 15:19 msbrown Tag Attached: tc1-2008
2011-06-15 15:23 Don Cragun Interp Status --- => Pending
2011-06-15 15:23 Don Cragun Note Added: 0000837
2011-06-15 15:23 Don Cragun Status Under Review => Interpretation Required
2011-06-15 15:23 Don Cragun Resolution Future Enhancement => Accepted
2011-06-15 15:24 Don Cragun Final Accepted Text => See Note: 0000837
2011-06-15 15:29 Don Cragun Note Edited: 0000837
2011-06-15 15:40 Don Cragun Note Edited: 0000837
2011-06-15 15:47 eblake Desired Action Updated
2011-06-16 10:19 ajosey Interp Status Pending => Proposed
2011-06-16 10:19 ajosey Note Added: 0000856
2011-07-29 06:11 ajosey Interp Status Proposed => Approved
2011-07-29 06:11 ajosey Note Added: 0000896
2013-04-16 13:06 ajosey Status Interpretation Required => Closed


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker