0000221: poor wording about even quotes in double quoted parameter expansion

Notes
(0000399) geoffclare (manager) 2010-03-18 16:23	Here is a new proposal based on discussions on the mailing list. Change Within the string of characters from an enclosed "${" to the matching '}', an even number of unescaped double-quotes or single-quotes, if any, shall occur. A preceding <backslash> character shall be used to escape a literal '{' or '}' . The rule in [xref to 2.6.2 Parameter Expansion] shall be used to determine the matching '}'. to For the four varieties of parameter expansion that provide for substring processing (see [xref to 2.6.2 Parameter Expansion]), within the string of characters from an enclosed "${" to the matching '}', the double-quotes within which the expansion occurs shall have no effect on the handling of any special characters. For parameter expansions other than the four varieties that provide for substring processing, within the string of characters from an enclosed "${" to the matching '}', the double-quotes within which the expansion occurs shall preserve the literal value of all characters, with the exception of the characters double-quote, backquote, <dollar-sign>, and <backslash>. If any unescaped double-quote characters occur within the string, other than in embedded command substitutions, the behavior is unspecified. The backquote and <dollar-sign> characters shall follow the same rules as for characters in double-quotes described in this section. The <backslash> character shall follow the same rules as for characters in double-quotes described in this section except that it shall additionally retain its special meaning as an escape character when followed by '}' and this shall prevent the escaped '}' from being considered when determining the matching '}' (using the rule in [xref to 2.6.2 Parameter Expansion]). At page 2306 line 72702 section 2.6.2 delete The matching closing brace shall be determined by counting brace levels, skipping over enclosed quoted strings, and command substitutions. At page 2306 line 72721 section 2.6.2 change the expansion of word shall be assigned to parameter to quote removal shall be performed on the expansion of word and the result shall be assigned to parameter At page 2311 line 72905 section 2.6.7 change unless they have themselves been quoted to unless their literal values were preserved (by being quoted themselves) Cross-volume changes to XRAT... At page 3649 line 124027-124050 section C.2.2.3 change The requirement that double-quotes be matched inside "${...}" within double-quotes [...] rather than: ${foo-"'}'"} to Shell implementations differ widely in their handling of unescaped double-quote characters inside "${...}" (except for the four substring-processing variants). Hence this standard leaves the behavior unspecified. Single-quotes are ordinary characters in this context, and so cannot be used to quote a '}' within "${...}". However, <backslash> can be used to escape a '}'. For example, the value of foo assigned by the following commands is '}': unset bar foo="${bar-\}}" When <backslash> is used in this way it is a special character and is therefore removed during quote removal, even though it would not be removed in: foo="\}" Differences in processing the "${...}" form led to inconsistencies between the historical System V shell, BSD, and KornShells, and the text in the Shell and Utilities volume of POSIX.1-2008 is an attempt to converge them without breaking too many applications. The only alternative to this compromise between shells would be to make the behavior unspecified not just for unescaped double-quote but also for unescaped single-quote, '{', or '}'. The chosen requirements provide the maximum consistency between normal double-quote behavior and parameter expansion within double-quotes; the only real difference being the ability to escape a '}' with <backslash>. After page 3656 line 124322 section C.2.6.2 add a new paragraph Quote removal is performed when assigning the value in the ${parameter:=word} form of expansion in order that a subsequent expansion of the same parameter produces the same value as the original expansion. That is, the commands: unset parameter foo=${parameter:=word} bar=${parameter} assign the same value to foo and bar. A consequence of this is that the expansions ${parameter:=word} and ${parameter:-word} can produce different results for the same word. For example, with parameter unset or empty: ${parameter:-a\ b} expands to a single field "a b", whereas: ${parameter:=a\ b} expands to two fields 'a' and 'b' (because parameter is assigned the value "a b" before its value is substituted). At page 3660 line 124496 section C.2.6.7 change There is no additional rationale provided for this section. to The golden rule in quote removal is that if a quote character was treated as special in the original word, it is removed; if it was treated as a literal character, it is not removed.

(0000404) Don Cragun (manager) 2010-03-25 15:39 edited on: 2011-11-03 15:11	Interpretation response ------------------------ The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Rationale: ------------- None. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- Make the changes suggested in Note: 0000399

(0000516) ajosey (manager) 2010-07-30 09:42	Comments/objections on the proposed interpretation are due by 31 Aug 2010

(0003296) ajosey (manager) 2016-07-14 15:38	Interpretation approved: 2010-09-03

(0004217) kre (reporter) 2019-01-23 04:44	The accepted text in this note is incorrect (if only there was a version of the final text available rather than having to piece it together out of all of these ancient notes, this might have been detected sooner...) The wording: If any unescaped double-quote characters occur within the string, other than in embedded command substitutions, the behavior is unspecified. is incomplete (the "other than"). The behaviour should still be specified if double quotes occur in an embedded variable expansion using one of the substring operators. That is, this word "${var+${var#"**"}}" does not need to, and should not, become unspecified simply because it has those embedded double quotes (and no, the earlier paragraph which exempts the substring var expansions from the rules does not fix this, here we're deciding upon the validity of the "${var+word}" expansion which is where the "unescaped double quote characters" not "other than in embedded command substitutions" would make the behaviour unspecified. Also, while it is not incorrect as written, I believe that the paragraph that exempts the substring expansions could use a little extra clarity, to make it clear that double-quote is one of the "special characters" whose handling is unaffected (not just '' etc), and that any form of quoting can be used within the pattern word.

Issue History
Date Modified	Username	Field	Change
2010-02-23 10:20	geoffclare	New Issue
2010-02-23 10:20	geoffclare	Status	New => Under Review
2010-02-23 10:20	geoffclare	Assigned To	=> ajosey
2010-02-23 10:20	geoffclare	Name	=> Geoff Clare
2010-02-23 10:20	geoffclare	Organization	=> The Open Group
2010-02-23 10:20	geoffclare	Section	=> 2.2.3
2010-02-23 10:20	geoffclare	Page Number	=> 2298
2010-02-23 10:20	geoffclare	Line Number	=> 72382
2010-02-23 10:20	geoffclare	Interp Status	=> ---
2010-03-18 16:23	geoffclare	Note Added: 0000399
2010-03-25 15:39	Don Cragun	Note Added: 0000404
2010-03-25 15:40	Don Cragun	Interp Status	--- => Pending
2010-03-25 15:40	Don Cragun	Final Accepted Text	=> Note: 0000404
2010-03-25 15:40	Don Cragun	Resolution	Open => Accepted As Marked
2010-07-30 09:41	ajosey	Note Edited: 0000404
2010-07-30 09:42	ajosey	Interp Status	Pending => Proposed
2010-07-30 09:42	ajosey	Note Added: 0000516
2010-07-30 09:42	ajosey	Status	Under Review => Interpretation Required
2010-09-03 06:21	ajosey	Interp Status	Proposed => Approved
2010-09-24 11:14	geoffclare	Tag Attached: issue8
2011-10-21 10:44	geoffclare	Relationship added	related to 0000500
2011-11-03 15:11	Don Cragun	Note Edited: 0000404
2015-09-18 08:34	geoffclare	Relationship added	related to 0000985
2015-10-15 15:38	nick	Relationship added	related to 0000989
2016-07-14 15:38	ajosey	Note Added: 0003296
2016-07-14 15:56	Don Cragun	Relationship replaced	has duplicate 0000989
2019-01-23 04:44	kre	Note Added: 0004217
2020-01-31 12:07	geoffclare	Status	Interpretation Required => Applied
2024-06-11 08:53	agadmin	Status	Applied => Closed

Aardvark Mark IV