Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000221 [1003.1(2008)/Issue 7] Shell and Utilities Objection Error 2010-02-23 10:20 2020-01-31 12:07
Reporter geoffclare View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Applied  
Name Geoff Clare
Organization The Open Group
User Reference
Section 2.2.3
Page Number 2298
Line Number 72382
Interp Status Approved
Final Accepted Text Note: 0000404
Summary 0000221: poor wording about even quotes in double quoted parameter expansion
Description Section 2.2.3 Double-Quotes says:

    Within the string of characters from an enclosed "${" to the
    matching '}', an even number of unescaped double-quotes or
    single-quotes, if any, shall occur.

This wording is poor for two reasons:

1. It is a requirement on the application, but is not worded as such.

2. It is ambiguous, in that it could be taken to mean the total
number of unescaped single-quotes and double-quotes, taken together,
must be even. The intention is for the requirement to apply
separately to single-quotes and double-quotes.
Desired Action Change

    an even number of unescaped double-quotes or single-quotes, if
    any, shall occur.

to

    the application shall ensure that an even number of unescaped
    double-quotes, if any, occurs and an even number of unescaped
    single-quotes, if any, occurs.
Tags issue8
Attached Files

- Relationships
has duplicate 0000989Closedajosey 1003.1(2008)/Issue 7 Contradicting claims for shell parameter substitution 
related to 0000500Appliedajosey 1003.1(2008)/Issue 7 Special meaning of <backslash> inside double quotes 
related to 0000985Applied 1003.1(2013)/Issue7+TC1 quote removal missing from case statement patterns and alternative expansions 

-  Notes
(0000399)
geoffclare (manager)
2010-03-18 16:23

Here is a new proposal based on discussions on the mailing list.

Change

    Within the string of characters from an enclosed "${" to the
    matching '}', an even number of unescaped double-quotes or
    single-quotes, if any, shall occur. A preceding <backslash>
    character shall be used to escape a literal '{' or '}' . The rule
    in [xref to 2.6.2 Parameter Expansion] shall be used to determine
    the matching '}'.

to

    For the four varieties of parameter expansion that provide for
    substring processing (see [xref to 2.6.2 Parameter Expansion]),
    within the string of characters from an enclosed "${" to the
    matching '}', the double-quotes within which the expansion occurs
    shall have no effect on the handling of any special characters.

    For parameter expansions other than the four varieties that provide
    for substring processing, within the string of characters from an
    enclosed "${" to the matching '}', the double-quotes within which
    the expansion occurs shall preserve the literal value of all
    characters, with the exception of the characters double-quote,
    backquote, <dollar-sign>, and <backslash>. If any unescaped
    double-quote characters occur within the string, other than in
    embedded command substitutions, the behavior is unspecified.
    The backquote and <dollar-sign> characters shall follow the same
    rules as for characters in double-quotes described in this section.
    The <backslash> character shall follow the same rules as for
    characters in double-quotes described in this section except that
    it shall additionally retain its special meaning as an escape
    character when followed by '}' and this shall prevent the escaped
    '}' from being considered when determining the matching '}' (using
    the rule in [xref to 2.6.2 Parameter Expansion]).

At page 2306 line 72702 section 2.6.2 delete

    The matching closing brace shall be determined by counting brace
    levels, skipping over enclosed quoted strings, and command
    substitutions.

At page 2306 line 72721 section 2.6.2 change

    the expansion of word shall be assigned to parameter

to

    quote removal shall be performed on the expansion of word and the
    result shall be assigned to parameter

At page 2311 line 72905 section 2.6.7 change

    unless they have themselves been quoted

to

    unless their literal values were preserved (by being quoted
    themselves)

Cross-volume changes to XRAT...

At page 3649 line 124027-124050 section C.2.2.3 change

    The requirement that double-quotes be matched inside "${...}"
    within double-quotes
    [...]
    rather than:
    ${foo-"'}'"}

to

    Shell implementations differ widely in their handling of unescaped
    double-quote characters inside "${...}" (except for the four
    substring-processing variants). Hence this standard leaves the
    behavior unspecified. Single-quotes are ordinary characters in
    this context, and so cannot be used to quote a '}' within "${...}".
    However, <backslash> can be used to escape a '}'. For example,
    the value of foo assigned by the following commands is '}':

    unset bar
    foo="${bar-\}}"

    When <backslash> is used in this way it is a special character
    and is therefore removed during quote removal, even though it
    would not be removed in:

    foo="\}"

    Differences in processing the "${...}" form led to inconsistencies
    between the historical System V shell, BSD, and KornShells, and
    the text in the Shell and Utilities volume of POSIX.1-2008 is an
    attempt to converge them without breaking too many applications.
    The only alternative to this compromise between shells would be to
    make the behavior unspecified not just for unescaped double-quote
    but also for unescaped single-quote, '{', or '}'. The chosen
    requirements provide the maximum consistency between normal
    double-quote behavior and parameter expansion within
    double-quotes; the only real difference being the ability to
    escape a '}' with <backslash>.

After page 3656 line 124322 section C.2.6.2 add a new paragraph

    Quote removal is performed when assigning the value in the
    ${parameter:=word} form of expansion in order that a subsequent
    expansion of the same parameter produces the same value as the
    original expansion. That is, the commands:

    unset parameter
    foo=${parameter:=word}
    bar=${parameter}

    assign the same value to foo and bar. A consequence of this
    is that the expansions ${parameter:=word} and ${parameter:-word}
    can produce different results for the same word. For example,
    with parameter unset or empty:

    ${parameter:-a\ b}

    expands to a single field "a b", whereas:

    ${parameter:=a\ b}

    expands to two fields 'a' and 'b' (because parameter is assigned
    the value "a b" before its value is substituted).

At page 3660 line 124496 section C.2.6.7 change

    There is no additional rationale provided for this section.

to

    The golden rule in quote removal is that if a quote character was
    treated as special in the original word, it is removed; if it was
    treated as a literal character, it is not removed.
(0000404)
Don Cragun (manager)
2010-03-25 15:39
edited on: 2011-11-03 15:11

Interpretation response
------------------------
The standard is unclear on this issue, and no conformance
distinction can be made between alternative implementations based
on this. This is being referred to the sponsor.

Rationale:
-------------
None.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Make the changes suggested in Note: 0000399

(0000516)
ajosey (manager)
2010-07-30 09:42

Comments/objections on the proposed interpretation are due by 31 Aug 2010
(0003296)
ajosey (manager)
2016-07-14 15:38

Interpretation approved: 2010-09-03
(0004217)
kre (reporter)
2019-01-23 04:44

The accepted text in this note is incorrect (if only there was a version of
the final text available rather than having to piece it together out of all
of these ancient notes, this might have been detected sooner...)

The wording:

    If any unescaped
    double-quote characters occur within the string, other than in
    embedded command substitutions, the behavior is unspecified.

is incomplete (the "other than"). The behaviour should still be
specified if double quotes occur in an embedded variable expansion
using one of the substring operators.

That is, this word
          "${var+${var#"***"}}"
does not need to, and should not, become unspecified simply because
it has those embedded double quotes (and no, the earlier paragraph
which exempts the substring var expansions from the rules does not
fix this, here we're deciding upon the validity of the "${var+word}"
expansion which is where the "unescaped double quote characters" not
"other than in embedded command substitutions" would make the behaviour
unspecified.

Also, while it is not incorrect as written, I believe that the paragraph
that exempts the substring expansions could use a little extra clarity,
to make it clear that double-quote is one of the "special characters"
whose handling is unaffected (not just '*' etc), and that any form of
quoting can be used within the pattern word.

- Issue History
Date Modified Username Field Change
2010-02-23 10:20 geoffclare New Issue
2010-02-23 10:20 geoffclare Status New => Under Review
2010-02-23 10:20 geoffclare Assigned To => ajosey
2010-02-23 10:20 geoffclare Name => Geoff Clare
2010-02-23 10:20 geoffclare Organization => The Open Group
2010-02-23 10:20 geoffclare Section => 2.2.3
2010-02-23 10:20 geoffclare Page Number => 2298
2010-02-23 10:20 geoffclare Line Number => 72382
2010-02-23 10:20 geoffclare Interp Status => ---
2010-03-18 16:23 geoffclare Note Added: 0000399
2010-03-25 15:39 Don Cragun Note Added: 0000404
2010-03-25 15:40 Don Cragun Interp Status --- => Pending
2010-03-25 15:40 Don Cragun Final Accepted Text => Note: 0000404
2010-03-25 15:40 Don Cragun Resolution Open => Accepted As Marked
2010-07-30 09:41 ajosey Note Edited: 0000404
2010-07-30 09:42 ajosey Interp Status Pending => Proposed
2010-07-30 09:42 ajosey Note Added: 0000516
2010-07-30 09:42 ajosey Status Under Review => Interpretation Required
2010-09-03 06:21 ajosey Interp Status Proposed => Approved
2010-09-24 11:14 geoffclare Tag Attached: issue8
2011-10-21 10:44 geoffclare Relationship added related to 0000500
2011-11-03 15:11 Don Cragun Note Edited: 0000404
2015-09-18 08:34 geoffclare Relationship added related to 0000985
2015-10-15 15:38 nick Relationship added related to 0000989
2016-07-14 15:38 ajosey Note Added: 0003296
2016-07-14 15:56 Don Cragun Relationship replaced has duplicate 0000989
2019-01-23 04:44 kre Note Added: 0004217
2020-01-31 12:07 geoffclare Status Interpretation Required => Applied


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker