Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000217 [1003.1(2008)/Issue 7] Shell and Utilities Objection Error 2010-02-10 12:21 2013-04-16 13:06
Reporter geoffclare View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Closed  
Name Geoff Clare
Organization The Open Group
User Reference
Section 2.6.3
Page Number 2309
Line Number 72840
Interp Status Approved
Final Accepted Text Note: 0000403
Summary 0000217: The "single subshell" rule for $(( is not sufficient
Description In the description of command substitution the standard says:

    If the command substitution consists of a single subshell, such as:

    $( (command) )

    a conforming application shall separate the "$(" and '(' into two
    tokens (that is, separate them with white space). This is required
    to avoid any ambiguities with arithmetic expansion.

In a discussion on the mailing list in December 2009, Philip Guenther
demonstrated that there are ambiguous cases other than the "single
subshell" case. He gave the example:

    cat=1; EOH=3; echo $(( cat <<EOH
    + ( (
    EOH
    ) && ( cat <<EOH
    ) ) + 1 +
    EOH
    ))

Current shell implementations treat this as an arithmetic expansion.

Thus the requirement on applications quoted above should be extended
to cover all ambiguous cases, not just the "single subshell" case.

The same requirement should also apply to nested subshells, so that
shells which implement the "(( arithmetic expression ))" extension
can apply the same disambiguation rules consistently to $((...)) and
((...)).
Desired Action Change

    If the command substitution consists of a single subshell, such as:

    $( (command) )

    a conforming application shall separate the "$(" and '(' into two
    tokens (that is, separate them with white space). This is required
    to avoid any ambiguities with arithmetic expansion.

to

    If a character sequence beginning with $(( and ending with )) is
    syntactically valid as both an arithmetic expansion and a
    command substitution, the shell shall perform an arithmetic
    expansion. A conforming application shall ensure that it
    separates the "$(" and '(' into two tokens (that is, separate
    them with white space) in a command substitution if the command
    to be substituted is syntactically valid as an arithmetic
    expression. For example, a command substitution containing a
    single subshell could be written as:

    $( (command) )

After page 2321 line 73276 add a new paragraph to the description of
"(compound-list)"

    If a character sequence beginning with (( and ending with )) would
    be syntactically valid as an arithmetic expansion if preceded by
    a '$', shells which implement an extension whereby ((expression))
    is evaluated as an arithmetic expression may treat the sequence
    as an arithmetic evaluation instead of a grouping command. A
    conforming application shall ensure that it separates the two
    leading '(' characters with white space if a grouping command
    beginning with '(' contains a compound-list beginning with '('
    and ending with ')', and the grouping command is syntactically
    valid as an arithmetic expression.

Cross-volume changes to XRAT:

At page 3657 line 124359 section C.2.6.3 change

    The requirement to separate "$(" and '(' when a single subshell is
    command-substituted is to avoid any ambiguities with arithmetic
    expansion.

to

    Arithmetic expansions have precedence over command substitutions.
    That is, if $(( introduces a syntactically valid arithmetic
    expansion, then an arithmetic expansion will be performed.
    Otherwise if $(( introduces a syntactically valid command
    substitution, then a command substitution will be performed.
    If the syntax is valid for neither, then it is unspecified what
    kind of syntax error the shell reports.

    It is recommended that application writers always separate the
    "$(" and '(' with white space when a command substitution starts
    with a subshell, even if it is not a the simple case of a single
    subshell. More complicated ambiguous cases are possible, such
    as:

        $(( cat <<EOH
        + ( (
        EOH
        ) && ( cat <<EOH
        ) ) + 1 +
        EOH
        ))

    This is syntactically valid an arithmetic expansion, with cat and
    EOH as the names of shell variables.

After page 3667 line 124772 section C.2.9.4 add a new paragraph

    The requirement for applications to separate two leading '('
    characters with white space if a grouping command is
    syntactically valid as an arithmetic expression is to
    allow shells which implement the "(( arithmetic expression ))"
    extension to apply the same disambiguation rules consistently
    to $((...)) and ((...)). See [xref to C.2.6.3].
Tags tc1-2008
Attached Files

- Relationships

-  Notes
(0000398)
geoffclare (manager)
2010-03-05 15:38

 Here is an updated proposal which (I hope) addresses the problems
raised on the mailing list.

Change

    If the command substitution consists of a single subshell, such as:

    $( (command) )

    a conforming application shall separate the "$(" and '(' into two
    tokens (that is, separate them with white space). This is required
    to avoid any ambiguities with arithmetic expansion.

to

    The syntax of the shell command language has an ambiguity for
    expansions beginning with "$((", which can introduce an arithmetic
    expansion or a command substitution that starts with a subshell.
    Arithmetic expansion has precedence; that is, the shell shall
    first determine whether it can parse the expansion as an
    arithmetic expansion and shall only parse the expansion as a
    command substitution if it determines that it cannot parse the
    expansion as an arithmetic expansion. The shell need not evaluate
    nested expansions when performing this determination. If it
    encounters the end of input without already having determined that
    it cannot parse the expansion as an arithmetic expansion, the
    shell shall treat the expansion as an incomplete arithmetic
    expansion and report a syntax error. A conforming application
    shall ensure that it separates the "$(" and '(' into two tokens
    (that is, separate them with white space) in a command
    substitution that starts with a subshell. For example, a command
    substitution containing a single subshell could be written as:

    $( (command) )

After page 2321 line 73276 add a new paragraph to the description of
"(compound-list)"

    If a character sequence beginning with "((" would be parsed by the
    shell as an arithmetic expansion if preceded by a '$', shells
    which implement an extension whereby "((expression))" is evaluated
    as an arithmetic expression may treat the "((" as introducing as
    an arithmetic evaluation instead of a grouping command. A
    conforming application shall ensure that it separates the two
    leading '(' characters with white space to prevent the shell
    from performing an arithmetic evaluation.

Cross-volume changes to XRAT:

At page 3657 line 124359 section C.2.6.3 change

    The requirement to separate "$(" and '(' when a single subshell is
    command-substituted is to avoid any ambiguities with arithmetic
    expansion.

to

    Arithmetic expansions have precedence over command substitutions.
    That is, if the shell can parse an expansion beginning with "$(("
    as an arithmetic expansion then it will do so. It will only
    parse the expansion as a command substitution (that starts with
    a subshell) if it determines that it cannot parse the expansion
    as an arithmetic expansion. If the syntax is valid for neither
    type of expansion, then it is unspecified what kind of syntax
    error the shell reports.

    How well the shell performs this determination is a quality of
    implementation issue. Current shell implementations use
    heuristics. In particular, the shell need not evaluate nested
    expansions when determining whether it can parse an expansion
    beginning with "$((" as an arithmetic expansion. For example:

        $((a $op b))

    is always an arithmetic expansion if "$op" expands to, say, '+',
    but if "$op" expands to '(' then the shell might still parse the
    expansion as an arithmetic expansion (resulting in a syntax
    error due to unbalanced parentheses) or it might perform a
    command substitution.

    This standard requires that conforming applications always separate
    the "$(" and '(' with white space when a command substitution starts
    with a subshell. This is because implementations may support
    extensions in arithmetic expressions which could result in the
    shell parsing the input as an arithmetic expansion even though
    a minimally conforming shell would not. For example, many shells
    support arrays with the array index (which can be an expression)
    in square brackets. Therefore, the presence of "myfile[0-9]"
    within an expansion beginning "$((" is no guarantee that it will
    be parsed as a command substitution.

    The ambiguity is not restricted to the simple case of a single
    subshell. More complicated ambiguous cases are possible (even
    with just the standard shell syntax), such as:

        $(( cat <<EOH
        + ( (
        EOH
        ) && ( cat <<EOH
        ) ) + 1 +
        EOH
        ))

    This can be parsed as an arithmetic expansion, with cat and
    EOH as the names of shell variables. Ambiguous cases also
    exist where the end of the expansion is at a different location
    for the arithmetic expansion and the command substitution:

        $((cat <<EOF
        +((((
        EOF
        ) && (
        cat <<EOF
        +
        EOF
        ))

    This is an incomplete arithmetic expansion, but would have been a
    (complete) command substitution if it could not have been parsed
    as an arithmetic expansion. If this expansion occurs at the end
    of input then the shell reports a syntax error; it does not
    parse it as a command substitution.

After page 3667 line 124772 section C.2.9.4 add a new paragraph

    The requirement for conforming applications to separate two
    leading '(' characters with white space if a grouping command
    would be parsed as an arithmetic expansion if preceded by a '$' is
    to allow shells which implement the "(( arithmetic expression ))"
    extension to apply the same disambiguation rules consistently
    to $((...)) and ((...)). See [xref to C.2.6.3].
(0000403)
Don Cragun (manager)
2010-03-25 15:33
edited on: 2010-04-16 10:16

 Interpretation response
------------------------

The standard is unclear on this issue, and no conformance
 distinction can be made between alternative implementations based
 on this. This is being referred to the sponsor.

Rationale
----------
None.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Make the change proposed in Note: 0000398

- Issue History
Date Modified Username Field Change
2010-02-10 12:21 geoffclare New Issue
2010-02-10 12:21 geoffclare Status New => Under Review
2010-02-10 12:21 geoffclare Assigned To => ajosey
2010-02-10 12:21 geoffclare Name => Geoff Clare
2010-02-10 12:21 geoffclare Organization => The Open Group
2010-02-10 12:21 geoffclare Section => 2.6.3
2010-02-10 12:21 geoffclare Page Number => 2309
2010-02-10 12:21 geoffclare Line Number => 72840
2010-02-10 12:21 geoffclare Interp Status => ---
2010-03-05 15:38 geoffclare Note Added: 0000398
2010-03-25 15:33 Don Cragun Interp Status --- => Pending
2010-03-25 15:33 Don Cragun Final Accepted Text => Note: 0000398
2010-03-25 15:33 Don Cragun Note Added: 0000403
2010-03-25 15:33 Don Cragun Status Under Review => Interpretation Required
2010-03-25 15:33 Don Cragun Resolution Open => Accepted As Marked
2010-03-25 15:33 Don Cragun Description Updated
2010-03-25 15:35 Don Cragun Note Edited: 0000403
2010-03-25 15:36 Don Cragun Final Accepted Text Note: 0000398 => Note: 0000403
2010-04-16 10:16 ajosey Note Edited: 0000403
2010-04-16 10:16 ajosey Interp Status Pending => Proposed
2010-05-28 14:03 ajosey Interp Status Proposed => Approved
2010-09-24 17:09 Don Cragun Tag Attached: tc1-2008
2013-04-16 13:06 ajosey Status Interpretation Required => Closed

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker