Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000654 [1003.1(2013)/Issue7+TC1] Shell and Utilities Objection Clarification Requested 2013-02-07 08:01 2019-06-10 08:55
Reporter rhansen View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Closed  
Name Richard Hansen
Organization
User Reference
Section 2.9.1, 2.9.5, 2.14 (intro), description of 'set -a' option in 2.14, description of the 'export' special built-in utility in 2.14, XBD 4.21
Page Number 2339, 2347, 2356, 2372, 2380, 118
Line Number 74354-74355, 74675-74676, 75105-75106, 75584-75587, 75814-75817, 3292-3296
Interp Status Approved
Final Accepted Text See Note: 0001562.
Summary 0000654: unclear behavior of in-line variable assignments preceding functions, special built-ins
Description The behavior of in-line variable assignments preceding a command is
unclear when the command is a function or special built-in utility. For
example, the intended output of the following script is not well specified:

    showfoo() {
        printf %s "foo=${foo-<unset>}"
        sh -c 'printf %s\\n " foo=${foo-<unset>}"'
    }
    unset foo
    showfoo
    foo=foo showfoo
    showfoo

Specific points:

  * it is unclear whether the variable assignment should persist
    after the function or special built-in utility returns
  * it is unclear whether the variable should be marked for export
    during the execution of the function or special built-in utility
  * if the variable assignment is intended to persist after the
    function or special built-in utility returns, it is unclear
    whether the variable should be marked for export after the utility
    or special built-in utility returns

Issue7+TC1 Section 2.9.5 (page 2347 lines 74675-74676) says:

    When a function is executed, it shall have the syntax-error and
    variable-assignment properties described for special built-in
    utilities in the enumerated list at the beginning of Section 2.14.

Note the phrase "when a function is executed". Does "when a function is
executed" include the in-line variable assignments preceding the
invocation of the function? If so, the text in Section 2.14 becomes
relevant. Issue7+TC1 Section 2.14 (page 2356 lines 75105-75106) says:

    2. Variable assignments specified with special built-in utilities
       remain in effect after the built-in completes; this shall not
       be the case with a regular built-in or other utility.

In this text, the meaning of "specified with" is unclear. Originally I
interpreted "specified with" to refer to assignments in the
specification of the special built-in utility (e.g., assignments made by
'export' or in code passed to 'eval'), not in-line assignments preceding
a special built-in utility.

Before TC1, Section 2.9.1 said:

    If no command name results, variable assignments shall affect the
    current execution environment. Otherwise, the variable
    assignments shall be exported for the execution environment of the
    command and shall not affect the current execution environment
    (except for special built-ins).

The meaning of this was unclear; the exception in the parenthetical
could apply to the whole sentence or to just the "shall not affect the
current execution environment" part, and the meaning of "when a function
is executed" in Section 2.9.5 would determine whether the exception
applied to functions.

In TC1, this is more clear. Issue7+TC1 Section 2.9.1 (page 2339 lines
74354-74355) says:

    If no command name results, or if the command name is a special
    built-in or function, variable assignments shall affect the
    current execution environment.

This change, plus some of the text in the description of the 'set -a'
option, leads me to believe that the specification intended to specify
the following behavior for variable assignments preceding special
built-in utilities:

  * the variables do not gain the export attribute (although if the
    export attribute was already set it remains set) except with 'set
    -a'
  * the assignments affect the current shell execution environment
    (they persist after the utility or function returns)

Issue7+TC1 Section 2.14 'set' (page 2380 lines 75813-75817) says:

    If the assignment precedes a utility name in a command, the export
    attribute shall not persist in the current execution environment
    after the utility completes, with the exception that preceding one
    of the special built-in utilities causes the export attribute to
    persist after the built-in has completed.

This suggests that when the '-a' option is on, variable assignments
preceding a special built-in utility get the export attribute during the
duration of utility execution and the attribute persists after the
built-in returns, which implies that the assignment persists after the
built-in completes. It does not indicate what should happen if the '-a'
option is off.

However, I don't believe this same behavior was intended for functions.
 Issue7+TC1 XBD 4.21 "Utility" (page 118 lines 3292-3296) says:

    The system may implement certain utilities as shell functions (see
    XCU Section 2.9.5, on page 2346) or built-in utilities, but only
    an application that is aware of the command search order (as
    described in XCU Section 2.9.1.1, on page 2339) or of performance
    characteristics can discern differences between the behavior of
    such a function or built-in utility and that of an executable
    file.

This makes me believe that assignments preceeding functions should
behave like assignments preceding regular utilities. If they behave
like they do with special built-in utilities, then standard utilities
can't be implemented as functions in a transparent manner (unless there
are two kinds of functions -- one only available to the implementation
for use in implementing utilities).

In addition, if assignments before functions were treated like
assignments before special built-in utilities, it would be impossible to
write a function that transparently wraps a utility to do some
before/after automation or perform a transformation on the arguments.
For example, the two commands in this script would not produce the same
output (assuming GNU ls):

    ll() { ls -Fla "$@"; }
    BLOCKSIZE=\'1 ls -Fla file
    BLOCKSIZE=\'1 ll file

Also, only two implementations (that I am aware of) treat assignments
preceding functions like assignments preceding special built-in
utilities: ksh93 and dash. The following is the output of different
shells with the following script:

    showfoo() {
        printf %s "foo=${foo-<unset>}"
        sh -c 'printf %s\\n " foo=${foo-<unset>}"'
    }
    unset foo
    showfoo
    foo=foo showfoo
    showfoo

  * bash:

        foo=<unset> foo=<unset>
        foo=foo foo=foo
        foo=<unset> foo=<unset>

    (this is the only correct behavior in my opinion)

  * bash --posix, mksh:

        foo=<unset> foo=<unset>
        foo=foo foo=foo
        foo=foo foo=foo

  * dash, ksh93:

        foo=<unset> foo=<unset>
        foo=foo foo=<unset>
        foo=foo foo=<unset>

  * NetBSD's ash:

        foo=<unset> foo=<unset>
        foo=foo foo=<unset>
        foo=<unset> foo=<unset>

  * pdksh:

        foo=<unset> foo=<unset>
        foo=foo foo=foo
        foo=foo foo=<unset>

  * Bourne shell:

        foo=<unset> foo=<unset>
        foo=<unset> foo=<unset>
        foo=<unset> foo=<unset>
Desired Action For issue 7 tc2, given the lack of consistency among implementations, clean up the imprecise wording in 2.9.1, 2.9.5, and 2.14 to ensure that the following points are clear (precise suggested wording will be provided later):

  - an in-line variable assignment preceding a function or special
    built-in utility invocation is in the environment of the function
    or special built-in utility during its execution (at least)

  - the following is unspecified:

    * whether the variable is marked for export during the execution
      of the function or special built-in utility

    * whether the variable assignment persists after the function or
      special built-in utility returns

    * if the variable assignment does persist, whether the variable is
      exported after the function or special built-in utility returns

For issue 8, if implementors agree, I suggest something like this:

  - Dynamically scoped: The binding (value and attributes)
    temporarily affects the current execution environment. If the
    variable is not modified, the binding lasts only until the
    function or special built-in utility returns (on return, the
    temporary binding goes away and the variable's value and
    attributes are as they were before the command).

  - Exported: The temporary variable binding has the export attribute
    set (until cleared by the function or special built-in utility, if
    it chooses to do so).

  - Modification makes the binding persist: Any modification to the
    variable binding (setting the export attribute even if it is
    already set, clearing the export attribute even if it is already
    clear, setting the readonly attribute even if it is already set,
    setting the value even if it is already set to the same value,
    unsetting the variable even if it is already unset) by a function
    or special built-in utility causes the binding (with whatever
    attributes it has after modification) to persist after the
    function or special built-in utility returns. For example:

        foo() { bar=baz; }
        unset bar
        bar=bif foo
        printf %s\\n "${bar}"
        sh -c 'printf %s\\n "${bar}"'

    should output:

        baz
        baz
Tags tc2-2008
Attached Files

- Relationships
related to 0000352Closedajosey 1003.1(2008)/Issue 7 behavior when assignments precede export 
related to 0000854Applied 1003.1(2013)/Issue7+TC1 requirement for additional built-in utilities to be searched for via $PATH was not and is not existing practice 
related to 0001009Applied 1003.1(2013)/Issue7+TC1 "remain in affect after" and "during the execution" are problematic when built-in/function modifies the variable 

-  Notes
(0001484)
rhansen (manager)
2013-03-07 23:09

 Relevant discussion threads:
  http://thread.gmane.org/gmane.comp.standards.posix.austin.general/6728 [^]
  http://thread.gmane.org/gmane.comp.standards.posix.austin.general/6922 [^]
(0001485)
rhansen (manager)
2013-03-07 23:26
edited on: 2016-05-26 15:07

 There is another issue spotted by Robert Elz:

XBD 4.21 (page 118 lines 3267-3271) says:
The system may implement certain utilities as shell functions (see XCU Section 2.9.5, on page 2324) or built-in utilities, but only an application that is aware of the command search order (as described in XCU Section 2.9.1.1, on page 2317) or of performance characteristics can discern differences between the behavior of such a function or built-in utility and that of an executable file.
If the behavior of variable assignments before functions does not match the behavior of variable assignments before non-function non-special utilities, then an application is able to discern the difference between a non-function non-special utility and a utility implemented as a function. As far I can tell, this would violate XBD 4.21.

This implies that either assignments before functions should behave like assignments before utilities, or function-based utilities are a distinct type of function.
(0001559)
rhansen (manager)
2013-05-02 04:39
edited on: 2013-05-02 15:11

 Proposed changes:

In Issue7+TC1 XCU Section 2.9.1 at page 2339 lines 74354-74361, change:

    If no command name results, or if the command name is a special
    built-in or function, variable assignments shall affect the
    current execution environment. Otherwise, the variable
    assignments shall be exported for the execution environment of the
    command and shall not affect the current execution environment
    except as a side-effect of the expansions performed in step 4. In
    this case it is unspecified:

      * Whether or not the assignments are visible for subsequent
        expansions in step 4

      * Whether variable assignments made as side-effects of these
        expansions are visible for subsequent expansions in step 4, or
        in the current shell execution environment, or both

to:

    Variable assignments shall be performed as follows:

      * If no command name results, variable assignments shall affect
        the current execution environment.

      * If the command name is not a special built-in utility or
        function, the variable assignments shall be exported for the
        execution environment of the command and shall not affect the
        current execution environment except as a side-effect of the
        expansions performed in step 4. In this case it is
        unspecified:

          - Whether or not the assignments are visible for subsequent
            expansions in step 4

          - Whether variable assignments made as side-effects of these
            expansions are visible for subsequent expansions in step
            4, or in the current shell execution environment, or both

      * If the command name is a standard utility implemented as a
        function (see XBD Section 4.21), the effect of variable
        assignments shall be as if the utility was not implemented as
        a function.

      * If the command name is a special built-in utility, variable
        assignments shall affect the current execution environment.
        Unless the 'set -a' option is on (see the 'set' special
        built-in utility in Section 2.14), it is unspecified:

          - Whether or not the variables gain the export
            attribute during the execution of the special built-in
            utility

          - Whether or not export attributes gained as a result of the
            variable assignments persist after the completion of
            the special built-in utility

      * If the command name is a function that is not a standard
        utility implemented as a function, variable assignments shall
        affect the current execution environment during the execution
        of the function. It is unspecified:

          - Whether or not the variable assignments persist
            after the completion of the function

          - Whether or not the variables gain the export
            attribute during the execution of the function

          - Whether or not export attributes gained as a result of the
            variable assignments persist after the completion of
            the function (if variable assignments persist after the
            completion of the function)

In Issue7+TC1 XCU Section 2.9.5 at page 2347 lines 74675-74676, change:

    When a function is executed, it shall have the syntax-error and
    variable-assignment properties described for special built-in
    utilities in the enumerated list at the beginning of Section 2.14.

to:

    When a function is executed, it shall have the syntax-error
    properties described for special built-in utilities in the first
    item in the enumerated list at the beginning of Section 2.14.

In Issue7+TC1 XCU Section 2.14 at page 2356 lines 75105-75106, change:

      2. Variable assignments specified with special built-in
         utilities remain in effect after the built-in completes; this
         shall not be the case with a regular built-in or other
         utility.

to:

      2. As described in Section 2.9.1, variable assignments preceding
         the invocation of a special built-in utility remain in effect
         after the built-in completes; this shall not be the case with
         a regular built-in or other utility.

In Issue7+TC1 XCU Section 2.14 'export' at page 2372 lines 75584-75587, change:

    When no arguments are given, the results are unspecified. If a
    variable assignment precedes the command name of export but that
    variable is not also listed as an operand of export, then that
    variable shall be set in the current shell execution environment
    after the completion of the export command, but it is unspecified
    whether that variable is marked for export.

to:

    When no arguments are given, the results are unspecified.

(Note that this change reverts the changes made for issue #352, but the changes for XCU Section 2.9.1 result in no net change in the specified behavior of the 'export' special built-in utility.)
(0001560)
Don Cragun (manager)
2013-05-02 05:49

 Moved from Issue 7 to Issue 7 + TC1 at the request of the submitter.
(0001562)
Don Cragun (manager)
2013-05-02 16:29

 Interpretation response
------------------------
The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.

Rationale:
-------------
None.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Make the changes proposed in Note: 0001559
(0001799)
ajosey (manager)
2013-09-06 04:51

 Interpretation Proposed 6 Sep 2013
(0001884)
ajosey (manager)
2013-10-14 13:04

 Interpretation approved 14 October 2013

- Issue History
Date Modified Username Field Change
2013-02-07 08:01 rhansen New Issue
2013-02-07 08:01 rhansen Status New => Under Review
2013-02-07 08:01 rhansen Assigned To => ajosey
2013-02-07 08:01 rhansen Name => Richard Hansen
2013-02-07 08:01 rhansen Section => 2.9.1, 2.9.5, 2.14, description of 'set -a' option in 2.14
2013-02-07 08:01 rhansen Page Number => 2317, 2324, 2334, 2357
2013-02-07 08:01 rhansen Line Number => 73102-73104, 73126-73128, 73139-73140, 73391-73393, 73826-73827, 74513-74516
2013-02-07 08:05 rhansen Relationship added related to 0000352
2013-03-07 23:09 rhansen Note Added: 0001484
2013-03-07 23:26 rhansen Note Added: 0001485
2013-05-02 02:24 rhansen Note Edited: 0001485
2013-05-02 02:24 rhansen Note Edited: 0001485
2013-05-02 04:39 rhansen Note Added: 0001559
2013-05-02 05:48 Don Cragun Project 1003.1(2008)/Issue 7 => 1003.1(2013)/Issue7+TC1
2013-05-02 05:49 Don Cragun Note Added: 0001560
2013-05-02 15:11 rhansen Note Edited: 0001559
2013-05-02 15:16 Don Cragun Section 2.9.1, 2.9.5, 2.14, description of 'set -a' option in 2.14 => 2.9.1, 2.9.5, 2.14 (intro), description of 'set -a' option in 2.14, description of the 'export' special built-in utility in 2.14, XBD 4.21
2013-05-02 15:16 Don Cragun Page Number 2317, 2324, 2334, 2357 => 2339, 2347, 2356, 2372, 2380, 118
2013-05-02 15:16 Don Cragun Line Number 73102-73104, 73126-73128, 73139-73140, 73391-73393, 73826-73827, 74513-74516 => 74354-74355, 74675-74676, 75105-75106, 75584-75587, 75814-75817, 3292-3296
2013-05-02 15:16 Don Cragun Interp Status => ---
2013-05-02 16:17 Don Cragun Description Updated
2013-05-02 16:29 Don Cragun Interp Status --- => Pending
2013-05-02 16:29 Don Cragun Note Added: 0001562
2013-05-02 16:29 Don Cragun Status Under Review => Interpretation Required
2013-05-02 16:29 Don Cragun Resolution Open => Accepted As Marked
2013-05-02 16:29 Don Cragun Final Accepted Text => See Note: 0001562.
2013-05-02 16:30 Don Cragun Tag Attached: tc2-2008
2013-05-02 20:41 msbrown Description Updated
2013-09-06 04:51 ajosey Interp Status Pending => Proposed
2013-09-06 04:51 ajosey Note Added: 0001799
2013-10-14 13:04 ajosey Interp Status Proposed => Approved
2013-10-14 13:04 ajosey Note Added: 0001884
2015-01-09 17:43 eblake Relationship added related to 0000854
2015-11-20 01:01 rhansen Relationship added related to 0001009
2016-05-26 15:07 rhansen Note Edited: 0001485
2016-05-26 15:07 rhansen Note Edited: 0001485
2019-06-10 08:55 agadmin Status Interpretation Required => Closed

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker