Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001852 [1003.1(2024)/Issue8] Shell and Utilities Editorial Clarification Requested 2024-08-14 23:38 2024-10-14 12:48
Reporter steffen View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Interpretation Required  
Name steffen
Organization
User Reference
Section 2.52
Page Number 2479
Line Number 80376 ff.
Interp Status Approved
Final Accepted Text Note: 0006874
Summary 0001852: Clarify "$@[:$@:]" (with $# -eq 0)
Description ..where [::] means repetition, aka "$@$@$@" etc.
The standard says

  80376 unspecified. If there are no positional parameters, the expansion of '@' shall generate zero
  80377 fields, even when '@' is within double-quotes; however, if the expansion is embedded
  80378 within a word which contains one or more other parts that expand to a quoted null string,
  80379 these null string(s) shall still produce an empty field, except that if the other parts are all
  80380 within the same double-quotes as the '@', it is unspecified whether the result is zero fields
  80381 or one empty field.

In a test file

  one() { echo "$# 1<$1>"; }
  two() { one "$@"; }
  twox() { one "$@$@"; }
  two
  two x
  twox
  twox x

shells will give different results for the argument-less call to twox().
(Here on my box bash(1) is the only one which expands "$@$@" to nothing.)
Desired Action Please add wording that covers successive "empty field" expansions.
Tags tc1-2024
Attached Files

- Relationships

-  Notes
(0006862)
geoffclare (manager)
2024-08-15 09:30

The standard clearly requires "$@$@" to generate zero fields if there are no positional parameters. The quoted text "if the expansion is embedded within a word which contains one or more other parts that expand to a quoted null string, ..." does not apply because there are no other parts that expand to a quoted null string. (The second $@ is not a candidate for being such a part because "the expansion of '@' shall generate zero fields".)

If this does not match existing behaviour in some shells, we could consider making it unspecified, but the first step should be to alert the maintainers of those shells to the issue and find out whether they are willing to change their shell to comply with the standard.
(0006865)
steffen (reporter)
2024-08-15 18:38

..i'll pump up my balls and repost this to some lists where most of them linger..
(0006866)
McDutchie (reporter)
2024-08-19 20:16

ksh93 does this wrong, with inconsistent behaviour (e.g., for zero PPs, ''"$@" correctly yields one empty field, ''"$@$@" yields zero, and ''"$@$@$@" and on yield one again, but ''''"$@$@$@" yields zero -- etc.).

I found the bug and it will be fixed in the next release (ksh 93u+m/1.0.11).
(0006869)
geoffclare (manager)
2024-09-02 15:21

It is looking like there aren't going to be any objections to keeping the current requirement. I'd propose that we add some new cases to the existing rationale that shows various expansions of $@ (and $*) with no positional parameters.

On XRAT page 3877 line 134466 section C.2.5.2, change:
printf '[%s]\n' foo $@
[foo]
printf '[%s]\n' foo "$@"
[foo]
printf '[%s]\n' foo ''$@
[foo]
[]
printf '[%s]\n' foo ''"$@"
[foo]
[]
printf '[%s]\n' foo "$novar$@$(echo)"
[foo]
[] </tt>(this line of output is optional)<tt>
printf '[%s]\n' foo ''"$novar$@$(echo)"
[foo]
[]

to:
printf '[%s]\n' foo $@
[foo]
printf '[%s]\n' foo "$@"
[foo]
printf '[%s]\n' foo "$@$@"
[foo]
printf '[%s]\n' foo "$@""$@"
[foo]
printf '[%s]\n' foo "$@$*"
[foo]
[] </tt>(this line of output is optional)<tt>
printf '[%s]\n' foo "$@""$*"
[foo]
[]
printf '[%s]\n' foo ''$@
[foo]
[]
printf '[%s]\n' foo ''"$@"
[foo]
[]
printf '[%s]\n' foo "$novar$@$(echo)"
[foo]
[] </tt>(this line of output is optional)<tt>
printf '[%s]\n' foo ''"$novar$@$(echo)"
[foo]
[]
(0006870)
hvd (reporter)
2024-09-02 15:55
edited on: 2024-09-02 16:18

> It is looking like there aren't going to be any objections to keeping the current requirement.

That ignores the response to steffen's mail that it really is not at all clear what the standard is saying in the case of "$@$@" more generally: https://lore.kernel.org/dash/20240827002809.UTh0pE75@steffen%25sdaoden.eu/T/#m2fa3ab6795031f46f30ae668c81afefec2a2f5fd [^]

I also disagree that the standard currently makes it clear that "$@$@" should expand to no fields when there are no positional arguments. In fact, I think the standard currently does not make it clear that "$@" should expand to no fields in that case, even though we all know it must. The standard is clear that the expansion of $@ in that case results in no fields, but it says nothing about the surrounding "s. Normally, the quotes would by themselves already result in a field, and the wording for "$@" does not say the "s should be handled specially, it only says how the inner $@ should be expanded.

There has not been a response from the dash maintainer and I do not think I can predict whether he is going to object. In dash, "$@" is implemented as a special expansion of unquoted $* that always performs a special kind of field splitting, with a special exception that pre-expansion, if there are no positional arguments, the literal sequence opening-quote, dollar, at, closing-quote, is removed entirely. Without this special exception, "$@" would expand the same as "", and that special exception does not cover "$@$@". It would not be too difficult to extend this to allow multiple instances of dollar, at. However, it is not clear to me whether this would be sufficient to conform, or whether there are other cases where POSIX intends to expand to no fields but dash expands to one field.

(Edit: I should mention that technically, the special exception actually acts on dash's internal representation of unexpanded words, but that does not really affect anything.)

(0006871)
McDutchie (reporter)
2024-09-02 19:35

Here is a more comprehensive regression test I wrote. Expected output: none. Feel free to use any or all of it. I hereby dedicate it to the public domain as per Creative Commons CC-0: https://creativecommons.org/publicdomain/zero/1.0/ [^]

for e in 3 '"$@"' \
    5 '"$@$@"' \
    7 '"$@$@$@"' \
    9 '"$@$@$@$@"' \
    11 '"$@$@$@$@$@"' \
    13 '"$@$@$@$@$@$@"' \
    5 '"$@""$@"' \
    7 '"$@""$@""$@"' \
    9 '"$@""$@""$@""$@"' \
    11 '"$@""$@""$@""$@""$@"' \
    13 '"$@""$@""$@""$@""$@""$@"'
do case $e in
    [0-9]*) i=$e
        continue ;;
    esac
    set -- # set zero PPs
    eval "set -- $e"
    test "$#" -eq 0 || echo "$e does not yield zero fields for" \
        "zero positional parameters (got $#)"
    set -- one two three
    eval "set -- $e"
    test "$#" -eq "$i" || echo "$e does not yield $i fields for" \
        "3 positional parameters (got $#)"
    for q in "''" '""'
    do for q in "$q" "$q$q" "$q$q$q" "$q$q$q$q" \
        "$q$q$q$q$q" "$q$q$q$q$q$q"
        do for E in "$q$e" "$e$q" "$q$e$q"
            do set -- # set zero PPs
                eval "set -- $E"
                test "$#" -eq 1 || echo "$E does not" \
                    "yield one field for zero" \
                    "positional parameters (got $#)"
                set -- one two three
                eval "set -- $E"
                test "$#" -eq "$i" || echo "$E does not" \
                    "yield $i fields for 3" \
                    "positional parameters (got $#)"
            done
        done
    done
done
(0006872)
geoffclare (manager)
2024-09-03 08:29

Re Note: 0006870

> it really is not at all clear what the standard is saying in the case of "$@$@" more generally: https://lore.kernel.org/dash/[...] [^]

This bug is specifically about the behaviour with $# -eq 0 as per the summary line. The linked email is about behaviour with $# -ne 0 so I see it as a digression and not directly relevant to this bug. Having said that, I wouldn't object if a clarification for that case ends up being included in the resolution here.

> The standard is clear that the expansion of $@ in that case results in no fields, but it says nothing about the surrounding "s.

Actually it does. You need to read the $@ description in 2.5.2 in combination with the text in 2.6 that references it:
The shell shall create multiple fields or no fields from a single word only as a result of field splitting, pathname expansion, or the following cases:

  1. Parameter expansion of the special parameters '@' and '*', as described in Section 2.5.2, can create multiple fields or no fields from a single word.

When this says "single word" it is referring to the whole word (which includes any quotes).
(0006874)
geoffclare (manager)
2024-09-05 14:44
edited on: 2024-09-05 15:18

Interpretation response
------------------------
The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.

Rationale:
-------------
The standard states "if the parameter being expanded was embedded within a word, the first field shall be joined with the beginning part of the original word and the last field shall be joined with the end part of the original word." It is not clear how this applies when there are no positional parameters as the "first field" and "last field" do not exist in that case.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------

On page 2479 line 80372 section 2.5.2, change:
If one of these conditions is true, the initial fields shall be retained as separate fields, except that if the parameter being expanded was embedded within a word, the first field shall be joined with the beginning part of the original word and the last field shall be joined with the end part of the original word. In all other contexts the results of the expansion are unspecified. If there are no positional parameters, the expansion of '@' shall generate zero fields, even when '@' is within double-quotes; however, if the expansion is embedded within a word which contains one or more other parts that expand to a quoted null string, these null string(s) shall still produce an empty field, except that if the other parts are all within the same double-quotes as the '@', it is unspecified whether the result is zero fields or one empty field.

to:
If one of these conditions is true, the initial fields (if any) shall be retained as separate fields, except that if the parameter being expanded was embedded within a word and either the beginning part (the part before the parameter being expanded) or the end part (the part after the parameter being expanded) of the original word was non-null:

  • If there is at least one positional parameter, the first field shall be joined with the beginning part of the original word, or with the results of expansions already performed on it if there were any, and the last field shall be joined with the end part of the original word.

  • If there are no positional parameters, the beginning part of the original word, or the results of expansions already performed on it if there were any, shall be joined with the end part of the original word.

In all other contexts the results of the expansion are unspecified. If there are no positional parameters and the parameter being expanded was not embedded in a word with other parts whose expansion is non-null, the expansion of '@' shall generate zero fields, even when '@' is within double-quotes; however, if the expansion is embedded within a word which contains one or more other parts that expand to a quoted null string, these null string(s) shall still produce an empty field, except that if the other parts are all within the same double-quotes as the '@', it is unspecified whether the result is zero fields or one empty field.

On XRAT page 3877 line 134466 section C.2.5.2, change:
printf '[%s]\n' foo $@
[foo]
printf '[%s]\n' foo "$@"
[foo]
printf '[%s]\n' foo ''$@
[foo]
[]
printf '[%s]\n' foo ''"$@"
[foo]
[]
printf '[%s]\n' foo "$novar$@$(echo)"
[foo]
[] </tt>(this line of output is optional)<tt>
printf '[%s]\n' foo ''"$novar$@$(echo)"
[foo]
[]

to:
printf '[%s]\n' foo $@
[foo]
printf '[%s]\n' foo a$@
[foo]
[a]
printf '[%s]\n' foo "$@"
[foo]
printf '[%s]\n' foo "$@b"
[foo]
[b]
printf '[%s]\n' foo "$@$@"
[foo]
printf '[%s]\n' foo "$@""$@"
[foo]
printf '[%s]\n' foo "$@$*"
[foo]
[] </tt>(this line of output is optional)<tt>
printf '[%s]\n' foo "$@""$*"
[foo]
[]
printf '[%s]\n' foo ''$@
[foo]
[]
printf '[%s]\n' foo ''"$@"
[foo]
[]
printf '[%s]\n' foo "$novar$@$(echo)"
[foo]
[] </tt>(this line of output is optional)<tt>
printf '[%s]\n' foo ''"$novar$@$(echo)"
[foo]
[]


(0006877)
agadmin (administrator)
2024-09-12 12:31

Interpretation proposed: 12 Sep 2024
(0006915)
agadmin (administrator)
2024-10-14 12:48

Interpretation approved: 14 October 2024

- Issue History
Date Modified Username Field Change
2024-08-14 23:38 steffen New Issue
2024-08-14 23:38 steffen Name => steffen
2024-08-14 23:38 steffen Section => 2.52
2024-08-14 23:38 steffen Page Number => 2479
2024-08-14 23:38 steffen Line Number => 80376 ff.
2024-08-15 09:30 geoffclare Note Added: 0006862
2024-08-15 18:38 steffen Note Added: 0006865
2024-08-15 19:40 salewski Issue Monitored: salewski
2024-08-19 20:16 McDutchie Note Added: 0006866
2024-09-02 15:21 geoffclare Note Added: 0006869
2024-09-02 15:55 hvd Note Added: 0006870
2024-09-02 16:18 hvd Note Edited: 0006870
2024-09-02 19:35 McDutchie Note Added: 0006871
2024-09-03 08:29 geoffclare Note Added: 0006872
2024-09-05 14:44 geoffclare Note Added: 0006874
2024-09-05 15:18 geoffclare Note Edited: 0006874
2024-09-05 15:18 geoffclare Interp Status => Pending
2024-09-05 15:18 geoffclare Final Accepted Text => Note: 0006874
2024-09-05 15:18 geoffclare Status New => Interpretation Required
2024-09-05 15:18 geoffclare Resolution Open => Accepted As Marked
2024-09-05 15:20 geoffclare Tag Attached: tc1-2024
2024-09-12 12:31 agadmin Interp Status Pending => Proposed
2024-09-12 12:31 agadmin Note Added: 0006877
2024-10-14 12:48 agadmin Interp Status Proposed => Approved
2024-10-14 12:48 agadmin Note Added: 0006915


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker