Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000898 [1003.1(2013)/Issue7+TC1] Shell and Utilities Editorial Clarification Requested 2014-12-02 02:54 2019-06-10 08:54
Reporter tedu View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Closed  
Name Ted Unangst
Organization OpenBSD
User Reference
Section test
Page Number 3250
Line Number 108968-108969
Interp Status Approved
Final Accepted Text Note: 0002573
Summary 0000898: Ambiguity regarding necessity of ]
Description The test utility has two documented syntaxes. When invoked as test, no ending ] is required or permitted. When invoked as [ an ending ] is required. It is unclear what happens when it is invoked as /bin/[. Is the ending bracket required or not?

I believe the second part of the description tries to address this with "the application shall ensure that the square brackets are separate arguments." However, I simply interpreted that to mean that the ending ] must be separated from the last argument. i.e. [ -f file ] and not [ -f file].

If the intention is that the argument must consist entirely of the string "[", that should be clarified. Or is /bin/[ -f file ] expected to work?

Considering that [ is often a shell builtin, it can be difficult to invoke a separate [ utility except by calling it /bin/test, in which case the [ name seems a little orphaned.
Desired Action 1. Pick the desired syntax
/bin/[ -f file
or
/bin/[ -f file ]

2. Clarify text.
Tags tc2-2008
Attached Files

- Relationships

-  Notes
(0002469)
eblake (manager)
2014-12-02 16:38

You can always use "env [ ..." instead of "/bin/[" for invoking the non-builtin counterpart, without having to worry about whether adding slashes changes the interpretation of a literal argument content of just "[".

In practice, executing "/bin/[" is the same as executing "[" - both spellings require a trailing "]" as an independent argument. This is true for all implementations I am aware of.

It gets fuzzier if you do 'ln -s /bin/[ foo': at least GNU coreutils provides two separate binaries (the behavior is chosen at compile-time whether the binary was compiled as "test" or "[", independent of argv[0]); so you can make 'foo' have either behavior depending on whether it links to the "test" or the "[" binary. On the other hand, some implementations go solely off the value of argv[0] having an appropriate basename ('ln -s /usr/sbin/busybox /tmp/[' creates a /tmp/[ that requires a trailing ']' argument, but the busybox binary requires argv[0] to have a known basename, and attempting to create a foo symlink will fail because 'foo' is not a known busybox applet).

Can you propose wording that would offer the clarification you seek? Ideally, it could state something about the basename of the utility being [ being the determining factor on when a ] is required, and leave it unspecified if the utility is invoked through a basename other than '[' or 'test'.
(0002470)
tedu (reporter)
2014-12-02 18:39

Ah, so OpenBSD was looking at the entirety of argv[0], hence the question.

Perhaps:

In the second form, which is used when the utility's basename is "[", the application shall ensure that the square brackets are separate arguments.

I think that's clear. Symlinks with other names seems like "you're on your own".
(0002471)
geoffclare (manager)
2014-12-03 09:44

The description of test states:

    In the second form of the utility, which uses "[]" rather than
    test, the application shall ensure that the square brackets are
    separate arguments.

If an application executes /bin/[ it does not meet this requirement
because the left bracket is not a separate argument, and therefore
the behaviour is undefined.
(0002472)
tedu (reporter)
2014-12-03 19:47

In that case, an even simpler revision would clarify things for me. Add the word "complete".

the application shall ensure that the square brackets are
    complete, separate arguments.
(0002474)
shware_systems (reporter)
2014-12-04 16:18

In either case 'test' and '[' are simple command names, not arguments. Referring to both as arguments appears to be an error, therefore. The ']' for the second form is an argument that balances the '[' textually, that's all.

As keywords have precedence over operands in token recognition of a simple command suffix, afaik, by grammar Rule #1 I've always thought of ']' as optional for either form, required as allowed to be present to keep scripts prettier but otherwise superfluous. I agree the Synopsis has it as required, but does this match existing practice?

I'd expect
if test $V = V then command1;
if [ $V = V then command1;
if test $V = V ] then command1;
if [ $V = V ] then command1;

to behave the same, whether in a script or entered interactively, or if foo is linked to either binary in /bin, or 'env [ ...' or 'env test ...' is used.
(0002475)
eblake (manager)
2014-12-04 17:55

It's universal that implementations REQUIRE ] when invoked as [, and REJECT ] when invoked as test; observe with bash, ksh, and coreutils:

$ bash -c 'test a ]'
bash: line 0: test: a: unary operator expected
$ bash -c '[ a '
bash: line 0: [: missing `]'
$ ksh -c 'test a ]'
ksh: test: argument expected
$ ksh -c '[ a '
ksh: [: ']' missing
$ env [ a
[: missing ']'
$ env test a ]
test: missing argument after ‘]’

Furthermore, the reason this is universal is because of the definition of the utility having different behavior based on 0, 1, 2, 3, or 4 operands. If trailing ']' were optional instead of required/forbidden, then constructs like:

test "$a"
[ "$a"

would become ambiguous when $a contains "]". Put another way, "test ]" must return status 0, as "]" is non-empty (and not status 1 as shorthand for test with 0 arguments), while "[ ]" must return status 1, as there were no arguments supplied (and not status 0 for a 1-argument test of "]" being non-empty with no closing "]"). The trailing "]" cannot be optional in either form of the command, due to the fact that it cannot be semantically distinguished from valid commands that end with a trailing "]" as one of the real arguments.

An additional proof that the two command forms are distinct: coreutils uses the fact that '[' requires a trailing ']' to implement options as an extension; whereas the 'test' form has no escape ('[ --help' and '[ --version' do the usual GNU help/version text, because they are undefined by the standard without the trailing ']', while 'test --help' and 'test --version' are required by the standard to silently give status 0 with no room for output text).
(0002478)
geoffclare (manager)
2014-12-05 10:11

In Note: 0002474 Mark has a valid point about 'test' and '[' being the command name part of the command, not an argument. I suppose the text was written based on the assumption that the command name counts as an argument based on argv[0], but XBD 3.21 says "An argument is one of the options, option-arguments, or operands following the command name."

The reason for having the statement cover both brackets is to point out that all of these don't work:

[-f file]
[-f file ]
[ -f file]

However, in the first two cases the utility name is "[-f" so anything stated on the test/[ utility page doesn't apply anyway.

I think we should change:

In the second form of the utility, which uses "[]" rather than test, the application shall ensure that the square brackets are separate arguments.

to:

In the second form of the utility, where the utility name used is '[' rather than test, the application shall ensure that the closing square bracket is a separate argument.

We could also put something in APPLICATION USAGE to warn about "[-f file]" and "[-f file ]" if desired, stating that they execute a utility called "[-f" if one exists, not the '[' utility.
(0002573)
geoffclare (manager)
2015-03-05 17:31

Interpretation response
------------------------
The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.

Rationale:
-------------
None.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------

On Page: 3250 Line: 108974 Section: test, change:

In the second form of the utility, which uses "[ ]" rather than test, the application shall ensure that the square brackets are separate arguments.

to:

In the second form of the utility, where the utility name used is [ rather than test, the application shall ensure that the closing square bracket is a separate argument. The test and [ utilities may be implemented as a single linked utility which examines the basename of the zeroth command line argument to determine whether to behave as the test or [ variant. Applications using the exec() family of functions to execute these utilities shall ensure that the argument passed in arg0 or argv[0] is '[' when executing the [ utility and has a basename of "test" when executing the test utility.

Add to Application Usage after page 3255, line 109200:

Note that none of the following examples are permitted by the syntax described
[-f file]
[-f file ]
[ -f file]
[ -f file
test -f file ]

In the first two cases, if a utility named [-f exists, that utility would be invoked, and not test.
In the remaining cases, the brackets are mismatched, and the behavior is unspecified. However
test ! ]

does have a defined meaning, and must exit with status 1. Similarly,
test ]

must exit with status 0.

At page 785 line 26351 (exec RATIONALE), add a sentence:

Also, note that the test and [ utilities require specific strings for the argv[0] argument to have deterministic behavior across all implementations.
(0002629)
ajosey (manager)
2015-04-17 09:53

Interpretation Proposed: 17 April 2015
(0002671)
ajosey (manager)
2015-05-18 09:07

Interpretation Approved: 18 May 2015
(0003621)
kre (reporter)
2017-03-17 00:57

I was looking for a (totally unrelated) possible bug report, and happened
across this one...

While I agree with everything in the (adopted) resolution (from almost 2
years ago) this seems to be yet another case where the issue raised initially
has been simply forgotten, and the resolution solves some other problem.

The question asked (the problem raised) was that it is unspecified what
happens when argv[0] is "/bin/[".

Telling applications that exec [ or test that they must use exactly "[" or
"test" as argv[0], whatever the user/script actually said, is frankly,
absurd. Shells are a common application which does this, and I cannot
believe that you really expect the shell to try and guess whether the '['
final component of a command name represents the '[' command that requires
this magic treatment? Should it be done for $HOME/bin/[ ?

If not, what if PATH is $HOME/bin:/bin:... and the user just runs [ ?

Claiming that env [ solves the problem for applications first assumes that
the problem is what you believe it is (the reason an application might
want to run /bin/[ ... if it isn't a shell doing it, it won't have [ built
in probably, so avoiding the builtin is not necessarily it), and it gives
no assistance at all to test implementers.

Couldn't the resolution of this issue please address the issue raised?
Something to actually provide guidance to people implementing test (aka [)
commands? What is expected, basename(argv[0]) or just argv[0] when
looking for '[' ? Or perhaps if (strchr(argv[0], '[') != NULL) ??

kre
(0003638)
nick (manager)
2017-03-17 23:12

Note that TC2 is now published, with this interpretation now in the accepted text.

In TC2, this is on page 3288, lines 110611-110617.

You should open a new issue against TC2 (with a proposed resolution rather than a question if possible).

- Issue History
Date Modified Username Field Change
2014-12-02 02:54 tedu New Issue
2014-12-02 02:54 tedu Status New => Under Review
2014-12-02 02:54 tedu Assigned To => ajosey
2014-12-02 02:54 tedu Name => Ted Unangst
2014-12-02 02:54 tedu Organization => OpenBSD
2014-12-02 02:54 tedu Section => test
2014-12-02 03:07 Don Cragun Page Number => 3250
2014-12-02 03:07 Don Cragun Line Number => 108968-108969
2014-12-02 03:07 Don Cragun Interp Status => ---
2014-12-02 16:38 eblake Note Added: 0002469
2014-12-02 18:39 tedu Note Added: 0002470
2014-12-03 09:44 geoffclare Note Added: 0002471
2014-12-03 19:47 tedu Note Added: 0002472
2014-12-04 16:18 shware_systems Note Added: 0002474
2014-12-04 17:55 eblake Note Added: 0002475
2014-12-05 10:11 geoffclare Note Added: 0002478
2015-03-05 17:31 geoffclare Note Added: 0002573
2015-03-05 17:32 geoffclare Project 1003.1(2008)/Issue 7 => 1003.1(2013)/Issue7+TC1
2015-03-05 17:33 geoffclare Interp Status --- => Pending
2015-03-05 17:33 geoffclare Final Accepted Text => Note: 0002573
2015-03-05 17:33 geoffclare Status Under Review => Interpretation Required
2015-03-05 17:33 geoffclare Resolution Open => Accepted As Marked
2015-03-05 17:33 geoffclare Tag Attached: tc2-2008
2015-04-17 09:53 ajosey Interp Status Pending => Proposed
2015-04-17 09:53 ajosey Note Added: 0002629
2015-05-18 09:07 ajosey Interp Status Proposed => Approved
2015-05-18 09:07 ajosey Note Added: 0002671
2017-03-17 00:57 kre Note Added: 0003621
2017-03-17 23:12 nick Note Added: 0003638
2019-06-10 08:54 agadmin Status Interpretation Required => Closed


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker