|Anonymous | Login||2023-10-03 20:44 UTC|
|Main | My View | View Issues | Change Log | Docs|
|Viewing Issue Simple Details|
|ID||Category||Severity||Type||Date Submitted||Last Update|
|0001389||[1003.1(2016/18)/Issue7+TC2] Shell and Utilities||Objection||Error||2020-08-13 19:52||2021-01-28 16:57|
|Section||220.127.116.11 1.c (18.104.22.168 1.c in Issue8 Draft 1), 22.214.171.124 1.d, 126.96.36.199 1.e.i.a|
|Page Number||2367 (2335 in Issue8 draft 1)|
|Line Number||75564-73 75577-78 (74159-64 74168-71 in Issue8 draft 1)|
|Final Accepted Text|
|Summary||0001389: Shell command search procedure is incorrect|
POSIX currently specifies that normal built-in standard utilities be
invoked only when the PATH search locates that utility.
There are all kinds of problems with this, but the most important
is that it does not document the standard behaviour (rather it
documents what it seems some people would like the standard behaviour
I set PATH to a list of directories that does not include /bin
(which, on my system, is where the standard utilities ls and test
are to be found - for completeness here, printf is in /usr/bin which
remained in $PATH, though the results below do not show that).
(I am not showing the value of my PATH with /bin removed, as what's
there isn't actually relevant for anything here).
Then I ran the following script, which tests every (posixish) shell
I have access to that I could remember I have...
for SHELL in /bin/sh /bin/ksh bash 'bash -o posix' fbsh dash bosh yash \
pdksh mksh ksh93 zsh 'zsh --emulate sh'
$SHELL -c '
printf "\n\nTesting %s : \n" '"'$SHELL'"'
for p in $PATH
[ -x $p/test ] && echo found test in $p
test x = x || echo weird
done >/tmp/results 2>&1
It needs full pathnames for the standard NetBSD shells /bin/sh and /bin/ksh
as /bin isn't in PATH any more - /bin/ksh and pdksh are more or less the same
thing, though probably have slightly different sets of bugs fixed.
fbsh is the FreeBSD shell, the rest I think are well known. I don't
have a copy of ksh88.
In that the initial printf is just for documentation, the two ls
lines that follow are to show first that "ls" is not found (as /bin
is not in $PATH, and second that /bin/test exists - that is, that
is where the test utility would normally be found). From those
two lines I would expect (in every shell) to see some kind of error
about "ls" not being found, and then "/bin/test" as the output
from the 2nd /bin/ls command (and indeed, that is what happens,
as shown below).
IFS is set to break apart $PATH, and the for loop simply
verifies that there is no other "test" command anywhere else
that might be invoked. If there is another test, the printf
would tell us where it was located, if there isn't, then there
will be no output. Obviously all shells should do the same thing
here, and they do, none finds a "test" anywhere else.
Note that there is actually a possible issue here with the '['
command (which also lives in /bin) so a shell might complain that
'[' is not found. None do.
The final line is the important one, if a shell implemented the
specification in 188.8.131.52 1.c and 184.108.40.206 1.e.i.a "correctly", there
should be a "test not found" error (in some format or other,
just like the "ls not found" error) here, as the built-in version
of the standard utility test is only supposed to be located if test
is found in $PATH (which we have just seen it will not be).
The "|| echo weird" is just in case some non-standard test were
to be invoked, and there "echo" (which also lives in /bin) could
also result in "command not found" - this we don't know, as there
is no attempt made to execute that "echo" command, the test
command (in every shell I tested - I had hoped that in one or
two might not have test built in) is a built-in command, and
is executed (IMO correctly) here (and operates correctly,
returning a status of 0, so the || part is never attempted).
The results showing all of this appear bust below. To me this
is clear evidence that the current specification is not specifying
the standard behaviour, and should be fixed.
Testing /bin/sh :
ls: not found
Testing /bin/ksh :
/bin/ksh: ls: not found
Testing bash :
bash: ls: command not found
Testing bash -o posix :
bash: ls: command not found
Testing fbsh :
fbsh: ls: not found
Testing dash :
dash: 1: ls: not found
Testing bosh :
bosh: ls: not found
Testing yash :
yash: no such command `ls'
Testing pdksh :
pdksh: ls: not found
Testing mksh :
mksh: ls: not found
Testing ksh93 :
ksh93: ls: not found [No such file or directory]
Testing zsh :
zsh:1: command not found: ls
Testing zsh --emulate sh :
zsh:1: command not found: ls
First, if considering Issue8 draft 1, all references below to
section 220.127.116.11 should be interpreted as references to section 18.104.22.168
In section 22.214.171.124 1.c (page 2367) delete the sentences:
If the implementation has provided a standard utility
Replace the entire text of section 126.96.36.199 1.d (which differs dramatically
between Issue 7 and Issue 8 draft 1, so the old text is not quoted
here), (page 2367) with:
If the command name matches the name of a utility built
In section 188.8.131.52 1.e replace the entire first part of the text of
sub-section i (including the entire contents of sub-sub-sections 'a' and 'b')
down to, but not including the paragraph that begins
Once a utility has been searched for and found
The text to be removed is not included here, as it differs between
Issue 7 and Issue 8 draft 1. The replacement text should be:
If the search is successful the shell shall execute the
[This is using the section numbering for the reference as it appears
in Issue 8 draft 1, where 184.108.40.206 is the section entitled:
Non-built-in Utility Execution.]
These changes will make the standard match what shells actually
implement, rather than some pipe dream about what they should implement.
|Tags||No tags attached.|
|Although it goes into more detail, this is effectively a repeat of bug 0000854 and should be closed as a duplicate.|
edited on: 2020-08-19 10:36
Just in case this is not known, ksh93 implements PATH search for builtin commands and this was a requirement for ksh93 integration into OpenSolaris in 2008 after a long and heaty discussion. The reason for that requirement was to retain POSIX compliance even though basic standard utilities are built into the standard shell.
If a shell does not implement PATH search for additional builtins, we would need to forbid shells to add more builtins than the minimal set.
ksh93 added many additional builtins like cat, chown, cmp, ... and POSIX grants that a user that if he has private version of system utilities, the shell needs to use them if they are first in PATH.
If a shell adds more builtins than currently documented, this would no longer work as historically builtin are always found first.
As a consequence, we created a list of tainted utilities (see 0000854) that already may be builtins.
If you like to add more builtins from the list of standard utilities, you either need to implement PATH search for these builtins, or you are forbidden to implement these utilities as builtins.
In ksh93, builtin commands are managed with the builtin named "builtin".
Call builtin for a list....e.g.
Note that the builtins bound to /usr/ast/bin are not fully POSIX compliant and there fore are not bound to a path that appears in the default PATH.
If you do not care about POSIX compliance or if you do not like to add more builtins to your shell, you may do what ever you like. If you however like to add more builtins, you need to follow long existing rules about PATH search.
ksh93 does not follow the POSIX rules either, and never has done:
(1) ksh93 has several builtins (including builtin aliases, something POSIX doesn't allow either), not bound to any path search, that are "not allowed" by this rule, i.e.: which are neither a standard utility, nor in the list of names reserved as "unspecified" in XCU 220.127.116.11(b). They are: alarm, compound, enum, float, functions, integer, nameref, r, redirect.
(2) Several POSIX standard regular builtins are *never* subject to a path search in ksh93, although POSIX specifies that they must be. These are: [, echo, printf, sleep, test, type, ulimit.
What's more, path-bound builtins in ksh93 behave exactly like external commands. The only difference is their path doesn't actually exist in the file system. But that is a mere implementation detail. POSIX has no business specifying or regulating implementation details.
So, if this path search rule exists because of ksh93, then it is based on a fundamental misunderstanding of what ksh93 does.
In the real world, there are zero shells that actually manage to follow this rule. Not even the deliberately pedantic 'yash -o posix' fully manages, though it tries hard. Bash, Busybox ash, dash, FreeBSD sh, NetBSD sh, mksh, pdksh (OpenBSD sh), and zsh all ignore this rule completely. So does yash in its sane default mode.
This entire path search rule for builtins should quite simply be removed. I suppose it won't do much damage if POSIX decides to keep it; I'm confident the overwhelming majority of shells will continue to ignore it completely. The real damage is to POSIX's reputation as a standard to be taken seriously.
|2020-08-13 19:52||kre||New Issue|
|2020-08-13 19:52||kre||Name||=> Your Name Here|
|2020-08-13 19:52||kre||Section||=> (section number or name, can be interface name)|
|2020-08-13 19:52||kre||Page Number||=> (page or range of pages)|
|2020-08-13 19:52||kre||Line Number||=> 75564-73 75577-78 (74159-64 74168-71 in Issue8 draft 1)|
|2020-08-13 21:17||Don Cragun||Name||Your Name Here => Robert Elz|
|2020-08-13 21:17||Don Cragun||Interp Status||=> ---|
|2020-08-13 21:17||Don Cragun||Relationship added||related to 0001390|
|2020-08-13 21:18||Don Cragun||Relationship added||related to 0001391|
|2020-08-13 21:24||Don Cragun||Severity||Editorial => Objection|
|2020-08-13 21:24||Don Cragun||Type||Clarification Requested => Error|
|2020-08-13 21:30||Don Cragun||Section||(section number or name, can be interface name) => 18.104.22.168 1.c (22.214.171.124 1.c in Issue8 Draft 1), 126.96.36.199 1.d, 188.8.131.52 1.e.i.a|
|2020-08-13 21:30||Don Cragun||Page Number||(page or range of pages) => 2367 (2335 in Issue8 draft 1)|
|2020-08-14 08:47||geoffclare||Note Added: 0004920|
|2020-08-14 08:47||geoffclare||Relationship added||duplicate of 0000854|
|2020-08-14 10:39||joerg||Note Added: 0004924|
|2020-08-14 10:40||joerg||Note Edited: 0004924|
|2020-08-14 10:40||joerg||Note Edited: 0004924|
|2020-08-14 10:41||joerg||Note Edited: 0004924|
|2020-08-19 10:36||joerg||Note Edited: 0004924|
|2020-08-20 15:31||McDutchie||Note Added: 0004934|
|2021-01-28 16:57||Don Cragun||Status||New => Closed|
|2021-01-28 16:57||Don Cragun||Resolution||Open => Duplicate|
|Mantis 1.1.6[^] Copyright © 2000 - 2008 Mantis Group|