Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001063 [1003.1(2013)/Issue7+TC1] Shell and Utilities Editorial Clarification Requested 2016-07-23 22:13 2024-06-11 08:56
Reporter izabera View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Closed  
Name Isabella
Organization
User Reference
Section sh
Page Number 3189
Line Number 106700-106701
Interp Status ---
Final Accepted Text Note: 0003881
Summary 0001063: missing specification for sh -h
Description The synopsis in
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html [^] specifies
several options for sh, including -h.

Then the options section in the same page says:

> "The -a, -b, -C, -e, -f, -m, -n, -o option, -u, -v, and -x options are
> described as part of the set utility in Special Built-In Utilities."

Someone forgot to add -h in that list.
Desired Action Change this sentence

> "The -a, -b, -C, -e, -f, -m, -n, -o option, -u, -v, and -x options are
> described as part of the set utility in Special Built-In Utilities."

to this

> "The -a, -b, -C, -e, -f, -h, -m, -n, -o option, -u, -v, and -x options are
> described as part of the set utility in Special Built-In Utilities."
Tags tc3-2008
Attached Files

- Relationships

-  Notes
(0003306)
kre (reporter)
2016-07-24 02:23

Why not just mark -h as deprecated, and forget about this change?

It is, after all, a more or less useless (and underspecified) option
as it is now. Its definition is:

    Locate and remember utilities invoked by functions as those
    functions are defined

but as best I can tell, as a function is being defined, no utilities
are invoked, ever, so there is nothing to remember. Nor is it
specified anywhere what is the point (or effect) of this memory if
there was something to remember.

When if this language were to be corrected to say what it probably
means, which is, while parsing, to remember the location of utilities
that will be invoked by a function when it is invoked sometime later,
how is that supposed to be done in general, consider ...

    fn() {
        "$@"
    }

what does -h cause to be remembed in that case?

Then, if we can ever work out just what we should remember (if anything)
we then need to figure out what to do with that...

For normal command invocation, the file system location (that is,
from the PATH search) of commands with no /'s in their names may
be remembered, until PATH is changed, when they are all forgotten
(this is really just an efficiency hack, and aside from perhaps
allowing the shell to fail to notice a utility newly installed
earlier in PATH than where the old one was previously found, should
be invisible to the user.) If this is the intent of -h , then
there's really no need, remembering the location of commands actually
invoked when the function is invoked, is a better solution.

So, that cannot be it, the only possible intent for this would be
so that something like

    PATH=.:other:stuff

    fn() {
        cmd
    }

where ./cmd exists, would turn "cmd" into the value of ${PWD}/cmd
at the time of definition. I can see some utility in that, but it
isn't reliable if any change to PATH would cause the memory to be
deleted. Nor is it a good idea (IMO) to create something which is
only practiclaly useful when "." is in PATH. If "cmd" was "./cmd"
then remembering its location (replacing "." with ${PWD}) would be
a most unusual operation.

There are all kinds of other issues - such as the relationship
between aliases and functions - most shells (I believe) interpret
aliases during function definition, so that in...

    fn() { w true; d sleep 10; dd; }
    alias w=while
    alias d=do
    alias dd=done

the aliases do not affect the function (which would run the 'w',
'd', amd 'dd' commands) but if the function is defined after the
aliases, then the function instead contains a while loop. But there
is nothing in the definition of aliases that requires this that I'm
aware of, it would be OK for a shell to take the above and implement
a while loop (expanding the aliases as the function is invoked.)
In that case, the word which is just "true" here is a utility invoked
by the function, so -h (if defined properly) should cause its location
to be remembered. But when the function was defined, the shell could
not possibly know that. The same for "sleep".

To use the same example, if -h were in effect, would that prevent the
aliases being expanded, as (on most systems) 'w' and 'dd' are utilities
that a PATH search will locate (and we could use something else instead
of 'd' and make it 3 out of 3...) If the function has 'dd' replaced
by /usr/bin/dd then the 'dd' alias would never be seen. Let's
say the function was instead

    fn() { wh true; d sleep 10; dd; }

and the 'w' alias (defined later) was a 'wh" alias instead. Given
normal system commands, the lookup for 'wh' and 'd' will fail, so there is
nothing to remember in that case, so their aliases could be expanded
later, but 'dd' is a utility that should be found, so now we have
a function containing a while loop, with no "done" - but which was
not detected as a syntax error during parsing?

This actually makes another (unspecified) assumption of course,
which is that when a function is defined (with -h set) that a
utility that will (or might) later be invoked, which cannot be
found (at the time of function definition) should simply be
left as it is, and searched for later. But nothing anywhere
says that is what should happen, perhaps when -h is set, utilities
to be invoked by the function which cannot be found are intended
to give a "command not found" error at function definition time,
and the function definition should exit with status 127 ?

All of this is simply a gigantic quagmire, and (apart from that set
of shells that treat tne -h definition literally, and so make the
option a no-op - which is what the NetBSD shell does, and always
has (since it learned about -h at all), and always will, I expect)
I really cannot believe that there are any two (unrelated) shells that
implement -h the same way in all respects. That is, there is no
existing standard -h option that can be defined, just a whole bunch
of different ones.

That makes it a prime example for doing to it the same as was done
recently to "-o nolog", and retain the existence of the option but
explicitly state that shells may ignore it, and perhaps even make
the operation of the shell unspecified if the option is set.

In that case, it would be better to remove -h from the synposis,
than to add it to the later list.
(0003307)
nick (manager)
2016-07-25 16:57

It also seems that different shells implement -h with different meanings:

bash: Remember the location of commands as they are looked up for execution. This is enabled by default. (Unclear if this is what POSIX suggests)

zsh: Do not enter command lines into the history list if they are duplicates of the previous event. (Not what POSIX requires)

ksh: Each command becomes a tracked alias when first encountered. (As POSIX)

yash: When a function is defined when this option is enabled, the shell immediately performs command path search for each command that appears in the function and caches the command’s full path. (As POSIX)

dash: Does not support -h at all

mksh: If the trackall option is set (i.e. set -o trackall or set -h), the shell tracks all commands. This option is set automatically for non-interactive shells. (As POSIX?)
(0003308)
shware_systems (reporter)
2016-07-25 18:58

For bash and mksh, I'd say no, it's not what POSIX suggests; as the text doesn't mention setting or changing alias definitions that ksh version suspect also. I tend to agree with kre in Note 3306 that it should be unspecified behavior if aliases used in the definition or the PATH variable change before the function is invoked. I feel it should be clarified too that 1) aliases are expanded both at definition and execution time when tokenizing the compund_list elements of a definition's compound_command, in XCU 2.9.5 and 2) executing set +h does not affect definitions created with -h in effect; an unset of fname also required. With those I'd agree the yash behavior most conforming as to what is suggested. As an optimization that can speed up execution of functions called in a loop I'd prefer not to deprecate or remove it, however.
(0003309)
joerg (reporter)
2016-07-26 09:25

Maybe it makes sense to look at the Bourne Shell, where the feature has been taken from.

The Bourne Shell tries to enter all command names into the hash table while parsing the command line.
(0003881)
geoffclare (manager)
2017-11-16 17:26

Along with desired action, Change in set -h P2410, L77147 of 2016 edition:
    Locate and remember utilities invoked by functions as those
    functions are defined (the utilities are normally located when
    the function is executed).
to:
    [OB] Setting this option may speed up PATH searches (see [xref to XBD Chapter 8]).
    This option may be enabled by default.[/OB]

On page 2413 line 77273 of 2016 edition (set RATIONALE), change:

    The −h flag is related to command name hashing. See [xref to hash].

to:

    The −h option is related to command name hashing. See [xref to hash]. The normative description is deliberately vague because the way this option works varies between shell implementations.


    Earlier versions of this standard specified -h as a way to locate and remember utilities to be invoked by functions as those functions are defined (the utilities are normally located when the function is executed). However, this did not match existing practice in most shells.


On page 2847, line 93798 of 2016 edition (last paragraph of hash DESCRIPTION), change:

    Utilities provided as built-ins to the shell shall not be reported by hash.

to:

    Utilities provided as built-ins to the shell and functions shall not be reported by hash.


On page 2848, line 93830-93834 of 2016 edition (hash STDOUT), add to the end of the paragraph:

    This list shall be cleared when the contents of the PATH environment variable are changed.


On page 3226 line 108276-108280 of 2016 edition (sh SYNOPSIS) add OB shading to all six occurrences of the h option letter.

- Issue History
Date Modified Username Field Change
2016-07-23 22:13 izabera New Issue
2016-07-23 22:13 izabera Name => Isabella
2016-07-23 22:13 izabera Section => (section number or name, can be interface name)
2016-07-23 22:13 izabera Page Number => (page or range of pages)
2016-07-23 22:13 izabera Line Number => (Line or range of lines)
2016-07-23 22:53 Don Cragun Section (section number or name, can be interface name) => sh
2016-07-23 22:53 Don Cragun Page Number (page or range of pages) => 3189
2016-07-23 22:53 Don Cragun Line Number (Line or range of lines) => 106700-106701
2016-07-23 22:53 Don Cragun Interp Status => ---
2016-07-24 02:23 kre Note Added: 0003306
2016-07-25 16:57 nick Note Added: 0003307
2016-07-25 18:58 shware_systems Note Added: 0003308
2016-07-26 09:25 joerg Note Added: 0003309
2017-11-16 17:26 geoffclare Note Added: 0003881
2017-11-16 17:27 geoffclare Final Accepted Text => Note: 0003881
2017-11-16 17:27 geoffclare Status New => Resolved
2017-11-16 17:27 geoffclare Resolution Open => Accepted As Marked
2017-11-16 17:28 geoffclare Tag Attached: tc3-2008
2019-10-28 10:53 geoffclare Status Resolved => Applied
2024-06-11 08:56 agadmin Status Applied => Closed


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker