0001340: PATH specification has an ambiguity.

Notes
(0004846) kre (reporter) 2020-05-04 19:31	While I would not object to the definition of PATH being rewritten in a slightly more logical form, nothing in the Description here makes any sense. I have no idea what discussions related to 0001208 happened to be, or why an issue relating ti chdir in posix_spawn should care in the least about PATH, but the "pathname being sought" is typically "ls" or "cat" or "sh" none of which contains a '/' - so the first sentence is not "in error", there is no prefix, just the command name (you do understand how PATH works, right?) When the pathname begins with a slash there is nothing to do, no search is made (that's for when someone says /bin/sh) - when there is some other slash in the pathname, resolution happens the same as for any other filename reference - since there is no leading '/' (we know that, that case was already handled) the path lookup starts at the current working directory. There is no ambiguity here, and while, once again, the paragraph could be more clearly written (and should xref XCU 2.9.1.1) it doesn't contain any actual errors.

(0004847) kre (reporter) 2020-05-04 19:33	Actually, are you perhaps under the impression that the paragraph in question is about pathnames in general ? It is about the environment variable PATH - a very specific thing.

(0004848) shware_systems (reporter) 2020-05-04 20:05	The Desired Action has already been discussed and reached consensus, maybe with a couple wording tweaks. This report is simply the formal record the change is considered required, for the reasons given in the Description. This came up because sh does do PATH searches and may use *spawn() to exec utilities. The new Final Text for Bug 1208 is related mostly to how PATH is applicable.

(0004849) kre (reporter) 2020-05-05 03:00	Then the resolution is wrong, and wasn't properly considered. You could add a sentence like that which is proposed, perhaps even replacing the final sentence, but the earlier text (while it could be improved) is not incorrect and should not be deleted.

(0004850) shware_systems (reporter) 2020-05-05 03:16	In that context filename refers to the desired utility, pathname is prefix (possibly with appended slash) concatenated with filename. It was argued filename might replace pathname and keep the basic statement, but that just made it obvious it was redundant so is considered superfluous.

(0004851) kre (reporter) 2020-05-05 04:39 edited on: 2020-05-05 05:13	All filenames are pathnames. But yes, I agree the switch in terminology is one of the things I would fix if rewording this is considered worthwhile. But it is clear that the "pathname" in the later sections is intended to mean the same thing as the "filename" in the initial part. The text is redundant (to some extent) if and only if an xref to XCU 2.9.1.1 is added to replace it... but I am not sure that that section (which explains how sh uses PATH for lookups) necessarily applies precisely to all other uses, as it includes references to built in utilities, functions, etc, so I think it would be better to retain the XBD definition for those other purposes, and simply include an xref to add clarity - perhaps something like See [xref XCU 2.9.1.1] for a description of how sh [xref XCU 2] handles PATH searches. (wordsmithed as appropriate). ps: Not relevant directly here, but to respond to a comment made in Note: 0004848 it is hard to see how a sh could ever use posix_spawn as Note: 0004830 implies while retaining the view that some people have of how and when builtin utilities should be executed. It would be possible if the shell simply looks for a builtin before starting the PATH search, but if it needs to combine the PATH search with builtin recognition, if it was relying upon posix_spawn to do the PATH search, I fail to see how a builtin would ever be able to be executed - the only way would be to do the PATH search twice, and I don't think that works either (the only truly reliable way to test whether a PATH element + filename combination works (from userland) is to try to exec it, and there's no returning from that if it does work. It also isn't clear how a posix_spawn PATH scanning implementation would ever maintain the hash table properly (unless of course, in both cases, the PATH resolution is done as a separate step, and posix_spawn is only ever called (from sh) with a fully qualified pathname (ie: just to replace the fork/exec sequence, and fd/dir manipulations that fit between those) but in that case, there would be no motivation for fiddling the definition of PATH based upon what posix_spawn sees, as it would only every hit the "contains a slash" and "begins with a slash" sections (ie: PATH is irrelevant, and what's more, Note: 0004830 is incorrect, see: Note: 0004830 is incorrect">0001341) - so if you were to believe that this definition is solely for the purposes of posix_spawn then I guess the change would make sense. But it isn't, it is also used by popen() system() find env ... [Aside: mantis seems to have issues formatting this note correctly, I have tried several different ways to write the text, yet it always seems to mangle that final "see Note: 0004830 is incorrect">0001341" reference. Oh, I see the trigger now, it is because the referenced bug (1341) contains bugid and bugnote references in its Summary line; is that not supported - it works in the issue itself?]

(0004854) geoffclare (manager) 2020-05-05 08:40	The description in this bug is reasonably close to what was discussed in the telconference. However, the desired action is most definitely not what we thought Mark was going to put in this bug. The statement "If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed" is both erroneous and redundant. It is erroneous because the thing that is sought is a filename, not a pathname, and therefore by definition cannot contain a slash. It is redundant because all of the places in the standard that require a PATH search to be done already state that no PATH search is done if there is a slash in the string that would be the subject of the search. I.e the description of PATH never needs to be read in the context of something with a slash being sought. The statement "If the pathname begins with a <slash>, the specified path is resolved" is problematic because it only requires pathname resolution to be done for absolute pathnames, not relative ones. The meeting agreed that this needs to be changed in some way so that it applies to relative pathnames as well. Having thought about the issue a little more, I believe just changing that sentence is not the right solution to the second problem. The time when pathname resolution is done is during the search, not after the pathname has been found. (I.e. it can only be "found" by performing pathname resolution on each constructed pathname.) I will add another note with a new proposed resolution.

(0004855) geoffclare (manager) 2020-05-05 08:46 edited on: 2020-05-05 08:47	New proposed resolution... On line 5851 change: The list shall be searched from beginning to end, applying the filename to each prefix, until an executable file with the specified name and appropriate execution permissions is found. If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed. If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111). to: The list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found.

(0004861) shware_systems (reporter) 2020-05-05 16:19	I've no objection to the new wording; it was my understanding all the text preceding the two sentences quoted was considered adequate so I left it alone for original proposal.

(0004862) kre (reporter) 2020-05-05 16:21	Re Note: 0004854 To take the first couple of issues in reverse order, as it makes more sense (to me) this way... It is redundant because all of the places in the standard that require a PATH search to be done already state that no PATH search is done if there is a slash in the string that would be the subject of the search. To me that has produced exactly the inverse to the correct response. If everything that uses PATH needs to specify that it is only used when there is no slash in the name, then lets simplify all of those, and codify the rules for PATH processing in a single place, so everyone gets a known consistent interpretation and we don't have people quibbling about why the wording for the use in context X is subtly different from the wording for contest Y, and deducing from that, that the behaviour is supposed to be different. And if we're certain that the identical wording is used in every case, then that's an even better reason for consolidating it and removing the duplication. Further, this is the safe way to make the change - if we have missed one of the uses (and so don't delete its redundant wording) then we're left with some redundant text, but that's generally harmless. On the other hand if we have missed a use which turns out not to specify that no path search is done when the pathname being sought contains a slash, and we delete the text in the definition of PATH which deals with that, then we've just caused the standard to be broken. Since the standard is very large, and I'm sure the word PATH appears in it very many times, I'd hate to guarantee (as in, my life depends upon it) that we know every place where a path search is specified, and if we are not 100% certain that we have found every single one, then neither can we be certain that every single one has the "no slash" qualification. And last, it means that someone who simply wants to know what this environment variable PATH is all about, and (reasonably) turns to XBD 8.3 to get the answer, will actually discover the whole picture, and not end up missing the crucial element that a search for a/b only looks for "a/b" and doesn't try PATH[1]/a/b PATH[2]/a/b (etc) (using invented notation for the ':' delimited sub-strings of PATH). Next: It is erroneous because the thing that is sought is a filename, not a pathname, and therefore by definition cannot contain a slash. This is part of the "poor wording" I remarked on in earlier notes. It isn't really as bad (and isn't actually incorrect) it is just hard to read correctly easily (and currently depends upon the reader correctly understanding the difference between pathname and filename). PATH What we do about it depends upon the resolution of the previous point. If we agree to consolidate the definition of how PATH works (all of it) in XBD 8.3, which is what I'd suggest should be done, then the thing that is sought is a pathname, not a filename (as they're defined in XBD 3.170 and 3.271), and the correct response to this problem is to fix that, and use "filename" only after we have excepted the case where there's a slash in the pathname - and I'd make the wording be clear about the change of terminology, and why something perhaps like: PATH This variable shall represent the sequence of path prefixes that certain functions and utilities apply in searching for an executable file. The prefixes shall be separated by a <colon> (':'). If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list of prefixes shall be searched from beginning to end, applying the filename to each prefix, until an executable file with the specified name and appropriate execution permissions is found. When a non-zero-length prefix is applied to the filename, a <slash> shall be inserted between the prefix and the filename if the prefix did not end in <slash>. A zero-length prefix is a legacy feature that indicates the current working directory. It appears as two adjacent <colon> characters ("::"), as an initial <colon> preceding the rest of the list, or as a trailing <colon> following the rest of the list. A strictly conforming application shall use an actual pathname (such as .) to represent the current working directory in PATH. If the pathname being sought contains a <slash>, and so is not a filename, the search through the path prefixes shall not be performed. and then continue as it is now, as amended by the feed for ... The statement "If the pathname begins with a <slash>, the specified path is resolved" is problematic because it only requires pathname resolution to be done for absolute pathnames, not relative ones. More poor wording, and I have no problem with fixing that. However, I don't think we should discard mention of what happens with a PATH search when the pathname being sought contains a '/' and rely upon the reader just understanding that since it says "filename" thatmust be true, while being left in the dark (until they find a use of PATH search) about what happens if there is a '/'. And then wonder why it is divided up that way, and if perhaps different applications of PATH searching might have different rules for what to do in that case. That is, I don't believe the proposed resolution in Note: 0004855 is good enough.

(0004864) geoffclare (manager) 2020-05-06 08:18	Re Note: 0004862 Seems I didn't look hard enough for the places that require a PATH search. I have now found a place that doesn't say PATH is not searched if there is a slash in the search string: find -exec (and -ok), where the requirement that a PATH search is done is only made via the PATH entry in ENVIRONMENT VARIABLES - it's not in the description of -exec. There is also a slight problem with fc -e and xargs because they both state that the utility to be invoked is specified by its name, implying that portable applications can't use fc -e or xargs to invoke a utility by means of a pathname that includes a slash. Once those are fixed to allow a pathname, they would also need to specify the no-slash rule for PATH searching, unless it is covered by the description of PATH. So I am now in agreement that the description of PATH should handle this. Your suggestion of "If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list of prefixes shall be searched ..." is okay with me as a way to do that. (I didn't compare the rest of your new PATH description with the current one to see what else you changed - it would be helpful if you could propose the changes you want in smaller chunks so there is less unchanged text to compare.) The problems with fc -e and xargs could either be addressed here or in a separate bug.

(0004866) kre (reporter) 2020-05-06 14:39	Here are the editing instructions to get to text shown in Note: 0004862 as requested in Note: 0004864 Al of this applies to the description of the environment variable PATH in XBD 8.3, on page 178 (2016 edition). In lines 5843-4 delete the words "known only by a filename". In line 5844, after the sentence that ends "...separated by a <colon> (':')." insert the words (beginning a new sentence) If the pathname being sought contains no slash ('/') characters, and hence is a filename, and then follow that by moving the sentence that starts on line 5851 The list shall be searched... (up to the end of that sentence, on line 5854) ...permissions is found. While moving that sentence, change the capital 'T' in the leading "The" to lower case, as it is no longer the beginning of a sentence, just a claise. That's it, the rest of the text is unchanged (since in Note: 0004862 I did not attempt to fix the "resolve the path" issue. What I might do for that is to replace the sentence that runs from line 5855-7, viz: If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111). and replace it with In each case, either the result of appending the filename to each prefix from PATH when the pathname (filename) contains no slash characters, or the pathname when it does contain one or more slash characters, is resolved (see Section 4.13, on page 111). but I am less happy with that wording than the earlier part. Someone please suggest something better.

(0004868) geoffclare (manager) 2020-05-07 08:50 edited on: 2020-05-07 08:51	Here's a suggestion that combines kre's solution from Note: 0004866 for the filename/pathname problem with my suggestion from Note: 0004855 for the pathname resolution problem... All of this applies to the description of the environment variable PATH in XBD 8.3, on page 178 (2016 edition). In lines 5843-4 delete the words "known only by a filename". In line 5844, after the sentence that ends "...separated by a <colon> (':')." insert the new sentence If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found. Delete the sentence that starts on line 5851 The list shall be searched... (up to the end of that sentence, on line 5854) ...permissions is found. Replace the two sentences that run from line 5854-7, viz: If the pathname being sought contains a <slash>, the search through the path prefixes shall not be performed. If the pathname begins with a <slash>, the specified path is resolved (see Section 4.13, on page 111). with: If the pathname being sought contains any <slash> characters, the search through the path prefixes shall not be performed and the pathname shall be resolved as described in Section 4.13, on page 111. The end result of these edits is the following as the first paragraph of the PATH description: This variable shall represent the sequence of path prefixes that certain functions and utilities apply in searching for an executable file. The prefixes shall be separated by a <colon> (':'). If the pathname being sought contains no slash ('/') characters, and hence is a filename, the list shall be searched from beginning to end, applying the filename to each prefix and attempting to resolve the resulting pathname (see Section 4.13, on page 111), until an executable file with appropriate execution permissions is found. When a non-zero-length prefix is applied to this filename, a <slash> shall be inserted between the prefix and the filename if the prefix did not end in <slash>. A zero-length prefix is a legacy feature that indicates the current working directory. It appears as two adjacent <colon> characters ("::"), as an initial <colon> preceding the rest of the list, or as a trailing <colon> following the rest of the list. A strictly conforming application shall use an actual pathname (such as .) to represent the current working directory in PATH. If the pathname being sought contains any <slash> characters, the search through the path prefixes shall not be performed and the pathname shall be resolved as described in Section 4.13, on page 111. If PATH is unset or is set to null, the path search is implementation-defined.

(0004869) kre (reporter) 2020-05-07 10:40	That (Note: 0004868) is fine with me.

(0004871) geoffclare (manager) 2020-05-11 15:19	Interpretation response ------------------------ The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Rationale: ------------- The current text does not require constructed relative pathnames to be resolved, only absolute ones. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- Make the changes in Note: 0004868

(0004872) ajosey (manager) 2020-05-11 15:27	Interpretation proposed: 11 May 2020

(0004888) ajosey (manager) 2020-06-12 09:09	Interpretation approved: 12 June 2020

Issue History
Date Modified	Username	Field	Change
2020-05-04 16:28	shware_systems	New Issue
2020-05-04 16:28	shware_systems	Name	=> Mark Ziegast
2020-05-04 16:28	shware_systems	Organization	=> SHware Systems Dev.
2020-05-04 16:28	shware_systems	Section	=> XBD 8.3
2020-05-04 16:28	shware_systems	Page Number	=> 178
2020-05-04 16:28	shware_systems	Line Number	=> 5854-7
2020-05-04 19:31	kre	Note Added: 0004846
2020-05-04 19:33	kre	Note Added: 0004847
2020-05-04 20:05	shware_systems	Note Added: 0004848
2020-05-05 03:00	kre	Note Added: 0004849
2020-05-05 03:16	shware_systems	Note Added: 0004850
2020-05-05 04:39	kre	Note Added: 0004851
2020-05-05 05:03	kre	Note Edited: 0004851
2020-05-05 05:04	kre	Note Edited: 0004851
2020-05-05 05:11	kre	Note Edited: 0004851
2020-05-05 05:13	kre	Note Edited: 0004851
2020-05-05 08:40	geoffclare	Note Added: 0004854
2020-05-05 08:46	geoffclare	Note Added: 0004855
2020-05-05 08:47	geoffclare	Note Edited: 0004855
2020-05-05 16:19	shware_systems	Note Added: 0004861
2020-05-05 16:21	kre	Note Added: 0004862
2020-05-06 08:18	geoffclare	Note Added: 0004864
2020-05-06 14:39	kre	Note Added: 0004866
2020-05-07 08:50	geoffclare	Note Added: 0004868
2020-05-07 08:51	geoffclare	Note Edited: 0004868
2020-05-07 10:40	kre	Note Added: 0004869
2020-05-11 15:19	geoffclare	Note Added: 0004871
2020-05-11 15:20	geoffclare	Interp Status	=> Pending
2020-05-11 15:20	geoffclare	Final Accepted Text	=> Note: 0004871
2020-05-11 15:20	geoffclare	Status	New => Interpretation Required
2020-05-11 15:20	geoffclare	Resolution	Open => Accepted As Marked
2020-05-11 15:20	geoffclare	Tag Attached: tc3-2008
2020-05-11 15:27	ajosey	Interp Status	Pending => Proposed
2020-05-11 15:27	ajosey	Note Added: 0004872
2020-06-12 09:09	ajosey	Interp Status	Proposed => Approved
2020-06-12 09:09	ajosey	Note Added: 0004888
2020-06-17 14:02	geoffclare	Status	Interpretation Required => Applied
2024-06-11 09:08	agadmin	Status	Applied => Closed

Relationships

Aardvark Mark IV