View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update | 
|---|---|---|---|---|---|
| 0001913 | 1003.1(2024)/Issue8 | Shell and Utilities | public | 2025-03-12 03:33 | 2025-10-30 16:44 | 
| Reporter | calestyo | Assigned To | geoffclare | ||
| Priority | normal | Severity | Editorial | Type | Enhancement Request | 
| Status | Under Review | Resolution | Reopened | ||
| Name | Christoph Anton Mitterer | ||||
| Organization | |||||
| User Reference | Shell & Utilities | ||||
| Section | 2.7.5, 2.7.6 | ||||
| Page Number | 2497 | ||||
| Line Number | 81097-81118 | ||||
| Interp Status | Pending | ||||
| Final Accepted Text | |||||
| Summary | 0001913: clarify/define the meaning of n<&n and m>&m redirections | ||||
| Description | Hey. This originated from a thread that started at https://collaboration.opengroup.org/operational/mailarch.php?soph=N&action=show&archive=austin-group-l&num=38114&limit=100&offset=100&sid= In short: The motivation was whether it's possible to portably differentiate whether a non-zero exit status originated from a utility or failed redirection on that. The idea was that, with the clarifications from #1879 and assuming that there is at least on exit status which a utility is known not to use, a construct like the following: ( command exec <some redirections> || exit 125; utility ) where the subshell is merely to clean up any redirections, can be used to differentiate between a redirection error (which above would be indicated by 125) or a non-zero exit status from the utility. For file descriptors less than or equal to 2, this should already be guaranteed to work portably, as POSIX demands those to be passed on to the utility. For FDs greater than two, this is no longer guaranteed, though. An idea was brought up by Harald van Dijk, that n<&n and m>&m could be used e.g.: ( command exec <some redirections> || exit 125; utility n<&n... m>&m... ) in order to "manually" pass on on any such FDs. Questions remained: a) Does that behaviour even follow from POSIX (in an obvious way where it's really clear that shells should behave like this, not just some wobbly way) b) Does it even solve the original problem, or could e.g. such a n<&n respectively m>&m fail itself (not e.g. because a file doesn't exist, but because of something like resource exhaustion, etc.) When the original thread was continued a bit later at: https://collaboration.opengroup.org/operational/mailarch.php?soph=N&action=show&archive=austin-group-l&num=38279&limit=100&offset=0&sid= it apparently turned out that n<&n and m>&m redirections are not defined because POSIX uses the wordings: "The redirection operator: [n]<&word shall duplicate one input file descriptor from another" respectively "The redirection operator: [n]>&word shall duplicate one output file descriptor from another" i.e. one from another, implying the two must not be the same, as Geoff Clare pointed out. | ||||
| Desired Action | 1. It shall be clarified whether or not the n<&n and m>&m forms are covered by the current wording of the standard. (One could perhaps also interpret the "one from another" as the "one" being the FD from the utility's PoV, and the "another" being the FD of the same number from the shell's PoV.) 2. If possible - i.e. no conflicting behaviour of shell implementations - it would be nice if the definition could be change in such a way, that it actually allows portably for the goals described above. I have no strong opinion one how this definition should look like, one suggestion on the list was that implementations that the n<&n and m>&m forms shalll be no-ops for implementations which *do* pass on FDs > 2, and for those that don't, it shall have the meaning of explicitly passing the given FD on. It would be even nicer, if the standard mention in a sentence that this is explicitly meant to be usable for the purpose of differentiating between utility and redirection errors. Thanks, Chris. | ||||
| Tags | tc1-2024 | ||||
|  | Desired Action: It would be even nicer, if the standard mention in a sentence that this is explicitly meant to be usable for the purpose of differentiating between utility and redirection errors.The standard does not exist for your personal benefit, so I do not think it should bless your highly specific use case. However, it would make sense to briefly mention the closing-high-FDs behavior as motivation. Something along the lines of Geoff's wording in the thread would be more than sufficient: So I would support updating the standard to require that n<&n and n>&n are always a no-op if fd n is open, except that if the shell normally closes fds > 2, that were opened with exec, when it executes a non-built-in utility, then applying n<&n or n>&n to such commands causes fd n to remain open. | 
|  | The standard does not exist for your personal benefit, so I do not think it should bless your highly specific use case. I don't think I was asking to change it to "my personal benefit" (actually my own use case doesn't need FDs > 2, so I'm already happy with that.). 
 What I at least would want to avoid is that people might ever come across this issue or the corresponding mailing list thread and assume that this is now the way to portably differentiate between utility non-zero exit status and redirection error, when wouldn't be really the case. If you don't think it is, fine for me,... we can still make the change here, but should also mention that this cannot be expected to portably allow the above. If you think it is and if you think that "briefly mentioning the closing-high-FDs behavior as motivation" is enough to also make sure that this is understood by any shell implementer, then I'm all good. My idea to mention the deeper purpose was merely for the case that would be needed. | 
|  | In the Mar 13, 2025 teleconference the following wording was agreed, but the bug is being left open for now for feedback. On page 2497 line 81097-81118 replace sections 2.7.5 and 2.7.6 with: 2.7.5 Duplicating a File Descriptor (Added after the meeting) A change to merge XRAT C.2.7.5 and C.2.7.6 will also be needed, plus a change to lines 135455-135459 in C.2.9.3. | 
|  | The only two things that came to my mind: 1) Over-pedantic and probably not really needed to take care of: This text is now for both <& and >& ... and the wording uses "or standard input or standard output" as well as "or if n is not specified and word evaluates to 0 or 1",... so what about things like <1 or >0 ? 2) As far as I understand the wording about the n = word case, wouldn't that also mean that e.g.: exec 8<some file; exec 8<8; utility would now be expected to keep FD 8 open when invoking utility, and not just a construct like: exec 8<some file; utility 8<8 ? | 
|  | 0001913:0007117: so what about things like <1 or >0 ?The former is covered by section 2.7.1 and redirects stdin from a file literally named "1"; the latter is covered by section 2.7.2 and redirects stdout to a file literally named "0". This report isn't about those forms of redirection. As far as I understand the wording about the n = word case, wouldn't that also mean that e.g.:I don't see how, assuming you actually mean exec 8<somefile; exec 8<&8; utilityThe proposed wording says (emphasis mine): if the shell would have closed the file descriptor because it was opened using exec and has a value greater than 2, when the redirection is being performed in a command that will execute a non-built-in utility, the file descriptor shall instead remain open when the utility is executedYour second exec command isn't executing a non-built-in utility. | 
|  | 0001913:0007112: What I at least would want to avoid is that people might ever come across this issue or the corresponding mailing list thread and assume that this is now the way to portably differentiate between utility non-zero exit status and redirection error, when wouldn't be really the case.Why not? Is it because the redirections can't be absolutely guaranteed to succeed, as Geoff mentioned on the mailing list? If it is a no-op then it can't fail. So the only possible failure case would be in the "remain open" requirement. In practice this will involve calling fcntl() to clear the FD_CLOEXEC flag, which could indeed fail because of something like resource exhaustion, but I don't see that it increases the likelihood of internal shell failure significantly. Any command execution can fail within the shell because of resource exhaustion (e.g. fork() failure) before it gets as far as doing the exec.https://www.mail-archive.com/austin-group-l@opengroup.org/msg13640.html If you don't think it is, fine for me,... we can still make the change here, but should also mention that this cannot be expected to portably allow the above.I don't think the standard should mention your use case at all. If you think it is and if you think that "briefly mentioning the closing-high-FDs behavior as motivation" is enough to also make sure that this is understood by any shell implementer, then I'm all good.Alright. The readers I had in mind were application developers who might otherwise assume that n<&n serves no purpose whatsoever. | 
|  | > This text is now for both <& and >& ... and the wording uses "or standard input or standard output" as well as "or if n is not specified and word evaluates to 0 or 1", The proposed text is careful to use "respectively" where necessary. > so what about things like <1 or >0 ? I assume this was meant to be "like <&1 or >&0", in which case the answer is "a redirection error may result" (assuming that 0 is open readonly and 1 is open writeonly). | 
|  | Interpretation response ------------------------ The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Rationale: ------------- As the submitter points out, the wording in the standard "one input file descriptor from another" implies that they are two different file descriptors, and so the behavior of n<&n is currently unspecified (implicitly). Likewise for m>&m. There is an example in XRAT C.2.9.3 Lists (under "Asynchronous AND-OR Lists") which shows that the intention is for <&0 (and therefore also 0<&0) to be a no-op, but since this is non-normative it does not affect the requirements of the standard. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- On page 2497 line 81097-81118 replace sections 2.7.5 and 2.7.6 with: 2.7.5 Duplicating a File Descriptorand renumber 2.7.7 to 2.7.6.The input redirection operator:[n]<&wordand output redirection operator:[n]>&wordshall duplicate one input file descriptor or output file descriptor, respectively, from another, or shall close one. On page 2506 line 81501 section 2.9.3.1 Asynchronous AND-OR Lists, change: If, and only if, job control is disabled, the standard input for the subshell in which an asynchronous AND-OR list is executed shall initially be assigned to an open file description that behaves as if /dev/null had been opened for reading only. This initial assignment shall be overridden by any explicit redirection of standard input within the AND-OR list.to: If, and only if, job control is disabled, the standard input for the subshell in which an asynchronous AND-OR list is executed shall be assigned to an open file description that behaves as if /dev/null had been opened for reading only, except that: On page 3893 line 135146 section C.2.7.5, change the section heading: Duplicating an Input File Descriptorto: Duplicating a File Descriptor After page 3893 line 135152 section C.2.7.5, add a paragraph: If word and n evaluate to the same open file descriptor, the operation is a no-op except in shells which set the close-on-exec flag for file descriptors greater than 2 opened using [xref to exec]. In these shells, a redirection of this form can be used to clear the close-on-exec flag so that the file descriptor will remain open when executing a non-built-in utility. For example:exec 3<infile 4>outfile utility 3<&3 4>&4One use for this feature, together with command and exec, is to differentiate between utility and redirection errors. For example: On page 3893 line 135155-135158, delete section C.2.7.6 Duplicating an Output File Descriptor On page 3894 line 135159, change section number C.2.7.7 to C.2.7.6 On page 3901 line 135455 section C.2.9.3 Lists, change: Since the connection of the input to the equivalent of /dev/null is considered to occur before redirections, the following script would produce no output:to:exec < /etc/passwd cat <&0 & wait The assignment of standard input from an open file description that behaves like /dev/null is not overridden by an explicit <tt><&0</tt> redirection because this redirection does not perform a duplication and thus has no effect on where standard input comes from. This was the original Korn Shell behavior but was not clearly required by versions of this standard earlier than Issue 8 TC1, although in all those versions there was rationale stating that the following script would produce no output:exec < /etc/passwd cat <&0 & wait | 
|  | Interpretation Proposed: 23rd October 2025 | 
|  | Currently, in multiple shells, even the non-close-on-exec shells, 8<&8 is not a no op if fd 8 was closed: it results in an error exactly as 0<&8 would. By my reading of the proposed wording, in shells that do not set close-on-exec, this would be required to be silently accepted, and in shells that do set close-on-exec, this would effectively be required to result in an error (an error is the natural consequence of the required behaviour). This inconsistency does not look useful to me. This should be accepted in all shells, an error in all shells, or unspecified whether it results in an error, not tied to close-on-exec. | 
|  | Sorry, I did not read the thread, but to detect redirection error, you can do 
there_was_a_redirection_error=true
{
   there_was_a_redirection_error=false
   utility
} > this < that 2> redirection
if "$there_was_a_redirection_error"; then
  echo There was a redirection error and utility was not run
fi
 | 
|  | Re: 0001913:0007277 > If the file descriptor denoted by word represents an open file descriptor that is not open for input or open for output, respectively, a redirection error may result Why? Is there any shell that would report an error on 3<&4 if 4 was open in O_WRONLY? I've not come across any myself where both 3<&4 and 3>&4 were not both just doing just a dup2(4, 3) or equivalent. | 
|  | Re: 0001913:0007293 > Why? Is there any shell that would report an error on 3<&4 if 4 was open in O_WRONLY? I've not come across any myself where both 3<&4 and 3>&4 were not both just doing just a dup2(4, 3) or equivalent. This is not the subject of this bug, this wording came about because of 0001536. Previously, the wording specified "if the digits in word do not represent a file descriptor already open for input, a redirection error shall result" and likewise for >&. That wording was adjusted to no longer require that to reflect the reality that shells just did not implement that. The previously required behaviour continues to be permitted, making it so that POSIX doesn't suddenly start saying "echo error <&2" is just peachy when the only effect over >&2 would be to confuse readers. | 
|  | Re 0001913:0007291 > Currently, in multiple shells, even the non-close-on-exec shells, 8<&8 is not a no op if fd 8 was closed: it results in an error exactly as 0<&8 would. By my reading of the proposed wording, in shells that do not set close-on-exec, this would be required to be silently accepted Your reading is wrong. Taking the stated conditions at the start of each paragraph in the new text for 2.7.5 in turn: "If word consists of one or more decimal digits which evaluate to a value not equal to n" is not satisfied because word is equal to n. "If word and n evaluate to the same open file descriptor" is not satisfied because n is not open. "If word evaluates to '-'" is not satisfied. That just leaves "If word evaluates to something else, the behavior is unspecified.". So the behaviour is unspecified. | 
|  | Re: 0001913:0007295 Thanks, good to know it's meant to be allowed to result in an error. That same "open" is not present in the immediately following "or if n is not specified and word evaluates to 0 or 1" though, so if that is the key word, 'exec <&-; exec 0<&0' would be allowed to result in an error, but 'exec <&-; exec <&0' would not be. | 
|  | > That same "open" is not present in the immediately following "or if n is not specified and word evaluates to 0 or 1" though I suppose we could add a suitable condition to the 0 or 1 case, but I'm not sure it is worth making the text even more complicated just for that contrived case. A better way to handle it would be to make the current statement in XSH exec apply to other forms of execution by changing the paragraph about closing a file descriptor to: If word evaluates to '-', then file descriptor n (or standard input or standard output, respectively, if n is not specified) shall be closed. Attempts to close a file descriptor that is not open shall not constitute an error. If a standard utility or a conforming application is executed with file descriptor 0 not open for reading or with file descriptor 1 or 2 not open for writing, the environment in which the utility or application is executed shall be deemed non-conforming, and consequently the utility or application might not behave as described in this standard. (last sentence copied from XSH exec). | 
|  | > I suppose we could add a suitable condition to the 0 or 1 case, but I'm not sure it is worth making the text even more complicated just for that contrived case. Wording that covers it wouldn't need to be more complicated and could be based on the previous paragraph: If word consists of one or more decimal digits which evaluate to a value not equal to n (or 0 or 1, respectively, if n is not specified), [...] If word evaluates to the same open file descriptor as n (or 0 or 1, respectively, if n is not specified), [...] I'd be concerned about changing the requirements on closing fds. sh -c 'exec 3<&0; exec <&-; exec <&3' should be fine (it's sensible and works in existing shells). sh -c 'exec <&-; exec <&0' should be unspecified (it's an error in several existing shells). Note that neither executes any standard utility with fd 0 closed. Tweaks to the wording on closing fd 0/1/2 would, I think, either disallow both, or allow both. | 
|  | > If word evaluates to the same open file descriptor as n (or 0 or 1, respectively, if n is not specified), [...] That works, thanks for the suggestion. > I'd be concerned about changing the requirements on closing fds. At the moment there is only some non-normative text about this in C.2.7: Applications should not use the [n]<&- or [n]>&- operators to execute a utility or application with file descriptor 0 not open for reading or with file descriptor 1 or 2 not open for writing, as this might cause the executed program (or shell built-in) to misbehave. I think there ought to be some normative text to back this up. > sh -c 'exec 3<&0; exec <&-; exec <&3' should be fine (it's sensible and works in existing shells). I disagree that it's sensible. The "exec <&-" does not achieve anything useful and can just be omitted. Although, having said that, I would not object to an exception for "an exec command that reopens the closed file descriptor" in my proposed normative addition. | 
|  | New suggested Interpretation response ------------------------ The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Rationale: ------------- As the submitter points out, the wording in the standard "one input file descriptor from another" implies that they are two different file descriptors, and so the behavior of n<&n is currently unspecified (implicitly). Likewise for m>&m. There is an example in XRAT C.2.9.3 Lists (under "Asynchronous AND-OR Lists") which shows that the intention is for <&0 (and therefore also 0<&0) to be a no-op, but since this is non-normative it does not affect the requirements of the standard. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- On page 2497 line 81097-81118 replace sections 2.7.5 and 2.7.6 with: 2.7.5 Duplicating a File Descriptorand renumber 2.7.7 to 2.7.6.The input redirection operator:[n]<&wordand output redirection operator:[n]>&wordshall duplicate one input file descriptor or output file descriptor, respectively, from another, or shall close one. On page 2506 line 81501 section 2.9.3.1 Asynchronous AND-OR Lists, change: If, and only if, job control is disabled, the standard input for the subshell in which an asynchronous AND-OR list is executed shall initially be assigned to an open file description that behaves as if /dev/null had been opened for reading only. This initial assignment shall be overridden by any explicit redirection of standard input within the AND-OR list.to: If, and only if, job control is disabled, the standard input for the subshell in which an asynchronous AND-OR list is executed shall be assigned to an open file description that behaves as if /dev/null had been opened for reading only, except that: After page 3891 line 135051 section C.2.7, add: When a utility is executed with a redirection, for example:utility < infile > outfileit is generally not possible to tell from the exit status of the command whether a non-zero value is caused by failure of the utility or failure of the redirection. One method of differentiating the two is to apply the redirection to a compound command and include a command within the compound command that affects the current shell environment, such as setting a shell variable. For example: On page 3893 line 135146 section C.2.7.5, change the section heading: Duplicating an Input File Descriptorto: Duplicating a File Descriptor After page 3893 line 135152 section C.2.7.5, add a paragraph: If word and n evaluate to the same open file descriptor, the operation is a no-op except in shells which set the close-on-exec flag for file descriptors greater than 2 opened using [xref to exec]. In these shells, a redirection of this form can be used to clear the close-on-exec flag so that the file descriptor will remain open when executing a non-built-in utility. For example:exec 3<infile 4>outfile utility 3<&3 4>&4One use for this feature, together with command and exec, is an alternative method to the one described in [xref to C.2.7] of differentiating between utility and redirection errors. For example: On page 3893 line 135155-135158, delete section C.2.7.6 Duplicating an Output File Descriptor On page 3894 line 135159, change section number C.2.7.7 to C.2.7.6 On page 3901 line 135455 section C.2.9.3 Lists, change: Since the connection of the input to the equivalent of /dev/null is considered to occur before redirections, the following script would produce no output:to:exec < /etc/passwd cat <&0 & wait The assignment of standard input from an open file description that behaves like /dev/null is not overridden by an explicit <tt><&0</tt> redirection because this redirection does not perform a duplication and thus has no effect on where standard input comes from. This was the original Korn Shell behavior but was not clearly required by versions of this standard earlier than Issue 8 TC1, although in all those versions there was rationale stating that the following script would produce no output:exec < /etc/passwd cat <&0 & wait | 
| Date Modified | Username | Field | Change | 
|---|---|---|---|
| 2025-03-12 03:33 | calestyo | New Issue | |
| 2025-03-12 07:00 | larryv | Note Added: 0007111 | |
| 2025-03-13 02:41 | calestyo | Note Added: 0007112 | |
| 2025-03-13 16:12 | geoffclare | Note Added: 0007115 | |
| 2025-03-13 17:48 | calestyo | Note Added: 0007117 | |
| 2025-03-13 20:20 | larryv | Note Added: 0007119 | |
| 2025-03-13 20:43 | larryv | Note Added: 0007120 | |
| 2025-03-14 09:44 | geoffclare | Note Edited: 0007115 | |
| 2025-03-18 12:30 | geoffclare | Note Added: 0007125 | |
| 2025-03-20 14:49 | geoffclare | Note Edited: 0007115 | |
| 2025-09-25 11:34 | geoffclare | Note Added: 0007277 | |
| 2025-10-23 15:34 | geoffclare | Note Edited: 0007277 | |
| 2025-10-23 15:36 | geoffclare | Note Edited: 0007277 | |
| 2025-10-23 15:37 | geoffclare | Status | New => Interpretation Required | 
| 2025-10-23 15:37 | geoffclare | Resolution | Open => Accepted As Marked | 
| 2025-10-23 15:37 | geoffclare | Interp Status | => Pending | 
| 2025-10-23 15:37 | geoffclare | Final Accepted Text | => 0001913:0007277 | 
| 2025-10-23 15:38 | geoffclare | Tag Attached: tc1-2024 | |
| 2025-10-23 15:52 | ajosey | Interp Status | Pending => Proposed | 
| 2025-10-23 15:52 | ajosey | Note Added: 0007289 | |
| 2025-10-23 16:46 | hvd | Note Added: 0007291 | |
| 2025-10-23 21:17 | stephane | Note Added: 0007292 | |
| 2025-10-23 21:24 | stephane | Note Added: 0007293 | |
| 2025-10-25 18:47 | hvd | Note Added: 0007294 | |
| 2025-10-28 11:59 | geoffclare | Note Added: 0007295 | |
| 2025-10-28 12:35 | hvd | Note Added: 0007296 | |
| 2025-10-28 15:52 | geoffclare | Note Added: 0007297 | |
| 2025-10-28 16:30 | hvd | Note Added: 0007298 | |
| 2025-10-30 15:23 | geoffclare | Note Added: 0007299 | |
| 2025-10-30 16:41 | geoffclare | Assigned To | => geoffclare | 
| 2025-10-30 16:41 | geoffclare | Status | Interpretation Required => Under Review | 
| 2025-10-30 16:41 | geoffclare | Resolution | Accepted As Marked => Reopened | 
| 2025-10-30 16:41 | geoffclare | Note Added: 0007300 | |
| 2025-10-30 16:44 | geoffclare | Interp Status | Proposed => Pending | 
| 2025-10-30 16:44 | geoffclare | Final Accepted Text | 0001913:0007277 => | 
