|Anonymous | Login||2023-12-05 12:51 UTC|
|Main | My View | View Issues | Change Log | Docs|
|Viewing Issue Simple Details|
|ID||Category||Severity||Type||Date Submitted||Last Update|
|0000657||[1003.1(2008)/Issue 7] System Interfaces||Objection||Clarification Requested||2013-02-08 22:46||2023-11-21 10:35|
|Priority||normal||Resolution||Accepted As Marked|
|Final Accepted Text||Note: 0006535|
|Summary||0000657: Conditions under which fmemopen() write a NUL to the buffer are insufficiently specified|
As updated by XSH/TC1/D3/0149, the fmemopen() description now states:
When a stream open for writing is flushed or closed, a null byte shall
be written at the current position or at the end of the buffer, depending
on the size of the contents. If a stream open for update is flushed or
closed and the last write has advanced the current buffer size, a null
byte shall be written at the end of the buffer if it fits.
The first sentence does not specify _how_ the choice of where to write the NUL depends on the size of the contents. Therefore, an implementation is presumably conforming if it writes the NUL to the current position whenever the size of the contents is an even number, and to the end of the buffer when it is odd, for example.
The second sentence only indicates that a NUL is (required to be) written if the last write advanced the current buffer size. So, a program that does a write which advanced the buffer size, then seeks back and does another write, and _then_ flushes or closes it cannot depend on a NUL to be written.
Provide an actual specification for when and where the NUL is written in the "open for writing" case, presumably something about the minimum of the two offsets.
Change the second sentence to match what seems to be the glibc behavior:
If a stream open for update is flushed or closed and the current buffer
size has been advanced by a write since the stream was opened or flushed,
a null byte shall be written at the end of the buffer if it fits.
Another spot or two where glibc and the standard disagree:
I'm not sure why the text for these interfaces was put in the standard, given that the words apparently had little to do with the implementation that existed at the time. It's not clear they're converging either.
Might I suggest that if single-sourced APIs are going to be pulled into the standard, that that source be consulted with to verify what part of them can be considered stable for inclusion (vs left as unspecified or undefined behavior) and what level of "change control" they're willing to pass over? Is the submission done with the desire to improve the original implementation from the feedback (I think the *at() API family was like this) or is it pretty much cast in stone? If the final spec doesn't match the implementation, is the implementation going to change or will the spec need to change?
If the answer from the glibc people is "we'll update to match the requirements of the spec in a future release", all the better for POSIX. If their answer is "the spec is wrong on these points, but we'll work with you to bring the spec in line in a timely fashion with what we say the behavior is", that would be wonderful too, because then we would at least have a spec that matched reality.
If their answer is "sorry, we got here first and have too many apps depending on their current behavior and/or may make future changes to their behavior without consultation", then these interfaces should be deprecated from the spec with a note about why...so that the other existing implementations can go shred those pages in their copies of POSIX and then reverse engineer what glibc is doing.
Lacking visible word from the glibc maintainers on this, my assumption is that the latter is the case, but no one wants to actually say it.
edited on: 2023-10-16 16:09
The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.
Notes to the Editor (not part of this interpretation):
Page and line numbers are for Issue 8 draft 3.
Change "size" (when talking about the argument) to "max_size" throughout.
Change "size" (when talking about the last position in the buffer) to "end position" throughout.
Change on P961, L32741:
Open the stream for update (reading and writing). Truncate the buffer contents.to:
Open the stream for update (reading and writing).
Change P961, L32744-32745 from:
If the mode argument includes 'b', then the stream shall be in binary mode; otherwise the stream shall be in text mode.to:
If the mode argument begins with 'w' and max_size is not zero, the buffer contents shall be truncated by writing a null byte at the beginning. If the mode argument includes 'b', the results are implementation-defined.
Change on P961, L32758-32762:
The stream shall also maintain the size of the current buffer contents; use of fseek() or fseeko() on the stream with SEEK_END shall seek relative to this size. If mode starts with 'r' or includes 'b', the size shall be set to the value given by the size argument and shall not change. Otherwise, the stream is in text mode and writable, and the size shall be variable; for modes w and w+ the initial size shall be zero and for modes a and a+ the initial size shall be:to:
The stream shall also maintain the end position of the current buffer contents; use of fseek() or fseeko() on the stream with SEEK_END shall seek relative to this end position. If mode starts with 'r' the end position shall be set to the value given by the max_size argument and shall not change. Otherwise, the stream is writable and the end position shall be variable; for modes w and w+ the initial end position shall be zero and for modes a and a+ the initial end position shall be:
Change on P962, L32775-32776:
When a stream open for writing in text mode is flushed or closed, a null byte shall be written at the current position or at the end of the buffer, depending on the size of the contents. If a stream open for update in text mode is flushed or closed and the last write has advanced the current buffer size, a null byte shall be written at the end of the buffer if it fits. If a stream is opened in binary mode, no additional null byte shall be written.to:
When a stream open for update (the mode argument includes '+') or for writing only is successfully written and the write advances the current buffer end position, a null byte shall be written at the new buffer end position if it fits.
Change on P963, L32822-32828:
Unlike fopen(), where a 'b' in the mode argument is required to have no effect, fmemopen() distinguishes between text and binary modes. Text mode guarantees that the underlying memory will always be null terminated after any write operation, and tracks the growth of the largest position written to up to that point; while binary mode only modifies the underlying buffer according to direct writes, while seeking relative to the full buffer size. The combination of append and binary modes is not commonly used, since any attempt to write to such a stream will necessarily fail because the stream does not dynamically grow beyond the initial size.to:
Implementations differ as regards how a 'b' in the mode argument affects the behavior. For some the 'b' has no effect, as is required for fopen(); others distinguish between text and binary modes.
Was any sort of buy-in from glibc maintainers obtained?
Has anyone verified that the updated wording match the current glibc behavior?
Put another way: is there any reason to be believe that the wording here is a useful reference vs just reverse engineering glibc?
Don Cragun (manager)
edited on: 2023-10-16 22:21
re: Note: 0006536
Yes,. Except for the change of making mode "w" truncate the buffer (as well as "w+". Looking at the glibc code we found that the original implementation of fmemopen() did truncate in both cases, but that was changed along with a note saying the change was make to conform to POSIX. The original POSIX text was derived from the glibc man page for fmemopen() and it was not verified against the source at that time. We have asked the maintainers if they are willing to go back to the original implementation; if they are not we can change this back to the current behavior (or make it unspecified) before the interpretation is finalized. Note that the BSD implementation of fmemopen() does truncate the buffer for both "w" and "w+".
We made binary mode implementation-defined because glibc dropped binary mode, but BSD still supports it.
|Interpretation Proposed: 16 October 2023|
|Interpretation approved: 20 November 2023|
|2013-02-08 22:46||philip-guenther||New Issue|
|2013-02-08 22:46||philip-guenther||Status||New => Under Review|
|2013-02-08 22:46||philip-guenther||Assigned To||=> ajosey|
|2013-02-08 22:46||philip-guenther||Name||=> Philip Guenther|
|2013-02-08 22:46||philip-guenther||Organization||=> OpenBSD|
|2013-02-08 22:46||philip-guenther||Section||=> fmemopen|
|2013-02-08 22:46||philip-guenther||Page Number||=> 867|
|2013-02-08 22:46||philip-guenther||Line Number||=> 28775|
|2013-02-14 16:55||eblake||Relationship added||related to 0000396|
|2013-02-14 16:55||eblake||Relationship added||related to 0000456|
|2013-03-28 23:54||philip-guenther||Note Added: 0001506|
|2014-01-30 04:06||eblake||Relationship added||related to 0000818|
|2023-10-16 16:06||geoffclare||Note Added: 0006535|
|2023-10-16 16:07||geoffclare||Interp Status||=> Pending|
|2023-10-16 16:07||geoffclare||Final Accepted Text||=> Note: 0006535|
|2023-10-16 16:07||geoffclare||Status||Under Review => Interpretation Required|
|2023-10-16 16:07||geoffclare||Resolution||Open => Accepted As Marked|
|2023-10-16 16:08||geoffclare||Note Edited: 0006535|
|2023-10-16 16:09||geoffclare||Note Edited: 0006535|
|2023-10-16 16:09||geoffclare||Tag Attached: issue8|
|2023-10-16 18:29||philip-guenther||Note Added: 0006536|
|2023-10-16 22:15||Don Cragun||Note Added: 0006539|
|2023-10-16 22:19||Don Cragun||Note Edited: 0006539|
|2023-10-16 22:21||Don Cragun||Note Edited: 0006539|
|2023-10-17 05:02||agadmin||Interp Status||Pending => Proposed|
|2023-10-17 05:02||agadmin||Note Added: 0006541|
|2023-11-20 09:58||agadmin||Interp Status||Proposed => Approved|
|2023-11-20 09:58||agadmin||Note Added: 0006575|
|2023-11-21 10:35||geoffclare||Status||Interpretation Required => Applied|
|2023-11-21 10:36||geoffclare||Tag Attached: applied_after_i8d3|
|Mantis 1.1.6[^] Copyright © 2000 - 2008 Mantis Group|