View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000689 | 1003.1(2008)/Issue 7 | System Interfaces | public | 2013-05-05 15:37 | 2024-06-11 08:52 |
| Reporter | dalias | Assigned To | ajosey | ||
| Priority | normal | Severity | Editorial | Type | Clarification Requested |
| Status | Closed | Resolution | Accepted As Marked | ||
| Name | Rich Felker | ||||
| Organization | musl libc | ||||
| User Reference | |||||
| Section | XSH 2.5 | ||||
| Page Number | unknown | ||||
| Line Number | unknown | ||||
| Interp Status | Approved | ||||
| Final Accepted Text | 0000689:0006428 | ||||
| Summary | 0000689: Possibly unintended allowance for stdio deadlock | ||||
| Description | XSH 2.5 paragraph 2 reads: "When a stream is "unbuffered", bytes are intended to appear from the source or at the destination as soon as possible; otherwise, bytes may be accumulated and transmitted as a block. When a stream is "fully buffered", bytes are intended to be transmitted as a block when a buffer is filled. When a stream is "line buffered", bytes are intended to be transmitted as a block when a <newline> byte is encountered. Furthermore, bytes are intended to be transmitted as a block when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line-buffered stream that requires the transmission of bytes. Support for these characteristics is implementation-defined, and may be affected via setbuf() and setvbuf()." This intent, albeit implementation-defined and thus not in itself normative, reflects the traditional practice of having stdio input functions flush line-buffered streams, to accommodate lazy programming practices such as: printf("Prompt: "); scanf("%d", &x); with no intervening fflush. Unfortunately, encouraging or even permitting reads to flush all line-buffered output streams has some heavy locking consequences for multithreaded programs, including the possibility of deadlock. If thread A is holding a lock on a line-buffered output stream while waiting for a result from thread B, and thread B happens to use stdio for reading any file as part of its operation (unrelated to thread A's use of the line-buffered stream), the program will deadlock. This behavior seems highly undesirable and unintended. | ||||
| Desired Action | My understanding is that a definition of "multithreaded program" is being added to the standard, with the intent that certain legacy implementation practices (like the alarm-based sleep implementation) that are incompatible or problematic for multithreaded programs can be ruled out for multithreaded programs while still allowing them in singlethreaded programs. If so, I think it would make sense to make use of that here, by adding text along the lines of: "In a multithreaded program, performing an operation on a stream shall not cause any other open stream to be locked as if by flockfile. In particular, performing an input operation which results in bytes being transferred shall not cause line-buffered streams to be flushed in a multithreaded program." | ||||
| Tags | applied_after_i8d3, issue8 | ||||
|
|
Removing the automatic flush of output streams sounds like a rather severe solution that may surprise people negatively (something stops working after creating a thread). Note that this problem may also occur without explicit use of flockfile(), for example if two threads each wrap a pipe or socket in a FILE (directly or indirectly connected), set the appropriate buffering, one thread writes so much that the write operation blocks and the other thread tries to read. Even without the lock, it is impossible to flush the write buffer before reading. XSH 3 flockfile says that all functions that "reference (FILE *) objects" (except the *_unlocked ones) shall lock and unlock the FILE. Is the automatic flush a "reference" in this sense? If not, an implementation can use a flag for files that are already locked, to ask the current lock owner to flush the file. The flag would be checked at an explicit or implicit funlockfile() on a writable line-buffered file and possibly elsewhere. |
|
|
I would be perfectly happy with wording that still allows or encourages the traditional behavior as long as it's clear that the deadlock is not permitted. This could probably be implemented safely via a trylock approach. As this is at least the second issue that's come up as a result of incomplete and ambiguous specification of the implicit FILE locking (specifically, the fact that "referencing" is not defined anywhere), I think an additional part of the resolution should be adding text that clearly defines the intended automatic locking behavior. So far, the only places I've found where FILE objects are locked when the FILE pointer has not been passed as an argument by the application are fflush(NULL), exit(), and now this implicit flushing of line-buffered streams. In the first two cases, the specification of fflush and exit makes it clear that they act upon (and thus in a sense, "reference") all open FILE objects. But in the latter case, there's no text in the specification of fgetc or other input functions indicating that they "reference" all line-buffered streams. |
|
|
The quoted text is taken from the C Standard. Now that C11 has threads, it also has this issue. We should liaise with the C committee on a solution. |
|
|
Interpretation response ------------------------ The standard does not speak to this issue, and as such no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Rationale: ------------- There are possible deadlocks in multi-threaded applications that were not considered when POSIX added support for threading. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- After Issue 8 draft 3 page 520 line 18489 section 2.5: All functions that read, write, position, or query the position of a stream, except those with names ending _unlocked, shall lock the stream as if by a call to flockfile( ) before accessing it and release the lock as if by a call to funlockfile( ) when the access is complete.add: [CX]If the lock is not immediately available, the function shall wait for it to become available, except in the following circumstances. If the stream is line buffered and is open for writing or for update, and the reason the function is attempting to lock the stream is because it is going to request input on another stream that is unbuffered, or is line buffered and requires the transmission of characters from the host environment (see above), then the function shall attempt to determine whether a deadlock situation exists. If a deadlock situation is found to exist, the function shall fail. If the function is able to establish that a deadlock situation does not exist, it shall wait for the lock to become available. If the function does not establish whether or not a deadlock situation exists, it shall continue as if it had already locked the stream, found its buffer to be empty, and released the lock.[/CX] |
|
|
Interpretation proposed: 14 August 2023 |
|
|
Interpretation approved: 2 October 2023 |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2013-05-05 15:37 | dalias | New Issue | |
| 2013-05-05 15:37 | dalias | Status | New => Under Review |
| 2013-05-05 15:37 | dalias | Assigned To | => ajosey |
| 2013-05-05 15:37 | dalias | Name | => Rich Felker |
| 2013-05-05 15:37 | dalias | Organization | => musl libc |
| 2013-05-05 15:37 | dalias | Section | => XSH 2.5 |
| 2013-05-05 15:37 | dalias | Page Number | => unknown |
| 2013-05-05 15:37 | dalias | Line Number | => unknown |
| 2013-05-05 17:13 | jilles | Note Added: 0001588 | |
| 2013-05-05 18:11 | dalias | Note Added: 0001589 | |
| 2013-05-07 11:08 | geoffclare | Note Added: 0001591 | |
| 2023-08-10 16:29 | geoffclare | Note Added: 0006428 | |
| 2023-08-10 16:31 | geoffclare | Interp Status | => Pending |
| 2023-08-10 16:31 | geoffclare | Final Accepted Text | => 0000689:0006428 |
| 2023-08-10 16:31 | geoffclare | Status | Under Review => Interpretation Required |
| 2023-08-10 16:31 | geoffclare | Resolution | Open => Accepted As Marked |
| 2023-08-10 16:31 | geoffclare | Tag Attached: issue8 | |
| 2023-08-14 16:07 | agadmin | Interp Status | Pending => Proposed |
| 2023-08-14 16:07 | agadmin | Note Added: 0006432 | |
| 2023-10-02 14:30 | agadmin | Interp Status | Proposed => Approved |
| 2023-10-02 14:30 | agadmin | Note Added: 0006504 | |
| 2023-10-10 09:13 | geoffclare | Status | Interpretation Required => Applied |
| 2023-10-10 09:13 | geoffclare | Tag Attached: applied_after_i8d3 | |
| 2024-06-11 08:52 | agadmin | Status | Applied => Closed |
