Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000689 [1003.1(2008)/Issue 7] System Interfaces Editorial Clarification Requested 2013-05-05 15:37 2024-06-11 08:52
Reporter dalias View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Closed  
Name Rich Felker
Organization musl libc
User Reference
Section XSH 2.5
Page Number unknown
Line Number unknown
Interp Status Approved
Final Accepted Text Note: 0006428
Summary 0000689: Possibly unintended allowance for stdio deadlock
Description XSH 2.5 paragraph 2 reads:

"When a stream is "unbuffered", bytes are intended to appear from the source or at the destination as soon as possible; otherwise, bytes may be accumulated and transmitted as a block. When a stream is "fully buffered", bytes are intended to be transmitted as a block when a buffer is filled. When a stream is "line buffered", bytes are intended to be transmitted as a block when a <newline> byte is encountered. Furthermore, bytes are intended to be transmitted as a block when a buffer is filled, when input is requested on an unbuffered stream, or when input is requested on a line-buffered stream that requires the transmission of bytes. Support for these characteristics is implementation-defined, and may be affected via setbuf() and setvbuf()."

This intent, albeit implementation-defined and thus not in itself normative, reflects the traditional practice of having stdio input functions flush line-buffered streams, to accommodate lazy programming practices such as:

printf("Prompt: ");
scanf("%d", &x);

with no intervening fflush.

Unfortunately, encouraging or even permitting reads to flush all line-buffered output streams has some heavy locking consequences for multithreaded programs, including the possibility of deadlock. If thread A is holding a lock on a line-buffered output stream while waiting for a result from thread B, and thread B happens to use stdio for reading any file as part of its operation (unrelated to thread A's use of the line-buffered stream), the program will deadlock. This behavior seems highly undesirable and unintended.
Desired Action My understanding is that a definition of "multithreaded program" is being added to the standard, with the intent that certain legacy implementation practices (like the alarm-based sleep implementation) that are incompatible or problematic for multithreaded programs can be ruled out for multithreaded programs while still allowing them in singlethreaded programs. If so, I think it would make sense to make use of that here, by adding text along the lines of:

"In a multithreaded program, performing an operation on a stream shall not cause any other open stream to be locked as if by flockfile. In particular, performing an input operation which results in bytes being transferred shall not cause line-buffered streams to be flushed in a multithreaded program."
Tags applied_after_i8d3, issue8
Attached Files

- Relationships

-  Notes
(0001588)
jilles (reporter)
2013-05-05 17:13

 Removing the automatic flush of output streams sounds like a rather severe solution that may surprise people negatively (something stops working after creating a thread).

Note that this problem may also occur without explicit use of flockfile(), for example if two threads each wrap a pipe or socket in a FILE (directly or indirectly connected), set the appropriate buffering, one thread writes so much that the write operation blocks and the other thread tries to read. Even without the lock, it is impossible to flush the write buffer before reading.

XSH 3 flockfile says that all functions that "reference (FILE *) objects" (except the *_unlocked ones) shall lock and unlock the FILE. Is the automatic flush a "reference" in this sense? If not, an implementation can use a flag for files that are already locked, to ask the current lock owner to flush the file. The flag would be checked at an explicit or implicit funlockfile() on a writable line-buffered file and possibly elsewhere.
(0001589)
dalias (reporter)
2013-05-05 18:11

 I would be perfectly happy with wording that still allows or encourages the traditional behavior as long as it's clear that the deadlock is not permitted. This could probably be implemented safely via a trylock approach.

As this is at least the second issue that's come up as a result of incomplete and ambiguous specification of the implicit FILE locking (specifically, the fact that "referencing" is not defined anywhere), I think an additional part of the resolution should be adding text that clearly defines the intended automatic locking behavior. So far, the only places I've found where FILE objects are locked when the FILE pointer has not been passed as an argument by the application are fflush(NULL), exit(), and now this implicit flushing of line-buffered streams. In the first two cases, the specification of fflush and exit makes it clear that they act upon (and thus in a sense, "reference") all open FILE objects. But in the latter case, there's no text in the specification of fgetc or other input functions indicating that they "reference" all line-buffered streams.
(0001591)
geoffclare (manager)
2013-05-07 11:08

 The quoted text is taken from the C Standard. Now that C11 has threads, it also has this issue. We should liaise with the C committee on a solution.
(0006428)
geoffclare (manager)
2023-08-10 16:29

 Interpretation response
------------------------

The standard does not speak to this issue, and as such no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor.

Rationale:
-------------
There are possible deadlocks in multi-threaded applications that were not considered when POSIX added support for threading.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
After Issue 8 draft 3 page 520 line 18489 section 2.5:
All functions that read, write, position, or query the position of a stream, except those with names ending _unlocked, shall lock the stream as if by a call to flockfile( ) before accessing it and release the lock as if by a call to funlockfile( ) when the access is complete.
add:
[CX]If the lock is not immediately available, the function shall wait for it to become available, except in the following circumstances. If the stream is line buffered and is open for writing or for update, and the reason the function is attempting to lock the stream is because it is going to request input on another stream that is unbuffered, or is line buffered and requires the transmission of characters from the host environment (see above), then the function shall attempt to determine whether a deadlock situation exists. If a deadlock situation is found to exist, the function shall fail. If the function is able to establish that a deadlock situation does not exist, it shall wait for the lock to become available. If the function does not establish whether or not a deadlock situation exists, it shall continue as if it had already locked the stream, found its buffer to be empty, and released the lock.[/CX]
(0006432)
agadmin (administrator)
2023-08-14 16:07

 Interpretation proposed: 14 August 2023
(0006504)
agadmin (administrator)
2023-10-02 14:30

 Interpretation approved: 2 October 2023

- Issue History
Date Modified Username Field Change
2013-05-05 15:37 dalias New Issue
2013-05-05 15:37 dalias Status New => Under Review
2013-05-05 15:37 dalias Assigned To => ajosey
2013-05-05 15:37 dalias Name => Rich Felker
2013-05-05 15:37 dalias Organization => musl libc
2013-05-05 15:37 dalias Section => XSH 2.5
2013-05-05 15:37 dalias Page Number => unknown
2013-05-05 15:37 dalias Line Number => unknown
2013-05-05 17:13 jilles Note Added: 0001588
2013-05-05 18:11 dalias Note Added: 0001589
2013-05-07 11:08 geoffclare Note Added: 0001591
2015-05-07 08:34 dancol Issue Monitored: dancol
2023-08-10 16:29 geoffclare Note Added: 0006428
2023-08-10 16:31 geoffclare Interp Status => Pending
2023-08-10 16:31 geoffclare Final Accepted Text => Note: 0006428
2023-08-10 16:31 geoffclare Status Under Review => Interpretation Required
2023-08-10 16:31 geoffclare Resolution Open => Accepted As Marked
2023-08-10 16:31 geoffclare Tag Attached: issue8
2023-08-14 16:07 agadmin Interp Status Pending => Proposed
2023-08-14 16:07 agadmin Note Added: 0006432
2023-10-02 14:30 agadmin Interp Status Proposed => Approved
2023-10-02 14:30 agadmin Note Added: 0006504
2023-10-10 09:13 geoffclare Status Interpretation Required => Applied
2023-10-10 09:13 geoffclare Tag Attached: applied_after_i8d3
2024-06-11 08:52 agadmin Status Applied => Closed

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker