Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000498 [1003.1(2008)/Issue 7] System Interfaces Comment Enhancement Request 2011-10-02 21:20 2011-10-20 15:20
Reporter dtrebbien View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Resolved  
Name Daniel Trebbien
Organization
User Reference
Section _Exit, _exit
Page Number 545
Line Number 18923-18924
Interp Status ---
Final Accepted Text Note: 0000993
Summary 0000498: During process termination, open file descriptors should be closed as if by appropriate calls to close()
Description This relates to a question on Stack Overflow, What is the order in which a POSIX system clears the file locks that were not unlocked cleanly? <http://stackoverflow.com/q/7622279/196844>, [^] a copy of which is attached in PDF format.

The wording of the first-stated consequence of process termination is "All of the file descriptors, directory streams, conversion descriptors, and message catalog descriptors open in the calling process shall be closed." But, the details of how an open file descriptor is "closed" are not specified. Some details should be specified for the benefit of applications (notably SQLite) that need guarantees on the atomicity of the effects of closing each file descriptor in the case of regular files.

If an application implements a file locking-based synchronization method involving locks on multiple file segments of regular files, then it is important that all locks be removed atomically with respect to another thread's attempt to acquire a file segment lock. Because of section 2.9.7, Thread Interactions with Regular File Operations, the application is guaranteed atomicity if it closes a file descriptor uncleanly. However, that is not very useful because a well-written application likely closes all file locks anyway before closing the file descriptor. It would be much more useful to know what happens in the case of abnormal process termination.
Desired Action Under Consequences of Process Termination, replace the first item:


 * All of the file descriptors, directory streams, conversion descriptors, and message catalog descriptors open in the calling process shall be closed.


with two items:


 * All of the file descriptors open in the calling process shall be closed as if by appropriate calls to close().

 * All of the directory streams, conversion descriptors, and message catalog descriptors open in the calling process shall be closed.
Tags tc2-2008
Attached Files pdf file icon SO7622279.pdf [^] (139,140 bytes) 2011-10-02 21:20

- Relationships
related to 0000529Interpretation Requiredajosey fildes unspecified on close()'s [EINTR] 

-  Notes
(0000984)
Don Cragun (manager)
2011-10-06 20:33

I believe the intent is that process termination (normal or abnormal)
will close all file descriptors that were open in the process and
that releasing locks is an atomic operation no matter how the file
descriptor is closed. However, when a process terminates, the order
in which the file descriptors are closed is not specified and there
is clearly no requirement that all file descriptors will be closed
as an atomic operation.

The POSIX descriptions of directory streams, conversion descriptors,
and message catalog descriptors all say that they MAY use file
descriptors. The standard requires that file descriptors, directory
streams, conversion descriptors, and message catalog descriptors
all be closed on process termination. There is no requirement (and
there should not be any requirement) that file descriptors be closed
before directory streams, conversion descriptors, or message catalog
descriptors. If there is any reason that closing any object in
this list (other than a file descriptor) needs to have the underlying
file descriptor(s) open to successfully close the object (as viewed
by an external observer after the process has terminated), an
implementation needs to close that object before closing the
underlying file descriptor. Therefore, splitting the current item
in the list would be misleading.

Instead of making the change suggested by the submitter, I believe
the appropriate action is to change:

   All locks associated with a file for a given process shall be
   removed when a file descriptor for that file is closed by that
   process or the process holding that file descriptor terminates.

on P809, L26982-26983 in the DESCRIPTION section of fcntl() to:

   All locks associated with a file for a given process shall be
   removed atomically when a file descriptor for that file is closed
   by that process or the process holding that file descriptor
   terminates.
(0000985)
eblake (manager)
2011-10-06 22:44

It was also pointed out during the meeting that dup2() and FD_CLOEXEC can
both be used to close file descriptors, and that while exec mentions
effects as if by close() for FD_CLOEXEC (line 25752), dup leaves things
as merely the generic statement that things are closed (line 24872).
If we were to require things in terms of close(), then more places need
to be edited; which is why I think that the minimal edits proposed in
Note: 0000984, which document generic closing rather than specific ties
to close(), makes for a better solution.
(0000986)
geoffclare (manager)
2011-10-07 10:00

Section 2.9.7, Thread Interactions with Regular File Operations, makes
close() atomic with respect to many other file operations, not just
those involving locks. I think these should be addressed as well.

So instead of the change proposed for fcntl() in Note: 0000984 I suggest
appending something like the following to the last paragraph in 2.9.7
(P516 L17869):

    The requirement on the close() function shall also apply to
    all closing of file descriptors however caused (for example,
    as a consequence of process termination).
(0000987)
dtrebbien (reporter)
2011-10-07 12:22
edited on: 2011-10-07 12:24

I very much prefer the proposed wording change in Note: 0000986 because it
specifies behavior wherever the specification talks about a file descriptor
being "closed" without mentioning the intended atomic effects if the file
descriptor is associated with the file description of a regular file.

I also now think that a paragraph needs to be added to the description of
close(). At the end of the Description section, add the following paragraph:

    The effects of close() apply to all closing of file descriptors
    however caused (for example, as a consequence of process termination).

(0000988)
Don Cragun (manager)
2011-10-10 23:53
edited on: 2011-10-11 00:02

Note that if close() is interrupted by a signal, the state of the file
descriptor being closed is unspecified and any locks associated with the
file underlying the file descriptor may remain in place. When a process
terminates, no matter how termination happens, each file descriptor is
required to be closed and all locks associated with the files underlying
those file descriptors are required to be released. Since a signal
could be received during normal or abnormal process termination, I don't
believe saying that file descriptors are closed "as if by calling
close()" nor saying that "all of the effects of close() apply to all
closing of file descriptors however caused" guarantees that file
descriptors will be closed and file locks will be released on process
termination.

The standard needs to guarantee that file descriptors will be closed and
file locks will be released atomically on process termination. The
current wording in the description of Consequences of Process
Termination in the DESCRIPTION section of _Exit() and _exit() on P545,
L18923-18924 requires that file descriptors be closed on process
termination (unless this section is watered down as suggested in the
Desired Action for this bug report or made ambiguous by the change
suggested in Note: 0000986). Furthermore, the text in the description of
fcntl()'s description of advisory file locking (on P809, L26985-26986)
already requires that these locks be released when a file descriptor is
closed by the process or the process terminates (although atomic
unlocking is not clearly required). So, the only missing piece that has
been requested is to make it clear that all locks associated with a file
are released as an atomic operation when the file descriptor is closed
(no matter whether by calling close() or by process termination).


The wording change suggested in Note: 0000984
closes the loophole for process termination. The last sentence in the
first paragraph of the DESCRIPTION section of close() should also be
updated to clearly require that file locks for any file underlying a
file descriptor being closed by close() must be removed atomically.

So, make the change suggested in Note: 0000984
and also make a similar change in the DESCRIPTION of close().

On P676, L22869-22870 change:
    All outstanding record locks owned by the process on the file
    associated with the file descriptor shall be removed (that is,
    unlocked).
to:
    All outstanding record locks owned by the process on the file
    associated with the file descriptor shall be removed (that is,
    unlocked) atomically.

If there are additional actions being performed by close() that are
not currently documented in the list of Consequences of Process
Termination documented in the _Exit() function DESCRIPTION
section or in the description of whatever function enabled the
action in the first place, then the list of Consequences of Process
Termination actions needs to be updated as a separate bug report.

(0000989)
dtrebbien (reporter)
2011-10-11 12:11
edited on: 2011-10-11 12:38

I don't think that the additional wording change suggested in Note: 0000988 is
necessary because Section 2.9.7, Thread Interactions with Regular File Operations,
already makes very precise the atomic effects of close() for regular files;
unless, if the intent of Note: 0000988 is to extend the atomicity requirement
to all types of files, then something like the additional wording change
suggested in Note: 0000988 is required.

There seem to me to be three issues with the wording change suggested in Note: 0000984:

1. "shall be removed atomically" does not specify atomic with respect to what.
Does it mean that the effect (removal of all outstanding fcntl locks) is either
visible or hidden to threads that are attempting to lock a segment of *any*
file, or just the file associated with the file descriptor being closed?

2. As pointed out in Note: 0000985, it may not be enough to specify the
requirement of atomicity if "[the] file is closed by that process or the
process holding [the] file descriptor terminates". dup2() can "close" a file
descriptor as well. There might be other places in the specification that
describe a file descriptor being "closed", or, if none exist now, then perhaps
in the future.

3. The atomicity requirement would be extended to file descriptors associated
with all files, not just regular files. That may be a major change for
implementations to implement.

The reasons why I like the wording change suggested in Note: 0000986 include
that I do not believe it to have the same issues:

1. The wording is "The requirement", referring to *the* requirement specified
in the first paragraph of Section 2.9.7. So, there is no ambiguity about
atomic with respect to what because this is precisely specified by that first
paragraph.

2. The wording change makes clear that the atomicity requirement applies
"however" a file descriptor is closed. This includes the effect of dup2() as
well as possible future changes that describe the closing of a file descriptor.

3. The atomicity requirement is not extended to all files, just regular files,
as "The requirement" only applies to the effects of chmod(), close(), fchmod(),
fcntl(), fstat(), ftruncate(), lseek(), open(), read(), readlink(), stat(),
symlink(), and write() operating on regular files.

(0000993)
geoffclare (manager)
2011-10-20 15:18

Append the following to the last paragraph in 2.9.7 (P516 L17869):

     The requirement on the close() function shall also apply
     whenever a file descriptor is successfully closed however
     caused (for example, as a consequence of calling close(),
     calling dup2(), or of process termination).

- Issue History
Date Modified Username Field Change
2011-10-02 21:20 dtrebbien New Issue
2011-10-02 21:20 dtrebbien Status New => Under Review
2011-10-02 21:20 dtrebbien Assigned To => ajosey
2011-10-02 21:20 dtrebbien File Added: SO7622279.pdf
2011-10-02 21:20 dtrebbien Name => Daniel Trebbien
2011-10-02 21:20 dtrebbien Section => _Exit, _exit
2011-10-02 21:20 dtrebbien Page Number => unknown
2011-10-02 21:20 dtrebbien Line Number => unknown
2011-10-03 10:53 Don Cragun Page Number unknown => 545
2011-10-03 10:53 Don Cragun Line Number unknown => 18923-18924
2011-10-03 10:53 Don Cragun Interp Status => ---
2011-10-06 20:33 Don Cragun Note Added: 0000984
2011-10-06 22:44 eblake Note Added: 0000985
2011-10-07 10:00 geoffclare Note Added: 0000986
2011-10-07 12:22 dtrebbien Note Added: 0000987
2011-10-07 12:23 dtrebbien Note Edited: 0000987
2011-10-07 12:24 dtrebbien Note Edited: 0000987
2011-10-10 23:53 Don Cragun Note Added: 0000988
2011-10-11 00:02 Don Cragun Note Edited: 0000988
2011-10-11 12:11 dtrebbien Note Added: 0000989
2011-10-11 12:38 dtrebbien Note Edited: 0000989
2011-10-20 15:18 geoffclare Note Added: 0000993
2011-10-20 15:19 msbrown Tag Attached: tc2-2008
2011-10-20 15:20 msbrown Final Accepted Text => Note: 0000993
2011-10-20 15:20 msbrown Status Under Review => Resolved
2011-10-20 15:20 msbrown Resolution Open => Accepted As Marked
2012-02-17 02:18 eblake Relationship added related to 0000529


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker