Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000850 [1003.1(2013)/Issue7+TC1] System Interfaces Editorial Enhancement Request 2014-06-24 22:11 2014-07-10 18:22
Reporter mdempsky View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Resolved  
Name Matthew Dempsky
Organization OpenBSD
User Reference
Section mmap
Page Number
Line Number
Interp Status ---
Final Accepted Text See Note: 0002305.
Summary 0000850: Standardize MAP_ANON
Description It seems that all modern UNIX-like systems provide an mmap() flag to allow allocating anonymous memory directly without needing a file descriptor. Here are my notes from looking through various online manual pages (e.g., Solaris, Linux, *BSD, AIX, HPUX, UnixWare):

1. Most systems call it MAP_ANON, but HPUX, AIX, and UnixWare call it
MAP_ANONYMOUS. Some systems provide both; e.g., FreeBSD.

2. Mac OS X allows Mach VM flags to be passed via the "fd" parameter,
but recognizes "-1" as "no flags". Other OSes reject MAP_ANON(YMOUS)?
with fd != -1. Common practice seems to be EINVAL, but at least AIX
specifies EBADF instead.

3. FreeBSD requires offset==0 when using MAP_ANON; Linux and Mac OS X
explicitly document that it's ignored, and other OSes don't seem to
care.


First attempt at desired action below. I made a best effort at formatting it as I've seen done in previous bug reports. Two notes:

  1. Defining a new "anonymous memory object" type seemed the easiest way to tackle the idea, but the definition is kind of weak. Maybe there's a better way to define it.

  2. There are a *lot* of places in mmap() that mention "fildes", and it would be tedious to add "Unless MAP_ANON is specified" to all of them, so I tried to word MAP_ANON to make it clear that fildes shouldn't matter. Not sure if that's appropriate.
Desired Action In XBD 3 Definitions, add (and renumber subsequent entries as appropriate):

    3.XXX Anonymous Memory Object

        An object that represents memory not associated with any other memory objects.

In XBD 3 Definitions under 3.220 Memory Object, between "A file" and "A shared memory object", add:

    * An anonymous memory object (see <a>Anonymous Memory Object</a>).

In <sys/mman.h> under the "symbolic constants for use as flag options" section, add:

    MAP_ANON
        Map anonymous memory.

In mmap() under "The mmap() function shall be supported for the following memory objects", add:

    * Anonymous memory objects.

In mmap()'s table about MAP_* symbolic constants, add a row:

    MAP_ANON | Map anonymous memory.

In mmap() at an appropriate place (e.g., after the paragraph beginning with "If MAP_FIXED is specified and addr is non-zero"?), add a paragraph:

    If MAP_ANON is specified, fildes is -1, and off is 0, then mmap() shall ignore fildes and instead establish a mapping to a new anonymous memory object of size length. The effect of specifying MAP_ANON with other values of fildes or off is unspecified.

In msync() change:

    [SHM|TYM]The effect of msync() on a shared memory object or a typed memory object is unspecified.[/]

to:

    The effect of msync() on an anonymous memory object, [SHM]shared memory object[/], or [TYM]typed memory object[/] is unspecified.
Tags issue8
Attached Files

- Relationships

-  Notes
(0002274)
mdempsky (reporter)
2014-06-24 23:21

Two more thoughts:

  1. Does a lifetime for anonymous memory objects need to be defined? E.g., "Anonymous memory objects are destroyed once they're no longer mapped into any process's address space." in the definition?

  2. I suppose it should actually specify "new zero-filled anonymous memory object of size length" to clarify the initial memory contents.
(0002281)
rhansen (manager)
2014-06-26 16:40
edited on: 2014-07-10 15:58

On page 37 after line 1195 (XBD 3.15 definition of Angle Brackets), add a new section (and renumber subsequent entries as appropriate):

3.16 Anonymous Memory Object
An object that represents memory not associated with any other memory objects.


On page 68 after line 1972 (XBD 3.22 definition of Memory Object), add:

  • An anonymous memory object (see Section 3.16 on page 37).


On page 372 after line 12515 (XBD <sys/mman.h> description), add:

MAP_ANONYMOUS
Map anonymous memory.


On page 1322 after line 43724 (XSH mmap() description), add:

  • Anonymous memory objects.


On page 1322 line 43730 (XSH mmap() description) change:

The mmap() function shall establish a mapping between the address space of the process at an address pa for len bytes to the memory object represented by the file descriptor fildes at offset off for len bytes.


to:

The mmap() function shall establish a mapping between the address space of the process at an address pa for len bytes to the memory object represented by the file descriptor fildes at offset off for len bytes, or to an anonymous memory object of len bytes.


On page 1323 after line 43773 (XSH mmap() description, table about MAP_* symbolic constants), add a row:

MAP_ANONYMOUS | Map anonymous memory.


On page 1324 after line 43823 (XSH mmap() description), add a new paragraph:

If MAP_ANONYMOUS is specified, fildes is -1, and off is 0, then mmap() shall ignore fildes and instead establish a mapping to a new anonymous memory object of size len. The effect of specifying MAP_ANONYMOUS with other values of fildes or off is unspecified. Anonymous memory objects shall be initialized to all bits zero.


On page 1328 after line 43989 (XSH mmap() rationale), add a new paragraph:

Many existing implementations support MAP_ANON as a synonym for MAP_ANONYMOUS. MAP_ANONYMOUS was chosen for standardization because that spelling is supported in more existing implementations.


On page 1365 lines 45154-45155 (XSH msync() description) change:

[SHM|TYM]The effect of msync() on a shared memory object or a typed memory object is unspecified.[/]


to:

The effect of msync() on an anonymous memory object, [SHM]shared memory object[/], or [TYM]typed memory object[/] is unspecified.


(0002282)
rhansen (manager)
2014-06-26 16:50

This bug was discussed during today's teleconference but we ran out of time. We will continue the discussion at the next teleconference (two weeks from today). To encourage review and feedback, the current draft is in Note: 0002281. That bugnote might be edited in the future.
(0002283)
geoffclare (manager)
2014-06-26 16:51

The mmap() ERRORS section is going to need some work, notably for EACCES, EBADF, and ENODEV.
(0002284)
mdempsky (reporter)
2014-06-26 17:49

Just curious, can you share the rationale from the conference call on why MAP_ANONYMOUS was chosen instead of MAP_ANON? I suppose because MAP_ANONYMOUS is more common among UNIX 03 certified systems?
(0002285)
mdempsky (reporter)
2014-06-26 18:09

Oops, nevermind, reading the proposed wording I see that rationale was given.
(0002292)
geoffclare (manager)
2014-06-28 08:08

Some info regarding the two names...

On searchcode.com MAP_ANON gets 3289 hits and MAP_ANONYMOUS gets 2206. Since
MAP_ANON matches both names this implies that MAP_ANON by itself only has 1083.

Interestingly, the first hit for MAP_ANONYMOUS is this code snippet from the python source:

/* Prefer MAP_ANONYMOUS since MAP_ANON is deprecated according to man page. */
#if !defined(MAP_ANONYMOUS) && defined(MAP_ANON)
# define MAP_ANONYMOUS MAP_ANON
#endif
(0002295)
mdempsky (reporter)
2014-06-30 19:02

For what it's worth, there's a widespread mmap-anon.m4 autoconf macro package from the Free Software Foundation that makes this claim at the top:

# Detect how mmap can be used to create anonymous (not file-backed) memory
# mappings.
# - On Linux, AIX, OSF/1, Solaris, Cygwin, Interix, Haiku, both MAP_ANONYMOUS
# and MAP_ANON exist and have the same value.
# - On HP-UX, only MAP_ANONYMOUS exists.
# - On Mac OS X, FreeBSD, NetBSD, OpenBSD, only MAP_ANON exists.
# - On IRIX, neither exists, and a file descriptor opened to /dev/zero must be
# used.
(0002296)
rhansen (manager)
2014-06-30 19:38

According to [1], FreeBSD now supports MAP_ANONYMOUS as a synonym for MAP_ANON. The NetBSD, OpenBSD, and Mac OS X man pages do not mention MAP_ANONYMOUS.

[1] http://www.freebsd.org/cgi/man.cgi?query=mmap&sektion=2 [^]
(0002298)
rhansen (manager)
2014-07-01 03:16

NetBSD just committed a change adding MAP_ANONYMOUS as a synonym for MAP_ANON.

http://mail-index.netbsd.org/source-changes/2014/06/30/msg055938.html [^]
(0002299)
mdempsky (reporter)
2014-07-01 22:18

OpenBSD would still prefer to see MAP_ANON standardized, as that's the more historic name and (slightly) more widely available, even if Linux's man pages chose to document it as "deprecated". (MAP_ANON was added to BSD's <sys/mman.h> header by Kirk McKusick on December 5, 1990; MAP_ANONYMOUS was added to Linux's <linux/mman.h> header by Linus on Jan 18, 1994.) But as long as the semantics remain compatible, I don't object to MAP_ANONYMOUS.

Also, I've submitted an Apple Bug Report (17525652) to alert Apple's engineering team about this proposal. Hopefully they'll chime in directly, but if not I'll proxy anything I hear back from them.
(0002300)
mdempsky (reporter)
2014-07-01 23:37

One more data point: the third edition of "Advanced Programming in the UNIX environment" (published 2013) still recommends using MAP_ANON: http://books.google.com/books?id=_kpsAQAAQBAJ&lpg=PA578&pg=PA578 [^]

According to http://apuebook.com/about3e.html [^] (since the Preface section is missing from the Google Books preview): "The four platforms used to test the examples in the book include FreeBSD 8.0, Linux 3.2.0, Mac OS X 10.6.8 and Solaris 10."
(0002305)
rhansen (manager)
2014-07-10 05:23
edited on: 2014-07-10 15:56

On page 37 after line 1195 (XBD 3.15 definition of Angle Brackets), add a new section (and renumber subsequent entries as appropriate):

3.16 Anonymous Memory Object
An object that represents memory not associated with any other memory objects.


On page 68 after line 1972 (XBD 3.22 definition of Memory Object), add:

  • An anonymous memory object (see Section 3.16 on page 37).


On page 372 after line 12515 (XBD <sys/mman.h> description), add:

MAP_ANON
Synonym for MAP_ANONYMOUS. MAP_ANON shall have the same value as MAP_ANONYMOUS.

MAP_ANONYMOUS
Map anonymous memory.


On page 1322 after line 43724 (XSH mmap() description), add:

  • Anonymous memory objects.


On page 1322 line 43730 (XSH mmap() description) change:

The mmap() function shall establish a mapping between the address space of the process at an address pa for len bytes to the memory object represented by the file descriptor fildes at offset off for len bytes.


to:

The mmap() function shall establish a mapping between the address space of the process at an address pa for len bytes to the memory object represented by the file descriptor fildes at offset off for len bytes, or to an anonymous memory object of len bytes.


On page 1323 after line 43773 (XSH mmap() description, table about MAP_* symbolic constants), add two new rows:

MAP_ANON | Synonym for MAP_ANONYMOUS.
MAP_ANONYMOUS | Map anonymous memory.


On page 1324 after line 43823 (XSH mmap() description), add a new paragraph:

If MAP_ANONYMOUS (or its synonym MAP_ANON) is specified, fildes is -1, and off is 0, then mmap() shall ignore fildes and instead establish a mapping to a new anonymous memory object of size len. The effect of specifying MAP_ANONYMOUS (or MAP_ANON) with other values of fildes or off is unspecified. Anonymous memory objects shall be initialized to all bits zero.


On page 1325 lines 43881-43882 change:

[ENXIO]
MAP_FIXED was specified in flags and the combination of addr, len, and off is invalid for the object specified by fildes.


to:

[ENXIO]
MAP_FIXED was specified in flags and the combination of addr, len, and off is invalid for the specified object.


Add before page 1326 line 43887:

In addition, if MAP_ANONYMOUS (or MAP_ANON) is not set in flags, the mmap() function shall fail if:


and then move the EACCES, EBADF, ENODEV and EOVERFLOW errors, and the first ENXIO error (line 43880), to after this new text.

On page 1328 after line 43989 (XSH mmap() rationale), add a new paragraph:

Some historical implementations only supported MAP_ANON, some only supported MAP_ANONYMOUS, and some supported both spellings. This standard includes both spellings partly for application compatibility and partly because neither spelling was clearly more popular than the other at the time this feature was considered for standardization.


On page 1365 lines 45154-45155 (XSH msync() description) change:

[SHM|TYM]The effect of msync() on a shared memory object or a typed memory object is unspecified.[/]


to:

The effect of msync() on an anonymous memory object, [SHM]shared memory object[/], or [TYM]typed memory object[/] is unspecified.


(0002306)
EdSchouten (updater)
2014-07-10 09:14

Sorry for hijacking this discussion, but would there be some interest in unifying this with adding support for anonymous shared memory objects?

FreeBSD has a special value for shm_open()'s path argument called SHM_ANON. It can be used to create a nameless shared memory object that is automatically removed when the last reference (descriptor or memory mapping) is removed.

Ed
(0002308)
mdempsky (reporter)
2014-07-10 18:22

Ed: I think FreeBSD's SHM_ANON is interesting and worth discussing in a new issue, but I'd think it would be defined simply as returning a shared memory object without a name (similar to a shared memory object that has been shm_unlink()'d), not related to anonymous memory objects.

- Issue History
Date Modified Username Field Change
2014-06-24 22:11 mdempsky New Issue
2014-06-24 22:11 mdempsky Name => Matthew Dempsky
2014-06-24 22:11 mdempsky Organization => OpenBSD
2014-06-24 22:11 mdempsky Section => mmap
2014-06-24 23:10 eadler Issue Monitored: eadler
2014-06-24 23:21 mdempsky Note Added: 0002274
2014-06-26 16:40 rhansen Note Added: 0002281
2014-06-26 16:50 rhansen Note Added: 0002282
2014-06-26 16:51 geoffclare Note Added: 0002283
2014-06-26 17:49 mdempsky Note Added: 0002284
2014-06-26 18:09 mdempsky Note Added: 0002285
2014-06-28 08:08 geoffclare Note Added: 0002292
2014-06-30 19:02 mdempsky Note Added: 0002295
2014-06-30 19:38 rhansen Note Added: 0002296
2014-07-01 03:16 rhansen Note Added: 0002298
2014-07-01 22:18 mdempsky Note Added: 0002299
2014-07-01 23:37 mdempsky Note Added: 0002300
2014-07-10 05:23 rhansen Note Added: 0002305
2014-07-10 05:25 rhansen Note Edited: 0002305
2014-07-10 09:14 EdSchouten Note Added: 0002306
2014-07-10 15:50 rhansen Note Edited: 0002305
2014-07-10 15:54 rhansen Note Edited: 0002305
2014-07-10 15:56 rhansen Note Edited: 0002305
2014-07-10 15:57 rhansen Note Edited: 0002281
2014-07-10 15:58 Don Cragun Interp Status => ---
2014-07-10 15:58 Don Cragun Final Accepted Text => See bugnote2305.
2014-07-10 15:58 Don Cragun Status New => Resolved
2014-07-10 15:58 Don Cragun Resolution Open => Accepted As Marked
2014-07-10 15:58 Don Cragun Final Accepted Text See bugnote2305. => See Note: 0002305.
2014-07-10 15:58 Don Cragun Tag Attached: issue8
2014-07-10 15:58 rhansen Note Edited: 0002281
2014-07-10 18:22 mdempsky Note Added: 0002308
2015-04-23 23:09 emaste Issue Monitored: emaste
2016-01-10 16:56 ch3root Issue Monitored: ch3root


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker