Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001218 [1003.1(2013)/Issue7+TC1] System Interfaces Editorial Enhancement Request 2018-12-08 21:03 2021-05-07 15:53
Reporter alanc View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Applied  
Name Alan Coopersmith
Organization Oracle Solaris Engineering
User Reference
Section reallocarray
Page Number (page or range of pages)
Line Number (Line or range of lines)
Interp Status ---
Final Accepted Text Note: 0005340
Summary 0001218: Add reallocarray()
Description OpenBSD created a reallocarray() function to offer overflow checking for array size calculations in the library instead of every caller:

https://man.openbsd.org/reallocarray [^] defines it as:

  void *reallocarray(void *ptr, size_t nmemb, size_t size);

  Designed for safe allocation of arrays, the reallocarray() function is
  similar to realloc() except it operates on /nmemb/ members of size /size/
  and checks for integer overflow in the calculation /nmemb/ * /size/.

Other platforms have since adopted it, including GNU libc in version 2.26.
Desired Action Add reallocarray() interface in the next edition of the standards.
Tags issue8
Attached Files

- Relationships
related to 0000374Appliedajosey 1003.1(2008)/Issue 7 malloc(0) and realloc(p,0) must not change errno on success 

-  Notes
(0004175)
schwarze (reporter)
2018-12-09 02:09

Here is the authoritative manual page:

  http://man.openbsd.org/reallocarray.3 [^]


Supplementary information about systems already implementing this interface:

FreeBSD provides it for three years now:
https://svnweb.freebsd.org/base/head/lib/libc/stdlib/reallocarray.c [^]

NetBSD also provides it for three years:
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/stdlib/reallocarray.c [^]

illumos provides it since last year:
http://src.illumos.org/source/history/illumos-gate/usr/src/lib/libc/port/gen/reallocarray.c [^]

The NetBSD implementation suffers from hiding it behind _OPENBSD_SOURCE even though it is no longer OpenBSD-specific for quite some time now. That sometimes causes issues in build systems like automatic configuration tests not finding it with subsequent symbol clashes between the libc symbol and a replacement implementation that the build system included. Including the function in the standard might convince NetBSD to no longer hide their implementation, making life easier for people who try to design portable build systems.
(0004453)
alanc (reporter)
2019-06-26 18:52

Also now in Solaris 11.4.10 & later:
https://docs.oracle.com/cd/E88353_01/html/E37843/reallocarray-3c.html [^]
(0004511)
shware_systems (reporter)
2019-08-07 21:47
edited on: 2019-08-07 21:48

I've no objection to adding this, and the recalloc_array() variant, but the man page cited is not in a format suitable for direct inclusion into XSH.

Additonally, FreeBSD implements dereferences of zero size allocations or reallocs to raise SIGSEGV. Any proposal should have this as implementation-defined behavior, imo, not as a requirement. This could be extended, as a generalized behavior, to dereferences into any space reserved past size, e.g. to maintain alignments.

(0004512)
eblake (manager)
2019-08-07 21:47
edited on: 2019-08-07 21:49

On page 360 after line 12313 (<stdlib.h>), insert with CX shading:
<CX>void *reallocarray(void *, size_t, size_t);<CX>

On page 480 line 16535 (2.2.2 The Name Space), add reallocarray in sorted order to the list of reserved identifiers.

On page 636 line 22019 (calloc ERRORS), change:
<CX>[ENOMEM] Insufficient memory is available.</CX>
to:
<CX>[ENOMEM] Insufficient memory is available, including the case when nelem * elsize would overflow.</CX>
[The above change may be worth doing separately for TC3]

On page 932 line 31675 (free DESCRIPTION), change:
has been deallocated by a call to free( ) or realloc( ),
to:
has been deallocated by a call to free( ), realloc( ), <CX>or reallocarray( )</CX>,

On page 1788 line 87858 (realloc NAME), change:
realloc -- memory reallocator
to:
realloc, reallocarray -- memory reallocators

On page 1788 after line 47861 (realloc SYNOPSIS), add with CX shading:
<CX>void *reallocarray(void *ptr, size_t nelem, size_t elsize);</CX>

On page 1788 after line 57873 (realloc DESCRIPTION), add a new paragraph with CX shading:
<CX>The reallocarray( ) function shall be equivalent to the call <code>realloc(ptr, nelem * elsize)</code> except that overflow in the multiplication shall be an error.</CX>

On page 1788 lines 57874, 57877, 57884, 57890, 57891, and 57894, change "realloc( )" to "realloc( ) <CX>or reallocarray( )</CX>". (The set of lines changed may depend on the resolution to 0000374; in particular, whatever is said for realloc(x,0) applies to reallocarray if either nelem or elsize is 0)

On page 1788 line 57875, change:
by calloc( ), malloc( ), or realloc( )
to:
as if by malloc( )
[The above change may be worth doing separately for TC3]

On page 1788 line 57876, change:
free( ) or realloc( ),
to:
free( ), realloc( ), <CX>or reallocarray( ),<CX>

On page 1788 after line 57895 (realloc ERRORS), add a paragraph:
The reallocarray( ) function shall fail if:
<CX>[ENOMEM] The calculation nelem * elsize would overflow.</CX>

On page 3790 line 130049 (E.1 Subprofiling Option Groups), add reallocarray in sorted order under POSIX_C_LIB_EXT

(0004513)
eblake (manager)
2019-08-07 22:05

Responding to Note: 0004511:
"the recalloc_array() variant" - that was not mentioned in the original post; it might be better to do that as a separate request rather than trying to declare it in scope for this bug, particularly since there are definitely existing implementations with reallocarray but not recalloc_array() (glibc, for example). Do you have a pointer to man pages from implementations with this additional function (which I'm assuming is intended to be similar in use to a combination of reallocarray and calloc, in that any new memory allocated is guaranteed to be zero-initialized while preserving any earlier contents in the existing pointer)?

"FreeBSD implements dereferences of zero size allocations or reallocs to raise SIGSEGV. Any proposal should have this as implementation-defined behavior, imo, not as a requirement." - This should already be the case, by deferring to the C standard, which already states it is undefined behavior to dereference any pointer returned by malloc(0), and the fact that reallocarray() is identical to realloc() other than for overflow, where in turn realloc() (typically) allocates memory as if by malloc(), then the FreeBSD behavior is already covered. However, the resolution of 0000374 is probably the more appropriate place to worry about what POSIX mandates for any acceptable response to realloc(x,0).
(0004514)
alanc (reporter)
2019-08-07 22:27

The original definition of recallocarray would be from OpenBSD:
https://man.openbsd.org/recallocarray.3 [^]

It has also been implemented by illumos:
https://illumos.org/man/3c/recallocarray [^]

but I agree with tackling that (and any other of the new functions listed on the
OpenBSD page that are desirable) as a separate task.
(0004515)
shware_systems (reporter)
2019-08-07 22:55
edited on: 2019-08-07 23:11

Re: 4513
OpenBSD has it, documented on same man page as reallocarray() with zero-fill of added space if size argument larger than current size, or ptr is NULL, but the behavior of reallocarray otherwise. As such, if you going to add the one, may as well add both. I agree the other interfaces are better left to other bug reports.

As to the latter, any string interface that does not have a buffer size parameter, relying on NUL termination, may be passed such a pointer. To it this looks the same as a pointer to any object of size > 0. Telling an application not to do something is fine but I don't see this absolves an implementation from documenting what it will do if that advice is ignored, especially if it may abort an application as the default action of SIGSEGV requires.

(0004516)
eblake (manager)
2019-08-08 00:40

Passing the result of malloc(0) to a function that requires a string with NUL termination is undefined behavior because the C standard says you cannot dereference such a pointer, but only pass it to free() - yet a function that looks for a NUL termination must dereference the incoming pointer. And undefined behavior means just that - we don't have to document what it will or can do, and it if causes SIGSEGV, it is your fault for violating the C standard by writing buggy code, and not something that the standard has to provide any further details about, nor a problem with the library for letting it cause SIGSEGV, or letting it reformat your hard drive, or silently succeeding, or anything else it might do. We do not need to burden implementations with documenting what the C standard has already declared to be undefined (because requiring documentation would mean it is no longer undefined, but implementation-defined), and this is no different from any other number of undefined behaviors that have been known to cause SIGSEGV in some setups (such as passing a null pointer to a function that requires a NUL-terminated string).
(0005058)
geoffclare (manager)
2020-10-23 14:05

The reallocarray() addition has been made in the Issue8NewAPIs branch in gitlab, based on Note: 0004512.

The suggested addition to the list of reserved identifiers in XSH 2.2.2 was not applied, as that list only contains identifiers reserved by the C standard.

Also, the calloc() change was not applied, as that is a separate issue not related to the addition of reallocarray(). If this does not get done in a separate bug, it could be handled here by making the eventual resolution of this bug something like, "Make the changes from The Open Group document Cxxx and also make the following change to the calloc() page ..."
(0005340)
geoffclare (manager)
2021-04-29 15:45

Make the changes from "Additional APIs for Issue 8, Part 1" (Austin/1110) and also make the following change:
    
On page 636 line 22019 (calloc ERRORS), change:
[ENOMEM] Insufficient memory is available.
to:
[ENOMEM] Insufficient memory is available, including the case when nelem * elsize would overflow.

- Issue History
Date Modified Username Field Change
2018-12-08 21:03 alanc New Issue
2018-12-08 21:03 alanc Name => Alan Coopersmith
2018-12-08 21:03 alanc Organization => Oracle Solaris Engineering
2018-12-08 21:03 alanc Section => reallocarray
2018-12-08 21:03 alanc Page Number => (page or range of pages)
2018-12-08 21:03 alanc Line Number => (Line or range of lines)
2018-12-09 02:09 schwarze Note Added: 0004175
2019-06-26 18:52 alanc Note Added: 0004453
2019-08-07 21:01 eblake Relationship added related to 0000374
2019-08-07 21:47 shware_systems Note Added: 0004511
2019-08-07 21:47 eblake Note Added: 0004512
2019-08-07 21:48 shware_systems Note Edited: 0004511
2019-08-07 21:49 eblake Note Edited: 0004512
2019-08-07 22:05 eblake Note Added: 0004513
2019-08-07 22:27 alanc Note Added: 0004514
2019-08-07 22:55 shware_systems Note Added: 0004515
2019-08-07 23:11 shware_systems Note Edited: 0004515
2019-08-08 00:40 eblake Note Added: 0004516
2020-10-23 14:05 geoffclare Note Added: 0005058
2021-04-29 15:45 geoffclare Note Added: 0005340
2021-04-29 15:46 geoffclare Interp Status => ---
2021-04-29 15:46 geoffclare Final Accepted Text => Note: 0005340
2021-04-29 15:46 geoffclare Status New => Resolved
2021-04-29 15:46 geoffclare Resolution Open => Accepted As Marked
2021-04-29 15:47 geoffclare Tag Attached: issue8
2021-05-07 15:53 geoffclare Status Resolved => Applied


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker