Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000374 [1003.1(2008)/Issue 7] System Interfaces Objection Omission 2011-02-03 17:12 2021-08-16 11:09
Reporter eblake View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Applied  
Name Eric Blake
Organization Red Hat
User Reference ebb.malloc
Section malloc
Page Number 1268
Line Number 41715
Interp Status ---
Final Accepted Text Note: 0004507
Summary 0000374: malloc(0) and realloc(p,0) must not change errno on success
Description Since the standard explicitly allows malloc(0) and realloc(p,0) to return
either NULL or a pointer, an application that wishes to know whether a
return of NULL is the result of an implementation that does not allocate
(successful) or an implementation that tries to allocate but had no memory
(errno == ENOMEM). However, the standard is also explicit that errno cannot
be inspected upon success unless documented, and that no function can reset
errno to 0. The standard also allows malloc to fail for reasons unrelated
to ENOMEM (although this is unlikely to happen).

Requiring errno to remain unchanged when a pointer is returned may invalidate
existing implementations, so this proposal limits the wording to just the
case of a 0-size allocation request.
Desired Action At line 21395 [XSH calloc DESCRIPTION], change:

If the size of the space requested is 0, the behavior is
implementation-defined: the value returned shall be either a null pointer or
a unique pointer.

to:

If the size of the space requested is 0, the behavior is
implementation-defined: either no allocation attempt is made, <CX>errno shall
be unchanged</CX>, and NULL is returned; or an allocation attempt is made
which results in a valid pointer if successful and in a NULL return <CX>and a
valid errno</CX> if it fails. <CX>Since a return of NULL for a size of 0 may
indicate either success or failure, an application wishing to determine which
of the two implementations is in use should set errno to 0, then call
calloc( ), then check errno.</CX>

At line 21410 [XSH calloc RATIONALE], change "None" to:

Note that the value of errno is undefined if a pointer is returned; it is
only required to be unchanged or valid when NULL is returned.

At line 41713 [XSH malloc DESCRIPTION], change:

If the size of the space requested is 0, the behavior is
implementation-defined: the value returned shall be either a null pointer or
a unique pointer.

to:

If the size of the space requested is 0, the behavior is
implementation-defined: either no allocation attempt is made, <CX>errno shall
be unchanged</CX>, and NULL is returned; or an allocation attempt is made
which results in a valid pointer if successful and in a NULL return <CX>and a
valid errno</CX> if it fails. <CX>Since a return of NULL for a size of 0 may
indicate either success or failure, an application wishing to determine which
of the two implementations is in use should set errno to 0, then call
malloc( ), then check errno.</CX>

At line 41729 [XSH malloc RATIONALE], change "None" to:

Note that the value of errno is undefined if a pointer is returned; it is
only required to be unchanged or valid when NULL is returned.

At line 56036 [XSH realloc DESCRIPTION], change:

If ptr is a null pointer, realloc( ) shall be equivalent to malloc( ) for
the specified size.

to:

If ptr is a null pointer, realloc( ) shall be equivalent to malloc( ) for
the specified size. In particular, if the size argument is 0, the behavior
is implementation-defined: either no allocation attempt is made, <CX>errno
shall be unchanged</CX>, and NULL is returned; or an allocation attempt is
made which results in a valid pointer if successful and in a NULL return
<CX>and a valid errno</CX> if it fails. <CX>Since a return of NULL for a
null pointer and size of 0 may indicate either success or failure, an
application wishing to determine which of the two implementations is in
use should set errno to 0, then call realloc( ), then check errno.</CX>

At line 56058 [XSH realloc RATIONALE], change "None" to:

Note that the value of errno is undefined if a pointer is returned; it is
only required to be unchanged or valid when NULL is returned.
Tags c99, issue8
Attached Files

- Relationships
related to 0000526Closedajosey 1003.1(2008)/Issue 7 Adopt C99 wording for zero size calloc(), malloc() et al. 
related to 0000688Closed 1003.1(2013)/Issue7+TC1 realloc(NULL, 0) vs. malloc(0) 
related to 0001218Applied 1003.1(2013)/Issue7+TC1 Add reallocarray() 
child of 0000400Closedajosey 1003.1(2008)/Issue 7 realloc wording conflicts with C99 

-  Notes
(0000664)
eblake (manager)
2011-02-03 17:25

Concerns were raised during the 3 Feb 2011 conference about line 56034 of
realloc; is realloc(ptr, 0) allowed to return a non-NULL value, and therefore,
should a NULL return when ptr is freed also need special treatment for
whether errno is unchanged on success. This will require feedback from the
C committee.
(0000708)
nick (manager)
2011-03-17 12:30
edited on: 2011-03-17 16:09

This was considered during the WG14 meeting in London.

On Feb 20, 2011, at 1:48 AM, DAGwyn@aol.com wrote:

> In a message dated 2/19/2011 7:45:08 P.M. Eastern Standard Time,
> msebor@gmail.com writes:
> > 1) there are implementations that do not behave according
> > to the C99 requirement and return NULL after deallocating
> > the space, and
> > 2) the POSIX standard specifically permits such implementations
> > 3) there likely are programs that rely on the behavior of such
> > implementations.
> > Declaring such implementations non-conforming and expecting them
> > to change may not be feasible because doing so would introduce
> > memory leaks into programs that rely on their behavior.
>
> Such programs already are not portable without memory leaks,
> since on a C99-conforming platform space is not deallocated
> when NULL is returned from realloc. If they need to cope with
> both C99-conforming and non-conforming implementations as
> described in (1), they cannot rely on the deallocation anyway;
> change:
> // old code which leaks using C99-conforming implementations:
> if ((newp = realloc(p, newsize)) == NULL)
> if (newsize > 0)
> ... // still have previous data, recovery method A
> else
> ... // non-conforming realloc discarded previous data,
> // recovery method B
> else {
> p = newp;
> size = newsize;
> }
> to:
> // new code which doesn't leak regardless of implementation:
> if (newsize == 0) {
> p = NULL;
> size = 0;
> } else if ((newp = realloc(p, newsize)) == NULL)
> ... // still have previous data, recovery method A
> else {
> p = newp;
> size = newsize;
> }
> Notice that recovery method B, which is likely to be quite
> awkward since the current data has been lost, is no longer
> necessary using the second approach. It is likely that the vast
> majority of programs using realloc already use something like
> the second approach, which avoids realloc(,0) altogether and
> which requires less and simpler recovery coding.
>
> There are already several things wrong with the current POSIX
> specification for realloc, as I noted in a previous message. It
> should be changed to not contradict the C standard, the simplest
> method being to use the C standard's wording except not to say
> anything special about the case of size 0. That would remove
> permission for implementations to gratuitously report failure to
> allocate when space is in fact available. Then the only time that
> reasonably-coded programs that had assumed the non-C99
> behavior (1) could leak memory would be when in fact memory
> wasn't available, in which case most programs probably adopt a
> "recovery" scheme that doesn't continue execution for much
> longer, and the leak will evaporate when the program terminates.
>
>

The C committee considered three possibilities:

1. Do nothing and accept the fact that some C99 implementations
   may not be able to conform to the requirements of the C99
   standard in this regard for compatibility reasons.
2. Change C to more closely align with the POSIX requirements
   so as to make the existing implementatins conforming. This
   resolution is being sought by the POSIX group and is also
   preferred by the implementers of the affected implementations.
3. Change both the C and POSIX standards to require realloc()
   to always attempt to allocate space even for zero-size
   requests. This would simplify the specification and, at
   least in theory, also make it easier to write portable
   programs. This option would require the support of the
   implementers of the affected implementations. Without such
   support this option becomes equivalent to option 1 above.

The outcome of this discussion is that if realloc returns NULL, it has failed, and the original memory has not been freed.
The POSIX words came from C90, and these were changed for C99. POSIX failed to pick up the change

See N872 item 19c.

Unfortunately this breaks several existing implementations. However, those implementer present (Sun/Oracle, Apple, IBM, HP) all agreed that they were prepared to change their implementations.

POSIX Change requested by C99:
Page 1754 line 56030 Change:

The realloc() function shall change the size of the memory object pointed to by ptr to the size specified by size. The contents of the object shall remain unchanged up to the lesser of the new and old sizes. If the new size of the memory object would require movement of the object, the space for the previous instantiation of the object is freed. If the new size is larger, the contents of the newly allocated portion of the object are unspecified. If size is 0 and ptr is not a null pointer, the object pointed to is freed.
to
The realloc() function shall deallocate the old object pointed to by ptr and return a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.


Page 1754 line 56046 change
Upon successful completion with a size not equal to 0, realloc() shall return a pointer to the (possibly moved) allocated space.

to

Upon successful completion, realloc() shall return a pointer to the (possibly moved) allocated space.

At line 56047-8 change:

If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned.
to
If size is 0, either:
  * a null pointer shall be returned <CX>and errno set to EINVAL</CX>, or
  * unique pointer that can be successfully passed to free() shall be returned, and the memory object pointed to by ptr shall be freed.

Add after line 56049:
If realloc() returns NULL, the memory referenced by ptr is not changed.

Add after 56052:

The realloc() function may fail if:
EINVAL size is 0 and the implementation does not support 0 sized allocations.


NOTE: Since the text at the top of realloc() states:
   The functionality described on this reference page is aligned with the ISO C standard. Any conflict
    between the requirements described here and the ISO C standard is unintentional. This volume of
    POSIX.1-2008 defers to the ISO C standard.
I am not sure we have the wiggle room to do anything but DEFECT situation.

(0000722)
eblake (manager)
2011-03-24 16:41

The issue of leaving errno unchanged on successful allocation is only
appropriate for Issue 8; and this bug should not be resolved until
after 0000400 regarding the conflict between C99 and POSIX on realloc
behavior has been fixed first in TC1.
(0000737)
eblake (manager)
2011-04-07 20:45

There are at least three existing realloc behaviors when NULL is
returned; the differences only occur for a size of 0 (for
non-zero size, all three implementations set errno to ENOMEM when
returning NULL, even though that is not required by C99).

AIX: realloc(NULL,0) always returns NULL, errno is EINVAL
      realloc(ptr,0) always returns NULL, ptr freed, errno is EINVAL
BSD: realloc(NULL,0) only gives NULL on alloc failure, errno is ENOMEM
      realloc(ptr,0) only gives NULL on alloc failure, ptr unchanged,
        errno is ENOMEM
glibc:realloc(NULL,0) only gives NULL on alloc failure, errno is ENOMEM
      realloc(ptr,0) always returns NULL, ptr freed, errno unchanged

During the 7 Apr 2011 teleconference, the idea was reiterated that the
WG14 intentions were to require BSD behavior, although the wording in
C99 has a loophole in implementation-defined behavior that still appears
to permit AIX and glibc behaviors. The C1x draft carries the same
wording loophole, so the planned course of action is to raise a defect
against C1x once it completes standardization, where the outcome of that
defect will either be that C1x tightens the wording to eliminate the
loophole or relaxes the wording to align with existing practice.
Therefore, the behavior of errno in Issue 8 should be deferred until
after any C1x defect has been resolved.
(0001001)
nick (manager)
2011-11-03 15:22

This was further discussed at the Washington DC meeting of WG14, and C defect report 400 has been accepted. This will not be resolved for at least another 2 meetings - i.e. at least one year.

This bug should remain open in the meantime.
(0001130)
nick (manager)
2012-02-16 16:13

See Note: 0001129
(0002221)
geoffclare (manager)
2014-04-10 15:57

This was discussed in the 10th April teleconference. We plan to accept these changes in spirit, but since there is overlap with several other bugs, we will wait until after TC2 has been merged into Issue 7 in order to have a clean starting point for the detailed edits.
(0002222)
eblake (manager)
2014-04-10 15:59

http://open-std.org/JTC1/SC22/WG14/www/docs/dr_400.htm [^]
(0004507)
geoffclare (manager)
2019-08-02 11:18

We need to decide how to handle the overlap between this bug and the work to align Issue 8 with C17.

A major rewrite of the realloc() DESCRIPTION and RETURN VALUE sections is needed to align with C17. This will mean most of the changes requested in Note: 0000708 are taken care of by that. The only thing left for this bug to do is to address any changes to CX functionality that we want to make.

The original request to leave errno unchanged when a memory allocation function is called with size zero and a null pointer is returned because the implementation did not attempt an allocation appears to conflict with C17, which says (7.22.3 para 1; emphasis added):
If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned to indicate an error, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object.


So if we want applications to be able to distinguish between implementations that attempt allocation and those that don't, we will need to specify two different non-zero errno values -- Note: 0000708 already suggests using EINVAL for this for realloc() and this seems like an appropriate way to handle it in all the functions.

Note also that C17 makes calling realloc() with size 0 an obsolescent feature.

I suggest that we should resolve this bug first and then the C17 alignment changes can incorporate the resolution into the rewritten realloc() DESCRIPTION and RETURN VALUE sections.

Proposed changes (page and line numbers are for the 2018 edition) ...

On page 637 line 22010 section calloc(), change:
Upon successful completion with both nelem and elsize non-zero, calloc() shall return a pointer to the allocated space. If either nelem or elsize is 0, then either:

  • A null pointer shall be returned [CX]and errno may be set to an implementation-defined value[/CX], or

  • A pointer to the allocated space shall be returned. The application shall ensure that the pointer is not used to access an object.


Otherwise, it shall return a null pointer [CX]and set errno to indicate the error[/CX].

to:
Upon successful completion, calloc() shall return a pointer to the allocated space; if either nelem or elsize is 0, the application shall ensure that the pointer is not used to access an object.

Otherwise, it shall return a null pointer [CX]and set errno to indicate the error[/CX].


On page 637 line 22019 section calloc(), add:
The calloc() function may fail if:
[EINVAL] nelem or elsize is 0 and the implementation does not support 0 sized allocations.


On page 1295 line 43152 section malloc(), change:
Upon successful completion with size not equal to 0, malloc() shall return a pointer to the allocated space. If size is 0, then either:

  • A null pointer shall be returned [CX]and errno may be set to an implementation-defined value[/CX], or

  • A pointer to the allocated space shall be returned. The application shall ensure that the pointer is not used to access an object.


Otherwise, it shall return a null pointer [CX]and set errno to indicate the error[/CX].

to:
Upon successful completion, malloc() shall return a pointer to the allocated space; if size is 0, the application shall ensure that the pointer is not used to access an object.

Otherwise, it shall return a null pointer [CX]and set errno to indicate the error[/CX].


On page 1295 line 43161 section malloc(), add:
The malloc() function may fail if:
[EINVAL] size is 0 and the implementation does not support 0 sized allocations.


On page 1788 line 57886 section realloc(), change:
A null pointer shall be returned [CX]and, if ptr is not a null pointer, errno shall be set to an implementation-defined value[/CX].

to:
A null pointer shall be returned [CX]and, if ptr is not a null pointer, errno shall be set to [EINVAL][/CX].


On page 1788 line 57895 section realloc(), add:
The realloc() function may fail if:
[EINVAL] size is 0 and the implementation does not support 0 sized allocations.
(0004510)
Florian Weimer (reporter)
2019-08-07 12:10

The C17 change to realloc looks like an editorial mistake. I brought this very issue to the WG14 reflector in May 2018. I was told that the intent was to make a wide range of existing implementation behavior conforming to the standard, and not to require new changes to implementation for maintaining conformance. (In general, C17 is supposed to contain editorial changes only.)

glibc will not change realloc for (perceived) C17 conformance. glibc and GCC continue to support zero-sized objects as an extension to the standard.
(0004517)
geoffclare (manager)
2019-08-08 08:31

Re: Note: 0004510 It was a deliberate decision by WG14, not any kind of accident or editorial mistake. Their decision is stated in Note: 0000708 (also repeated in bug 0000400 which is where the current POSIX requirement comes from), in particular:

"The outcome of this discussion is that if realloc returns NULL, it has failed, and the original memory has not been freed."

and

"Unfortunately this breaks several existing implementations. However, those implementer present (Sun/Oracle, Apple, IBM, HP) all agreed that they were prepared to change their implementations."

As regards C17 containing editorial changes only, according to Note: 0000708 WG14 believed that this change in requirement happened between C90 and C99 (and "POSIX failed to pick up the change"), in which case the later addition of "to indicate an error" can be seen as an editorial clarification.
(0005445)
geoffclare (manager)
2021-08-16 11:09

When applying this bug I noticed that the resolution was missing CX shading on the new EINVAL errors and I included this shading.

I also worded the realloc() EINVAL slightly differently so that it can apply to reallocarray() as well: "The requested allocation size is 0 and ..."

- Issue History
Date Modified Username Field Change
2011-02-03 17:12 eblake New Issue
2011-02-03 17:12 eblake Status New => Under Review
2011-02-03 17:12 eblake Assigned To => ajosey
2011-02-03 17:12 eblake Name => Eric Blake
2011-02-03 17:12 eblake Organization => Red Hat
2011-02-03 17:12 eblake User Reference => ebb.malloc
2011-02-03 17:12 eblake Section => malloc
2011-02-03 17:12 eblake Page Number => 1268
2011-02-03 17:12 eblake Line Number => 41715
2011-02-03 17:12 eblake Interp Status => ---
2011-02-03 17:15 nick Tag Attached: c99
2011-02-03 17:23 eblake Desired Action Updated
2011-02-03 17:25 eblake Note Added: 0000664
2011-03-17 12:30 nick Note Added: 0000708
2011-03-17 12:31 nick Note Added: 0000709
2011-03-17 16:07 nick Note Deleted: 0000709
2011-03-17 16:09 nick Note Edited: 0000708
2011-03-24 16:31 eblake Tag Attached: issue8
2011-03-24 16:38 eblake Relationship added child of 0000400
2011-03-24 16:41 eblake Note Added: 0000722
2011-04-07 20:45 eblake Note Added: 0000737
2011-11-03 15:22 nick Note Added: 0001001
2012-02-16 16:13 nick Note Added: 0001130
2012-02-16 17:15 nick Relationship added related to 0000526
2013-05-09 15:57 eblake Relationship added child of 0000688
2013-05-09 16:23 eblake Relationship deleted child of 0000688
2013-05-09 16:23 eblake Relationship added related to 0000688
2014-04-10 15:57 geoffclare Note Added: 0002221
2014-04-10 15:59 eblake Note Added: 0002222
2019-08-02 11:18 geoffclare Note Added: 0004507
2019-08-07 12:10 Florian Weimer Note Added: 0004510
2019-08-07 12:11 Florian Weimer Issue Monitored: Florian Weimer
2019-08-07 21:01 eblake Relationship added related to 0001218
2019-08-08 08:31 geoffclare Note Added: 0004517
2021-08-12 15:44 geoffclare Final Accepted Text => Note: 0004507
2021-08-12 15:44 geoffclare Status Under Review => Resolved
2021-08-12 15:44 geoffclare Resolution Open => Accepted As Marked
2021-08-16 11:09 geoffclare Note Added: 0005445
2021-08-16 11:09 geoffclare Status Resolved => Applied


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker