Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000374 [1003.1(2008)/Issue 7] System Interfaces Objection Omission 2011-02-03 17:12 2014-04-10 15:59
Reporter eblake View Status public  
Assigned To ajosey
Priority normal Resolution Open  
Status Under Review  
Name Eric Blake
Organization Red Hat
User Reference ebb.malloc
Section malloc
Page Number 1268
Line Number 41715
Interp Status ---
Final Accepted Text
Summary 0000374: malloc(0) and realloc(p,0) must not change errno on success
Description Since the standard explicitly allows malloc(0) and realloc(p,0) to return
either NULL or a pointer, an application that wishes to know whether a
return of NULL is the result of an implementation that does not allocate
(successful) or an implementation that tries to allocate but had no memory
(errno == ENOMEM). However, the standard is also explicit that errno cannot
be inspected upon success unless documented, and that no function can reset
errno to 0. The standard also allows malloc to fail for reasons unrelated
to ENOMEM (although this is unlikely to happen).

Requiring errno to remain unchanged when a pointer is returned may invalidate
existing implementations, so this proposal limits the wording to just the
case of a 0-size allocation request.
Desired Action At line 21395 [XSH calloc DESCRIPTION], change:

If the size of the space requested is 0, the behavior is
implementation-defined: the value returned shall be either a null pointer or
a unique pointer.

to:

If the size of the space requested is 0, the behavior is
implementation-defined: either no allocation attempt is made, <CX>errno shall
be unchanged</CX>, and NULL is returned; or an allocation attempt is made
which results in a valid pointer if successful and in a NULL return <CX>and a
valid errno</CX> if it fails. <CX>Since a return of NULL for a size of 0 may
indicate either success or failure, an application wishing to determine which
of the two implementations is in use should set errno to 0, then call
calloc( ), then check errno.</CX>

At line 21410 [XSH calloc RATIONALE], change "None" to:

Note that the value of errno is undefined if a pointer is returned; it is
only required to be unchanged or valid when NULL is returned.

At line 41713 [XSH malloc DESCRIPTION], change:

If the size of the space requested is 0, the behavior is
implementation-defined: the value returned shall be either a null pointer or
a unique pointer.

to:

If the size of the space requested is 0, the behavior is
implementation-defined: either no allocation attempt is made, <CX>errno shall
be unchanged</CX>, and NULL is returned; or an allocation attempt is made
which results in a valid pointer if successful and in a NULL return <CX>and a
valid errno</CX> if it fails. <CX>Since a return of NULL for a size of 0 may
indicate either success or failure, an application wishing to determine which
of the two implementations is in use should set errno to 0, then call
malloc( ), then check errno.</CX>

At line 41729 [XSH malloc RATIONALE], change "None" to:

Note that the value of errno is undefined if a pointer is returned; it is
only required to be unchanged or valid when NULL is returned.

At line 56036 [XSH realloc DESCRIPTION], change:

If ptr is a null pointer, realloc( ) shall be equivalent to malloc( ) for
the specified size.

to:

If ptr is a null pointer, realloc( ) shall be equivalent to malloc( ) for
the specified size. In particular, if the size argument is 0, the behavior
is implementation-defined: either no allocation attempt is made, <CX>errno
shall be unchanged</CX>, and NULL is returned; or an allocation attempt is
made which results in a valid pointer if successful and in a NULL return
<CX>and a valid errno</CX> if it fails. <CX>Since a return of NULL for a
null pointer and size of 0 may indicate either success or failure, an
application wishing to determine which of the two implementations is in
use should set errno to 0, then call realloc( ), then check errno.</CX>

At line 56058 [XSH realloc RATIONALE], change "None" to:

Note that the value of errno is undefined if a pointer is returned; it is
only required to be unchanged or valid when NULL is returned.
Tags c99, issue8
Attached Files

- Relationships
related to 0000526Resolvedajosey 1003.1(2008)/Issue 7 Adopt C99 wording for zero size calloc(), malloc() et al. 
related to 0000688Resolved 1003.1(2013)/Issue7+TC1 realloc(NULL, 0) vs. malloc(0) 
child of 0000400Closedajosey 1003.1(2008)/Issue 7 realloc wording conflicts with C99 

-  Notes
(0000664)
eblake (manager)
2011-02-03 17:25

Concerns were raised during the 3 Feb 2011 conference about line 56034 of
realloc; is realloc(ptr, 0) allowed to return a non-NULL value, and therefore,
should a NULL return when ptr is freed also need special treatment for
whether errno is unchanged on success. This will require feedback from the
C committee.
(0000708)
nick (manager)
2011-03-17 12:30
edited on: 2011-03-17 16:09

This was considered during the WG14 meeting in London.

On Feb 20, 2011, at 1:48 AM, DAGwyn@aol.com wrote:

> In a message dated 2/19/2011 7:45:08 P.M. Eastern Standard Time,
> msebor@gmail.com writes:
> > 1) there are implementations that do not behave according
> > to the C99 requirement and return NULL after deallocating
> > the space, and
> > 2) the POSIX standard specifically permits such implementations
> > 3) there likely are programs that rely on the behavior of such
> > implementations.
> > Declaring such implementations non-conforming and expecting them
> > to change may not be feasible because doing so would introduce
> > memory leaks into programs that rely on their behavior.
>
> Such programs already are not portable without memory leaks,
> since on a C99-conforming platform space is not deallocated
> when NULL is returned from realloc. If they need to cope with
> both C99-conforming and non-conforming implementations as
> described in (1), they cannot rely on the deallocation anyway;
> change:
> // old code which leaks using C99-conforming implementations:
> if ((newp = realloc(p, newsize)) == NULL)
> if (newsize > 0)
> ... // still have previous data, recovery method A
> else
> ... // non-conforming realloc discarded previous data,
> // recovery method B
> else {
> p = newp;
> size = newsize;
> }
> to:
> // new code which doesn't leak regardless of implementation:
> if (newsize == 0) {
> p = NULL;
> size = 0;
> } else if ((newp = realloc(p, newsize)) == NULL)
> ... // still have previous data, recovery method A
> else {
> p = newp;
> size = newsize;
> }
> Notice that recovery method B, which is likely to be quite
> awkward since the current data has been lost, is no longer
> necessary using the second approach. It is likely that the vast
> majority of programs using realloc already use something like
> the second approach, which avoids realloc(,0) altogether and
> which requires less and simpler recovery coding.
>
> There are already several things wrong with the current POSIX
> specification for realloc, as I noted in a previous message. It
> should be changed to not contradict the C standard, the simplest
> method being to use the C standard's wording except not to say
> anything special about the case of size 0. That would remove
> permission for implementations to gratuitously report failure to
> allocate when space is in fact available. Then the only time that
> reasonably-coded programs that had assumed the non-C99
> behavior (1) could leak memory would be when in fact memory
> wasn't available, in which case most programs probably adopt a
> "recovery" scheme that doesn't continue execution for much
> longer, and the leak will evaporate when the program terminates.
>
>

The C committee considered three possibilities:

1. Do nothing and accept the fact that some C99 implementations
   may not be able to conform to the requirements of the C99
   standard in this regard for compatibility reasons.
2. Change C to more closely align with the POSIX requirements
   so as to make the existing implementatins conforming. This
   resolution is being sought by the POSIX group and is also
   preferred by the implementers of the affected implementations.
3. Change both the C and POSIX standards to require realloc()
   to always attempt to allocate space even for zero-size
   requests. This would simplify the specification and, at
   least in theory, also make it easier to write portable
   programs. This option would require the support of the
   implementers of the affected implementations. Without such
   support this option becomes equivalent to option 1 above.

The outcome of this discussion is that if realloc returns NULL, it has failed, and the original memory has not been freed.
The POSIX words came from C90, and these were changed for C99. POSIX failed to pick up the change

See N872 item 19c.

Unfortunately this breaks several existing implementations. However, those implementer present (Sun/Oracle, Apple, IBM, HP) all agreed that they were prepared to change their implementations.

POSIX Change requested by C99:
Page 1754 line 56030 Change:

The realloc() function shall change the size of the memory object pointed to by ptr to the size specified by size. The contents of the object shall remain unchanged up to the lesser of the new and old sizes. If the new size of the memory object would require movement of the object, the space for the previous instantiation of the object is freed. If the new size is larger, the contents of the newly allocated portion of the object are unspecified. If size is 0 and ptr is not a null pointer, the object pointed to is freed.
to
The realloc() function shall deallocate the old object pointed to by ptr and return a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.


Page 1754 line 56046 change
Upon successful completion with a size not equal to 0, realloc() shall return a pointer to the (possibly moved) allocated space.

to

Upon successful completion, realloc() shall return a pointer to the (possibly moved) allocated space.

At line 56047-8 change:

If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned.
to
If size is 0, either:
  * a null pointer shall be returned <CX>and errno set to EINVAL</CX>, or
  * unique pointer that can be successfully passed to free() shall be returned, and the memory object pointed to by ptr shall be freed.

Add after line 56049:
If realloc() returns NULL, the memory referenced by ptr is not changed.

Add after 56052:

The realloc() function may fail if:
EINVAL size is 0 and the implementation does not support 0 sized allocations.


NOTE: Since the text at the top of realloc() states:
   The functionality described on this reference page is aligned with the ISO C standard. Any conflict
    between the requirements described here and the ISO C standard is unintentional. This volume of
    POSIX.1-2008 defers to the ISO C standard.
I am not sure we have the wiggle room to do anything but DEFECT situation.

(0000722)
eblake (manager)
2011-03-24 16:41

The issue of leaving errno unchanged on successful allocation is only
appropriate for Issue 8; and this bug should not be resolved until
after 0000400 regarding the conflict between C99 and POSIX on realloc
behavior has been fixed first in TC1.
(0000737)
eblake (manager)
2011-04-07 20:45

There are at least three existing realloc behaviors when NULL is
returned; the differences only occur for a size of 0 (for
non-zero size, all three implementations set errno to ENOMEM when
returning NULL, even though that is not required by C99).

AIX: realloc(NULL,0) always returns NULL, errno is EINVAL
      realloc(ptr,0) always returns NULL, ptr freed, errno is EINVAL
BSD: realloc(NULL,0) only gives NULL on alloc failure, errno is ENOMEM
      realloc(ptr,0) only gives NULL on alloc failure, ptr unchanged,
        errno is ENOMEM
glibc:realloc(NULL,0) only gives NULL on alloc failure, errno is ENOMEM
      realloc(ptr,0) always returns NULL, ptr freed, errno unchanged

During the 7 Apr 2011 teleconference, the idea was reiterated that the
WG14 intentions were to require BSD behavior, although the wording in
C99 has a loophole in implementation-defined behavior that still appears
to permit AIX and glibc behaviors. The C1x draft carries the same
wording loophole, so the planned course of action is to raise a defect
against C1x once it completes standardization, where the outcome of that
defect will either be that C1x tightens the wording to eliminate the
loophole or relaxes the wording to align with existing practice.
Therefore, the behavior of errno in Issue 8 should be deferred until
after any C1x defect has been resolved.
(0001001)
nick (manager)
2011-11-03 15:22

This was further discussed at the Washington DC meeting of WG14, and C defect report 400 has been accepted. This will not be resolved for at least another 2 meetings - i.e. at least one year.

This bug should remain open in the meantime.
(0001130)
nick (manager)
2012-02-16 16:13

See Note: 0001129
(0002221)
geoffclare (manager)
2014-04-10 15:57

This was discussed in the 10th April teleconference. We plan to accept these changes in spirit, but since there is overlap with several other bugs, we will wait until after TC2 has been merged into Issue 7 in order to have a clean starting point for the detailed edits.
(0002222)
eblake (manager)
2014-04-10 15:59

http://open-std.org/JTC1/SC22/WG14/www/docs/dr_400.htm [^]

- Issue History
Date Modified Username Field Change
2011-02-03 17:12 eblake New Issue
2011-02-03 17:12 eblake Status New => Under Review
2011-02-03 17:12 eblake Assigned To => ajosey
2011-02-03 17:12 eblake Name => Eric Blake
2011-02-03 17:12 eblake Organization => Red Hat
2011-02-03 17:12 eblake User Reference => ebb.malloc
2011-02-03 17:12 eblake Section => malloc
2011-02-03 17:12 eblake Page Number => 1268
2011-02-03 17:12 eblake Line Number => 41715
2011-02-03 17:12 eblake Interp Status => ---
2011-02-03 17:15 nick Tag Attached: c99
2011-02-03 17:23 eblake Desired Action Updated
2011-02-03 17:25 eblake Note Added: 0000664
2011-03-17 12:30 nick Note Added: 0000708
2011-03-17 12:31 nick Note Added: 0000709
2011-03-17 16:07 nick Note Deleted: 0000709
2011-03-17 16:09 nick Note Edited: 0000708
2011-03-24 16:31 eblake Tag Attached: issue8
2011-03-24 16:38 eblake Relationship added child of 0000400
2011-03-24 16:41 eblake Note Added: 0000722
2011-04-07 20:45 eblake Note Added: 0000737
2011-11-03 15:22 nick Note Added: 0001001
2012-02-16 16:13 nick Note Added: 0001130
2012-02-16 17:15 nick Relationship added related to 0000526
2013-05-09 15:57 eblake Relationship added child of 0000688
2013-05-09 16:23 eblake Relationship deleted child of 0000688
2013-05-09 16:23 eblake Relationship added related to 0000688
2014-04-10 15:57 geoffclare Note Added: 0002221
2014-04-10 15:59 eblake Note Added: 0002222


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker