Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000400 [1003.1(2008)/Issue 7] System Interfaces Objection Error 2011-03-24 16:37 2014-04-10 16:00
Reporter eblake View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Closed  
Name Eric Blake
Organization Red Hat
User Reference ebb.realloc
Section realloc
Page Number 1754
Line Number 56027
Interp Status Approved
Final Accepted Text Note: 0000772
Summary 0000400: realloc wording conflicts with C99
Description The C committee considered three possibilities:

1. Do nothing and accept the fact that some C99 implementations
   may not be able to conform to the requirements of the C99
   standard in this regard for compatibility reasons.
2. Change C to more closely align with the POSIX requirements
   so as to make the existing implementatins conforming. This
   resolution is being sought by the POSIX group and is also
   preferred by the implementers of the affected implementations.
3. Change both the C and POSIX standards to require realloc()
   to always attempt to allocate space even for zero-size
   requests. This would simplify the specification and, at
   least in theory, also make it easier to write portable
   programs. This option would require the support of the
   implementers of the affected implementations. Without such
   support this option becomes equivalent to option 1 above.

The outcome of this discussion is that if realloc returns NULL, it has failed, and the original memory has not been freed.
The POSIX words came from C90, and these were changed for C99. POSIX failed to pick up the change

See N872 item 19c.

Unfortunately this breaks several existing implementations. However, those implementer present (Sun/Oracle, Apple, IBM, HP) all agreed that they were prepared to change their implementations.

NOTE: Since the text at the top of realloc() states:
   The functionality described on this reference page is aligned with the ISO C standard. Any conflict
    between the requirements described here and the ISO C standard is unintentional. This volume of
    POSIX.1-2008 defers to the ISO C standard.
I am not sure we have the wiggle room to do anything but DEFECT situation.
Desired Action POSIX Change requested by C99:
Page 1754 line 56030 Change:

The realloc() function shall change the size of the memory object pointed to by ptr to the size specified by size. The contents of the object shall remain unchanged up to the lesser of the new and old sizes. If the new size of the memory object would require movement of the object, the space for the previous instantiation of the object is freed. If the new size is larger, the contents of the newly allocated portion of the object are unspecified. If size is 0 and ptr is not a null pointer, the object pointed to is freed.
to
The realloc() function shall deallocate the old object pointed to by ptr and return a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.


Page 1754 line 56046 change
Upon successful completion with a size not equal to 0, realloc() shall return a pointer to the (possibly moved) allocated space.

to

Upon successful completion, realloc() shall return a pointer to the (possibly moved) allocated space.

At line 56047-8 change:

If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned.
to
If size is 0, either:
  * a null pointer shall be returned <CX>and errno set to EINVAL</CX>, or
  * unique pointer that can be successfully passed to free() shall be returned, and the memory object pointed to by ptr shall be freed.

Add after line 56049:
If realloc() returns NULL, the memory referenced by ptr is not changed.

Add after 56052:

The realloc() function may fail if:
EINVAL size is 0 and the implementation does not support 0 sized allocations.
Tags c99, tc1-2008
Attached Files

- Relationships
parent of 0000374Closedajosey 1003.1(2008)/Issue 7 malloc(0) and realloc(p,0) must not change errno on success 
parent of 0000524Closedajosey 2008-TC1 The 2nd of 2 'If size is 0, either:' option was split into 2 options. 
related to 0000526Closedajosey 1003.1(2008)/Issue 7 Adopt C99 wording for zero size calloc(), malloc() et al. 
related to 0000688Closed 1003.1(2013)/Issue7+TC1 realloc(NULL, 0) vs. malloc(0) 

-  Notes
(0000723)
eblake (manager)
2011-03-24 17:08

 glibc is impacted by the outcome of this bug:
http://sourceware.org/bugzilla/show_bug.cgi?id=12547 [^]
(0000725)
geoffclare (manager)
2011-03-25 09:48

 The line 56049 addition should be worded:

    If realloc() returns a null pointer, the memory referenced by ptr
    shall not be changed.

I think the EINVAL error should be "shall fail", not "may fail",
because although the behaviour of returning an EINVAL error for size 0
is optional, the implementation choice is stated in the error
description: "... and the implementation does not support 0 sized
allocations."
(0000726)
eblake (manager)
2011-03-25 20:22

 A fourth possibility not covered by the C99's recommendations:

Change line 56034 [realloc DESCRIPTION]:
If size is 0 and ptr is not a null pointer, the object pointed to is freed.
to:
<CX>If size is 0 and ptr is not a null pointer, behavior is unspecified;
portable applications shall ensure that a size of 0 is only used when
ptr is a null pointer.</CX>

Change line 56047 [RETURN VALUE]:
If size is 0,
to:
If size is 0 and ptr is a null pointer,

Change line 56058 [RATIONALE]:
None.
to two paragraphs:
The C99 standard requires that for a non-null pointer ptr, realloc(ptr,0)
will either return a zero-size allocation (as if by malloc(0)) or leave
ptr unchanged (because a NULL return indicates failure). However, there
are existing implementations that take a third approach of freeing ptr
while still returning NULL. While these implementations do not comply
with C99, they are still permitted by this version of POSIX. An
application that assumes C99 semantics by calling free(ptr) after a NULL
return from realloc(ptr,0) failure will end up with undefined behavior
of freeing an already-freed pointer. Therefore, this standard completely
avoids the portability issues by instead requiring that applications
never use realloc with a non-null pointer but zero size.

This restriction does not represent a conflict with the ISO C standard.
An implementation may choose whether it wishes to comply to both C99
and POSIX (use of size 0 will have some guarantees per C99, although
a conforming POSIX application will not trigger those guarantees), or
just POSIX.



Note that while there are several places in the standard where <CX>
provides tighter implementation constraints (not all C99 implementations
comply with POSIX), I think this would be the first case of the converse
(a case where <CX> provides looser implementation constraints, that is,
where a POSIX-compliant implementation need not obey C99. On the other
hand, where this standard is tighter than C99, it implies that an
application that conforms to this standard will not necessarily behave
the same way in all other C99 environments; whereas this requirement
to avoid the unspecified behavior of the looser implementation guarantees
that an application written to this standard will never trigger
portability problems whether run on a C99 environment or a legacy
environment. Given the extreme reluctance to change existing
implementations, this may be the only solution for POSIX to continue to
cater to implementations such as glibc, while still permitting
implementations that obey C99.
(0000727)
eblake (manager)
2011-03-25 20:26

 Note also that this standard has already had a track record of marking
some C99 requirements as obsolescent and not to be relied on by portable
programs, such as gets(), which gives more precedence to the notion
of declaring realloc(ptr,0) as <OB>.
(0000734)
nick (manager)
2011-04-01 22:29

 Sent to the SC22WG14 email reflector:


The Austin Group has been studying this problem (see http://austingroupbugs.net/view.php?id=400) [^] and trying to resolve any conflict with C. This was also examined by WG14 during the London meeting, where the resolution was "POSIX should conform to the requirements of C". The Austin Group agrees with this resolution.

However, during our discussions, we have found the following in the C standard (using refs to C99, not C1x):

Point 1: Objects cannot be zero sized.
6.2.6.1(2) Except for bit-fields, objects are composed of contiguous sequences of one or more bytes,

Point 2: realloc(NULL, 0) is required to behave like malloc(0), and malloc states "7.20.3.3(2) The malloc function allocates space for an object whose size is specified by size and whose value is indeterminate." Since objects cannot be zero sized, returning a non-null pointer is at best questionable. However, this argues that realloc(p, 0) should always fail.

Point 3: Explicit permission to succeed AND return NULL is granted: "7.20.3(1) If the size of the space requested is zero, the behavior is implementation- defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object." [this refers to calloc, malloc and realloc]. It has been claimed that if realloc(p,0) returns a null pointer, that indicates that "the new object cannot be allocated" and so the old object remains alive. However, that interpretation is not clear, and the two separate sentences in 7.20.3.1 about null pointers are distinct cases, and the second does not necessarily imply the first.

The POSIX words currently follow the original C89 text, and not the C99 update. They state: "If size is 0 and ptr is not a null pointer,
the object pointed to is freed. ...RETURNS... If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned." The C99 words state "7.20.3.4(2) The realloc function deallocates the old object pointed to by ptr and returns a pointer to a new object that has the size specified by size." This would appear to allow an implementation to deallocate the space and then return NULL (without setting errno). Additionally, "7.20.3(3) ... or if the space has been deallocated by a call to the free or realloc function..." strongly suggests that realloc(ptr, 0) is a valid way to free ptr.

So, the Austin Group currently does not believe that POSIX conflicts with C, and that it is permissible for realloc(p, 0) (where p is non-NULL) to free the space allocated and return NULL. In POSIX, it is acceptable to set errno to zero before the call, and test it afterwards to see if the pointer has been freed or not.

Our current survey has shown that the following implementations free the pointer and return NULL :

Glibc (GNU/Linux)
AIX
HP-UX
Solaris
OSF/1

There is strong resistance from at least one of these to changing their implementation, in the belief that this would make current, memory leak free, conforming code become both non-conforming and leaking.

--
Nick Stoughton
Austin Group/WG14 Liaison
(0000735)
nick (manager)
2011-04-01 22:30

 Reply on SC22WG14 reflector:

What WG 14 has been saying is that that interpretation is
incorrect. Referencing the latest C1x draft:

1. 7.22.3.5p4, "Returns," states "The realloc function returns a
pointer to the new object (which may have the same value as a pointer to
the old object), or a null pointer if the new object could not be
allocated." In other words, null means failure.

2. 7.22.3p1 states "If the space cannot be allocated, a null pointer is
returned. If the size of the space requested is zero, the behavior is
implementation-defined: either a null pointer is returned, or the
behavior is as if the size were some nonzero value, except that the
returned pointer shall not be used to access an object." This, together
with the previous quote, means that the function can either succeed and
return a valid pointer, or fail and return a null pointer.

3. 7.22.3.5p1 states "If memory for the new object cannot be allocated,
the old object is not deallocated and its value is unchanged." This,
together with the previous two quotes, means that if a null pointer is
returned, the operation has failed and the old object is not deallocated.

      You mentioned the fact that objects cannot be zero sized. That is
why the wording is the way it is. If we had zero-sized objects, we
could just say that an allocation of size zero returns a pointer to just
past the end of a zero-sized object (and that is why the pointer cannot
be dereferenced). Instead, we have to say the same thing without
talking about zero-sized objects.
(0000746)
nick (manager)
2011-04-21 15:25
edited on: 2011-04-28 15:17

 Current proposed wording:


Page 1754 line 56030 Change:

The realloc() function shall change the size of the memory object pointed to by ptr to the size specified by size. The contents of the object shall remain unchanged up to the lesser of the new and old sizes. If the new size of the memory object would require movement of the object, the space for the previous instantiation of the object is freed. If the new size is larger, the contents of the newly allocated portion of the object are unspecified. If size is 0 and ptr is not a null pointer, the object pointed to is freed.

to

The realloc() function shall deallocate the old object pointed to by ptr and return a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values. If the size of the space requested is zero, the behavior shall be implementation-defined: either a null pointer is returned, or the behavior shall be as if the size were some nonzero value, except that the returned pointer shall not be used to access an object.



Page 1754 line 56046 change
Upon successful completion with a size not equal to 0, realloc() shall return a pointer to the (possibly moved) allocated space.

to

Upon successful completion, realloc() shall return a pointer to the (possibly moved) allocated space.

At line 56047-8 change:

If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned.

to

If size is 0, either:
  * a null pointer shall be returned <CX>and errno set to an implementation defined value</CX>, or
  * unique pointer that can be successfully passed to free() shall be returned, and the memory object pointed to by ptr shall be freed. The application shall ensure that the pointer is not used to access an object.

Add after line 56049:
If realloc() returns NULL <CX>and errno has been set to a ENOMEM</CX>, the memory referenced by ptr shall not be changed.

At line 56056 (Application Usage), change:

None.

to

The description of realloc() has been modified from previous versions of this standard to align with C99. Previous versions explicitly permitted a call to realloc(p, 0) to free the space pointed to by p and return NULL. While this behavior could be interpreted as permitted by this version of the standard, the C language committee have indicated that this interpretation is incorrect. Applications should assume that if realloc returns a null pointer, the space pointed to be p has not been freed. Since this could lead to double-frees, implementations should also set errno if a null pointer actually indicates a failure, and applications should only free the space if errno was changed.


Change at line 56060 (Future Directions) from:

None.

to

This standard defers to the C standard. While that standard currently
has language that might permit realloc(p, 0), where p is not a null
pointer, to free p while still returning a null pointer, the committee
responsible for that standard is considering clarifying the language to
explicitly prohibit that alternative.
(0000772)
msbrown (manager)
2011-04-28 15:25

 Interpretation response
------------------------
The standard states its realloc specification , and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
The C standards WG has given us guidance on this issue.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Take the actions as described in Note: 0000746
(0000845)
ajosey (manager)
2011-06-16 10:16

 Interpretation proposed 16 June 2011 for final 30 day review
(0000907)
ajosey (manager)
2011-07-29 06:14

 The interpretation is now approved.
(0002223)
eblake (manager)
2014-04-10 16:00

 http://open-std.org/JTC1/SC22/WG14/www/docs/dr_400.htm [^]

- Issue History
Date Modified Username Field Change
2011-03-24 16:37 eblake New Issue
2011-03-24 16:37 eblake Status New => Under Review
2011-03-24 16:37 eblake Assigned To => ajosey
2011-03-24 16:37 eblake Name => Eric Blake
2011-03-24 16:37 eblake Organization => Red Hat
2011-03-24 16:37 eblake User Reference => ebb.realloc
2011-03-24 16:37 eblake Section => realloc
2011-03-24 16:37 eblake Page Number => 1754
2011-03-24 16:37 eblake Line Number => 56027
2011-03-24 16:37 eblake Interp Status => ---
2011-03-24 16:38 eblake Relationship added parent of 0000374
2011-03-24 16:38 eblake Tag Attached: c99
2011-03-24 16:38 eblake Tag Attached: tc1-2008
2011-03-24 17:08 eblake Note Added: 0000723
2011-03-25 09:48 geoffclare Note Added: 0000725
2011-03-25 20:22 eblake Note Added: 0000726
2011-03-25 20:26 eblake Note Added: 0000727
2011-03-31 15:29 msbrown Note Added: 0000731
2011-03-31 15:30 msbrown Note Deleted: 0000731
2011-04-01 22:29 nick Note Added: 0000734
2011-04-01 22:30 nick Note Added: 0000735
2011-04-21 15:25 nick Note Added: 0000746
2011-04-28 15:17 nick Note Edited: 0000746
2011-04-28 15:17 nick Note Edited: 0000746
2011-04-28 15:25 msbrown Interp Status --- => Pending
2011-04-28 15:25 msbrown Note Added: 0000772
2011-04-28 15:25 msbrown Status Under Review => Interpretation Required
2011-04-28 15:25 msbrown Resolution Open => Accepted As Marked
2011-04-28 15:26 msbrown Final Accepted Text => Note: 0000746
2011-04-28 15:28 msbrown Final Accepted Text Note: 0000746 => Note: 0000772
2011-06-16 10:16 ajosey Interp Status Pending => Proposed
2011-06-16 10:16 ajosey Note Added: 0000845
2011-07-29 06:14 ajosey Interp Status Proposed => Approved
2011-07-29 06:14 ajosey Note Added: 0000907
2011-12-06 16:12 nick Relationship added parent of 0000524
2011-12-12 10:27 geoffclare Relationship added related to 0000526
2013-04-16 13:06 ajosey Status Interpretation Required => Closed
2013-05-09 15:56 eblake Relationship added related to 0000688
2014-04-10 16:00 eblake Note Added: 0002223

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker