Anonymous | Login | 2023-12-02 07:45 UTC |
Main | My View | View Issues | Change Log | Docs |
Viewing Issue Simple Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||
ID | Category | Severity | Type | Date Submitted | Last Update | ||
0000934 | [1003.1(2013)/Issue7+TC1] Base Definitions and Headers | Objection | Omission | 2015-04-05 02:52 | 2019-06-10 08:54 | ||
Reporter | dannyniu | View Status | public | ||||
Assigned To | |||||||
Priority | normal | Resolution | Accepted As Marked | ||||
Status | Closed | ||||||
Name | DannyNiu/NJF | ||||||
Organization | |||||||
User Reference | |||||||
Section | <netinet/in.h> - Internet address family and <sys/un.h> - definitions for UNIX domain sockets | ||||||
Page Number | 306,406 | ||||||
Line Number | 10224-10225, 10238-10239, 13611-13612 | ||||||
Interp Status | Approved | ||||||
Final Accepted Text | See Note: 0002695. | ||||||
Summary | 0000934: Requirement for zeroing the sockaddr_in and sockaddr_un structures. | ||||||
Description |
The netinet/in.h says that sockaddr_in6 should be memset'zero before being used by application, as there may be implementation-specific fields. But the standard didn't make the same requirement for sockaddr_in structure which is meant for IPv4 communication when it should. The standard didn't require it for sockaddr_un either. I could be wrong, but as far as my knowledge reaches, zeroing these structs prior to using them is the de facto standard practice of portability, and without mandating it in the POSIX standard, de jure portability may not be fully achieved. |
||||||
Desired Action |
Add the following description for sockaddr_in, sockaddr_in6, and sockaddr_un (hence denoted as sockaddr_*) to page netinet/in.h and sys/un.h. The sockaddr_* structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_*. |
||||||
Tags | tc2-2008 | ||||||
Attached Files | |||||||
|
![]() |
|||||||||||||||||||||||||
|
![]() |
|
(0002610) Don Cragun (manager) 2015-04-05 05:12 edited on: 2015-04-05 05:12 |
The requested text is already present for sockaddr_6 on P306, L10238-10239. Add the following new paragraph on the <netinet/in.h> DESCRIPTION section after P306, L10225: The sockaddr_in structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_in. Add the following new paragraph on the <sys/un.h> DESCRIPTION section after P406, L13612: The sockaddr_un structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_un. |
(0002611) philip-guenther (reporter) 2015-04-05 19:00 |
Are there implementation where sockaddr_in or sockaddr_un have additional "must be initialized" members**? Given the number of programs I've seen that don't zero those out but rather just set the family and either port and addr or path members, I doubt it. Is adding a new requirement that renders those working applications non-compliant in order to permit systems to add "must be initialized" members to the existing sockaddr_{in,un} really a good idea? It just seems to increase the reliance on unspecified behavior, which is bad training for developers. "What problem are we trying to solve?" ** BSD's sin_len and sun_len do *not* need to be initialized by userspace when using the POSIX APIs |
(0002622) dannyniu (reporter) 2015-04-14 13:42 edited on: 2015-04-14 14:53 |
Well Mr. Guenther, the problem we're trying to solve, is as mentioned in my original post, that is how to achieve de-jure portability for sockaddr_*. I'll first object "in order to permit them to have "must be initialized" field". We already gave them permission to do that when we said sockaddr_* contain "at least" the following. BSD did with the 'len' field. What I'm doing here, is trying to provide a 'standard' way of dealing with them. Second, in answer to your questions, The reason those apps don't zero those out, is that POSIX didn't mandate it. If one fear that the new requirement would render working applications non-compliant, then maybe we can change the wording to "should" instead of "shall". The biggest benefit of mendating pre-zeroing, I is that it provides a way for the applications to initialize sockaddr_* in a way that they can be certain of, which is the purpose and the whole meaning of POSIX - portability. The downside of *not* mandating it, I think would be that the applications will not be certain that, ** one way of initializing sockaddr_* that works on one system, will still work on another system **. And mandating it would be able to solve it. Plus, if the app was to be used on one system exclusively, POSIX don't need to concern for him because he doesn't need portability. And if an app do want portability, he would've already thought of this problem and inserted '#ifdef's. What's more, the code example from the "FreeBSD Developer's Handbook" does it and says so. As socket originated from BSD and that FreeBSD has the largest usage share, I think it can serve as an authorative reference. |
(0002623) dannyniu (reporter) 2015-04-14 14:18 |
A few arguments I failed to present in my previous note. There aren't implementation that has additional must-be-initialized fields right now, doesn't mean there isn't going to be in the future. With this mandate is in place, those future implementations would make sure if system received zero'd fields, they won't do anything the apps didn't intend, thus increasing portability. Without it, when a programmer ports it to the new system, he'll have to debug and find out about it, then he'll have to add '#ifdef's. Thus losing portability. And on existing systems, unless the system developer is insane, ** I don't think he would add new fields ** as it obviously breaks compatibility. So existing application that'll be rendered non-conforming to POSIX, will nine in ten still be working programs on those system. |
(0002624) dannyniu (reporter) 2015-04-14 14:29 edited on: 2015-04-14 14:31 |
From developer's perspective, If I want to write a portable program that communicates using Unix-domain socket. I would first consult POSIX on how to use sockaddr_un. But turns out it there could be additional fields on specific implementations. So I have to go through all of their man pages, and add a lot of '#ifdef's. This is right now the most secure way, and it's became totally unportable. So the blame is on POSIX for omiting it. Second, sockaddr_* are unlike other structs in the standard. Other structs such as dirent and stat are filled by the system, as these information were to be surveyed by the apps. But sockaddr_* are like forms that apps submit to system to make IPC calls. If a human finds a field in the form he doesn't know about, he'll ask, but how can apps ask? So we should obviously provide a uniform method to safely initialize sockaddr_*. There is an existing practice of pre-zeroing the sockaddr_* structs. And this practice is specified in the "FreeBSD Developer's Handbook". Because sockets originate from BSD, and that FreeBSD has the largest usage share within BSD systems, I believe it can serve as an authorative reference. |
(0002630) philip-guenther (reporter) 2015-04-18 23:21 |
> Well Mr. Guenther, the problem we're trying to solve, is as mentioned in my > original post, that is how to achieve de-jure portability for sockaddr_*. The de-jure *and* de-facto portability of sockaddr_{in,un} initialization is that applications only have to initialize the members in the specification. > I'll first object "in order to permit them to have "must be initialized" field". > We already gave them permission to do that when we said sockaddr_* contain > "at least" the following. BSD did with the 'len' field. The standard currently does not require applications to memset the structures before passing them to POSIX APIs, therefore if a system fails to implement the specified behavior of POSIX APIs when other members are uninitialized then that _system_ does not comply with POSIX; the application is fine. The BSD len fields are not a problem for this because the system ignores them on input. > sockaddr_* are unlike other structs in the standard. I do not see evidence of this in the standard. In a quick scan, the only structure other than sockaddr_in6 that I see which must be initialized to zero is the struct addrinfo passed as the 'hints' argument to getaddrinfo(). For other calls the standard describes explicitly the behavior based on how the standardized members are set, leaving no room for a member not described to change the behavior of the interface. For structures where there is an explicit initialization function (e.g., pthread_attr_t and pthread_attr_init()), the requirement that the application use the initialization function means that systems can add members that affect the behavior as long as the standard's initialization function sets them to a 'no-op' value. Systems can also, of course, add 'output only' members, such as the various additions to struct stat out there, as well as BSD's 'len' member in sockaddr*. |
(0002631) dannyniu (reporter) 2015-04-19 05:51 edited on: 2015-04-20 08:12 |
Well. Thank you for your patience Mr. Guenther. I think at this point, I can get your point, and I believe you can get mine as well. So allow me to summerize here and propose a solution that we can both agree upon. Please do correct me if you feel to. I get that the current applications does not "pre-zero sockaddr_*", the standard "officially" allowed them to, and they should continue to be allowed to do so without worrying they'd be non-conforming. I believe you get my point, that I think apps should pre-zero sockaddr_* structs, for reasons including 1) a very authorative reference says to so, 2) I'd believe it increases portability, (even on some systems that you may consider "unPOSIXly"). So as a compromise, **I propose** that in "Application Usage", the practice of pre-zeroing be mentioned, along with the reasons for apps to do so as well as the reason it's not included in the normative sections. A bit of thing I forgot to mention. I'm someone who always like to follow the intuition. And I think the most **obvious** reason we should require pre-zeroing is that we already did for sockaddr_in6. And since all sockaddr_in, sockaddr_un, and sockaddr_in6 are derived from the generic sockaddr structure, we should require it for all of them. Although, I admit that this would be inconsiderate of many things. |
(0002632) geoffclare (manager) 2015-04-20 08:47 |
I see that the description and one of the comments in this bug mention the use of memset() to "zero" a structure. I would like to point out that for the two structures which applications are required to initialize, namely the getaddrinfo() hints argument and sockaddr_in6, the use of memset() to initialize them results in undefined behaviour. This is very clear for getaddrinfo(), perhaps less clear for sockaddr_in6 (and should be clarified). For getaddrinfo(), the standard says "In this hints structure every member other than ai_flags, ai_family, ai_socktype, and ai_protocol shall be set to zero or a null pointer." Since the C standard does not require that null pointers have an all-bits zero representation, the portable way to perform the required initialisation, as per 0000918 is: struct addrinfo hints = {0}; and the use of memset() might set any pointers in the structure to invalid values. (An alternative portable method is static struct addrinfo hints_init; struct addrinfo hints = hints_init;). For sockaddr_in6, the standard says "The sockaddr_in6 structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_in6." Although it is not clear what "set to zero" means, the fact that additional fields are given as the reason, and that those fields could include pointers, means using memset() has the same issue as for addrinfo. |
(0002633) dannyniu (reporter) 2015-04-21 09:14 edited on: 2015-04-22 03:01 |
Allow me to correct you Mr. Geoffclare (pardon me if I misspelled). Although the C standard didn't require NULL to be 0, POSIX did, in <stddef.h>. And I haven't read the official C99 spec, but according to draft n1256's section 6.3.2.3. NULL is also 0. It could be in case such as 1's complement representation that NULL is -0, but I suppose 0 means +0 and no sane people would use -0, and save 1024 words of explanation here. So I suppose NULL is all-bits-zero in the context of POSIX, snd I think "set to zero" in correct here for both addrinfo and sockaddr_*. Okay, I see your point in the next note. Yeah, if you're referring to the scenario where an insane developer would exploit the loophole in section 6.2.6. of C99 and randomize every pointer address, then I guess it'd make sense. |
(0002634) geoffclare (manager) 2015-04-21 09:32 |
In response to Note: 0002633, I'm afraid that far from "correcting" me, all you have done is demonstrate that your knowledge of the C language is somewhat lacking. My point in Note: 0002632 has nothing to do with the NULL macro; it is about the representation of pointer types. If you want to discuss this further, please do so by email to the mailing list, rather than cluttering up this bug with additional irrelevant notes. |
(0002635) dannyniu (reporter) 2015-04-23 04:13 |
Okay, I did took some time studying on Note: 0002632 and Note 0002634. And I would like now to propose a solution, that we mandate NULL pointer to be represented all-bits-zero. And I think the following reasons may be adequate to justify it: 1. It's almost conventionalized for implementors to make pointers have same representation as size_t and intptr_t types. Mandating it would render hardly any implementation non-conforming. 2. We made a similar requirement: that the <NUL> character be all-bits-zero. 3. Mandating it is just much simpler! And **isn't simplicity one of the philosophies of Unix**? If we have mandate all-bits-zero NULL, we'll be able to memset(,0,) it like we've always did. |
(0002637) geoffclare (manager) 2015-04-24 09:45 |
In the April 23rd teleconference it was decided to clarify the requirements for initialising getaddrinfo() hints. This resulted in 0000939 (the change in the desired action was worked out during the call). There are similar issues with the requirements for initialising sockaddr_in6 and a change along the lines of the one in 0000939 should also be made for this structure. Additional problems with the sockaddr_in6 case are: * It is not clear what "prior to using it" means. For example, passing a pointer to it as an argument to memset() could be considered to be "using it". * It is not necessary to set the fields to zero/null if the structure has been properly initialised by some other means (e.g. from data returned by getaddrinfo()). |
(0002674) rhansen (manager) 2015-05-21 15:51 edited on: 2015-06-04 16:33 |
[Note: this old response is superseded by Note: 0002695] Interpretation response ------------------------ The standard clearly states that applications need not initialize non-standard members of the sockaddr_in and sockaddr_un structures, and conforming implementations must conform to this. Rationale: ------------- Historically some applications initialize just the standard members, and some initialize the whole structure (using default initialization or memset()). There may be applications which do the latter for sockaddr_in and then rely on the zero value of the sin_family member being set to AF_UNSPEC. Therefore the standard should require that AF_UNSPEC has the value 0. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- On Page: 306 Line: 10238 Section: <netinet/in.h>, change from: The sockaddr_in6 structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_in6.to: Prior to calling a function in this standard which reads values from a sockaddr_in6 structure (for example, bind() or connect()), the application shall ensure that all members of the structure, including any additional non-standard members, if any, are initialized. If the sockaddr_in6 structure has a non-standard member, and that member has a value other than the value that would result from default initialization, the behavior of any function in this standard that reads values from the sockaddr_in6 structure is implementation-defined. All functions in this standard that return data in a sockaddr_in6 structure (for example, getaddrinfo() or accept()) shall initialize the structure in a way that meets the above requirements, and shall ensure that each non-standard member, if any, has a value equal to the value that would result from default initialization.On Page: 309 Line: 10323 change APPLICATION USAGE from: None.to: Although applications are required to initialize all members (including any non-standard ones) of a sockaddr_in6 structure, the same is not required for the sockaddr_in structure, since historically many applications only initialized the standard members. Despite this, applications are encouraged to initialize sockaddr_in structures in a manner similar to the required initialization of sockaddr_in6 structures.On page 388 after line 13000 insert the following new paragraph: The value of AF_UNSPEC shall be 0.On page 406 after line 13620 (sys/un.h APPLICATION USAGE) insert the following new paragraph: Although applications are required to initialize all members (including any non-standard ones) of a sockaddr_in6 structure (see <netinet/in.h>), the same is not required for the sockaddr_un structure, since historically many applications only initialized the standard members. Despite this, applications are encouraged to initialize sockaddr_un structures in a manner similar to the required initialization of sockaddr_in6 structures. |
(0002677) ajosey (manager) 2015-05-24 12:59 |
Interpretation Proposed: 24 May 2015 |
(0002695) Don Cragun (manager) 2015-06-04 15:48 edited on: 2015-06-04 15:50 |
Interpretation response ------------------------ The standard clearly states that applications need not initialize non-standard members of the sockaddr_in and sockaddr_un structures, and conforming implementations must conform to this. Rationale: ------------- Historically some applications initialize just the standard members, and some initialize the whole structure (using default initialization or memset()). There may be applications which do the latter for sockaddr_in and then rely on the zero value of the sin_family member being set to AF_UNSPEC. Therefore the standard should require that AF_UNSPEC has the value 0. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- On Page: 306 Line: 10238 Section: <netinet/in.h>, change from: The sockaddr_in6 structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_in6. to: Prior to calling a function in this standard which reads values from a sockaddr_in6 structure (for example, bind() or connect()), the application shall ensure that all members of the structure, including any additional non-standard members, if any, are initialized. If the sockaddr_in6 structure has a non-standard member, and that member has a value other than the value that would result from default initialization, the behavior of any function in this standard that reads values from the sockaddr_in6 structure is implementation-defined. All functions in this standard that return data in a sockaddr_in6 structure (for example, getaddrinfo() or accept()) shall initialize the structure in a way that meets the above requirements, and shall ensure that each non-standard member, if any, has a value that produces the same behavior as default initialization would in all functions in this standard which read values from a sockaddr_in6 structure. On Page: 309 Line: 10323 change APPLICATION USAGE from: None. to: Although applications are required to initialize all members (including any non-standard ones) of a sockaddr_in6 structure, the same is not required for the sockaddr_in structure, since historically many applications only initialized the standard members. Despite this, applications are encouraged to initialize sockaddr_in structures in a manner similar to the required initialization of sockaddr_in6 structures. On page 388 after line 13000 insert the following new paragraph: The value of AF_UNSPEC shall be 0. On page 406 after line 13620 (sys/un.h APPLICATION USAGE) insert the following new paragraph: Although applications are required to initialize all members (including any non-standard ones) of a sockaddr_in6 structure (see <netinet/in.h>), the same is not required for the sockaddr_un structure, since historically many applications only initialized the standard members. Despite this, applications are encouraged to initialize sockaddr_un structures in a manner similar to the required initialization of sockaddr_in6 structures. On page 928 after line 31296 (freeaddrinfo() APPLICATION USAGE) insert the following new paragraph: Although it is common practice to initialize the hints structure using: On page 107 after line 2884, insert a new 4.x General Concepts section (and renumber subsequent sections as needed): Default Initialization |
(0002701) ajosey (manager) 2015-06-05 13:45 |
Interpretation Proposed: 5 June 2015 (restarting clock after revision to interpretation) |
(0002821) ajosey (manager) 2015-09-07 11:34 |
Interpretation approved: 7 Sep 2015 |
(0002887) geoffclare (manager) 2015-11-06 09:51 |
As an editorial matter, in order to keep the section numbering of XBD 4.x and XRAT A.4.x in sync, a new XRAT A.4.x Default Initialization section needs to be added. See 0001000. |
![]() |
|||
Date Modified | Username | Field | Change |
2015-04-05 02:52 | dannyniu | New Issue | |
2015-04-05 02:52 | dannyniu | Name | => DannyNiu/NJF |
2015-04-05 02:52 | dannyniu | Section | => netinet/in.h - Internet address family and sys/un.h - definitions for UNIX domain sockets |
2015-04-05 02:52 | dannyniu | Page Number | => basedefs/netinet_in.h.html and basedefs/sys_un.h.html |
2015-04-05 02:52 | dannyniu | Line Number | => Unsure. |
2015-04-05 05:12 | Don Cragun | Note Added: 0002610 | |
2015-04-05 05:12 | Don Cragun | Note Edited: 0002610 | |
2015-04-05 05:21 | Don Cragun | Section | netinet/in.h - Internet address family and sys/un.h - definitions for UNIX domain sockets => <netinet/in.h> - Internet address family and <sys/un.h> - definitions for UNIX domain sockets |
2015-04-05 05:21 | Don Cragun | Page Number | basedefs/netinet_in.h.html and basedefs/sys_un.h.html => 306,406 |
2015-04-05 05:21 | Don Cragun | Line Number | Unsure. => 10224-10225, 10238-10239, 13611-13612 |
2015-04-05 05:21 | Don Cragun | Interp Status | => --- |
2015-04-05 19:00 | philip-guenther | Note Added: 0002611 | |
2015-04-14 13:42 | dannyniu | Note Added: 0002622 | |
2015-04-14 14:18 | dannyniu | Note Added: 0002623 | |
2015-04-14 14:29 | dannyniu | Note Added: 0002624 | |
2015-04-14 14:29 | dannyniu | Note Added: 0002625 | |
2015-04-14 14:30 | dannyniu | Note Deleted: 0002625 | |
2015-04-14 14:31 | dannyniu | Note Edited: 0002624 | |
2015-04-14 14:31 | dannyniu | Note Edited: 0002624 | |
2015-04-14 14:47 | dannyniu | Note Edited: 0002622 | |
2015-04-14 14:48 | dannyniu | Note Edited: 0002622 | |
2015-04-14 14:53 | dannyniu | Note Edited: 0002622 | |
2015-04-15 01:10 | dannyniu | Issue Monitored: dannyniu | |
2015-04-18 23:21 | philip-guenther | Note Added: 0002630 | |
2015-04-19 05:51 | dannyniu | Note Added: 0002631 | |
2015-04-19 05:52 | dannyniu | Note Edited: 0002631 | |
2015-04-20 08:12 | dannyniu | Note Edited: 0002631 | |
2015-04-20 08:47 | geoffclare | Note Added: 0002632 | |
2015-04-20 08:47 | geoffclare | Relationship added | related to 0000918 |
2015-04-21 09:14 | dannyniu | Note Added: 0002633 | |
2015-04-21 09:32 | geoffclare | Note Added: 0002634 | |
2015-04-22 01:19 | dannyniu | Note Edited: 0002633 | |
2015-04-22 03:01 | dannyniu | Note Edited: 0002633 | |
2015-04-23 04:13 | dannyniu | Note Added: 0002635 | |
2015-04-24 09:45 | geoffclare | Note Added: 0002637 | |
2015-04-24 09:46 | geoffclare | Relationship added | related to 0000939 |
2015-04-26 13:57 | eblake | Relationship added | related to 0000940 |
2015-05-21 15:51 | rhansen | Note Added: 0002674 | |
2015-05-21 15:56 | rhansen | Note Edited: 0002674 | |
2015-05-21 15:58 | rhansen | Final Accepted Text | => see Note: 0002674 |
2015-05-21 15:58 | rhansen | Status | New => Resolved |
2015-05-21 15:58 | rhansen | Resolution | Open => Accepted As Marked |
2015-05-21 15:59 | rhansen | Tag Attached: tc2-2008 | |
2015-05-21 16:10 | rhansen | Note Edited: 0002674 | |
2015-05-21 16:11 | geoffclare | Interp Status | --- => Pending |
2015-05-21 16:11 | geoffclare | Status | Resolved => Interpretation Required |
2015-05-24 12:59 | ajosey | Interp Status | Pending => Proposed |
2015-05-24 12:59 | ajosey | Note Added: 0002677 | |
2015-06-04 15:48 | Don Cragun | Note Added: 0002695 | |
2015-06-04 15:50 | Don Cragun | Note Edited: 0002695 | |
2015-06-04 15:51 | Don Cragun | Final Accepted Text | see Note: 0002674 => See Note: 0002695. |
2015-06-04 16:33 | geoffclare | Note Edited: 0002674 | |
2015-06-05 13:45 | ajosey | Note Added: 0002701 | |
2015-09-07 11:34 | ajosey | Interp Status | Proposed => Approved |
2015-09-07 11:34 | ajosey | Note Added: 0002821 | |
2015-11-06 09:47 | geoffclare | Relationship added | related to 0001000 |
2015-11-06 09:51 | geoffclare | Note Added: 0002887 | |
2017-12-23 11:31 | dannyniu | Issue End Monitor: dannyniu | |
2019-06-10 08:54 | agadmin | Status | Interpretation Required => Closed |
Mantis 1.1.6[^] Copyright © 2000 - 2008 Mantis Group |