0000934: Requirement for zeroing the sockaddr_in and sockaddr_un structures.

Notes
(0002610) Don Cragun (manager) 2015-04-05 05:12 edited on: 2015-04-05 05:12	The requested text is already present for sockaddr_6 on P306, L10238-10239. Add the following new paragraph on the <netinet/in.h> DESCRIPTION section after P306, L10225: The sockaddr_in structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_in. Add the following new paragraph on the <sys/un.h> DESCRIPTION section after P406, L13612: The sockaddr_un structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_un.

(0002611) philip-guenther (reporter) 2015-04-05 19:00	Are there implementation where sockaddr_in or sockaddr_un have additional "must be initialized" members? Given the number of programs I've seen that don't zero those out but rather just set the family and either port and addr or path members, I doubt it. Is adding a new requirement that renders those working applications non-compliant in order to permit systems to add "must be initialized" members to the existing sockaddr_{in,un} really a good idea? It just seems to increase the reliance on unspecified behavior, which is bad training for developers. "What problem are we trying to solve?" BSD's sin_len and sun_len do not need to be initialized by userspace when using the POSIX APIs

(0002622) dannyniu (reporter) 2015-04-14 13:42 edited on: 2015-04-14 14:53	Well Mr. Guenther, the problem we're trying to solve, is as mentioned in my original post, that is how to achieve de-jure portability for sockaddr_. I'll first object "in order to permit them to have "must be initialized" field". We already gave them permission to do that when we said sockaddr_ contain "at least" the following. BSD did with the 'len' field. What I'm doing here, is trying to provide a 'standard' way of dealing with them. Second, in answer to your questions, The reason those apps don't zero those out, is that POSIX didn't mandate it. If one fear that the new requirement would render working applications non-compliant, then maybe we can change the wording to "should" instead of "shall". The biggest benefit of mendating pre-zeroing, I is that it provides a way for the applications to initialize sockaddr_* in a way that they can be certain of, which is the purpose and the whole meaning of POSIX - portability. The downside of not mandating it, I think would be that the applications will not be certain that, ** one way of initializing sockaddr_* that works on one system, will still work on another system **. And mandating it would be able to solve it. Plus, if the app was to be used on one system exclusively, POSIX don't need to concern for him because he doesn't need portability. And if an app do want portability, he would've already thought of this problem and inserted '#ifdef's. What's more, the code example from the "FreeBSD Developer's Handbook" does it and says so. As socket originated from BSD and that FreeBSD has the largest usage share, I think it can serve as an authorative reference.

(0002623) dannyniu (reporter) 2015-04-14 14:18	A few arguments I failed to present in my previous note. There aren't implementation that has additional must-be-initialized fields right now, doesn't mean there isn't going to be in the future. With this mandate is in place, those future implementations would make sure if system received zero'd fields, they won't do anything the apps didn't intend, thus increasing portability. Without it, when a programmer ports it to the new system, he'll have to debug and find out about it, then he'll have to add '#ifdef's. Thus losing portability. And on existing systems, unless the system developer is insane, I don't think he would add new fields as it obviously breaks compatibility. So existing application that'll be rendered non-conforming to POSIX, will nine in ten still be working programs on those system.

(0002624) dannyniu (reporter) 2015-04-14 14:29 edited on: 2015-04-14 14:31	From developer's perspective, If I want to write a portable program that communicates using Unix-domain socket. I would first consult POSIX on how to use sockaddr_un. But turns out it there could be additional fields on specific implementations. So I have to go through all of their man pages, and add a lot of '#ifdef's. This is right now the most secure way, and it's became totally unportable. So the blame is on POSIX for omiting it. Second, sockaddr_* are unlike other structs in the standard. Other structs such as dirent and stat are filled by the system, as these information were to be surveyed by the apps. But sockaddr_* are like forms that apps submit to system to make IPC calls. If a human finds a field in the form he doesn't know about, he'll ask, but how can apps ask? So we should obviously provide a uniform method to safely initialize sockaddr_. There is an existing practice of pre-zeroing the sockaddr_ structs. And this practice is specified in the "FreeBSD Developer's Handbook". Because sockets originate from BSD, and that FreeBSD has the largest usage share within BSD systems, I believe it can serve as an authorative reference.

(0002630) philip-guenther (reporter) 2015-04-18 23:21	> Well Mr. Guenther, the problem we're trying to solve, is as mentioned in my > original post, that is how to achieve de-jure portability for sockaddr_. The de-jure and* de-facto portability of sockaddr_{in,un} initialization is that applications only have to initialize the members in the specification. > I'll first object "in order to permit them to have "must be initialized" field". > We already gave them permission to do that when we said sockaddr_* contain > "at least" the following. BSD did with the 'len' field. The standard currently does not require applications to memset the structures before passing them to POSIX APIs, therefore if a system fails to implement the specified behavior of POSIX APIs when other members are uninitialized then that _system_ does not comply with POSIX; the application is fine. The BSD len fields are not a problem for this because the system ignores them on input. > sockaddr_* are unlike other structs in the standard. I do not see evidence of this in the standard. In a quick scan, the only structure other than sockaddr_in6 that I see which must be initialized to zero is the struct addrinfo passed as the 'hints' argument to getaddrinfo(). For other calls the standard describes explicitly the behavior based on how the standardized members are set, leaving no room for a member not described to change the behavior of the interface. For structures where there is an explicit initialization function (e.g., pthread_attr_t and pthread_attr_init()), the requirement that the application use the initialization function means that systems can add members that affect the behavior as long as the standard's initialization function sets them to a 'no-op' value. Systems can also, of course, add 'output only' members, such as the various additions to struct stat out there, as well as BSD's 'len' member in sockaddr*.

(0002631) dannyniu (reporter) 2015-04-19 05:51 edited on: 2015-04-20 08:12	Well. Thank you for your patience Mr. Guenther. I think at this point, I can get your point, and I believe you can get mine as well. So allow me to summerize here and propose a solution that we can both agree upon. Please do correct me if you feel to. I get that the current applications does not "pre-zero sockaddr_", the standard "officially" allowed them to, and they should continue to be allowed to do so without worrying they'd be non-conforming. I believe you get my point, that I think apps should pre-zero sockaddr_ structs, for reasons including 1) a very authorative reference says to so, 2) I'd believe it increases portability, (even on some systems that you may consider "unPOSIXly"). So as a compromise, I propose that in "Application Usage", the practice of pre-zeroing be mentioned, along with the reasons for apps to do so as well as the reason it's not included in the normative sections. A bit of thing I forgot to mention. I'm someone who always like to follow the intuition. And I think the most obvious reason we should require pre-zeroing is that we already did for sockaddr_in6. And since all sockaddr_in, sockaddr_un, and sockaddr_in6 are derived from the generic sockaddr structure, we should require it for all of them. Although, I admit that this would be inconsiderate of many things.

(0002632) geoffclare (manager) 2015-04-20 08:47	I see that the description and one of the comments in this bug mention the use of memset() to "zero" a structure. I would like to point out that for the two structures which applications are required to initialize, namely the getaddrinfo() hints argument and sockaddr_in6, the use of memset() to initialize them results in undefined behaviour. This is very clear for getaddrinfo(), perhaps less clear for sockaddr_in6 (and should be clarified). For getaddrinfo(), the standard says "In this hints structure every member other than ai_flags, ai_family, ai_socktype, and ai_protocol shall be set to zero or a null pointer." Since the C standard does not require that null pointers have an all-bits zero representation, the portable way to perform the required initialisation, as per 0000918 is: struct addrinfo hints = {0}; and the use of memset() might set any pointers in the structure to invalid values. (An alternative portable method is static struct addrinfo hints_init; struct addrinfo hints = hints_init;). For sockaddr_in6, the standard says "The sockaddr_in6 structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_in6." Although it is not clear what "set to zero" means, the fact that additional fields are given as the reason, and that those fields could include pointers, means using memset() has the same issue as for addrinfo.

(0002633) dannyniu (reporter) 2015-04-21 09:14 edited on: 2015-04-22 03:01	Allow me to correct you Mr. Geoffclare (pardon me if I misspelled). Although the C standard didn't require NULL to be 0, POSIX did, in <stddef.h>. And I haven't read the official C99 spec, but according to draft n1256's section 6.3.2.3. NULL is also 0. It could be in case such as 1's complement representation that NULL is -0, but I suppose 0 means +0 and no sane people would use -0, and save 1024 words of explanation here. So I suppose NULL is all-bits-zero in the context of POSIX, snd I think "set to zero" in correct here for both addrinfo and sockaddr_*. Okay, I see your point in the next note. Yeah, if you're referring to the scenario where an insane developer would exploit the loophole in section 6.2.6. of C99 and randomize every pointer address, then I guess it'd make sense.

(0002634) geoffclare (manager) 2015-04-21 09:32	In response to Note: 0002633, I'm afraid that far from "correcting" me, all you have done is demonstrate that your knowledge of the C language is somewhat lacking. My point in Note: 0002632 has nothing to do with the NULL macro; it is about the representation of pointer types. If you want to discuss this further, please do so by email to the mailing list, rather than cluttering up this bug with additional irrelevant notes.

(0002635) dannyniu (reporter) 2015-04-23 04:13	Okay, I did took some time studying on Note: 0002632 and Note 0002634. And I would like now to propose a solution, that we mandate NULL pointer to be represented all-bits-zero. And I think the following reasons may be adequate to justify it: 1. It's almost conventionalized for implementors to make pointers have same representation as size_t and intptr_t types. Mandating it would render hardly any implementation non-conforming. 2. We made a similar requirement: that the <NUL> character be all-bits-zero. 3. Mandating it is just much simpler! And isn't simplicity one of the philosophies of Unix? If we have mandate all-bits-zero NULL, we'll be able to memset(,0,) it like we've always did.

(0002637) geoffclare (manager) 2015-04-24 09:45	In the April 23rd teleconference it was decided to clarify the requirements for initialising getaddrinfo() hints. This resulted in 0000939 (the change in the desired action was worked out during the call). There are similar issues with the requirements for initialising sockaddr_in6 and a change along the lines of the one in 0000939 should also be made for this structure. Additional problems with the sockaddr_in6 case are: * It is not clear what "prior to using it" means. For example, passing a pointer to it as an argument to memset() could be considered to be "using it". * It is not necessary to set the fields to zero/null if the structure has been properly initialised by some other means (e.g. from data returned by getaddrinfo()).

(0002674) rhansen (manager) 2015-05-21 15:51 edited on: 2015-06-04 16:33	[Note: this old response is superseded by Note: 0002695] Interpretation response ------------------------ The standard clearly states that applications need not initialize non-standard members of the sockaddr_in and sockaddr_un structures, and conforming implementations must conform to this. Rationale: ------------- Historically some applications initialize just the standard members, and some initialize the whole structure (using default initialization or memset()). There may be applications which do the latter for sockaddr_in and then rely on the zero value of the sin_family member being set to AF_UNSPEC. Therefore the standard should require that AF_UNSPEC has the value 0. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- On Page: 306 Line: 10238 Section: <netinet/in.h>, change from: The sockaddr_in6 structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_in6. to: Prior to calling a function in this standard which reads values from a sockaddr_in6 structure (for example, bind() or connect()), the application shall ensure that all members of the structure, including any additional non-standard members, if any, are initialized. If the sockaddr_in6 structure has a non-standard member, and that member has a value other than the value that would result from default initialization, the behavior of any function in this standard that reads values from the sockaddr_in6 structure is implementation-defined. All functions in this standard that return data in a sockaddr_in6 structure (for example, getaddrinfo() or accept()) shall initialize the structure in a way that meets the above requirements, and shall ensure that each non-standard member, if any, has a value equal to the value that would result from default initialization. On Page: 309 Line: 10323 change APPLICATION USAGE from: None. to: Although applications are required to initialize all members (including any non-standard ones) of a sockaddr_in6 structure, the same is not required for the sockaddr_in structure, since historically many applications only initialized the standard members. Despite this, applications are encouraged to initialize sockaddr_in structures in a manner similar to the required initialization of sockaddr_in6 structures. On page 388 after line 13000 insert the following new paragraph: The value of AF_UNSPEC shall be 0. On page 406 after line 13620 (sys/un.h APPLICATION USAGE) insert the following new paragraph: Although applications are required to initialize all members (including any non-standard ones) of a sockaddr_in6 structure (see <netinet/in.h>), the same is not required for the sockaddr_un structure, since historically many applications only initialized the standard members. Despite this, applications are encouraged to initialize sockaddr_un structures in a manner similar to the required initialization of sockaddr_in6 structures.

(0002677) ajosey (manager) 2015-05-24 12:59	Interpretation Proposed: 24 May 2015

(0002695) Don Cragun (manager) 2015-06-04 15:48 edited on: 2015-06-04 15:50	Interpretation response ------------------------ The standard clearly states that applications need not initialize non-standard members of the sockaddr_in and sockaddr_un structures, and conforming implementations must conform to this. Rationale: ------------- Historically some applications initialize just the standard members, and some initialize the whole structure (using default initialization or memset()). There may be applications which do the latter for sockaddr_in and then rely on the zero value of the sin_family member being set to AF_UNSPEC. Therefore the standard should require that AF_UNSPEC has the value 0. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- On Page: 306 Line: 10238 Section: <netinet/in.h>, change from: The sockaddr_in6 structure shall be set to zero by an application prior to using it, since implementations are free to have additional, implementation-defined fields in sockaddr_in6. to: Prior to calling a function in this standard which reads values from a sockaddr_in6 structure (for example, bind() or connect()), the application shall ensure that all members of the structure, including any additional non-standard members, if any, are initialized. If the sockaddr_in6 structure has a non-standard member, and that member has a value other than the value that would result from default initialization, the behavior of any function in this standard that reads values from the sockaddr_in6 structure is implementation-defined. All functions in this standard that return data in a sockaddr_in6 structure (for example, getaddrinfo() or accept()) shall initialize the structure in a way that meets the above requirements, and shall ensure that each non-standard member, if any, has a value that produces the same behavior as default initialization would in all functions in this standard which read values from a sockaddr_in6 structure. On Page: 309 Line: 10323 change APPLICATION USAGE from: None. to: Although applications are required to initialize all members (including any non-standard ones) of a sockaddr_in6 structure, the same is not required for the sockaddr_in structure, since historically many applications only initialized the standard members. Despite this, applications are encouraged to initialize sockaddr_in structures in a manner similar to the required initialization of sockaddr_in6 structures. Although it is common practice to initialize a sockaddr_in6 structure using: struct sockaddr_in6 sa; memset(&sa, 0, sizeof sa); this method is not portable according to this standard, because the structure can contain pointer or floating point members that are not required to have an all-bits-zero representation after default initialization. Portable methods make use of default initialization, for example: struct sockaddr_in6 sa = { 0 }; or static struct sockaddr_in6 sa_init; struct sockaddr_in6 sa = sa_init; A future version of this standard may require that a pointer object with an all-bits-zero representation is a null pointer, and that sockaddr_in6 does not have any floating-point members if a floating-point object with an all-bits-zero representation does not have the value 0.0. On page 388 after line 13000 insert the following new paragraph: The value of AF_UNSPEC shall be 0. On page 406 after line 13620 (sys/un.h APPLICATION USAGE) insert the following new paragraph: Although applications are required to initialize all members (including any non-standard ones) of a sockaddr_in6 structure (see <netinet/in.h>), the same is not required for the sockaddr_un structure, since historically many applications only initialized the standard members. Despite this, applications are encouraged to initialize sockaddr_un structures in a manner similar to the required initialization of sockaddr_in6 structures. On page 928 after line 31296 (freeaddrinfo() APPLICATION USAGE) insert the following new paragraph: Although it is common practice to initialize the hints structure using: struct addrinfo hints; memset(&hints, 0, sizeof hints); this method is not portable according to this standard, because the structure can contain pointer or floating point members that are not required to have an all-bits-zero representation after default initialization. Portable methods make use of default initialization, for example: struct addrinfo hints = { 0 }; or static struct addrinfo hints_init; struct addrinfo hints = hints_init; A future version of this standard may require that a pointer object with an all-bits-zero representation is a null pointer, and that addrinfo does not have any floating-point members if a floating-point object with an all-bits-zero representation does not have the value 0.0. On page 107 after line 2884, insert a new 4.x General Concepts section (and renumber subsequent sections as needed): Default Initialization Default initialization causes an object to be initialized according to these rules: if it has pointer type, it is initialized to a null pointer; if it has arithmetic type, it is initialized to (positive or unsigned) zero; if it is an aggregate, every member is initialized (recursively) according to these rules; if it is a union, the first named member is initialized (recursively) according to these rules. For an object of aggregate type with an explicit initializer, the initialization shall occur in initializer list order, each initializer provided for a particular subobject overriding any previously listed initializer for the same subobject; all subobjects that are not initialized explicitly shall be initialized implicitly according to the rules for default initialization. Objects with static storage duration but no explicit initializer shall be initialized implicitly according to the rules for default initialization. An explicit initializer of <tt>{ 0 }</tt> works to perform explicit default initialization for any object of scalar or aggregate type, and for any storage duration. <small>Note: The C standard does not require a compiler to set any field alignment padding bits in a structure or array definition to a particular value. Because of this, a structure initialized using <tt>{ 0 }</tt> might not memcmp() as equal to the same structure initialized using memset() to zero. For consistent results, portable applications comparing structures should test each field individually. Note: If an implementation treats the all-zero bit pattern of a pointer object as a null pointer, and the all-zero bit pattern of a floating point object as equivalent to positive 0, then memset() to zero and calloc() have the same effects as default initialization for all named members of a structure. [MX]Implementations that define __STDC_IEC_559__ guarantee that the all-zero bit pattern of a floating point object represents 0.0.[/MX]</small>

(0002701) ajosey (manager) 2015-06-05 13:45	Interpretation Proposed: 5 June 2015 (restarting clock after revision to interpretation)

(0002821) ajosey (manager) 2015-09-07 11:34	Interpretation approved: 7 Sep 2015

(0002887) geoffclare (manager) 2015-11-06 09:51	As an editorial matter, in order to keep the section numbering of XBD 4.x and XRAT A.4.x in sync, a new XRAT A.4.x Default Initialization section needs to be added. See 0001000.

Issue History
Date Modified	Username	Field	Change
2015-04-05 02:52	dannyniu	New Issue
2015-04-05 02:52	dannyniu	Name	=> DannyNiu/NJF
2015-04-05 02:52	dannyniu	Section	=> netinet/in.h - Internet address family and sys/un.h - definitions for UNIX domain sockets
2015-04-05 02:52	dannyniu	Page Number	=> basedefs/netinet_in.h.html and basedefs/sys_un.h.html
2015-04-05 02:52	dannyniu	Line Number	=> Unsure.
2015-04-05 05:12	Don Cragun	Note Added: 0002610
2015-04-05 05:12	Don Cragun	Note Edited: 0002610
2015-04-05 05:21	Don Cragun	Section	netinet/in.h - Internet address family and sys/un.h - definitions for UNIX domain sockets => <netinet/in.h> - Internet address family and <sys/un.h> - definitions for UNIX domain sockets
2015-04-05 05:21	Don Cragun	Page Number	basedefs/netinet_in.h.html and basedefs/sys_un.h.html => 306,406
2015-04-05 05:21	Don Cragun	Line Number	Unsure. => 10224-10225, 10238-10239, 13611-13612
2015-04-05 05:21	Don Cragun	Interp Status	=> ---
2015-04-05 19:00	philip-guenther	Note Added: 0002611
2015-04-14 13:42	dannyniu	Note Added: 0002622
2015-04-14 14:18	dannyniu	Note Added: 0002623
2015-04-14 14:29	dannyniu	Note Added: 0002624
2015-04-14 14:29	dannyniu	Note Added: 0002625
2015-04-14 14:30	dannyniu	Note Deleted: 0002625
2015-04-14 14:31	dannyniu	Note Edited: 0002624
2015-04-14 14:31	dannyniu	Note Edited: 0002624
2015-04-14 14:47	dannyniu	Note Edited: 0002622
2015-04-14 14:48	dannyniu	Note Edited: 0002622
2015-04-14 14:53	dannyniu	Note Edited: 0002622
2015-04-15 01:10	dannyniu	Issue Monitored: dannyniu
2015-04-18 23:21	philip-guenther	Note Added: 0002630
2015-04-19 05:51	dannyniu	Note Added: 0002631
2015-04-19 05:52	dannyniu	Note Edited: 0002631
2015-04-20 08:12	dannyniu	Note Edited: 0002631
2015-04-20 08:47	geoffclare	Note Added: 0002632
2015-04-20 08:47	geoffclare	Relationship added	related to 0000918
2015-04-21 09:14	dannyniu	Note Added: 0002633
2015-04-21 09:32	geoffclare	Note Added: 0002634
2015-04-22 01:19	dannyniu	Note Edited: 0002633
2015-04-22 03:01	dannyniu	Note Edited: 0002633
2015-04-23 04:13	dannyniu	Note Added: 0002635
2015-04-24 09:45	geoffclare	Note Added: 0002637
2015-04-24 09:46	geoffclare	Relationship added	related to 0000939
2015-04-26 13:57	eblake	Relationship added	related to 0000940
2015-05-21 15:51	rhansen	Note Added: 0002674
2015-05-21 15:56	rhansen	Note Edited: 0002674
2015-05-21 15:58	rhansen	Final Accepted Text	=> see Note: 0002674
2015-05-21 15:58	rhansen	Status	New => Resolved
2015-05-21 15:58	rhansen	Resolution	Open => Accepted As Marked
2015-05-21 15:59	rhansen	Tag Attached: tc2-2008
2015-05-21 16:10	rhansen	Note Edited: 0002674
2015-05-21 16:11	geoffclare	Interp Status	--- => Pending
2015-05-21 16:11	geoffclare	Status	Resolved => Interpretation Required
2015-05-24 12:59	ajosey	Interp Status	Pending => Proposed
2015-05-24 12:59	ajosey	Note Added: 0002677
2015-06-04 15:48	Don Cragun	Note Added: 0002695
2015-06-04 15:50	Don Cragun	Note Edited: 0002695
2015-06-04 15:51	Don Cragun	Final Accepted Text	see Note: 0002674 => See Note: 0002695.
2015-06-04 16:33	geoffclare	Note Edited: 0002674
2015-06-05 13:45	ajosey	Note Added: 0002701
2015-09-07 11:34	ajosey	Interp Status	Proposed => Approved
2015-09-07 11:34	ajosey	Note Added: 0002821
2015-11-06 09:47	geoffclare	Relationship added	related to 0001000
2015-11-06 09:51	geoffclare	Note Added: 0002887
2017-12-23 11:31	dannyniu	Issue End Monitor: dannyniu
2019-06-10 08:54	agadmin	Status	Interpretation Required => Closed

Aardvark Mark IV