Austin Group Defect Tracker

Aardvark Mark III


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001264 [1003.1(2016)/Issue7+TC2] System Interfaces Objection Clarification Requested 2019-06-27 19:01 2019-07-29 15:42
Reporter shware_systems View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Interpretation Required  
Name Mark Ziegast
Organization SHware Systems Dev.
User Reference
Section newlocale(), others
Page Number 1392
Line Number 46280-2
Interp Status Pending
Final Accepted Text Note: 0004500
Summary 0001264: "default locale" inadequately specified in newlocale()
Description currently, the text reads:
    "If a completely new locale object is created, the data for all sections not requested by category_mask shall be taken from the default locale."

However, while applications are required to start using the POSIX locale as "default locale", per setlocale(), threads upon pthread_create() inherit LC_GLOBAL_LOCALE, per the return value of uselocale(), as their "default locale". As setlocale() may have been called to modify the global locale before pthread_create(), and therefore modifies LC_GLOBAL_LOCALE indirectly, this makes it indeterminate which "default locale" newlocale() should use; the POSIX locale, or LC_GLOBAL_LOCALE.

Complicating this further, on systems supporting the [LD] or [XSI] option groups, the implementation-defined native locale for the system may have default values for additional LC_ categories (as supported by its implementation of the localedef utility) that will not be reflected in a POSIX locale definition that only includes the categories required by XBD7. So, to ensure all new locale_t values include these additional categories' implementation-specific defaults, it may implement newlocale() to use the native locale as "default locale", whatever the current LC_GLOBAL_LOCALE value is, or what extensions may be in that platform's definition for the POSIX locale.

Therefore, it appears "default locale" has 3 definitions, and this makes the expectations for newlocale() unclear. Of the 3 only the native locale, by implication, can be required to have values for all categories defined in <locale.h> by the implementation.
Desired Action At P1392, L46280, Change:
"If a completely new locale object is created, the data for all sections not requested by category_mask shall be taken from the default locale."

to:
"If a completely new locale object is created, the data for all sections not requested by category_mask shall be taken from the global locale, as referenced by LC_GLOBAL_LOCALE, if a thread has previously called uselocale() or the native locale otherwise."

At P1902, after L61500, Insert:
[LD] This locale shall have appropriate default values assigned for all LC_* values defined by <locale.h> that are recognized by the localedef utility. Upon application startup the locale_t object referenced by LC_GLOBAL_LOCALE shall use these values for any implementation-defined categories and those from the POSIX locale definition for the categories defined in XBD 7.

------------
Other changes , in uselocale() and XBD, may be desirable.
Tags tc3-2008
Attached Files

- Relationships

-  Notes
(0004456)
Don Cragun (manager)
2019-06-28 03:36

Do current implementations of setlocale() and newlocale() behave as specified in the Desired Action?

What makes you think that an implementation defined locale is in any way required to provide definitions for all locale categories that are extensions to the POSIX required locale categories? What in the current standard requires this to happen?
(0004473)
shware_systems (reporter)
2019-07-04 17:09

The requirement stems from the wording at XBD 7.2, L3990:
All implementations shall define a locale as the default locale, to be invoked when no environment variables are set, or set to the empty string.
This default locale can be the POSIX locale or any other implementation-defined locale.

From this and XBD8, if LANG, LC_ALL, and an LC_* value for a new category (with LC_* and LC_*_MASK values defined in <locale.h>) are all unset or null, this locale is supposed to provide the values to be used. As a blanket statement about categories it does not exclude those new categories. As a side effect, which probably should be mentioned there for clarity, if localedef or any implementation-provided locale does support additional categories then the default locale can NOT be the POSIX locale.

This still applies basically to setlocale, as newlocale has it's thread local sensitivity requirements related to LC_GLOBAL_LOCALE as an additional possible override of the environment, which I try to take into account in the desired action. No, I haven't checked whether any implementations do it precisely this way, so it may be invention, but it's a starting point for discussion at least. K. Simonsen would be more conversant with existing practice, as chair of the ISO-TR 30112 effort, due to the work that's been done on that.
(0004474)
Don Cragun (manager)
2019-07-04 19:38
edited on: 2019-07-04 19:39

If I correctly understand Note: 0004473, you're saying that you are proposing significant changes to the POSIX internalization tools and you have no idea how any existing systems behave in this area and you have not asked Keld whether or not he thinks this is a good idea.

My understanding is that that Mantis is supposed to be used to propose changes to be made to the standard; not as a place to start a discussion about changes to existing requirements that may break every existing implementation currently meeting the standard's requirements. Isn't that what the malling list is for?

(0004475)
geoffclare (manager)
2019-07-05 08:39

Re: Note: 0004473 since that quote from XBD 7.2 is talking about the default that you get if you request that the locale is set from environment variables and the relevant environment variables are unset or empty, it only applies when you pass an empty string as the second argument to setlocale() or newlocale().

It's unfortunate that XBD 7.2 uses the phrase "default locale" for two different purposes (the other being on line 3986), but it is clear from the context what is meant in each case.

The use of "default locale" on the newlocale() page is clearly not intended to refer to the one related to environment variables, since it applies when the second argument is not "" (as well as when it is).

The only two things that could possibly be intended here are for "default locale" to mean the POSIX locale or the global locale. We need to find out which of those two is the one that implementations use.
(0004476)
geoffclare (manager)
2019-07-05 14:45

As per my email to the mailing list, glibc and Solaris both take this "default locale" to be the POSIX locale. So unless someone can identify a system that uses the global locale, I believe the only change needed here is...

On page 1392 line 46282 section newlocale(), change:
default locale
to:
POSIX locale
(0004477)
shware_systems (reporter)
2019-07-05 17:53

Re:4474, 4476
Apparently you don't understand. Both c99 and c17 have the allowance new LC_ constants may defined in <locale.h>, as POSIX extends this base set with LC_MESSAGES already. It is not a new requirement therefore, that implementations may have to support more than what the C standard requires, and as such no changes should be significant, however current implementations do it.

I am aware how current implementations try to avoid addressing it, by limiting their implementations of localedef to just the categories POSIX requires, so they can argue they are meeting the minimal requirements, but this is not in a forwards compatible fashion, or provide additional categories in an unsupported manner.

This is indicated also by Geoff's testing. glibc is apparently non-conforming in that it extends the C locale with additional categories from TR 14652, I expect under the assumption the alignment effort with 9945 would occur, but which disagrees with the current requirement of L3983:
"The behavior of standard utilities and functions in the POSIX locale shall be as if the locale was defined via the localedef utility with input data from the
POSIX locale tables in Section 7.3."

This does not have the allowance ", and any (or all) implementation-defined additional categories." which would include categories defined by other standards. For backwards compatibility, I don't see this clause should be added. These are only permitted for locale's other than the C locale, without that clause, the way I currently read things, and I feel it should stay this way. As such, glibc does not have a valid POSIX locale, so its use as the native or default locale by newlocale() is in the realm of undefined behavior, imo, and therefore not a valid reason to change the wording to that of 4476.

Side note: because it hard codes the C locale as this response, this blocks any platform-provided native locale from being accessed by applications using glibc on platforms other than LINUX, which qualifies as a bug in any ports to those platforms. Even for LINUX it can be seen as non-conforming to the C standard, which can be read as requiring the native locale to be a separate definition from the C locale, but which may happen to include copies of categories from the C locale.

Last point, Keld has already proposed inclusion of all or parts of TR 30112 (or TR 14652 as its final form) into the standard, I believe as part of that failed alignment effort, so I don't think I need to ask him again if he thinks it's a good idea. I don't support alignment either, due to the TR 14652 aspects considered controversial or that aren't backwards compatible enough, but the less controversial aspects I see as doable for Issue 8.
(0004500)
geoffclare (manager)
2019-07-29 15:41

Interpretation response
------------------------
The standard is unclear on this issue, and no conformance distinction can
be made between alternative implementations based on this. This is being
referred to the sponsor.

Rationale:
-------------
There are several different uses of "default" and it is not clear which one should apply here.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
On page 1392 line 46282 section newlocale(), change:
default locale
to:
POSIX locale

- Issue History
Date Modified Username Field Change
2019-06-27 19:01 shware_systems New Issue
2019-06-27 19:01 shware_systems Name => Mark Ziegast
2019-06-27 19:01 shware_systems Organization => SHware Systems Dev.
2019-06-27 19:01 shware_systems Section => newlocale(), others
2019-06-27 19:01 shware_systems Page Number => 1392
2019-06-27 19:01 shware_systems Line Number => 46280-2
2019-06-28 03:36 Don Cragun Note Added: 0004456
2019-07-04 17:09 shware_systems Note Added: 0004473
2019-07-04 19:38 Don Cragun Note Added: 0004474
2019-07-04 19:39 Don Cragun Note Edited: 0004474
2019-07-05 08:39 geoffclare Note Added: 0004475
2019-07-05 14:45 geoffclare Note Added: 0004476
2019-07-05 17:53 shware_systems Note Added: 0004477
2019-07-29 15:41 geoffclare Note Added: 0004500
2019-07-29 15:42 geoffclare Interp Status => Pending
2019-07-29 15:42 geoffclare Final Accepted Text => Note: 0004500
2019-07-29 15:42 geoffclare Status New => Interpretation Required
2019-07-29 15:42 geoffclare Resolution Open => Accepted As Marked
2019-07-29 15:42 geoffclare Tag Attached: tc3-2008


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker