Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001199 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Error 2018-08-12 19:23 2020-04-27 11:12
Reporter bhaible View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Applied  
Name Bruno Haible
Organization GNU
User Reference
Section strfmon
Page Number 2039 to 2043
Line Number 65327 to 65492
Interp Status Approved
Final Accepted Text Note: 0004324
Summary 0001199: strfmon, if standards compliant, produces highly misleading results
Description The following program shows strfmon results in the C locale:
=======================================================================
#include <locale.h>
#include <monetary.h>
#include <stdio.h>
int main ()
{
  char buf[80];
  printf ("currency = |%s|\n", localeconv()->currency_symbol);
  printf ("positive sign = |%s|\n", localeconv()->positive_sign);
  printf ("negative sign = |%s|\n", localeconv()->negative_sign);

  strfmon (buf, sizeof (buf), "%^#5.0n", 123.4);
  printf ("strfmon result for positive value = |%s|\n", buf);

  strfmon (buf, sizeof (buf), "%^#5.0n", -123.4);
  printf ("strfmon result for negative value = |%s|\n", buf);

  strfmon (buf, sizeof (buf), "%.2n", 123.5);
  printf ("strfmon result with 2 fractional digits: = |%s|\n", buf);
}
=======================================================================

The result on AIX 7.1 is:
currency = ||
positive sign = ||
negative sign = ||
strfmon result for positive value = | 123|
strfmon result for negative value = | 123|
strfmon result with 2 fractional digits: = |12350|

This is POSIX compliant, as far as I can see. However, without a distinction between positive and negative values, and without a visible decimal separator, this behaviour is useless, or even dangerous (if a programmer was not aware of this behaviour and has not worked around it).

The result on glibc 2.23 is:
currency = ||
positive sign = ||
negative sign = ||
strfmon result for positive value = | 123|
strfmon result for negative value = |- 123|
strfmon result with 2 fractional digits: = |123.50|

This is not POSIX compliant, because in the negative value case, it shows a '-' sign, although localeconv()->negative_sign is the empty string (which is mandated by POSIX http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html#tag_07_03_03_01 [^] ).

The result on Solaris 10 is:
currency = ||
positive sign = ||
negative sign = ||
strfmon result for positive value = | 123|
strfmon result for negative value = |- 123|
strfmon result with 2 fractional digits: = |123.50|

This is not POSIX compliant, 1. because of the '-' sign, 2. because it produces
a different number of characters in the negative and in the positive case, which is against what POSIX says for the '#' flag (lines 65387 to 65389):

  To ensure alignment, any characters appearing before or after the
  number in the formatted output such as currency or sign symbols are
  padded as necessary with <space> characters to make their positive
  and negative formats an equal length.

The result on macOS 10.13 is:
currency = ||
positive sign = ||
negative sign = ||
strfmon result for positive value = | 123 |
strfmon result for negative value = |( 123)|
strfmon result with 2 fractional digits: = |123.50|

This is not POSIX compliant, because the test program does not specify the '+' nor '(' flag, and POSIX says (lines 65363 to 65366):

  If '+' is specified, the locale's equivalent of '+' and '-' are
  used (for example, in many locales, the empty string if positive and '-'
  if negative). If '(' is specified, negative amounts are enclosed within
  parentheses. If neither flag is specified, the '+' style is used.

In summary, out of four implementations, only one is POSIX compliant, and it produces highly misleading / dangerous results.
Desired Action Remove strfmon and strfmon_l from the standard. Also remove <monetary.h> from the standard, since its only purpose is the declare these functions.

Or otherwise, change the standard so that the glibc results become the only compliant results.
Tags issue8
Attached Files

- Relationships
related to 0001241Applied Empty string values for LC_MONETARY, POSIX locale LC_MONETARY values 

-  Notes
(0004086)
geoffclare (manager)
2018-08-21 11:08
edited on: 2019-03-18 12:11

There seems to be a conflict between the strfmon() '+' flag and the values of p_sign_posn and n_sign_posn. If a locale has n_sign_posn = 0 and both positive_sign and negative_sign are empty, then this would mean that the only way to indicate sign is to enclose negative values in parentheses, in which case it should be an error to specify the '+' flag. It also seems odd that if neither '+' nor '(' is specified, the default is '+'; it would make more sense, if locales with both positive_sign and negative_sign empty can exist, for the default to be whatever style is specified by p_sign_posn and n_sign_posn (and if p_sign_posn is -1 and the value is positive, or n_sign_posn is -1 and the value is negative, strfmon() should return an error).
Update (2019-03-15): while working on proposed changes, it seemed to me that rather than have strfmon() be able to fail when neither '+' nor '(' is specified, it should fall back to doing something sensible. This seems to be what current implementations try to do, based on the test program output.

Note that where the bug description says:

> although localeconv()->negative_sign is the empty string (which is mandated by POSIX ...)

this is unclear because POSIX says, at the end of XBD 7.3.3.1 LC_MONETARY Category in the POSIX Locale, "All values are unspecified in the POSIX locale." (Not hugely relevant to this issue, since the test program prints the values of positive_sign and negative_sign, but I thought it worth pointing out.)
Update (2019-03-18): Looking into this further, I believe that "unspecified" is intended to be interpreted as "unavailable", as in the text about localeconv() on line 4846. See bug 0001241.

(0004322)
nick (manager)
2019-03-14 16:35
edited on: 2019-03-14 16:36

Note that page 155 lines 4848-4850 state:

Keywords that are not provided, string values set to the empty string (""), or integer keywords set to −1, are used to indicate that the value is not available in the locale.


So should it be invalid to request a value that is not available?

(0004323)
geoffclare (manager)
2019-03-15 11:53
edited on: 2019-03-15 16:59

Re: Note: 0004322 since it is common to have positive_sign = "" with negative_sign = "-", that would appear to be a problem with what those lines say about empty string values. The empty string for positive_sign in this case is specifying that the sign is an empty string, not saying it's "not available".

This should be addressed in a separate bug report targeted at TC3.

(0004324)
geoffclare (manager)
2019-03-15 16:53
edited on: 2019-03-18 15:15

Interpretation response
------------------------
The standard is unclear on this issue, and no conformance distinction can
be made between alternative implementations based on this. This is being
referred to the sponsor.

Rationale:
-------------
None.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------

On page 2039 line 65362 section strfmon(), change:
+ or (
Specify the style of representing positive and negative currency amounts. Only one of '+' or '(' may be specified. If '+' is specified, the locale's equivalent of '+' and '-' are used (for example, in many locales, the empty string if positive and '-' if negative). If '(' is specified, negative amounts are enclosed within parentheses. If neither flag is specified, the '+' style is used.
to:
+ or (
Specify the style of representing positive and negative currency amounts. Only one of '+' or '(' can be specified.

If '+' is specified, the locale's positive_sign and negative_sign values shall be used (for example, in many locales, the empty string if positive and '-' if negative). However, if both values would be returned by localeconv() as empty strings, strfmon() shall fail. The placement of the signs (if not empty) shall depend on the locale settings:
  • For the 'n' conversion specifier, the placement specified by the locale's p_sign_posn and n_sign_posn values shall be used.
  • For the 'i' conversion specifier, the placement specified by the locale's int_p_sign_posn and int_n_sign_posn values shall be used.
If a sign's placement cannot be determined from these locale values because a value that needs to be used would be returned by localeconv() as 0 or {CHAR_MAX}, the sign shall be placed as if the relevant value was 1.

If '(' is specified, negative amounts shall be enclosed within parentheses and the locale's positive_sign and negative_sign values shall not be used.

If neither flag is specified, the style used shall depend on the locale settings:
  • For the 'n' conversion specifier, the style specified by the locale's p_sign_posn and n_sign_posn values shall be used.
  • For the 'i' conversion specifier, the style specified by the locale's int_p_sign_posn and int_n_sign_posn values shall be used.
If the style cannot be determined from these locale values because a value that needs to be used would be returned by localeconv() as {CHAR_MAX}, the style used shall be that specified for the '+' flag; if this would cause strfmon() to fail because the locale's positive_sign and negative_sign values would both be returned by localeconv() as empty strings, strfmon() shall behave as if the negative_sign value was the string "-".

On page 2041 line 65415 section strfmon(), change:
If the total number of resulting bytes ...
to:
If all conversions are successful and the total number of resulting bytes ...

On page 2041 line 65421 section strfmon(), add:
[EINVAL]
The '+' flag was included in a conversion specification and the locale's positive_sign and negative_sign values would both be returned by localeconv() as empty strings.

On page 2042 line 65463 section strfmon(), change APPLICATION USAGE from:
None.
to:
The '+' flag should be used with care, because if the locale's positive_sign and negative_sign values are both empty strings, there is no way to distinguish negative from positive values with signs and therefore strfmon() fails. If the application has a preference for signs but parentheses are acceptable, it should try strfmon() with the '+' flag first, and if it fails with [EINVAL] then repeat the call without the '+' flag.

On page 2042 line 65465 section strfmon(), change RATIONALE from:
None.
to:
The [EINVAL] error condition applies only when the '+' flag is used because this flag indicates that the application requires the use of signs, and if there are no signs in the locale data then this requirement cannot be satisfied. When neither '+' nor '(' is used, the application is requesting whatever formatting is appropriate for the locale, and so strfmon() has a fallback of using a '-' sign for negative values in cases where the locale data does not indicate parentheses should be used and has no signs.

Cross-volume change to XBD ...

On page 157 line 4931 section 7.3.3 LC_MONETARY, add:
Certain combinations of the *_sign_posn, positive_sign and negative_sign values are invalid and shall not be accepted by localedef:
  • If p_sign_posn and n_sign_posn are both greater than 0, and positive_sign and negative_sign have the same value or are both either empty strings or omitted, this combination is invalid because it requires signs to be used but does not provide the means to distinguish negative from positive values using signs.
  • Likewise, if int_p_sign_posn and int_n_sign_posn are both greater than 0, and positive_sign and negative_sign have the same value or are both either empty strings or omitted, this combination is invalid.
  • If p_sign_posn and n_sign_posn are both 0, this combination is invalid because it requires parentheses to be used but does not provide the means to distinguish negative from positive values using parentheses.
  • Likewise, if int_p_sign_posn and int_n_sign_posn are both 0, this combination is invalid.


(0004331)
shware_systems (reporter)
2019-03-18 15:19

The way the LC_MONETARY category is set up for the POSIX locale reflects that the POSIX locale has neither a language or country designation that would otherwise control what constitutes a country's specific requirements of common cultural practice. At most the POSIX locale can be considered to be a subset of script category "Latn". As such, I don't see strfmon() should be considering both positive_sign and negative_sign as empty strings as an error case; the no country case is one where this is entirely legitimate, even if a language code is provided. For example, a generic "en" locale has no means of determining whether <USD>, <GBP> or <AUD> (among others) should be the int_currency designation. Now if ISO 4217 is updated to require particular values for these fields where no country information is available, then POSIX can defer to that. Otherwise I see the standard as being correct as is; any use as arbitrary defaults of a particular character name for signs or currency is parochial in nature, not international.

Where EINVAL might be legitimate is if the charset used by a locale does not have either a left or right parentheses character name in its repertoire. While this is not an issue for encodings that are supersets of the portable character set, it does apply for various symbol sets and the PFCS, if this is supplied as a charset.
(0004333)
geoffclare (manager)
2019-03-18 16:20

Re: Note: 0004331 what you say is a valid argument for conversions that have neither a '+' flag or a '(' flag. However, when an application uses the '+' flag it is specifically requesting the use of signs, and therefore if there are no signs this request cannot be satisfied and strfmon() should fail. I think the advice in the APPLICATION USAGE addition in Note: 0004324 is a reasonable way to handle that situation.
(0004335)
bhaible (reporter)
2019-03-18 20:49

Note: 0004324 is a good and clear fix of the issue. Thank you for working on that!

Of course, libc implementations will have to change a couple of lines of code in order to comply; but that will be for the better (that's the purpose of a standard).
(0004342)
agadmin (administrator)
2019-03-21 15:45

Interpretation Proposed: 21 March 2019
(0004390)
agadmin (administrator)
2019-05-09 09:27

Interpretation approved: 9 May 2019

- Issue History
Date Modified Username Field Change
2018-08-12 19:23 bhaible New Issue
2018-08-12 19:23 bhaible Name => Bruno Haible
2018-08-12 19:23 bhaible Organization => GNU
2018-08-12 19:23 bhaible Section => strfmon
2018-08-12 19:23 bhaible Page Number => 2039 to 2043
2018-08-12 19:23 bhaible Line Number => 65327 to 65492
2018-08-12 19:24 bhaible Issue Monitored: bhaible
2018-08-21 11:08 geoffclare Note Added: 0004086
2019-03-14 16:35 nick Note Added: 0004322
2019-03-14 16:36 nick Note Edited: 0004322
2019-03-15 11:47 geoffclare Note Edited: 0004086
2019-03-15 11:53 geoffclare Note Added: 0004323
2019-03-15 16:53 geoffclare Note Added: 0004324
2019-03-15 16:59 geoffclare Note Edited: 0004323
2019-03-15 17:11 geoffclare Note Edited: 0004324
2019-03-18 12:02 geoffclare Relationship added related to 0001241
2019-03-18 12:11 geoffclare Note Edited: 0004086
2019-03-18 15:15 geoffclare Note Edited: 0004324
2019-03-18 15:16 geoffclare Interp Status => Pending
2019-03-18 15:16 geoffclare Final Accepted Text => Note: 0004324
2019-03-18 15:16 geoffclare Status New => Interpretation Required
2019-03-18 15:16 geoffclare Resolution Open => Accepted As Marked
2019-03-18 15:16 geoffclare Tag Attached: issue8
2019-03-18 15:19 shware_systems Note Added: 0004331
2019-03-18 16:20 geoffclare Note Added: 0004333
2019-03-18 20:49 bhaible Note Added: 0004335
2019-03-21 15:45 agadmin Interp Status Pending => Proposed
2019-03-21 15:45 agadmin Note Added: 0004342
2019-05-09 09:27 agadmin Interp Status Proposed => Approved
2019-05-09 09:27 agadmin Note Added: 0004390
2020-04-27 11:12 geoffclare Status Interpretation Required => Applied


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker