Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000281 [Online Pubs] Shell and Utilities Editorial Enhancement Request 2010-07-09 23:29 2011-09-08 15:24
Reporter Roger Marquis View Status public  
Assigned To ajosey
Priority normal Resolution Accepted As Marked  
Status Resolved  
Name Roger Marquis
Organization PCBSD
User Reference
URL  http://pubs.opengroup.org/onlinepubs/009695399/utilities/sort.html [^]
Section sort(1)
Summary 0000281: sort "-M" flag incorrectly omitted.
Description The POSIX sort definition specifies that "The -M option, present in System V, was omitted because of non-portability in international usage.".

This is incorrect for a number of reasons but primarily because a common use of sort among systems administrators is with syslog-generated log files and their associated time stamps. These timestamps are an administrative function (not a user function) and so should not be changed by POSIX i.e., they should not be LOCALE dependent. For this reason "-M" is portable and so should not have been ommitted by POSIX.
Desired Action Delete the sentence noting:
  "The -M option, present in System V, was omitted because of non-portability in international usage."
and re-add the flag's definition in the DESCRIPTION section.

It should be noted that the historical behavior of "sort -M", on both System V and BSD systems, has been to sort on the three letter English month abbreviation, the subsequent one or two digit day of the month, AND the subsequent (C LOCALE) %H:%M:%S timestamp.

Developers should also be encouraged but not required to improve the behavior of "-M" by recognizing other timestamp formats including but not limited to A) historical formats plus a year field, and B) those specified as Interpreted FORMAT sequences in the date(1) man page.
Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0000459)
Don Cragun (manager)
2010-07-10 04:31

 Note that the Purpose of this standard does not include system administration.
(See the 5th bullet point in the Section titled "Purpose" on page vi in the front
matter in the standard) says:
   "No Superuser, No System Administration – There was no intention to specify
    all aspects of an operating system.  System administration facilities and
    functions are excluded from this standard, and functions usable only by the
    superuser have not been included. Still, an implementation of the standard
    interface may also implement features not in POSIX.1-2008. POSIX.1-2008 is
    also not concerned with hardware constraints or system maintenance."


The fact that syslog() is included as part of the XSI option (with no specification
of the date format used in the header of the produced log messages) for use by
application writers, however, makes this request reasonable if the arguments for
use by system administrators are ignored.
(0000482)
msbrown (manager)
2010-07-29 15:44

 Implementations are allowed to provide this function. Some implementations which offer this function internationalize it, while others which offer it do not.

Our consensus is to not require this function as a part of the specification.
(0000518)
Roger Marquis (reporter)
2010-07-30 18:29

 Apologies for not attending the teleconference where this was discussed.
The conference is held at a difficult hour for those of us on the US West
Coast.

The consensus was to not require support for "-M" but there does not seem
to be a rational provided for this decision. I cannot think of another
important function which is nearly universally implemented and, has
been for many years, which has similarly been left unstandardized. "sort
-M" works as expected in FreeBSD, Linux, and Solaris at a minimum. That
alone would cause an average person to expect that it would be
standardized, and to use it as such.

Before (re)closing this ticket would it bee too much to ask for a
rational?

Roger
(0000519)
eblake (manager)
2010-07-30 19:57

 Additional comments made during the teleconference were that:

1. Requiring the functionality now would render existing implementations non-compliant (since there are implementations that lack -M); as such, it would be better as a future enhancement request on the next revision than a defect on the current version.

2. Existing implementations that have -M differ in internationalized support - if we made any requirements on how -M must behave in all locales, then the other half of implementations would be rendered non-compliant. Conversely, requiring -M to exist, but with no specified behavior except in the POSIX locale, is pointless, since all the functionality of the POSIX locale can equally be obtained through other required options.

3. If it were trivial to implement (such as something that could leverage off of strftime for all locales), we wouldn't mind requiring it. However, existing implementations that do support full localization of -M are not that trivial, and we did not feel it appropriate to burden remaining implementations with a difficult implementation.

4. -M was historically associated with system administration, which is outside the scope of the POSIX charter.

A proposal to document more of these points into the Rationale section of sort might be appropriate; but for now, we felt that the existing comments about the existence of -M in many implementations and why it was omitted from the standard in the first place (line 106205), were adequate.
(0000520)
Roger Marquis (reporter)
2010-07-30 20:26

 Was it noted which implementations would be non-compliant? I am not
aware of an implementation that does not support "-M" at this time.

> 2. Existing implementations that have -M differ in internationalized
> support - if we made any requirements on how -M must behave in all locales,
> then the other half of implementations would be rendered non-compliant.

Are any implementations locale-aware? The historical behavior is
specific to the C locale and that should not change because of the its
use in syslog parsing i.e., systems administration.

If there are locale-aware implementations of syslogd these would be
non-compliant, or at least they should be if POSIX is consistent in its
recognition of systems administration interfaces.

> Conversely, requiring -M to exist, but with no specified behavior except in
> the POSIX locale, is pointless, since all the functionality of the POSIX
> locale can equally be obtained through other required options.

All functionality except backwards compatibility. Was any mention made
of backwards compatibility? This seems like an unsupportable assertion
given the extent and consistency of "-M" support.

> 4. -M was historically associated with system administration, which is
> outside the scope of the POSIX charter.

Is that not why it should not have been removed/deprecated from the sort
definition in the first place? It is why the ticket was opened as an
objection.

> A proposal to document more of these points into the Rationale section of
> sort might be appropriate; but for now, we felt that the existing comments
> about the existence of -M in many implementations and why it was omitted
> from the standard in the first place (line 106205), were adequate.

I would rather see POSIX better define the conditions under which systems
administration functions can be deprecated, how such decisions can be
implemented more consistently, and how bad decisions can be corrected.
Those definitions should both recognize the importance of
backwards-compatibility, and its equal importance to systems
administration (vs developers). Those conditions should also note the
importance of getting input from professional systems administrators when
considering changes or corrections to systems administration interfaces
such as syslogd and sort's "-M" flag.

Alternatively, could someone explain how the syslogd locale and sort "-M"
flags are not systems administration interfaces?
(0000530)
Don Cragun (manager)
2010-08-19 16:10

 As noted before, system administration is outside the scope of POSIX.1.
Therefore, sort -M has never been included in POSIX and should not be
added. Some implementations (including GNU Coreutils) that provide
-M do so in a locale dependant manner; others do not. This is another
reason why sort -M should not be specified by the standard.
(0000866)
Roger Marquis (reporter)
2011-06-23 17:16

 >As noted before, system administration is outside the scope of POSIX.1.

Which is why the -M flag should not have been deprecated by POSIX and why it has not been deprecated on Unix/Linux OS, despite the omission in POSIX' documentation.

Problem is, while the Austin Group does have a 'scope' or charter, it has no enforcement mechanism.
(0000953)
Don Cragun (manager)
2011-09-08 08:05

 This bug report was oringally misfiled against project
"1003.1(2008)/Issue 7" and e-mail related to this discussion revealed
that the submitter's problem was not with the standard but with the
on-line version of the 1003.1(2004)/Issue 6 description of the sort
utility now listed in the URL field for this bug report.

I believe that there are really two issues here:

First: Should on-line descriptions of headers, functions, and utilities
include a disclaimer or a link to the standard's front matter?

The submitter seems to have been confused by finding what looked to him
like a manual page for the sort utility while performing a web search.
What actually was found was a description for the sort utility
functionality that is required for conformance to a specific version of
the standard (in this case, the 2004 edition). His search yielded a
non-frame page. The frame version of the page would have included links
to the various volumes in the standard and the preface or volume
introduction sections which clearly spell out the target audience for
the standard. Even the 2008 edition of the sort utility description
might have been less confusing because it has links at the top and
bottom of the page that connect to the previous page, next page, and
home page (detailed Table of Contents) for the associated volume of the
standard. The 2004 edition of the on-line standard doesn't have these
links and does look like a stand-alone manual page to a casual observer.

I believe that it would help if (in addition to the previous page, home,
and next page links) every page in the on-line version of the standard
had:

A. a link to the non-frames version of the corresponding page in the
frames version of the standard AND a link to the frames version of the
corresponding page from non-frames version of the standard, and

B. a link at the top and bottom of every page (next to the current Home
link, possibly titled "Audience") that points to the frames version of
the Preface section in the Frontmatter of the standard.

Second: Should old versions of the on-line version of the contain a
pointer to a newer (or the latest) on-line version of the standard?

I believe that it would help users who have found a reference to a web
page for a version of the standard that is no longer the latest revision
to include a pointer to the home page for on-line version of the next
revision of the standard. (Note that I believe that keeping a chain of
pointers to on-line versions is preferrable to having all old verions
point to the latest revision. This allows users to more easily examine
changes as the standard evolves.) But, just having an indication on
every page of the old versions of the standard that a newer version of
the standard has been adopted would help.

Obviously, none of the suggestions here are a change to the standard;
only to the presentation of the on-line version of the standard. I
would encourage the editors of the on-line versions of the standards to
consider the suggestions above and implement as many of them as possible
given budgetary constraints.
(0000954)
nick (manager)
2011-09-08 15:24
edited on: 2011-09-08 15:24

 During September 8, 2011 conference call, it was agreed that Note: 0000953 was the correct course of action, and should be implemented by the editor as and when time and budget is available.

- Issue History
Date Modified Username Field Change
2010-07-09 23:29 Roger Marquis New Issue
2010-07-09 23:29 Roger Marquis Status New => Under Review
2010-07-09 23:29 Roger Marquis Assigned To => ajosey
2010-07-09 23:29 Roger Marquis Name => Roger Marquis
2010-07-09 23:29 Roger Marquis Organization => PCBSD
2010-07-09 23:29 Roger Marquis Section => sort(1)
2010-07-09 23:29 Roger Marquis Page Number => n/a
2010-07-09 23:29 Roger Marquis Line Number => n/a
2010-07-10 04:31 Don Cragun Page Number n/a => 3187
2010-07-10 04:31 Don Cragun Line Number n/a => 106205-106206
2010-07-10 04:31 Don Cragun Interp Status => ---
2010-07-10 04:31 Don Cragun Note Added: 0000459
2010-07-29 15:44 msbrown Note Added: 0000482
2010-07-29 15:44 msbrown Status Under Review => Closed
2010-07-29 15:44 msbrown Resolution Open => Rejected
2010-07-30 18:29 Roger Marquis Note Added: 0000518
2010-07-30 18:29 Roger Marquis Status Closed => Under Review
2010-07-30 18:29 Roger Marquis Resolution Rejected => Reopened
2010-07-30 19:57 eblake Note Added: 0000519
2010-07-30 20:26 Roger Marquis Note Added: 0000520
2010-08-19 16:10 Don Cragun Note Added: 0000530
2010-08-19 16:10 Don Cragun Resolution Reopened => Rejected
2011-06-23 15:34 msbrown Status Under Review => Closed
2011-06-23 17:16 Roger Marquis Note Added: 0000866
2011-06-23 17:16 Roger Marquis Status Closed => Under Review
2011-06-23 17:16 Roger Marquis Resolution Rejected => Reopened
2011-09-07 13:57 msbrown Project 1003.1(2008)/Issue 7 => Online Pubs
2011-09-07 13:58 msbrown URL => http://pubs.opengroup.org/onlinepubs/009695399/utilities/sort.html [^]
2011-09-07 13:58 msbrown Severity Objection => Editorial
2011-09-07 13:58 msbrown Type Error => Enhancement Request
2011-09-08 08:05 Don Cragun Note Added: 0000953
2011-09-08 15:21 nick Status Under Review => Resolved
2011-09-08 15:21 nick Resolution Reopened => Accepted As Marked
2011-09-08 15:24 nick Note Added: 0000954
2011-09-08 15:24 nick Note Edited: 0000954

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker