Viewing Issue Advanced Details
1538 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2021-12-05 06:48 2021-12-05 06:48
andras_farkas
 
normal  
New  
Open  
   
Andras Farkas
what
---
what -s is poorly described, uses the word "quit"
On:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/what.html [^]
The -s option for what is described as so:
Quit after finding the first occurrence of the pattern in each file.
I find the usage of the word 'quit' here unfortunate, as it can be read as exiting or terminating.
Both "in each file" and the behavior of what on various BSDs leads me to believe "quit" isn't the best way to describe the behavior of -s as "what -s foo bar" on the BSDs doesn't quit after finding a pattern in foo. (it also checks bar after) If I understand right, this may be a correct behavior according to the standard.

In the man pages on NetBSD and OpenBSD, they describe -s as follows:
If the -s option is specified, only the first occurrence of an identification string in each file is printed.
On FreeBSD, the following phrasing is used:
Stop searching each file after the first match.
I also checked the phrasing in Solaris 10 and AIX 7.2 but don't have access to test actual behavior. (So I don't know if on SysV, "what -s foo bar" doesn't check bar if a pattern is found in foo)

I don't know if this is an Issue 7 or Issue 8 sort of thing to fix, so I have this marked as Issue 7.
There may be a better wording than this, but this is what I recommend.

Change:
Quit after finding the first occurrence of the pattern in each file.
to:
For each file, print only the first occurrence of the pattern.

I wonder if there's a way to make this more clear. -s doesn't make "pathname:" not get printed, so "print only" can still be misinterpreted. I'd like feedback and suggestions on this.

On that topic, the normative STDOUT section doesn't specify how printing works when multiple patterns are found, while the informative EXAMPLES section does. (Alternatively, this could be read as the STDOUT and EXAMPLES section defining different outputs)
There are no notes attached to this issue.




Viewing Issue Advanced Details
1537 [Online Pubs] Main Index Editorial Error 2021-11-21 18:33 2021-11-21 18:33
andras_farkas
 
normal  
New  
Open  
   
Andras Farkas
https://pubs.opengroup.org/onlinepubs/9699919799/download/index.html [^]
Download
Downloads no longer work: receiving tiny files.
Currently, on
https://pubs.opengroup.org/onlinepubs/9699919799/download/index.html [^]
susv4-2018.tar, susv4-2018.tar.bz2, and susv4-2018.tgz all are less than a kilobyte. The .zip is unaffected.
Get the proper bundles back in the correct place.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1536 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-11-20 17:07 2021-11-20 17:07
kre
 
normal  
New  
Open  
   
Robert Elz
XCU 2;7;5, XCU 2.7.6
2363
75362-75377
---
Unimplemented requirements in fd duplication
Note: the text in question appears unchanged in Issue 8 Draft 2.1
but obviously the page & line numbers differ.

In XCU 2.7.5 (the <& redirect operator) it is stated:

    if the digits in word do not represent a file descriptor already open
    for input, a redirection error shall result;

Similarly in XCU 2.7.6 (the >& redirect operator):

    if the digits in word do not represent a file descriptor already open
    for output, a redirection error shall result;

I am unable to find a shell which implements that redirection error, where
the fd given by word is open, but not for input/output as specified.
(as the requirement is "error shall result" that means there are no
conforming shells, or none I have available to test ... I find it hard to
believe that ksh88 is any different).

Try:
    sh $ exec 4>/tmp/foo; exec 6<&4
    sh $ exec 4</tmp/foo; exec 6>&4

in your favourite shell and see how many redirection errors you experience.
(Errors on the redirect of fd 4 do not count for this purpose, it is easy
to set up a scenario where that redirect fails -- just make that one work).

This is hardly surprising, as all shells simply use either dup2() or
fcntl(F_DUPFD) to perform these operations, and as long as the source
fd is open, and there are sufficient available fd's for the new one
to be opened (and its fd number is within range) those operations succeed.
They don't care in the slightest whether the source fd is open for reading,
writing, both, or neither.

Requiring that an error be generated when the source fd is not open for
the direction of I/O implied by the operator in use, would require the
shell to first determine how the source fd was opened (fcntl(F_GETFL)
and verify it - which no-one does - but we would also need to invent a
<>& operator (which no-one has done yet) to make sure to be able to
correctly duplicate a file descriptor open for both read and write.

It would be tempting to simply delete the words "for input" and "for output"
and leave it like that, but that might lead to a suboptimal result, where
shells are (effectively) forbidden from generating errors from the example
code shown above, and while no-one currently does, making it effectively
impossible to ever do, which it would be if applications were told that it
makes no difference which fd duplication redirect operator they use, which
that change would effectively do, is probably not what we want.

Instead, I'd make it unspecified what happens if the input duplication
operator is applied to a fd not already open for input, or if the output
duplication operator is applied to a fd not already open for output.
That leaves current shells compliant, puts the onus on applications to
do the right thing (which is their only choice right now -- it doesn't
matter which duplication operator is applied, what matters is how the
resulting fd is used - it can only be used for operations that were
permitted to the source fd ("word"). It leaves it open for a shell to
actually verify correct usage, but such a shell would probably need to
invent <>& (in that, or some other, syntax) for fd's open read+write.
In line 75394 delete the words

for input


and append after the words (before the terminating period)

see Section 2.8.1


a new clause:

; if the file descriptor denoted by word represents an open
file descriptor that is not open for input, the behaviour is
unspecified


In line 75404 delete the words

for output


and append after the words (before the terminating period)

see Section 2.8.1


a new clause:

; if the file descriptor denoted by word represents an open
file descriptor that is not open for output, the behaviour is
unspecified


Somewhere in XRAT (probably) add a new section, or paragraph

<blockqouote>Redirection file descriptor duplication operators.

The file descriptor duplication redirection operators, <& and >&,
make a copy of one file desciptor as another.
The new file descriptor will traditionally have the same access permissions
as the source (old) file descriptor.
However this is not guaranteed by this standard.
Applications must ensure that they use the appropriate redirection
operator for the fle descriptor being duplicated, and only use the
new, duplicate, file descriptor for operations that would be permitted
to the original file descriptor.
Any other usage results in unspecified behaviour.

Or some words with that general intent.

Note that this solution does not require the <>& operator be
invented, a file descriptor open for input and output can be used by
either of the redirect/duplicate operators, and can then be used for
both input and output.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1535 [Issue 8 drafts] Shell and Utilities Objection Error 2021-11-18 16:13 2021-11-19 09:45
kre
 
normal  
New  
Open  
   
Robert Elz
2.9.1.1
2332
75291-75302
Poor description of declaration (all really) utility argument processing
XCU 2.9.1.1 (in Issue 8 Draft 2.1) says:

   When a given simple command is required to be executed [...]
   the following expansions, assignments, and redirections shall all
   be performed from the beginning of the command text to the end:

All that's relevant about that quote is that it makes the ordering
a requirement.

   1. [not relevant here]
   2. The words that are not variable assignments or redirections shall be
       expanded.

That's simple enough. The first thing we do (after moving redirects
and var-assigns out of the way - that's step 1 - is to expand the remaining
words.

      If any fields remain following their expansion, the first field shall
      be considered the command name and remaining fields are the arguments
      for the command.

This is where we get the command name, and it comes "following their [the
words] expansion" (assuming there are remaining fields, which for this issue
we shall do).

     If the command name is recognized as a declaration utility,

At this point we look and see if we have a declaration utility, and if
we do...

     then any remaining words that would be recognized as a variable
     assignment in isolation shall be expanded as a variable assignment

then we must look at the remaining (remember already expanded) words to
see if any look like a variable assignment, and if we find any, expand
them *again* (as variable assignments - the following (parenthesised)
text goes into what that means, but that's not relevant here.

     while words that would not be a variable assignment
     in isolation shall be subject to regular expansion.

and the other words also get expanded again, the regular way.

     For all other command names,

that is, commands that are not declaration utilities

     all subsequent words shall be subject to regular expansion

all the remaining words just get regular expansion (this is followed by
an explanation of what "regular expansion" means - not relevant here -
except to note in passing that this is the second occurrence of "regular
expansion", the first one didn't get explained - that's probably backwards.

That is, the words are expanded, the command name (first non-empty remaining
field) gets examined, and depending upon what we see, we expand all the
other args again in one way or another.

That's not how it is supposed to happen - not in anyone's world view.

Further, with this (precisely specified order of processing) this final
paragraph of 2.9.1.1...

    When determining whether a command name is a declaration utility, an
    implementation may use only lexical analysis.

It isn't really clear what that means, but lexical analysis is occurring
around the same time as alias expansion, so how the two interact would
probably need to be made clear, but this probably doesn't matter anyway
(or not with the current wording) as:

    It is unspecified whether assignment context will be used if the
    command name would only become recognized as a declaration utility
    after word expansions.

No it isn't, it is very precisely specified, as above - the word expansions
happen first, and then the result is examined to determine whether the command
name is a declaration utility or not. Nothing even slightly unspecified
about that, and adding contradictory text here can only confuse things.
Rewrite this nonsense into something that makes sense. Nothing is ever
expanded twice (other than via eval or similar) in the shell, once is all
anyone ever gets.

While doing that, make it unspecified whether any particular commands are
declaration utilities - I can understand why people want them, and know
that some shells implement this, but it is all new in the standard, and
other shells do not (some *never* will). Further, the concept is absolutely
not required for anything, instead of

        export PATH=~/bin:...

where the '~' would not be expanded by a regular expansion, one can
always just write

        PATH=~/bin:...
        export PATH

which works, with no changes to the standard, in every shell. It even
worked back when assignments in the export (etc) commands didn't work at all.
This is the way that applications should be advised to code, the safe way
that works.

Then shells that want to extend the syntax of these commands should be allowed
to do so, and applications should be warned that if this kind of modified
expansion is not desired, they should make sure to always quote the arguments
to these commands (which will prevent any of this magic expansions, as a
quoted "PATH=..." does not have the correct form for a variable assignment,
the name= part must be unquoted.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1533 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Editorial Enhancement Request 2021-11-08 22:04 2021-11-22 15:36
steffen
 
normal  
New  
Open  
   
steffen
time.h
425
14451
---
struct tm: add tm_gmtoff (and tm_zone) field(s)
Hello.

Regarding the MUA i maintain i was pinged by a user who needs to
use the timezone Europe/Dublin. He wrote

      In 2018, the tzdata maintainers (IANA) corrected a historical mistake
      with the Europe/Dublin timezone. The mistake was rooted in a
      misunderstanding of whether IST meant "Irish Summer Time" or "Irish
      Standard Time".

      The problem was discussed at great length
      (http://mm.icann.org/pipermail/tz/2018-January/thread.html) [^] and it was
      concluded that IST really meant Irish *Standard* Time (in constrast
      with, say, British *Summer* Time), and that this standard time is
      defined as UTC+0100.
[.]
      Once the question was settled, the only possible solution for keeping
      the Irish local time in sync with the rest of the world (for example,
      Belfast & London) was for IANA to _reverse_ the functioning of the DST
      flag for Ireland. The result is that in the current IANA timezone
      database (2021e), Europe/Dublin has DST applied in *winter*, with an
      adjustment of -1h (that is, negative).
[.]
      It turns out that the introduction of a negative DST adjustment caused
      all sorts of bugs back in 2018; in the source distribution of IANA's
      tzdata, one can spot this inside ./europe:

        # In January 2018 we discovered that the negative SAVE values in the
        # Eire rules cause problems with tests for ICU [...] and with tests
        # for OpenJDK [...]
        # To work around this problem, the build procedure can translate the
        # following data into two forms, one with negative SAVE values and the
        # other form with a traditional approximation for Irish timestamps
        # after 1971-10-31 02:00 UTC; although this approximation has tm_isdst
        # flags that are reversed, its UTC offsets are correct and this often
        # suffices. This source file currently uses only nonnegative SAVE
        # values, but this is intended to change and downstream code should
        # not rely on it.

      So, a temporary hack was put in place in order to allow distro
      maintainers to retain the old broken convention of IST and support
      buggy software, but it is clear that the current (and technically, and
      politically, correct) implementation of a negative DST adjustment for
      Ireland is there to stay.
      As a matter of fact, the distro maintainer can choose to compile
      tzdata to keep buggy software happy ("make DATAFORM=rearguard"),
      which replicates the behaviour of tzdata prior to 2018. Many distros
      seem to be doing that for one reason or another, while some have passed
      the upstream change down to their users (probably, without knowing).

Anyhow, all the simple minded software, including the MUA
i maintain, used to do something like

   if((t2 = mktime(gmtime(&t))) == (time_t)-1){
      t = 0;
      goto jredo;
   }
   tzdiff = t - t2;
   if((tmp = localtime(&t)) == NULL){
      t = 0;
      goto jredo;
   }

   tzdiff_hour = (int)(tzdiff / 60);
   tzdiff_min = tzdiff_hour % 60;
   tzdiff_hour /= 60;
   if (tmp->tm_isdst > 0)
      tzdiff_hour++;

Note the .tm_isdst plus positive summer time adjustment.
This was overly primitive, and i recognize that POSIX supports the
%z (and %Z) formats for strftime(3), and in general code as below
is used by projects, so doing it right is very expensive but
doable with POSIX as of today.

However, all BSDs and Linux with either of GNU and musl C library
support the .tm_gmtoff (and .tm_zone) members of "struct tm", in
general all users of the public domain (and standardized) IANA TZ
project can bake it in upon their own desire.. With .tm_gmtoff
being available, code gets as simple as
  s64
  time_tzdiff(s64 secsepoch, struct tm const *utcp_or_nil,
        struct tm const *localp_or_nil){
     struct tm tmbuf[2], *tmx;
     time_t t;
     s64 rv;
     UNUSED(utcp_or_nil);

     rv = 0;

     if(localp_or_nil == NIL){
        t = S(time_t,secsepoch);
        while((tmx = localtime(&t)) == NIL){
           if(t == 0)
              goto jleave;
           t = 0;
        }
        tmbuf[0] = *tmx;
        localp_or_nil = &tmbuf[0];
     }

  #ifdef HAVE_TM_GMTOFF
     rv = localp_or_nil->tm_gmtoff;

  #else
     if(utcp_or_nil == NIL){
        t = S(time_t,secsepoch);
        while((tmx = gmtime(&t)) == NIL){
           if(t == 0)
              goto jleave;
           t = 0;
        }
        tmbuf[1] = *tmx;
        utcp_or_nil = &tmbuf[1];
     }

     rv = ((((localp_or_nil->tm_hour - utcp_or_nil->tm_hour) * 60) +
           (localp_or_nil->tm_min - utcp_or_nil->tm_min)) * 60) +
           (localp_or_nil->tm_sec - utcp_or_nil->tm_sec);

     if((t = (localp_or_nil->tm_yday - utcp_or_nil->tm_yday)) != 0){
        s64 const ds = 24 * 60 * 60;

        rv += (t == 1) ? ds : -S(s64,ds);
     }
  #endif

  jleave:
     return rv;
  }
Insert at the given location

<code>
 int tm_gmtoff Seconds east of UTC
 const char *tm_zone Timezone abbreviation
</code>

Maybe cross-reference with strftime(3) and note relationship to the %z and %Z formats.
Notes
(0005529)
steffen   
2021-11-21 22:28   
P.S.: even though "Object or binary code portability" is outside the "1.1 Scope" of POSIX i want to add, just in case that the strftime(3) formats are yet part of the standard whereas tm_gmtoff is not even though it exists for long is because of binary compatibility nonetheless, that with dynamic library symbol versioning binary compatibility is overall not an issue, or, simpler via compile-time preprocessor switches, or even specific #pragma compiler directives (Solaris seems to use "#pragma redefine_extname") at least newly compiled code does not have issues.
As you all know, of course.
(0005530)
geoffclare   
2021-11-22 09:47   
Re Note: 0005529 The compatibility issue is not simply one of handling the increased structure size; there is also a behavioural difference.

On an implementation that lacks tm_gmtoff and tm_zone, if you change TZ after obtaining a struct tm from localtime(), the strftime() %z and %Z conversions use the values for the new TZ. Adding tm_gmtoff and tm_zone would change the behaviour such that %z and %Z use the values for the old TZ. This could break existing applications when they are recompiled and consequently start using the new ABI.
(0005531)
geoffclare   
2021-11-22 10:12   
I started wondering about where tm_zone points (since there is no function to free it), and came across this on the MacOS localtime(3) man page (under BUGS):
The tm_zone field of a returned tm structure points to a static array of characters, which will also be overwritten by any subsequent calls (as well as by subsequent calls to tzset(3) and tzsetwall(3)).
So perhaps what I wrote in my previous note about %Z using the old TZ is not true (at least for some implementations that support tm_zone).

If we want to add tm_gmtoff and tm_zone, the standard will need to go into this kind of detail.
(0005532)
geoffclare   
2021-11-22 10:50   
The desired action has tm_gmtoff as an int, but in glibc and MacOS it is a long. If there are implementations where it is an int, then to add it to the standard we would need to introduce a new type such as gmtoff_t.

MacOS also doesn't have the const on tm_zone, but I doubt if adding it would cause any problems.
(0005533)
steffen   
2021-11-22 15:26   
Re Note: 0005530 and Note: 0005531
I see. Yes, strftime(3) notes on page 2047 regarding %Z on TZ change!
Standard is hard .. i had a single file DB with a TOC of names at the end pointing to data at the front, if a non-builtin one was needed then all was there to stay after being loaded. I see.
musl does it like

<code>
const char *__tm_to_tzname(const struct tm *tm)
{
        const void *p = tm->__tm_zone;
        LOCK(lock);
        do_tzset();
        if (p != __utc && p != __tzname[0] && p != __tzname[1] &&
            (!zi || (uintptr_t)p-(uintptr_t)abbrevs >= abbrevs_end - abbrevs))
                p = "";
        UNLOCK(lock);
        return p;
}
</code>

therefore even using .tm_zone as a verification that %Z applies to the correct struct tm, which is an assertion beyond the usual "behaviour is undefined".

Re 0005532:
Oh! Indeed they use long not int, sorry! No, i do not know a system where this is int, actually -- sorry for this! I am afraid the standard cannot correct this; i personally never worked on a system where int was 16-bit, where that decision would make some sense. gmtoff_t does not seem to be needed:

<code>
#?0|kent:free-src.git$ git grep tm_gmtoff origin/main:include
origin/main:include:time.h: long tm_gmtoff; /* offset from UTC in seconds */
#?0|kent:free-src.git$ cd ../open-src.git/
#?0|kent:open-src.git$ git grep tm_gmtoff origin/master:include
origin/master:include:time.h: long tm_gmtoff; /* offset from UTC in seconds */
#?0|kent:open-src.git$ cd ../net-src.git/
#?0|kent:net-src.git$ git grep tm_gmtoff origin/trunk:include
origin/trunk:include:time.h: long tm_gmtoff; /* offset from UTC in seconds */
#?0|kent:net-src.git$ cd ../dfly-src.git/
#?0|kent:dfly-src.git$ git grep tm_gmtoff origin/master:include
origin/master:include:time.h: long tm_gmtoff; /* offset from UTC in seconds */
</code>

..even..

<code>
#?0|kent:unix-hist$ git grep tm_gmtoff BSD-SCCS-END
BSD-SCCS-END:usr/src/include/time.h: long tm_gmtoff; /* offset from CUT in seconds */
BSD-SCCS-END:usr/src/lib/libc/gen/ctime.3:long tm_gmtoff; /\(** offset from UTC in seconds \(**/
BSD-SCCS-END:usr/src/lib/libc/gen/ctime.3:.Fa tm_gmtoff
BSD-SCCS-END:usr/src/lib/libc/gen/ctime.c: tmp->tm_gmtoff = offset;
BSD-SCCS-END:usr/src/lib/libcompat/4.1/ftime.c: if (tm->tm_isdst) { /* tm_gmtoff has an offset applied */
BSD-SCCS-END:usr/src/lib/libcompat/4.1/ftime.c: tp->timezone = -localtime(&zero)->tm_gmtoff / 60;
BSD-SCCS-END:usr/src/lib/libcompat/4.1/ftime.c: tp->timezone = -tm->tm_gmtoff / 60;
BSD-SCCS-END:usr/src/share/zoneinfo/Makefile:# -DTM_GMTOFF=tm_gmtoff
BSD-SCCS-END:usr/src/share/zoneinfo/Makefile:# -DTM_GMTOFF=_tm_gmtoff
BSD-SCCS-END:usr/src/share/zoneinfo/Makefile:CFLAGS= -DTM_GMTOFF=tm_gmtoff -DTM_ZONE=tm_zone
BSD-SCCS-END:usr/src/usr.bin/chpass/util.c: tval -= lt->tm_gmtoff;
</code>

and

<code>
#?128|kent:unix-hist$ git blame BSD-SCCS-END -- usr/src/include/time.h
..
aec2dfdbbb2 (Keith Bostic 1989-03-08 23:45:57 -0800 49) long tm_gmtoff; /* offset from CUT in seconds */
aec2dfdbbb2 (Keith Bostic 1989-03-08 23:45:57 -0800 50) char *tm_zone; /* timezone abbreviation */
..

#?0|kent:unix-hist$ git blame aec2dfdbbb2^ -- usr/src/include/time.h
...
434fabe335c (Keith Bostic 1987-03-05 01:28:35 -0800 22) long tm_gmtoff;
434fabe335c (Keith Bostic 1987-03-05 01:28:35 -0800 23) char *tm_zone;
</code>

..and we come to..

<code>
commit 434fabe335c23dd3f16ab2d4ff3878b9bc69be50
Author: Keith Bostic <bostic@ucbvax.Berkeley.EDU>
AuthorDate: 1987-03-05 01:28:35 -0800
Commit: Keith Bostic <bostic@ucbvax.Berkeley.EDU>
CommitDate: 1987-03-05 01:28:35 -0800

    added Robert Elz's additions for new time package

    SCCS-vsn: include/time.h 1.2
---
 usr/src/include/time.h | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/usr/src/include/time.h b/usr/src/include/time.h
index 8eb62c0453..0233380029 100644
--- a/usr/src/include/time.h
+++ b/usr/src/include/time.h
@@ -1,4 +1,10 @@
-/* time.h 1.1 85/03/13 */
+/*
+ * Copyright (c) 1983, 1987 Regents of the University of California.
+ * All rights reserved. The Berkeley software License Agreement
+ * specifies the terms and conditions for redistribution.
+ *
+ * @(#)time.h 1.2 (Berkeley) %G%
+ */

 /*
  * Structure returned by gmtime and localtime calls (see ctime(3)).
@@ -13,6 +19,8 @@ struct tm {
        int tm_wday;
        int tm_yday;
        int tm_isdst;
+ long tm_gmtoff;
+ char *tm_zone;
 };

 extern struct tm *gmtime(), *localtime();
</code>

at which time i have to say Hello and a nice day dear Robert Elz!

So given this is from 1987 i would think 34 years give the guarantee that all possible UNIX systems go for long here. Apologies!
(0005534)
steffen   
2021-11-22 15:36   
P.S.: it seems to me not having "const" for tm_zone roots in incomplete transition to modern times here and there. ..Is this only a documentation miss or also a code negligence? Bit rot here, bit rot there, bit rot is just everywhere, ha-ha. Hrm, sorry. Was it Wheeler with "Programming is an iterative task"? Also applies to bit rot anyhow, const, volatile, restrict, atomic_t, ten years ago i was overrun when i said it needs to be volatile atomic_t instead of simple int or volatile int on some perl list/forum, just a few weeks ago i was corrected for not doing it, ... anyway this is all off-topic. const i always liked, today even rodata is put into the text section .. likely to have an excuse to create hundreds of specialized object file sections for whatever purpose. Terrible, volatile ok, but why atomic_t, aehh, ...




Viewing Issue Advanced Details
1532 [Issue 8 drafts] Shell and Utilities Objection Error 2021-11-03 14:15 2021-11-03 16:04
stephane
 
normal  
New  
Open  
   
Stephane Chazelas
0001053
stty utility
"stty -g" output should not have to be split
The resolution of 0001053, about the specification of "stty -g" changes:

-g

    Write to standard output all the current settings in an
    unspecified form that can be used as arguments to another
    invocation of the stty utility on the same system. The form
    used shall not contain any characters that would require
    quoting to avoid word expansion by the shell; see Section
    2.6 (on page 2353).

to:

-g

    Write to standard output all the current settings,
    optionally excluding the terminal window size, in an
    unspecified form that, when used as arguments to another
    invocation of the stty utility on the same system, attempts
    to apply those settings to the terminal. The form used shall
    not contain any sequence that would form an Informational
    Query, nor any characters that would require quoting to
    avoid word expansions, other than field splitting, by the
    shell; see Section 2.6 (on page 2353).

While the 2018 edition had:

-g
    Write to standard output all the current settings in an
    unspecified form that can be used as arguments to another
    invocation of the stty utility on the same system. The form
    used shall not contain any characters that would require
    quoting to avoid word expansion by the shell; see wordexp.

It's one stream of bytes that applications send to standard output.

For that stream of bytes to be "used as argument*S* to another
invocation of the stty utility", it implies that that output should
somehow be split into a list of arguments (and also implies the
result fits in the ARG_MAX limit and doesn't contain NUL bytes).

The reference to 2.6 suggests maybe "sh" should be involved to
perform that splitting. It almost suggests that the output
should be appended to "stty ", or possibly "stty -- " and fed to
sh as in (printf "stty -- "; stty -g) | sh

AFAIK, stty -g was introduced in SysIII circa 1980. It was
outputting on word on one line made of a ":"-separated of alnums
from the portable charset and followed by a newline character.
After it was specified by POSIX, it was also added to BSDs (some
4.3 variant), with "=" in the list of characters that may occur
(not in leading position) in the ":"-delimited list.
GNU stty has used a format similar to SysIII from at least as
far back as 1990.

The way to save and restore it was always to store that output
without the trailing newline characters, which in a shell can be
done with:

saveterm=$(stty -g)

And to restore it:

stty "$saveterm"

The "--" being unnecessary because the output of stty -g never
starts with "-" and with some (including GNU stty) not allowed.

All the documentations of stty I've seen talk of *one* (an)
argument to be passed to stty. They don't say the output may
(let alone should) be split in any way to be passed as several
arguments to stty.

In the unlikely event that there are indeed stty implementations
that require the output of stty -g to be split into arguments
before being fed back to stty, we'd need to specify how those
are meant to be split.
Change the description of the -g option to something like:

-g

    Write to standard output all the current settings,
    optionally excluding the terminal window size, in an
    unspecified form that, when stripped of trailing newline
    characters, and used as the one and only argument to another
    invocation of the stty utility on the same system, attempts
    to apply those settings to the terminal. The form used shall
    consist of one line of text consisting of only printable
    characters from the portable character set, excluding
    whitespace characters, NUL, and all characters that are
    special in the syntax of the sh language
    (`"$&*()[]{};'#~?<>\|).

    (alternatively, specify it as the
    ^[[:alnum:]][[:alnum:]=]*:[[:alnum:]=:]*\n
    extended regexp in the C locale.)

A clarifying example would help like:

saveterm=$(stty -g)

and:

[ -n "$saveterm" ] && stty "$saveterm"
[ -n "$saveterm" ] && stty $saveterm
eval "${saveterm:+stty $saveterm}"

Are three ways the settings may be restored, the first one
being the canonical one and the last one assuming none of the
characters allowed in the stty -g output are currently present
in $IFS (and therefore not recommended).

The example should be changed to

saveterm=$(stty −g) # save terminal state
restoresize=$(
  stty size | awk '{printf "stty rows %d cols %d", $1, $2}'
) # save terminal size
stty new settings # set new state
...
[ -n "$saveterm" ] && stty "$saveterm" # restore terminal state
eval "$ttysize" # restore terminal size

Also using awk instead of shell IFS-splitting+globbing as as
discussed on the mailing list, $IFS is not guaranteed to contain
all the blank characters in the locale (and anyway yash is the
only shell that considers multi-byte blanks as
IFS-whitespace-characters wrt IFS-splitting), while awk is
required to split fields on blanks (even though not all
implementations do).
Notes
(0005513)
stephane   
2021-11-03 14:23   
Sorry, a few typos above.

"It was outputting on word on one line made" should be "It was outputting one word on one line made"

The regexp was intended to be

^[[:alnum:]][[:alnum:]=]*:[[:alnum:]=:]*\n$

(starts with an alnum, contains at least 1 ":" and contains only alnums, ":" or "=" characters).

In:

> [ -n "$saveterm" ] && stty "$saveterm"
> [ -n "$saveterm" ] && stty $saveterm
> eval "${saveterm:+stty $saveterm}"

It's the second, not last that is dependant on IFS and should be avoided.
(0005514)
stephane   
2021-11-03 14:51   
> The reference to 2.6 suggests maybe "sh" should be involved to
> perform that splitting. It almost suggests that the output
> should be appended to "stty ", or possibly "stty -- " and fed to
> sh as in (printf "stty -- "; stty -g) | sh

That however wouldn't work as sh's stdin would then be a pipe and not the terminal . Not to mention the fact that if stty -g fails, the code becomes "stty " or "stty --" which requests tty settings.

Also, in:

> The form used shall
> consist of one line of text consisting of only printable
> characters from the portable character set, excluding
> whitespace characters, NUL, and all characters that are
> special in the syntax of the sh language
> (`"$&*()[]{};'#~?<>\|).

Since I've added "printable" (with the intention of excluding control characters) and "line of text", the "NUL, " is redundant.

(0005515)
stephane   
2021-11-03 16:01   
Also, on a not totally related note, IMO it's undesirable that stty "$saveterm" should restore the cols and rows settings.

In the now most common case of stty being used inside a terminal emulator in some windowing system, the rows and cols are typically being changed on the master side of the pty pair by the terminal emulator when the window of that emulator is resized.

The almost only use cases of "stty rows y cols x" is to fix those tty settings so they match those of the terminal when they are incorrect.

While the general use case for "stty -g" is so one can temporarily disable local echo or icanon while input is taken from the user. And the user could very well resize their terminal whilst they're being prompted for some input, and you don't want stty "$saveterm" to render "rows" and "cols" incorrect afterwards by restoring cols/rows from before the time the terminal window was resized.

If we allow implementations to restore the saved cols/rows upon stty "$saveterm", that means we now have to write:

{
  saveterm=$(stty -g)
  stty -echo
  printf 'Password: '
  IFS= read -r passwd
  printf '\n'

  restoresize=$(
    stty size | awk '{printf "stty rows %d cols %d", $1, $2}'
  )
  if [ -n "$saveterm" ]; then
    stty "$saveterm" # restore terminal state which may break cols/rows
    eval "$restoresize" # restore terminal size potentially broken by stty above
  fi
} 0<> /dev/tty >&0 2>&0


(which does have a race condition) to prompt for a password instead of just:

{
  saveterm=$(stty -g)
  stty -echo
  printf 'Password: '
  IFS= read -r passwd
  printf '\n'
  if [ -n "$saveterm" ]; then
    stty "$saveterm"
  fi
} 0<> /dev/tty >&0 2>&0
(0005516)
stephane   
2021-11-03 16:04   
Sorry again, yet another typo in my initial submission:

> saveterm=$(stty −g) # save terminal state
> restoresize=$(
> stty size | awk '{printf "stty rows %d cols %d", $1, $2}'
> ) # save terminal size
> stty new settings # set new state
> ...
> [ -n "$saveterm" ] && stty "$saveterm" # restore terminal state
> eval "$ttysize" # restore terminal size


Should be eval "$restoresize"




Viewing Issue Advanced Details
1531 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2021-10-30 15:56 2021-10-30 15:56
steffen
 
normal  
New  
Open  
   
steffen
time
3299
111068-111069
---
time: follow-up to issue #1440
""
on location as above change

 Alternatively, the following command can be used to apply time to a complex command:
  time sh −c 'complex-command-line'

to

  ...time sh −c -- 'complex-command-line'
There are no notes attached to this issue.




Viewing Issue Advanced Details
1530 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2021-10-30 15:55 2021-10-30 15:55
steffen
 
normal  
New  
Open  
   
steffen
nohup
3042
101217-101218
---
nohup: follow-up to issue #1440
""
on location as above change

 Alternatively, the following command can be used to apply nohup to a complex command:
   nohup sh −c 'complex-command-line' </dev/null

to

 ...nohup sh −c -- 'complex-command-line' </dev/null
There are no notes attached to this issue.




Viewing Issue Advanced Details
1529 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2021-10-30 15:54 2021-12-05 01:11
steffen
 
normal  
New  
Open  
   
steffen
ex
2732
89418-89420
---
ex: follow-up to issue #1440
""
on location as above change

  The ex utility shall execute the program named by the shell edit option. It shall pass two arguments to the program; the first shall be −c, and the second shall be the expanded arguments to the ! command as a single argument.

to

  ...three arguments to the program; the first shall be -c, the second --, and the third shall be...
Notes
(0005542)
mirabilos   
2021-12-05 00:51   
I’m not sure requiring the ["sh", "-c", "--", arg] behaviour in the standard is a good thing.

This is an implementation detail.

I agree with requiring it for system() and friends, but for user-space utilities, I would instead write something like “run it with sh -c in a safe way, for example by using a double-dash thingy”.

This leaves people who write portable software (such as MirBSD’s ed(1)) the option to do it by prefixing a space instead, which will work on both “old” and “new” systems; ed(1) has a leading character in the buffer which it can use for this anyway.

Other applications, where the arg comes in on its own, should not need copying, so using “sh -c --” is a good thing there (especially if the standard sh(1) supports that), but your proposed wording makes it harder for real-world systems.




Viewing Issue Advanced Details
1528 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2021-10-29 23:59 2021-10-29 23:59
steffen
 
normal  
New  
Open  
   
steffen
mailx
2958, 2961-2963
98057, 98174, 98250, 98195
---
mailx: document "sh(1) -c --" has to be used instead of "sh -c"
mailx(1) specific follow-up to issue #1440.

P.S.: page numbers and line numbers from C181. (I think this is not the newest?)
On page 2958 lines 98056-7 change

  Pipe the messages through the given command by invoking the command interpreter specified by SHELL with two arguments: −c and command. (See also sh −c.)

to

 ... three arguments: -c, -- and command...

Ditto:

page 2961, lines 98174-5 (!command),
page 2962, lines 98195-6 (~! command escape),
page 2963, lines 98250-1 (~| command escape).
There are no notes attached to this issue.




Viewing Issue Advanced Details
1527 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-10-28 00:24 2021-10-28 00:24
kre
 
normal  
New  
Open  
   
Robert Elz
XCU 4 / cd
2563
83023-83027
---
cd requires the impossible on standard output
The STDOUT section of XCU 4(cd) says:

     If a non-empty directory name from CDPATH is used, or if cd - is used,
     an absolute pathname of the new working directory shall be written to
     the standard output as follows:

Whether used as cd -P, or the ludicrous cd -L, this requires what might
be impossible.

From XCU 2.5.3 (shell vars):
     PWD Set by the shell and by the cd utility. [...]
          if there is insufficient permission on the current working
          directory, or on any parent of that directory, to determine
          what that pathname would be, the value of PWD is unspecified

So the standard clearly recognises that it is not always possioble to
determine the current working directory, which results in PWD being unspecified
but not in unspecified behaviour from cd or pwd (that's only if the user
modifies PWD).

Further, while not exactly common, nor is this a very rare event, it happens
from time to time, and is usually easily corrected by a simple cd to a fully
qualified path (one which works) which will result in establishing a value
for PWD, after which all is normal (which would not be able to be trusted if
the behaviour of cd was unspecified at that point).

In that state, if the user successfully does:

     mkdir foo foo/bar
     export CDPATH=foo
     cd bar

the text quoted above requires the cd command to print the full path
     /impossible/to/discover/foo/bar
which, of course, is impossible to do correctly.

It is possible to get into this state after the shell has started (and PWD
is set) as well, though in that case it is typically only cd -P which has the
problem (the shell just "knows" that $PWD is correct, and all it takes to fix
it is some string manipulation in the cd -L case ... but not always true,
as the filesystem may have altered state after PWD was set.)

What shells do in this situation varies. Some print an error or warning
which they're not supposed to do, as "The standard error shall be used only
for diagnostic messages." and that means (according to XCU 1.4) that the
exit status must indicate an error, but for cd (at least without the not
yet existing -e option) when cd has changed directory successfully, it must
exit 0 (-e doesn't solve the problem for cd -L with PWD unset, and no
ability to discover a path of the current working directory anyway, so it
is irrelevant here).

Aside from the technically incorrect error message that is sometimes printed,
most shells print the (adjusted) relative path (which is what it usually, though
not always, is in these cases) that was used in the chdir() call. In the
example above that would be "foo/bar" - not an absolute path by any means.
(zsh almost does that, at least in the copy I have currently, but pretends it
is an absolute path by prepending a '/' - /foo/bar - which is nonsense, and
most probably simply a bug). Other shells simply print nothing in this case.

In the case that PWD is set, and cd -P is used, and the resulting directory's
path cannot be determined, yash seems to treat it something like cd -L, treating
PWD as if it were the correct physical path, and adjusting it. The result is
an absolute path, and in one case I saw it, was actually correct. There was
no way yash could have known that however - it was a guess (even if a well
educated one).

Since the behaviour of the existing shells varies so much, I suspect that all
that it is possible to say in this situation is that the results are unspecified. Something similar may need to be said about what goes in PWD
(though there the existence of -e in issue8 might make a difference).

Whether it is also desirable to explicitly allow shells to issue an error in
this situation I will leave for future discussion (my shell does not...).


In lines 83024-83025 change the words


an absolute pathname of
the new working directory shall be written


to


and the absolute pathname of the new working directory
can be determined, that pathname shall be written


Between lines 83026 and 83027 insert a new paragraph.


If an absolute pathname of the new current working
cannot be determined, it is unspecified what, if anything,
shall be written to the standard output.


In line 83027 change the word
Otherwise
to:


If a non-empty directory name from CDPATH is not used,
and the directory argument is not <tt>'-'</tt>


This last change just because there are now two antecedent "if"
clauses, and an "Otherwise" would be ambiguous. Implementing the
necessary change in some other way might avoid that problem.

Additionally consider what might need to be altered elsewhere wrt how
PWD is to be set, or not, and any effects that might have. Given what is
in XCU 2.5.3 this might be considered unnecessary.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1526 [Issue 8 drafts] System Interfaces Objection Enhancement Request 2021-10-19 10:53 2021-10-21 08:48
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
fdopen()
823
28083-28093
Update fdopen() mode description to match new fopen() terminology
The new 'e' and 'x' mode string characters for fopen() have been accounted for on the fdopen() page using "prefix" terminology, which is being changed for fopen() via bug 0001302 (C17 alignment). The fdopen() page should change to match the new terminology, but the need for this change does not result (directly) from C17 alignment, so it is being handled by a separate bug instead of in 1302.

The current wording is also vague about what happens if there is a mismatch between the O_APPEND flag on the open file description and the use of 'w' versus 'a' in the mode. The only clue is that the rationale states "a good implementation of append (a) mode would cause the O_APPEND flag to be set", which implies that it is intentional that both behaviours are allowed when O_APPEND is clear and 'a' is used. I tried test programs on Linux (glibc), Solaris, MacOS and HP-UX, and the results were:

* Only Linux set O_APPEND if it was clear when using "a"
* All four left O_APPEND set if it was set when using "w"

Therefore the suggested changes make it unspecified for "a" but mandate leaving O_APPEND set for "w".

As an editorial matter, the use of the phrase "Open a file" is also problematic since fdopen() does not actually open a file.

Finally, the new changes include a typographical conventions change to use quotes around mode characters instead of italicising them. This change is also needed on the popen() page for consistency.
On page 823 line 28083 section fdopen(), change:
The mode argument points to a character string, which shall also be valid for fopen(). The string prefix shall have the following effects:

r or rb
Open a file for reading.
w or wb
Open a file for writing.
a or ab
Open a file for writing at end-of-file.
r+ or rb+ or r+b
Open a file for update (reading and writing).
w+ or wb+ or w+b
Open a file for update (reading and writing).
a+ or ab+ or a+b
Open a file for update (reading and writing) at end-of-file.
The meaning of these flags is exactly as specified in fopen(), except that modes beginning with w shall not cause truncation of the file, and the use of x shall have no effect. The FD_CLOEXEC flag of fildes shall be unchanged if e was not present, and shall be set if e is present.
to:
The mode argument points to a character string that is valid for fopen(). If the string begins with one of the following characters, then the stream shall be associated with fildes as specified. Otherwise, the behavior is undefined.

<tt>'r'</tt>
If mode includes <tt>'+'</tt>, the associated stream shall be open for update (reading and writing); otherwise, the stream shall be open for reading only.
<tt>'w'</tt>
If mode includes <tt>'+'</tt>, the associated stream shall be open for update (reading and writing); otherwise, the stream shall be open for writing only. The file shall not be truncated by the fdopen() call. If the open file description referenced by fildes has O_APPEND set, it shall remain set.
<tt>'a'</tt>
If mode includes <tt>'+'</tt>, the associated stream shall be open for update (reading and writing); otherwise, the stream shall be open for writing only. If the open file description referenced by fildes has O_APPEND clear, it is unspecified whether O_APPEND is set by the fdopen() call or remains clear.
The presence of <tt>'x'</tt> in mode shall have no effect. The FD_CLOEXEC flag of fildes shall be unchanged if <tt>'e'</tt> is not present, and shall be set by the fdopen() call if <tt>'e'</tt> is present.

On page 824 line 28124 section fdopen(), change:
The meanings of the mode arguments of fdopen() and fopen() differ. With fdopen(), open for write (w or w+) does not truncate, and append (a or a+) cannot create for writing. The mode argument formats that include a b are allowed for consistency with the ISO C standard function fopen(). The b has no effect on the resulting stream. Although not explicitly required by this volume of POSIX.1-202x, a good implementation of append (a) mode would cause the O_APPEND flag to be set. Since fdopen() does not create a file, the x mode modifier is silently ignored. The e mode modifier is ...
to:
The meanings of the mode arguments of fdopen() and fopen() differ. With fdopen(), write (<tt>'w'</tt>) mode cannot create or truncate a file, and append (<tt>'a'</tt>) mode cannot create a file. Inclusion of a <tt>'b'</tt> in the mode argument is allowed for consistency with fopen(); the <tt>'b'</tt> has no effect on the resulting stream. Implementations differ as to whether specifying append (<tt>'a'</tt>) mode causes the O_APPEND flag to be set if it was clear, but they are encouraged to do so. Since fdopen() does not create a file, the <tt>'x'</tt> mode modifier is silently ignored. The <tt>'e'</tt> mode modifier is ...

On page 1421 line 47441, 47447, 47453 section popen(), and
page 1422 line 47499 section popen(), and
page 1423 line 47508, 47509, 47525, 47532, 47533 section popen(), and
page 1428 line 47785 section popen():
change r, w, and e to <tt>'r'</tt>, <tt>'w'</tt>, and <tt>'e'</tt>, respectively.

On page 1422 line 47505 section popen(), change:
The behavior of popen() is specified for values of mode of r, w, re, and we. Other modes such as rb and wb might ...
to:
The behavior of popen() is specified for values of mode of <tt>"r"</tt>, <tt>"w"</tt>, <tt>"re"</tt>, and <tt>"we"</tt>. Other modes such as <tt>"rb"</tt> and <tt>"wb"</tt> might ...

On page 1423 line 47508 section popen(), change:
robert the robot
to:
<tt>"robert the robot"</tt>

On page 1423 line 47536 section popen(), change:
wf
to:
<tt>"wf"</tt>

On page 1423 line 47542 section popen(), change:
rf
to:
<tt>"rf"</tt>

Notes
(0005507)
kre   
2021-10-19 16:29   
(edited on: 2021-10-19 16:32)
What's the effect if fd has O_APPEND set and there is
     fdopen(fd, "r+");
??

That seems to be ignored in the suggested text. (O_APPEND shouldn't
matter for the "r" case, one way or the other).

I also wonder about the requirement that 'b' be ignored for fdopen().
That's certainly what will happen if binary made is something selected
at the kernel open level (O_BINARY) but is it forbidden for stdio to
implement different I/O mechanisms for text & binary modes? That is,
if stdio itself is doing something different based upon whether or not
'b' is in the open mode, why would it matter if the FILE* was created by
fopen() or fdopen()? The 'b' should have the same effect in both cases.
In some systems where O_BINARY does make a difference, it might also be
possible to turn that on or off dynamically, just like is suggested as
a possibility for O_APPEND with "a" mode).

Lastly, is there any prohibition (or stated to be undefined or unspecified
anywhere) on the same underlying fd being the fd for more than one stdio
stream? If there is not, then what is the effect of fdopen(fd, "w")
followed by fdopen(fd, "a") (resulting in 2 different FILE*'s). Obviously
it would be up to the application to sequence output so it makes sense,
but the underlying fd would either have O_APPEND (in a "good implementation"
set by the "a") or not, and that would affect the "w" FILE* as well.

If there is such a prohibition, how does that apply to the common case where
stdin/stdout/stderr all share one underling file (ie: they are fds 0, 1, and 2
but they get created by in = open("/dev/ttyX", O_RDWR); out=dup(in); err=dup(in);
(starting from a state where everything was closed). In that
state there's just one O_APPEND or O_BINARY for all 3 of them (only the O_CLO*
flags affect just the file descriptor). That's effectively sharing a fd
(even though the numbers vary - and for the purposes of the previous paragraph,
is just the same as if the fds were the same).

(0005508)
geoffclare   
2021-10-21 08:48   
> What's the effect if fd has O_APPEND set and there is
> fdopen(fd, "r+");

Good catch. I tried this (and r and w+ for completeness) on the same four systems and they all left O_APPEND set. So I think the statement for 'w': "If the open file description referenced by fildes has O_APPEND set, it shall remain set" should be added to 'r' as well.

> (O_APPEND shouldn't matter for the "r" case, one way or the other)

If fd has O_RDWR|O_APPEND then fdopen(fd, "r") shouldn't clear O_APPEND because the application could still write directly to fd and expect the data to be appended.

> I also wonder about the requirement that 'b' be ignored for fdopen().

It matches the requirement that 'b' is ignored for fopen() and freopen(). It's already required for fdopen() as well (via the normative reference to fopen(), but that reference is removed by these changes so this requirement now needs to be stated explicitly).

> Lastly, is there any prohibition (or stated to be undefined or unspecified
> anywhere) on the same underlying fd being the fd for more than one stdio
> stream? If there is not, then what is the effect of fdopen(fd, "w")
> followed by fdopen(fd, "a") (resulting in 2 different FILE*'s).

There is no prohibition, but there are rules (in XSH 2.5.1) that applications must follow when changing from one "handle" to another (in the case given, there are three handles: the two streams and the fd).

The second fdopen() must behave exactly as specified - the fact that there is already another stream that has the same fd underlying it has no bearing on how fdopen() behaves.




Viewing Issue Advanced Details
1525 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2021-10-15 16:07 2021-10-15 18:39
ben_pfaff
ajosey  
normal  
Under Review  
Open  
   
Ben Pfaff
close()
689
23562
---
only the close() of the last fd for a socket should destroy the socket
The description of close() repeatedly says that various actions only take place when the last fd for an object is closed. For example:

"When all file descriptors associated with a pipe or FIFO special file are closed..."

"When all file descriptors associated with an open file description have been closed, ..."

"The last close() for a STREAM..."

"If fildes refers to the master side of a pseudo-terminal, and this is the last close, ..."

But it does not say that for sockets: "If fildes refers to a socket, close() shall cause the socket to be destroyed." This suggests behavior that implementations do not (and should not) implement.
Change: "If fildes refers to a socket, close() shall cause the socket to be destroyed."

to: "When all file descriptors associated with a socket have been closed, the socket shall be destroyed."
There are no notes attached to this issue.




Viewing Issue Advanced Details
1524 [Issue 8 drafts] System Interfaces Objection Omission 2021-10-10 18:49 2021-10-11 09:12
Don Cragun
 
normal  
New  
Open  
   
Don Cragun
fopen()
881, 882
29973-29974, 30005-30013
open() flags used by fopen()
The lead-in to the table describing the open() flags used by fopen() on P882, L30005-30013 says:
The file descriptor associated with the opened stream shall be allocated and opened as if by a call to open( ) with the following flags:

but it doesn't say that additional flags can't silently be added to the given sets of flags. In fact, if you use the <tt>'e'</tt> or <tt>'x'</tt> characters in the mode string, other flags are required to be OR'ed into the flags listed in this table.

Implementations of fopen( ) should not be allowed to act as though flags such as O_DSYNC, O_NONBLOCK, etc. had been set when a file is opened.
Change the description of the <tt>'e'</tt> mode string character on P881, L29973-29974 from:
The underlying file descriptor shall have the FD_CLOEXEC flag atomically set, as if by the O_CLOEXEC flag to open( ).
to:
The underlying file descriptor shall have the FD_CLOEXEC flag atomically set.


A change similar to the above will be needed for the description of the <tt>'x'</tt> mode string character on P881, L29975-29977 but I won't specify the details here because that text is being modified by changes for 0001302.

In the second column of the table header on P882, L30007 change:
open( ) Flags
to:
Initial open( ) Flags


After the table on P882, L30005-30013 add three new paragraphs (all CX shaded):
If, and only if, the <tt>'e'</tt> mode string character is specified, the O_CLOEXEC flag shall be OR'ed into the initial open( ) flags specified in the above table.

Add a corresponding paragraph for the O_EXCL flag when <tt>'x'</tt> is in the mode string.

The implementation shall behave as if no other flags had been passed to open( ).
Notes
(0005501)
kre   
2021-10-10 23:45   
This, I think, is going too far:

    The implementation shall behave as if no other flags had been passed to open( ).

What it probably should say is something like:

    Unless other implementation defined characters are present in the mode
    string, the implementation shall behave as if no other flags had been
    passed to open().

or something like that. That is, implementations should be allowed to
make extensions, for example, an 'N' in the mode string could specify O_NONBLOCK
(etc) if the implementation had a reason to desire something like that.

That is, it isn't important that the implementation never add other open
flags, only that it not do so unless use of an implementation extension
requests it to do so, so code using only standard mode strings won't be surprised.
(0005502)
kre   
2021-10-10 23:51   
While not directly related to this bug, it is worth nothing that nothing I
have even seen in the standard specifies what modes the standard open fds
(stdin, stdout, stderr) should have set, or not set (aside from read/write
modes).

That is, there's nothing I can see that says that applications are not
required to operate correctly if started with stdin with O_CLOFORK set
(assuming that flag gets added) or O_NONBLOCK, or that stdout cannot
have O_APPEND set. Any of that would be unusual to say the least,
but is any of it forbidden?

[Aside, obviously O_CLOEXEC cannot be set on any of those fds ...]
(0005503)
Don Cragun   
2021-10-11 03:45   
Re Note: 0005501:
I would be happy to change the last paragraph of the additional text after the table to be:
When using mode strings specified by this standard, the implementation shall behave as if no other flags had been passed to open( ).
(0005504)
geoffclare   
2021-10-11 09:12   
The penultimate line of the desired action ("Add a corresponding paragraph ...") is part of the literal text to be added. It should be deleted and a statement added at the bottom to handle this in the same way as for the 'x' description - something along the lines of:

A paragraph similar to the one for O_CLOEXEC above will be needed for O_EXCL when the 'x' mode string character is used, but I won't specify the details here because that text is being modified by changes for bug 1302.




Viewing Issue Advanced Details
1523 [Online Pubs] System Interfaces Editorial Error 2021-10-09 14:42 2021-10-09 16:20
rillig
 
normal  
New  
Open  
   
Roland Illig
https://pubs.opengroup.org/onlinepubs/9699919799/functions/getopt.html [^]
DESCRIPTION
Wrong layout of getopt "-"
> If, when getopt() is called:
>
> argv[optind] is a null pointer*argv[optind] is not the character -
> argv[optind] points to the string "-"

I didn't understand the above excerpt, even after reading it multiple times.

My misunderstanding was due to the missing line break before the first '*' and by the additional space before the last line.
Change the rendered HTML text to be:

> If, when getopt() is called:
>
> argv[optind] is a null pointer
> argv[optind][0] is not the character '-'
> argv[optind] points to the string "-"

The changes are:
* additional line break between line 1 and line 2
* postfix array index in line 2 to make the lines look more similar
* single quotes around the character '-' in line 2
* no leading space in line 3

Having the HTML markup <tt> inside of
 looks strange, as if the text outside the <tt> was intended to be displayed in proportional font. That may or may not be an entirely different issue though.

While here, I'd prefer it if at the end of the
 sections, there were no additional newline, so that the continuation of the sentence "getopt() shall return -1" is closer to the list of conditions.	
Notes
(0005499)
rillig   
2021-10-09 14:52   
I just saw that Mantis didn't render the description as intended, so here is my second try, after reading the provided documentation that answered exactly my questions. :)

Change the rendered HTML text to be:

> If, when getopt() is called:
>
> argv[optind] is a null pointer
> argv[optind][0] is not the character '-'
> argv[optind] points to the string "-"

The changes are:
* additional line break between line 1 and line 2
* postfix array index in line 2 to make the lines look more similar
* single quotes around the character '-' in line 2
* no leading space in line 3

Having the HTML markup <tt> inside of <_pre> looks strange, as if the text outside the <tt> was intended to be displayed in proportional font. That may or may not be an entirely different issue though.

While here, I'd prefer it if at the end of the <_pre> sections, there were no additional newline, so that the continuation of the sentence "getopt() shall return -1" is closer to the list of conditions.
(0005500)
kre   
2021-10-09 16:20   
(edited on: 2021-10-09 16:25)
Of those proposed changes, the missing newline is clearly a
rendering into HTML issue, it isn't like that in the PDF version,
and should be easily fixed (says me who knows nothing about that process!)

The changes from *argv[optind] to argv[optind][0] and to put
quotes areound the minus sign on that line would require changes to
the PDF as well, and so need to be a bug report (editorial) against
issue7-TC2 - the first of those I see no point for, it is fine as it is
(when correctly rendered, as in the PDF), the second just maybe (it
is clear enough as it is, but single quotes wouldn't hurt).

The leading space in line 3 should also occur in line 1, and serves to
cause the three instances of argv[optind] to line up one above the other
(the 2nd line has the '*' in that position), and I see no reason to change
that either. In the HTML those should perhaps be &nbsp; rather than
just a space.

The blank line after the conditions also matches the PDF, and I see no
issue with that either.

One thing I might alter, if editorial (as aside from just rendering)
changes are considered, is to make it explicit that the three conditions
are "any of" choices, not "all of" (even though it is obvious that the
latter cannot possibly be intended upon careful reading), perhaps by
changing "If, when getopt() is called:" to "If, when getopt() is called any of:"
(to avoid needing to insert "or" somewhere, which would look ugly).





Viewing Issue Advanced Details
1522 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2021-10-08 14:31 2021-10-08 14:31
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
mkdir()
1316
43790
---
mkdir() and S_ISVTX
The mkdir() description states:
When bits in mode other than the file permission bits are set, the meaning of these additional bits is implementation-defined.
This is old POSIX.1-199x text from before the merge with SUS, after which the S_ISVTX bit has a specified meaning for directories (when the XSI option is supported), as per the <sys/stat.h> description of S_ISVTX: "On directories, restricted deletion flag". This needs to be accounted for in the mkdir() description.

The umask() page also has a similar problem.
On page 1316 line 43787 section mkdir(), change:
The mkdir() function shall create a new directory with name path. The file permission bits of the new directory shall be initialized from mode. These file permission bits of the mode argument shall be modified by the process' file creation mask.

When bits in mode other than the file permission bits are set, the meaning of these additional bits is implementation-defined.
to:
The mkdir() function shall create a new directory with name path. The file permission bits [XSI]and S_ISVTX bit[/XSI]of the new directory shall be initialized from mode. The file permission bits of the mode argument shall be modified by the file creation mask of the process.

When bits in mode other than the file permission bits [XSI] and S_ISVTX[/XSI] are set, the meaning of these additional bits is implementation-defined.

On page 2189 line 70020 section umask(), change:
Only the file permission bits of cmask (see <sys/stat.h>) are used; the meaning of the other bits is implementation-defined.
to:
Only the file permission bits of cmask (see <sys/stat.h>) shall be used; [XSI]the S_ISVTX bit shall be ignored, and[/XSI] the meaning of the other bits is implementation-defined.

On page 2189 line 70028 section umask(), change:
Bit positions that are set in cmask are cleared in the mode of the created file.
to:
Permission bit positions that are set in cmask are cleared in the mode of the created file.

On page 2189 line 70030 section umask(), change:
The file permission bits in the value returned by umask() shall be the previous value of the file mode creation mask. The state of any other bits in that value is unspecified, ...
to:
The file permission bits in the value returned by umask() shall be the previous value of the file mode creation mask. [XSI]The S_ISVTX bit in the returned value shall be clear.[/XSI] The state of any other bits in the returned value is unspecified, ...

There are no notes attached to this issue.




Viewing Issue Advanced Details
1521 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-09-09 15:34 2021-09-10 11:09
kre
 
normal  
New  
Open  
   
Robert Elz
XCU 2.7.4
2362
75362-75377
---
here document processing is underspecified
The specification for a Here-Document (XCU 2.7.4) is lacking
some precision, and needs to be improved.

First, there is no reason to limit the kind of file descriptor
to being a regular file, special file, or a pipe. Note that
the XBD 3.164 definition of "File" includes:

        File types include regular file, character special file, block special
        file, FIFO special file, symbolic link, socket, and directory.
        Other types of files may be supported by the implementation.

from which it appears that block/character/FIFO special files exist,
but that sockets, directories, symbolic links, and other types supported
by the implementation are not special files (though there is no actual
definition of a "special file"). While it is unlikely (to say the least)
that a symbolic link or directory would be a useful file type for a
here-document, sockets or other implementation types might be. The
wording of the standard should allow for that possibility.

Second, it is not precisely specified whether the "input lines"
which are to be stripped of tab characters are the lines in
the input, as presented to the shell, lines in the input, after
line joining (by the \<newline> combination, when that is applied)
has been carried out, or the lines to be input to the command to
which the redirection operator is applied.

Fortunately all shells seem to agree that tabs are stripped from
lines after line joining (when it applies) but before any expansions
are performed which might create more leading tab characters.

This should be precisely specified.

Third, it is not clear whether or not the end delimiter can be
formed from joined lines, or whether it must appear literally
in the input stream.

For this one shells are not in agreement, some initially treat
the end delimiter as part of the here document, and so perform
line joining, discovering the end delimiter after that, and then
removing it from the here document. Other shells look for the
end delimiter first (after joining previous lines, so a line
which looks like the end delimiter, but which is a continuation
of a previous line, will not be the end delimiter) and do not
permit it to be formed from joined lines. Since there is no
good reason for an application to ever split the end delimiter
over multiple lines in real code (as distinct from torture
tests) this can be made explicitly unspecified - applications
should not rely upon either behaviour.

Note that there is another open issue, relating to which <newline> is the
<newline> after which a here document begins, these two are orthoganal,
but this one, being simpler, should probably be processed first.
In line 75364 change the words
or a pipe

to
a pipe, or some other type of file



In line 75369 change the words
shall be expanded

to
shall be expanded as the redirection operator is
performed, each time it is performed,




In lines 75374-5 change the words
stripped from input lines and the line containing
the trailing delimiter.

to
stripped from input lines after <backslash><newline>
line joining (when it applies) has been performed and the line
containing the trailing delimiter.
It is unspecified whether the line containing the trailing delimiter
is subject to this line joining in any case.
Tab stripping occurs as the here document text is read from the shell
input stream, and is not repeated after any expansions that are
required have occurred.



Please adjust the actual wording applied to be more pleasing.
Notes
(0005497)
kre   
2021-09-09 15:37   
(edited on: 2021-09-09 15:42)
Oops, sorry, I see I missed a < /em > after the > after newline...

And it should say "has been performed" not "last been performed"
No idea how my fingers came up with that one.

(0005498)
geoffclare   
2021-09-10 08:41   
I have updated the desired action to fix the problems noted in Note: 0005497. (I removed the starting < em > rather than adding an ending < /em > as italics would be incorrect there anyway.)




Viewing Issue Advanced Details
1520 [Issue 8 drafts] Shell and Utilities Comment Enhancement Request 2021-09-07 09:44 2021-09-07 10:39
joerg
 
normal  
New  
Open  
   
Jörg Schilling
make
2947-2948
98896-98920
make should support the $^ internal macro
The $^ internal macro is supported by major make implementations since many
years but it is not mentioned in the POSIX standard.

SunPro Make, smake and gmake support $^ with the same meaning since at
least 25 years.

BSD make supports the same meaning under a different name ($>).
BSD make therefore would just need to add an alias for an existing
feature which seems to be a minor effort.

A possible reason for the missing $^ in POSIX may be that the SunPro Make
implementation supports that internal macro since a long time but did not
document it for unknown reason, causing the impression this may be a
gmake only extension.

On page 2947 after line 98895 add:

The $^ macro shall evaluate to the list of prerequisites for the current
target. It shall be evaluated for both target and inference rules.

If the list of prerequisites of the target contains .WAIT pseudo targets,
the .WAIT pseudo targets shall not be part of the list expanded from $^.

On page 2948 after line 98919 add:

$^ member.s2
Notes
(0005495)
joerg   
2021-09-07 10:39   
(edited on: 2021-09-07 11:36)
I am sorry, I just see that I was mistaken with the occurence of the
.WAIT pseudo target in the list created by $^. It seems that I did
not remember my previous test results correctly and the SunPro Make
documentation is still incomplete even though I enhance it on a
regular base.

SunPro Make includes .WAIT in that $^ list.

BSD make replaces the first .WAIT by .WAIT_1, the second .WAIT by .WAIT_2, ...
and includes that in the $> list.

It may thus need a discussion whether we make the exclusion of .WAIT
from the $^ list optional.

For the SunPro Make case, it would be sufficient to use this:

  $(^:.WAIT=)

instead of jist $^. For this reason, it would make sense to replace:

If the list of prerequisites of the target contains .WAIT pseudo targets,
the .WAIT pseudo targets shall not be part of the list expanded from $^.

by:

If the list of prerequisites of the target contains .WAIT pseudo targets,
it is unspecified whether .WAIT pseudo targets appear in the list expanded
from $^, it is recommended to use $(^:.WAIT=).

... if BSD make would implement $^ in a way that does not modify the .WAIT
entries.





Viewing Issue Advanced Details
1519 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Omission 2021-09-06 15:27 2021-09-06 15:27
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
sh
3226
108295
---
Mismatch between sh and set for -o and +o with no option-argument
The sh SYNOPSIS shows -o and +o as requiring an option-argument; however the description in OPTIONS refers to set for these options, and set has SYNOPSIS lines that allow them without an option-argument.

Some shell implementations complain if you invoke them with "sh -o" or "sh +o", but others do the equivalent of "set -o" or "set +o" (and then enter interactive mode).
Append to the second paragraph of OPTIONS (where set is referenced):
If the -o or +o option is specified without an option-argument, the behavior is unspecified.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1518 [Issue 8 drafts] Base Definitions and Headers Editorial Error 2021-08-30 11:17 2021-08-31 08:27
Vincent Lefevre
 
normal  
New  
Open  
   
Vincent Lefevre
Inria
<unistd.h>
433
15053, 15058
spelling inconsistency (programming vs programing)
The whole standard uses the spelling "programming" except at lines 15053 and 15058, where it uses "programing".
Spelling should be made consistent, i.e. change "programing" to "programming".
Notes
(0005483)
geoffclare   
2021-08-31 08:27   
When accepted this bug should be tagged tc3-2008 as Issue 7 has the problem.




Viewing Issue Advanced Details
1517 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2021-08-28 06:06 2021-08-28 06:06
andras_farkas
 
normal  
New  
Open  
   
Andras Farkas
lex
---
lex rationale references non-existent references to -c option
On
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/lex.html [^]
There's the following text in the rationale:
Even though the -c option and references to the C language are retained in this description, lex may be generalized to other languages, as was done at one time for EFL, the Extended FORTRAN Language.
However, the page has no reference of the -c option in the normative text. There's only one other mention of the -c option in the informative text, in the change history.
If the reference to the -c option in the change history isn't what that sentence meant, then change it to:
Even though references to the C language are retained in this description, lex may be generalized to other languages, as was done at one time for EFL, the Extended FORTRAN Language.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1516 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2021-08-24 08:51 2021-08-24 08:56
Don Cragun
 
normal  
New  
Open  
   
Don Cragun
self
catopen(), gencat, uucp, uustat, uux
649, 2819, 3355, 3366, 3369
22300-22305, 92796, 112986, 113372, 113489
---
XSI shading missing on several uses of the NLSPATH environment variable
The NLSPATH environment variable was only meant to be required on systems supporting the XSI option. When catopen() and gencat were moved from the XSI option to the base, their uses of NLSPATH should have still been marked with XSI shading. Similarly <uucp> and related utilities with NLSPATH in their lists of environment variables should have had them marked with XSI shading.
In the catopen() function DESCRIPTION section on P649, L22300-22305 change:
Otherwise, the environment variable NLSPATH is used with name substituted for the %N conversion specification (see XBD Chapter 8, on page 173); if NLSPATH exists in the environment when the process starts, then if the process has appropriate privileges, the behavior of catopen( ) is undefined. If NLSPATH does not exist in the environment, or if a message catalog cannot be found in any of the components specified by NLSPATH, then an implementation-defined default path shall be used.
to:blockquote>Otherwise, [XSI]the environment variable NLSPATH is used with name substituted for the %N conversion specification (see XBD Chapter 8, on page 173); if NLSPATH exists in the environment when the process starts, then if the process has appropriate privileges, the behavior of catopen( ) is undefined. If NLSPATH does not exist in the environment, or if a message catalog cannot be found in any of the components specified by NLSPATH, then [/XSI]an implementation-defined default path shall be used.

Shade the entire line describing NLSPATH in the ENVIRONMENT VARIABLES section of gencat on P2819, L92796 with the XSI margin code.

Shade the entire line describing NLSPATH in the ENVIRONMENT VARIABLES section of uucp on P3355, L112986 with the XSI margin code.

Shade the entire line describing NLSPATH in the ENVIRONMENT VARIABLES section of uustat on P3366, L113372 with the XSI margin code.

Shade the entire line describing NLSPATH in the ENVIRONMENT VARIABLES section of uux on P3369, L113489 with the XSI margin code.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1514 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Error 2021-08-20 09:46 2021-08-20 09:48
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
3.283 Positional Parameter
79
2267
---
More "positional parameter" problems
Bug 0001491 identified a problem with XCU 2.5.1 Positional Parameters. Searching the standard for other uses of "positional parameter" turned up three more problems:

1. The definition of positional parameter in XBD 3.283 needs to change in line with the change to XCU 2.5.1.

2. There is some text in the shell introduction in XCU that refers to "the positional parameter numbered 0".

3. The m4 rationale uses "positional parameter" when, for consistency with the normative text that it refers to, it should say "argument".
On page 79 line 2267 section 3.283 Positional Parameter, change:
In the shell command language, a parameter denoted by a single digit or one or more digits in curly braces.
to:
In the shell command language, a parameter denoted by a decimal representation of a positive integer.


Cross-volume changes to XCU ...

On page 2345 line 74687 section 2.1 Shell Introduction, change:
as the positional parameter numbered 0
to:
as special parameter 0

On page 2940 line 97373 section m4, change:
the eleventh positional parameter
to:
the eleventh argument

There are no notes attached to this issue.




Viewing Issue Advanced Details
1513 [Issue 8 drafts] Shell and Utilities Objection Omission 2021-08-19 11:05 2021-08-19 11:05
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
make
2934, 2943
98298, 98688, 98689
Changes for make missed when :::= was added
There are three places that were changed for ::= which needed to change again for :::= but were missed in bug 0001471.
On page 2934 line 98298 section make, change:
<tt>[macro[::]=value...]</tt>
to:
<tt>[macro[::[:]]=value...]</tt>

On page 2943 line 98688 section make, change:
before the <tt>=</tt>, <tt>::=</tt>, <tt>?=</tt>, or <tt>+=</tt>
to:
before the <tt>=</tt>, <tt>::=</tt>, <tt>:::=</tt>, <tt>?=</tt>, or <tt>+=</tt>

On page 2943 line 98689 section make, change:
precedes the <tt>::=</tt>, <tt>?=</tt>, or <tt>+=</tt>
to:
precedes the <tt>::=</tt>, <tt>:::=</tt>, <tt>?=</tt>, or <tt>+=</tt>

There are no notes attached to this issue.




Viewing Issue Advanced Details
1512 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-08-12 14:15 2021-08-12 14:16
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
what
3438
116073
---
Exit status 0 for the what utility needs changing
The what utility has a similar problem to the one reported for diff in bug 0001498.
Change:
0 Any matches were found.

1 Otherwise.
to:
0 One or more matches were found and the output specified in STDOUT was successfully written to standard output.

1 No matches were found or an error occurred.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1511 [Issue 8 drafts] Shell and Utilities Editorial Enhancement Request 2021-08-11 16:51 2021-08-12 09:32
kre
 
normal  
New  
Open  
   
Robert Elz
XCU 2.5.3
2315
74583-9
LINENO description claims to be a User Portability Utilities option var, but has no margin tag or shading
The description of the variable LINENO in XCU 2.5.3 concludes with
the sentence:

   This volume of POSIX.1-202x specifies the effects of the variable only
   for systems supporting the User Portability Utilities option.

(lines 74587-9)

but there is no shading, nor a UP margin tag, unlike, for example the
description of PS1 just below (lines 74598 and following) and others.


If anyone feels the need, the description of what LINENO actually does or
how it works is singularly useless, for example, it says (paraphrasing)
"counts lines from 1 within a script or function" which is fine except when
the function is in a script, and it isn't clear if LINENO starts from 1
inside the function, or continues counting from 1 in the script. Nor is
it clear when the "current sequentlial line number" is determined exactly
for example if the script contains (with the echo word being on line 1)

    echo ${LINENO} \
       ${LINENO}
    printf done

One could say that when the echo command is executed, the current sequential
line number is 3, as the shell has read past the \n at the end of the
continuation line of the echo command (but not yet parsed the printf command),
it must have done to know that the echo command has finished, which it needs
to know before it can be executed. But is printing 3 twice useful? And does
any shell actually do that? If not that, should it print 1, or 2 (twice)
or both 1 and 2 (once each) (which is what the NetBSD shell does, where
$LINENO gives the line number on which it appears, and it is very precisely
spefified just what that is for all of the weird cases - including when
LINENO has been exported).

But all of this is a giant can of worms, definitely not just editorial,
and not really part of this report (just documented here for posterity).
Not sure what the correct way to deal with this one is (nor how to specify
what should be done) - that is, it isn't clear if the intent is that LINENO
not be specified as existing at all if UP isn't included, or that it is
intended to exist, but it just isn't specified how it behaves in that case.
Notes
(0005438)
geoffclare   
2021-08-12 09:30   
In Issue 7 (C181) the effect of UP on PS1/2/4 is described the same way as for LINENO. It was changed in the Issue 8 drafts as part of the overhaul of PS1/2/4 in bug 0001006.

Therefore I think LINENO should be updated the same way, and since bug 1006 was tagged tc3-2008 so should this bug.

On 2018 edition page 2352 line 74935 section 2.5.3, add a new first paragraph:
[UP]The processing of the LINENO shell variable shall be supported if the system supports the User Portability Utilities option.[/UP]

On 2018 edition page 2352 line 74939 section 2.5.3, delete:

This volume of POSIX.1-2017 specifies the effects of the variable only for systems supporting the User Portability Utilities option.




Viewing Issue Advanced Details
1510 [Issue 8 drafts] Shell and Utilities Editorial Error 2021-08-11 16:15 2021-08-11 16:15
kre
 
normal  
New  
Open  
   
Robert Elz
XCU 1.14 / make
2954
99212-3
Typo in example in make
Lines 99212-3 in Innue-8 D2 are

    foo: y.tab.o lex.o main.o
       $(CC) $(CFLAGS) -o $@ t.tab.o lex.o main.o

The "t.tab.o" in line 99213 is probably intended to be y.tab.o
(fingers off by one...)
In line 99213 (page 2954) of Issue8 Draft 2, change the word

    t.tab.o

to be

    y.tab.o
There are no notes attached to this issue.




Viewing Issue Advanced Details
1509 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-08-10 15:25 2021-08-10 15:25
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
tty
3323
111896
---
tty EXIT STATUS needs some changes
The tty utility has a similar problem to the one reported for diff in bug 0001498.
Change:
0 Standard input is a terminal.

1 Standard input is not a terminal.
to:
0 Standard input is a terminal, and the output specified in STDOUT was successfully written to standard output.

1 Standard input is not a terminal, and the output specified in STDOUT was successfully written to standard output.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1508 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-08-10 15:17 2021-08-10 15:18
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
stty
3271
110029
---
Exit status 0 for the stty utility needs changing
The stty utility has a similar problem to the one reported for diff in bug 0001498.
Change:
0 The terminal options were read or set successfully.
to:
0 Successful completion.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1507 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-08-10 14:29 2021-08-12 22:51
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
mailx
2964
98259
---
Exit status 0 for mailx needs changing
The description of exit status 0 for the mailx utility assumes that mailx is always used to send one or more messages. In Send Mode that is true, but in Receive Mode it only sends messages if one of the forms of Follow Up, Mail, or Reply commands is used.
Change:
note that this status implies that all messages were sent, but it gives no assurances that any of them were actually delivered.
to:
note that this status implies that any messages that mailx was instructed to send were all successfully sent, but it gives no assurances that any of them were actually delivered.

Notes
(0005441)
steffen   
2021-08-12 22:51   
It is .. difficult.
You know, i personally start an instance of my mailx clone on Monday, and quit it on Saturday (Sunday that is already, mostly).
Should the exit status on Saturday reflect a failed delivery attempt on say Tuesday?

For my clone i added options like "set errexit", an "ignerr" command / "-" command escape prefix (to explicitly ignore errors even if "errexit" is active etc etc.
We also have per-command return/exit status and error numbers.
So if we survive errors, the next command will reset the exit status to 0.

It can be said that we are not compliant to the current state of affairs POSIX defines in at least receive mode, on the other hand.
Maybe i should make any occurrence of the send-error bit persistent, and restore it upon program exit, in at least send mode, and at least in conjunction with $POSIXLY_CORRECT aka "set posix".




Viewing Issue Advanced Details
1506 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-08-10 13:51 2021-08-10 13:51
geoffclare
 
normal  
New  
Open  
   
Geoff Clare
The Open Group
ln
2900
95791
---
Exit status 0 for ln needs changing
The description of exit status 0 for ln assumes that the source_file operands all refer to existing files, which isn't necessarily true if the -s option is specified.
Change:
All the specified files were linked successfully.
to:
Successful completion.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1505 [Issue 8 drafts] Shell and Utilities Comment Omission 2021-08-10 10:14 2021-08-11 10:55
quinq
 
normal  
New  
Open  
   
Quentin Rameau
EXTENDED DESCRIPTION
2935
98369
Make doesn't seem to specify unset macro expansion behaviour
Hello,

I was looking for an actual specification of
what happens when trying to expand an undefined macro.

Intuition would be that it expands to an empty string,
but this doesn't seem to be specified at all,
or maybe I missed that somewhere else.

Could an implementation be allowed
to generate an error on such case (or other behaviour)
without having this defined?
Specify that trying to expand an unset macro
results in an empty string.
Notes
(0005432)
geoffclare   
2021-08-10 11:23   
I agree the standard doesn't say what happens if make tries to expand an unset macro.

However, I'm not sure adding a requirement that it expands to an empty string would be useful. This is because applications cannot rely on any macro being unset just because they have not set it. Under "Default Rules" the standard says "Implementations may provide additional macros and rules." So any macro name the application tries to use without setting might happen to be the name of a macro the implementation provides, and thus could expand to something unexpected.

It would be better to state that the behaviour is unspecified and warn that applications should explicitly set macros to an empty string if they want them to expand to an empty string.
(0005433)
joerg   
2021-08-10 12:43   
We already have:

If the original value of the variable named by string1 is an empty string, the final result shall be an empty string.

in the current draft of Issue 8.
(0005434)
geoffclare   
2021-08-10 13:30   
Re: Note: 0005433 That's an addition related to pattern macro expansions. See https://austingroupbugs.net/view.php?id=519#c1206 [^]

And it has a mistake: "variable" should be "macro"!

We should fix that as part of the resolution of this bug.
(0005435)
joerg   
2021-08-10 13:44   
There is another mistake: it says "word", where np ns with no % in between is ment.
(0005436)
kre   
2021-08-10 14:28   
Re Note: 0005432

   I'm not sure adding a requirement that it expands to an empty string
   would be useful.

From the perspective you're looking at it, you're right, applications
should not depend upon a macro being undefined, and hence necessarily
an empty string when expanded, but that's not the only issue.

   It would be better to state that the behaviour is unspecified

That would be unfortunate, as, as one possible implementation, that could
lead to a situation where an implementation expanded undefined macros as
the name of the macro, rather than as an empty string (or an error).

Then when an application expects a macro to be defined (eg: CC) and does

     $(CC) file.c

in some rule, and the implementation instead of doing the expected invocation
of the C compiler, instead invokes the CC command (which might be anything
at all), that would have potential for very bad things (TM) to happen.

So, I think the standard needs to define how undefined macros are treated
when encountered - either as an error, or an empty string (and I think that
the empty string version is what is actually done in practice, so that one
should win), rather than simply say "unspecified". There can be an
APPLICATION USAGE note telling makefile writers not to depend upon a macro
being undefined, if anyone really believes this is a real problem - I mean,
why could anyone write
   whatever ${FOO} and more
expectig FOO to be undefined, and so to generate
   whatever and more
when they could have just written that in the first place? So I think
this issue most probably isn't real (ie; no-one expects macros to be
undefined) but one often expects them to be defined in situations where
they might in fact not be.
(0005437)
quinq   
2021-08-11 10:55   
Re: 0005432

That is a good point indeed about implementations being authorized to provide additional macros.

The solution of specifying that this is unspecified, and adding a warning note sounds good to me.

Re: 0005436

While I like the idea of having a defined behaviour, the bottom line is “don't do it” so it doesn't really matter there. As you said, it seems that most implementations (?) expand to an empty string, but $(MYTOOL) file could still expand to MYTOOL if an implementation decided to provide that specific macro anyway.

As for the use cases of having empty macros, for example that could happen on variable file inclusions where some macros are defined, or not.
Other cases could happen too, but in the end yes, correct solution is to explicitely set them if they are to be used.




Viewing Issue Advanced Details
1504 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-08-09 08:13 2021-12-02 17:27
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
kill
2881
95031
---
exit status 0 for kill does not cover the -l option
The description of exit status 0 for kill talks only about the pid operand(s) and sending a signal; it does not cover the -l option.
Change:
At least one matching process was found for each pid operand, and the specified signal was successfully processed for at least one matching process.
to:
The -l option was specified and the output specified in STDOUT was successfully written to standard output; or, the -l option was not specified, at least one matching process was found for each pid operand, and the specified signal was successfully processed for at least one matching process.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1503 [Issue 8 drafts] Base Definitions and Headers Editorial Enhancement Request 2021-08-07 20:59 2021-12-02 17:25
andras_farkas
 
normal  
Resolved  
Accepted As Marked  
   
Andras Farkas
math.h
Note: 0005541
Potential new M_ constants for math.h
There may be five useful new M_ constants to be included in the math.h header.

The latest standard for C++, known as C++20, introduces a header called <numbers>, which contains many math constants, and is similar in intent to XSI's M_ constants in math.h. There's lots of overlap, but the list of M_ constants and list of constants in <numbers> do differ.

Constants in both XSI and <numbers>:
M_E, e
M_LOG2E, log2e
M_LOG10E, log10e
M_LN2, ln2
M_LN10, ln10
M_PI, pi
M_1_PI, inv_pi
M_SQRT2, sqrt2

Constants only in XSI:
M_PI_2
M_PI_4
M_2_PI
M_2_SQRTPI
M_SQRT1_2

Constants only in <numbers>:
inv_sqrtpi (1 divided by the square root of pi, approximately 0.5641)
sqrt3 (Square root of 3, approximately 1.7320)
inv_sqrt3 (1 divided by the square root of 3, approximately 0.5773)
egamma (Euler-Mascheroni constant, approximately 0.5772)
phi (golden ratio, approximately 1.6180)

References:
XSI:
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/math.h.html [^]
C++20 draft standard (specifically section 26.9, page 1224):
https://isocpp.org/files/papers/N4860.pdf [^]
P0631R8 Math Constants (The original paper on <numbers>):
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p0631r8.pdf [^]
Notably, this paper references XSI.

I'm not a mathematician, so I'm not sure whether adding those five new constants from <numbers> would be considered useful, or considered clutter. Please chime in.
Personally, I've only ever used M_PI out of all the M_ constants, but I'm glad it's there when I do use it, and I'm glad all the constants are there.
I'd like for people to examine the usefulness of the five new constants. If they're judged useful:
Add to the section on math.h, immediately after the other M_ constants (or interspersed with them), and also XSI-marked:
M_1_SQRTPI
Value of 1/sqrt(pi)
M_SQRT3
Value of sqrt(3)
M_1_SQRT3
Value of 1/sqrt(3)
M_EGAMMA
Value of [lower-case gamma], Euler-Mascheroni constant
M_PHI
Value of [lower-case phi], ( (1 + sqrt(5))/2 ), golden ratio constant

Of course, the "Value of" text would use more fitting symbols than I was able to use here, for the values.
Notes
(0005430)
andras_farkas   
2021-08-07 21:32   
Actually, there could be some changes to my desired action proposal.
M_1_SQRT3 should certainly be M_SQRT1_3 in order to match M_SQRT1_2
M_1_SQRTPI could also be M_SQRT1_PI to match.
(0005431)
andras_farkas   
2021-08-07 21:41   
Thus, desiring to be consistent with the standard as it already exists, this is my new desired action:
Add to the section on math.h, immediately after the other M_ constants (or interspersed with them), and also XSI-marked:
M_SQRT1_PI
Value of 1/sqrt(pi)
M_SQRT3
Value of sqrt(3)
M_SQRT1_3
Value of 1/sqrt(3)
M_EGAMMA
Value of [lower-case gamma], Euler-Mascheroni constant
M_PHI
Value of [lower-case phi], ( (1 + sqrt(5))/2 ), golden ratio constant
(0005541)
geoffclare   
2021-12-02 17:24   
(edited on: 2021-12-02 17:39)
Add the requested new M_ symbolic constants to <math.h> using the names in Note: 0005431 in an order determined by the editor all with XSI shading.
Also add XSI shading to (2018 edition) P288, L9745-9746 and extend the shading down to L9759 (as a single shading block).





Viewing Issue Advanced Details
1502 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-08-06 15:10 2021-12-02 16:52
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
grep
2844
93706
---
Exit status 0 for the grep utility needs changing
The grep utility has a similar problem to the one reported for diff in bug 0001498.
Change:
One or more lines were selected.
to:
One or more lines were selected and the output specified in STDOUT was successfully written to standard output.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1501 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-08-06 14:40 2021-12-02 16:49
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
find
2801
92107-92108
---
find EXIT STATUS needs some changes
The find utility has a similar problem to the one reported for diff in bug 0001498.

In addition, the EXIT STATUS section does not reflect the effect of "-exec ... {} +" on find's exit status.
Change:
0 All path operands were traversed successfully.

>0 An error occurred.
to:
0 All path operands were traversed successfully, the output (if any) specified in STDOUT was successfully written to standard output, and all commands (if any) executed using the -exec primary punctuated by a <plus-sign> exited with exit status 0.

>0 A command executed using the -exec primary punctuated by a <plus-sign> exited with non-zero status, or an error occurred.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1500 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-08-06 13:00 2021-12-02 16:34
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
expr
2775
91106
---
expr EXIT STATUS needs some changes
The expr utility has a similar problem to the one reported for diff in bug 0001498.

In addition, exit status 2 should say that it is an error (so that a diagnostic message is required).
Change:
0 The expression evaluates to neither null nor zero.

1 The expression evaluates to null or zero.

2 Invalid expression.

>2 An error occurred.
to:
0 The expression evaluated to neither null nor zero, and the output specified in STDOUT was successfully written to standard output.

1 The expression evaluated to null or zero, and the output specified in STDOUT was successfully written to standard output.

2 Invalid expression error.

>2 Another error occurred.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1499 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-08-06 11:17 2021-12-02 16:31
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
1.4 Utility Description Defaults
2341, 2342
74553, 74558, 74561, 74563
---
Use of "non-zero" in 1.4 under CONSEQUENCES OF ERRORS is not specific enough
The CONSEQUENCES OF ERRORS subsection of 1.4 has rules that specify non-zero exit status, but this is not specific enough in the case of utilities for which there is a non-zero exit status that does not indicate an error occurred, such as diff, grep, and test.

If the utility is one for which the STDERR default rule comes into play, there is a chain of reasoning (see bug 0001498) that can be followed to deduce a more specific requirement, but it would be preferable for it to be stated directly.
On page 2341 line 74553,74558,74561, change:
exit status returned shall be non-zero
to:
exit status returned shall be one that indicates an error occurred

On page 2342 line 74563, change:
the utility shall exit with a non-zero exit status
to:
the utility shall exit with an exit status that indicates an error occurred

There are no notes attached to this issue.




Viewing Issue Advanced Details
1498 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-08-06 11:14 2021-12-02 16:27
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
diff
2663
86756
---
diff exit status when an error occurs while writing the differences
The EXIT STATUS section for diff includes:
1 Differences were found.

>1 An error occurred.
If differences are found but an error occurs when the differences are being written to standard output, then both of these descriptions are true, and the standard does not specify a precedence between them, which means that at first sight conforming implementations can exit with status 1 or >1.

When the rules from 1.4 Utility Description Defaults are brought into play, it is apparent that the exit status has to be >1, but it would be better if it was easily seen from just the diff EXIT STATUS section rather than having to follow this chain of reasoning:
1.4 under CONSEQUENCES OF ERRORS says "A diagnostic message shall be written to standard error whenever an error condition occurs."

diff STDERR says "The standard error shall be used only for diagnostic messages."

1.4 under STDERR says "When this section is listed as ``The standard error shall be used only for diagnostic messages.'', it means that, unless otherwise stated, the diagnostic messages shall be sent to the standard error only when the exit status indicates that an error occurred and the utility is used as described by this volume of POSIX.1-2017.

Therefore if an error occurs when writing the differences a diagnostic message has to be written, and because a diagnostic message was written the exit status has to be one that indicates an error occurred (if the utility is used as described...).

Change:
Differences were found.
to:
Differences were found and all differences were successfully output.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1497 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-08-05 14:25 2021-12-02 16:24
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
dd
2647
86120
---
Exit status 0 for the dd utility needs changing
The description of exit status 0 for dd doesn't cover usages where it does something other than just copy the input file.
Change:
The input file was copied successfully.
to:
Successful completion.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1496 [1003.1(2013)/Issue7+TC1] System Interfaces Comment Enhancement Request 2021-08-01 17:09 2021-12-05 12:48
alanc
 
normal  
New  
Open  
   
Alan Coopersmith
Oracle Solaris Engineering
printf()
(page or range of pages)
(Line or range of lines)
---
Add asprintf() and vasprintf()
The GNU C library has provided asprintf() and vasprintf() since at least 1991,
and many other OS'es have added them as well since. These variants of sprintf()
allocate new memory for the string, making it easier to have a safe usage
without having to re-implement much of the printf format string handling logic
to determine how much memory to allocate. These have been widely adopted in
software trying to avoid buffer overflows: codesearch.debian.net finds 1713
software packages referencing asprintf().

GNU C library documentation:
https://www.gnu.org/software/libc/manual/html_node/Dynamic-Output.html [^]

FreeBSD man page (added in 2.2 in 1997):
https://www.freebsd.org/cgi/man.cgi?query=asprintf [^]

OpenBSD man page (added in 2.3 in 1998):
https://man.openbsd.org/asprintf.3 [^]

NetBSD man page:
https://man.netbsd.org/asprintf.3 [^]

Solaris man page (added in Solaris 10 Update 10 in 2011):
https://docs.oracle.com/cd/E88353_01/html/E37843/asprintf-3c.html [^]

illumos man page:
https://illumos.org/man/3c/asprintf [^]
These functions should be added to the standard:

#include <stdio.h>

int asprintf(char **restrict strp, const char *restrict fmt, ...);
int vasprintf(char **restrict strp, const char *restrict fmt,
              va_list ap);
tr24731-2.pdf (79 KB) 2021-12-02 16:17
Notes
(0005427)
jsm28   
2021-08-02 16:31   
Note that these functions (and corresponding aswprintf, vaswprintf) are in ISO/IEC TR 24731-2:2010, so care should be taken to be compatible with the versions there and to raise any issues with the interfaces through the liaison with WG14.

Note also the known error handling issue (it's unspecified whether *ptr is set to NULL on error or left unchanged - apparently this is deliberate in TR 24731-2 to be compatible with multiple existing implementations that make different choices there).
(0005428)
alanc   
2021-08-03 19:19   
Thanks, I was unaware that asprintf() had been specified in that ISO TR.

Of the above listed implementations, all but GNU libc document that they will
set *ptr to NULL on error, while GNU libc explicitly states that you cannot
rely on it to do so (and looking at its code, sometimes it does, sometimes
it doesn't).
(0005540)
nick   
2021-12-02 16:19   
I have attached the final draft of TR24731-2, and since this is an ISO document, it is sponsored by ISO.
(0005543)
nick   
2021-12-05 12:48   
(edited on: 2021-12-05 12:50)
Add pointer page for asprintf referencing fprintf.
Add pointer page for vasprintf referencing vprintf.

On p909 line 30737 add asprintf to the title line.

On P 909, before line 30740, add
[CX]<tt>int asprintf(char ** restrict ptr, const char * restrict format, ...);</tt>

(the next line, dprintf() is already CX shaded, so this extends the shading backwards).

On line 30747, change
Excluding dprintf( ):

to
Excluding asprintf() and dprintf():


Before line 30754 insert a new paragraph, CX shaded:

The asprintf() function behaves as sprintf(),
except that the output string is written to
dynamically allocated memory, allocated as if by a call to malloc(), of sufficient
length to hold the resulting string, including a terminating null byte.
If the allocation is successful, the address of this dynamically allocated
string shall be stored in the location referenced by ptr.


At page 915 line 31014, change
Upon successful completion, the [CX]dprintf(),[/CX]

to
Upon successful completion, the [CX]asprintf(), dprintf(),[/CX]


Before line 31016, Add a new paragraph, CX shaded:

Upon successful completion, the asprintf() function shall return the number of bytes written to
the allocated string stored in ptr>, excluding the terminating null byte.


At line 31021 append a sentence CX shaded:

For asprintf(), if memory allocation was not possible, or if some other
error occurs, the function shall return a negative value, and the contents of
ptr are undefined.


Before line 31032 add (nb, 31032 is not CX shaded, so unclear as to shading for this):

The asprintf() function shall fail if:

[ENOMEM] Insufficient storage space is available.



On page 2212, line 70651 add vasprintf to the title line.

Insert before line 70655 (extending the CX shading backwards a line)
[CX]
<tt>int vasprintf(char ** restrict ptr, const char * restrict format, va_list arg);</tt>


At line 70666, change

The [CX}vdprintf()[/CX], ...

to

The [CX}vasprintf(), vdprintf()[/CX], ...


Though the ISO TR includes aswprintf, no current implementations appear to
provide this, so it is not included here. Additionally, such functionality
is available using open_wmemstream, so invention here is really unecessary.





Viewing Issue Advanced Details
1495 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-07-30 15:24 2021-12-02 16:08
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
at
2479
79612
---
Exit status 0 for the at utility needs rewriting
The exit status 0 description for the at utility has two problems:

1. It does not cover the case where "at -l" with no at_job_id operands writes no output because there are no jobs to list.

2. It implies that if two or more at_job_id operands are specified, there only needs to be one valid ID among them for the exit status to be 0.

The suggested new text for exit status 0 is rather long-winded. Depending on the outcome of bug 0001492, an acceptable alternative might be to change it to just "Successful completion".

The CONSEQUENCES OF ERRORS section also has a problem in that it refers to "the job" (singular), but with -l there can be zero or more jobs, and with -r there can be one or more jobs. The usual rule about continuing with later operands should apply for these.
On page 2479 line 79612 section at, change:
The at utility successfully submitted, removed, or listed a job or jobs.
to:
Neither the -l option nor the -r option was specified and a job was successfully submitted; or, the -l option was specified with no at_job_id operands and there were no jobs to be listed; or, the -l option was specified and all job listings were successfully output; or, the -r option was specified and all of the specified jobs were successfully removed.

On page 2479 line 79615 section at, change:
The job shall not be scheduled, removed, or listed.
to:
If neither the -l option nor the -r option was specified, the job shall not be scheduled. Otherwise, the default actions specified in [xref to 1.4 Utility Description Defaults] apply.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1494 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2021-07-30 09:38 2021-12-02 16:06
andras_farkas
 
normal  
Resolved  
Accepted  
   
Andras Farkas
tee
3285
110524
---
Missing parenthesis in tee.html
On this page:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/tee.html [^]
The following text is missing an end parenthesis:
If the -a option is not specified, output files shall be written (see File Read, Write, and Creation.
Add a closing parenthesis immediately in front of the period ending the sentence.
Notes
(0005425)
geoffclare   
2021-07-30 09:44   
It is also missing in the PDF, so I am moving this bug to the Issue7+TC2 project.




Viewing Issue Advanced Details
1493 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Enhancement Request 2021-07-29 14:33 2021-11-18 17:30
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
2.7
2360
75306
---
move XCU 2.7 definition of "file descriptor" into XBD 3
XCU 2.7 Redirection has a definition of "file descriptor" that should be incorporated into the definition in XBD chapter 3 instead of being tucked away there.

Also, the definitions of standard error/input/output currently say they are streams, but they are also used in XCU (and perhaps elsewhere) to refer to file descriptors.
On page 60 line 1783 section 3.166 File Descriptor, after:
A per-process unique, non-negative integer used to identify an open file for the purpose of file access.
append:
The values 0, 1, and 2 have special meaning and conventional uses, and are referred to as standard input, standard output, and standard error, respectively. Programs usually take their input from standard input, and write output on standard output. Diagnostic messages are usually written on standard error.

On page 92 line 2575 section 3.366 Standard Error, change:
An output stream usually intended to be used for diagnostic messages.
to:
In the context of file descriptors (see [xref to 3.166 File Descriptor]), file descriptor number 2.

In the context of standard I/O streams (see [xref to XSH 2.5]), an output stream usually intended to be used for diagnostic messages, and accessed using the global variable stderr.

Note: The file descriptor underlying stderr is initially 2, but it can be changed by freopen() to 0 or 1 (and implementations may have extensions that allow it to be changed to other numbers). Therefore, writing to the standard error stream does not always produce output on the standard error file descriptor.

On page 92 line 2577 section 3.367 Standard Input, change:
An input stream usually intended to be used for primary data input.
to:
In the context of file descriptors (see [xref to 3.166 File Descriptor]), file descriptor number 0.

In the context of standard I/O streams (see [xref to XSH 2.5]), an input stream usually intended to be used for primary data input, and accessed using the global variable stdin.

Note: The file descriptor underlying stdin is initially 0; this cannot change through the use of interfaces defined in this standard, but implementations may have extensions that allow it to be changed. Therefore, in conforming applications using extensions, reading from the standard input stream does not always obtain input from the standard input file descriptor.

On page 92 line 2579 section 3.368 Standard Output, change:
An output stream usually intended to be used for primary data output.
to:
In the context of file descriptors (see [xref to 3.166 File Descriptor]), file descriptor number 1.

In the context of standard I/O streams (see [xref to XSH 2.5]), an output stream usually intended to be used for primary data output, and accessed using the global variable stdout.

Note: The file descriptor underlying stdout is initially 1, but it can be changed by freopen() to 0 (and implementations may have extensions that allow it to be changed to other numbers). Therefore, writing to the standard output stream does not always produce output on the standard output file descriptor.

On page 2360 line 75294 section 2.7 Redirection, change:
The number n is an optional decimal number designating the file descriptor number; the application shall ensure it is delimited from any preceding text and immediately precede the redirection operator redir-op.
to:
The number n is an optional one or more digit decimal number designating the file descriptor number; the application shall ensure it is delimited from any preceding text and immediately precedes the redirection operator redir-op (with no intervening <blank> characters allowed).

On page 2360 line 75304 section 2.7 Redirection, change:
Open files are represented by decimal numbers starting with zero. The largest possible value is implementation-defined; however, all implementations shall support at least 0 to 9, inclusive, for use by the application. These numbers are called "file descriptors". The values 0, 1, and 2 have special meaning and conventional uses and are implied by certain redirection operations; they are referred to as standard input, standard output, and standard error, respectively. Programs usually take their input from standard input, and write output on standard output. Error messages are usually written on standard error. The redirection operators can be preceded by one or more digits (with no intervening <blank> characters allowed) to designate the file descriptor number.
to:
The largest file descriptor number supported in shell redirections is implementation-defined; however, all implementations shall support at least 0 to 9, inclusive, for use by the application.

Notes
(0005422)
kre   
2021-07-29 16:49   
I have no problem with this change in general, though there are a
couple of wording changes (just cleanups) I'll suggest in some later
note if no-one else gets there first, but this part:

   The file descriptor underlying stdin is initially 0; this cannot
   change through the use of interfaces defined in this standard,

(and the similar limitations expresseed for stdout and stderr) looks
to be simply wrong to me. I'm also not sure there's any need to
actually make this point, even if it were true.

But consider

     close(fileno(stdin));
     fd = open("/dev/null", 0);
     freopen("/my/file", "r", stdin);

(use fdopen() instead of open() if you prefer, but not fclose() as
any use of any stream after it has been fclose'd is undefined, as its
data struct may have been discarded).

I'm not sure what file descriptor you expect freopen() to assign in
that case (assuming the open of /my/file succeeds, etc, of course),
but it isn't usually going to be 0, and this is using only interfaces
defined by the standard, I believe.

Note that freopen() specifically says:

    If pathname is not a null pointer, freopen() shall close any file
    descriptor associated with stream. Failure to close the file descriptor
    successfully shall be ignored.

That is, freopen() is defined to work properly if the file descriptor has
already been closed (some systems may have other reasons for the close failing,
allowing for a different fd to be returned from the open, but that would be
using something beyond the standard interfaces I think).
(0005424)
geoffclare   
2021-07-30 08:53   
Re: Note: 0005422 Given the code:

     close(fileno(stdin));
     fd = open("/dev/null", O_RDONLY);
     freopen("/my/file", "r", stdin);

and assuming fileno(stdin) is 0 (which is the case the proposed text is referring to), freopen() will close fd 0 and then open "/my/file" on fd 0. The fd number underlying stdin will not change.

I included the notes for standard error and standard output as I thought it worth pointing out that "standard output" can refer to two different files depending on whether you mean the file descriptor or the stream. I have encountered people in the past who thought they were always the same, i.e. that freopen() preserved the fd number (if possible) instead of opening the lowest available. I included a similar note for standard input for consistency.
(0005426)
kre   
2021-07-30 15:05   
Re Note: 0005424 -- yes, of course, there's no way to get rid of the 0 from
fileno(stdin), so the close() in freopen() always does close(0) so the open
the succeeds it will always return 0.

Still not sure it is worth the notes though, if anything more complex than
to note that there's no guarantee that fileno(stdXX) is the sand as standard
XXput (or err).

Saying what values they must have will just lead to people (as I did) trying
to demonstrate that it isn't necessarily correct.




Viewing Issue Advanced Details
1492 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-07-29 13:53 2021-11-18 17:24
geoffclare
 
normal  
Resolved  
Accepted As Marked  
   
Geoff Clare
The Open Group
1.4 Utility Description Defaults
2341
74538
---
See Note: 0005439.
clarify what "successful completion" means in EXIT STATUS
There are some utilities for which the EXIT STATUS section describes exit status 0 as "Successful completion", but the utility is required to perform more than one action, meaning it is unclear whether all of those actions must succeed in order for the exit status to be 0.

The suggested change is to require that it means all actions must succeed, but also to stop using "Successful completion" for some utilities where this would not be appropriate.
On page 2341 line 74538 section 1.4 Utility Description Defaults, add to the end of EXIT STATUS:
Default Behavior: When the description of exit status 0 is ``Successful completion'', it means that exit status 0 shall indicate that all of the actions the utility is required to perform were completed successfully.

On page 2870 line 94579 section jobs, after:
When jobs reports the termination status of a job, the shell shall remove its process ID from the list of those ``known in the current shell execution environment''; see [xref to 2.9.3.1].
add:
If a write error occurs when jobs writes to standard output, some process IDs might have been removed from the list but not successfully reported.

On page 2872 line 94663 section jobs, change:
0 Successful completion
to:
0 The output specified in STDOUT was successfully written to standard output.

On page 2873 line 94704 section jobs, add to RATIONALE:
If jobs uses buffered writes to standard output, a write error could be detected when attempting to flush a buffer containing multiple reports of terminated jobs, resulting in some unreported jobs having their process IDs removed from the list of those known in the current shell execution environment (because they were removed when the report was added to the buffer).

On page 2982 line 99046 section make (EXIT STATUS with -q), change:
0 Successful completion

1 The target was not up-to-date.
to:
0 All specified targets were already up-to-date.

1 One or more targets were not up-to-date.

On page 2983 line 99050 section make (EXIT STATUS without -q), change:
0 Successful completion

>0 An error occurred.
to:
0 All specified targets were already up-to-date, or all commands executed to bring targets up-to-date either exited with status 0 or had a non-zero exit status that was specified (via the -i option, the special target .IGNORE, or a '-' command prefix) to be ignored.

>0 An error occurred, or at least one command executed to bring a target up-to-date exited with a non-zero exit status that was not specified to be ignored.

Notes
(0005439)
geoffclare   
2021-08-12 11:04   
I have noticed that the EXIT STATUS section for uniq uses a different wording for what amounts to the same thing as "Successful completion". In order for the updated default requirement to apply to uniq, it needs to use exactly matching wording. Therefore, in addition to the desired action we should make the following change:

On page 3347 line 112747 section uniq, change:
The utility executed successfully.
to:
Successful completion.




Viewing Issue Advanced Details
1491 [Issue 8 drafts] Shell and Utilities Editorial Error 2021-07-28 17:00 2021-08-16 13:38
mirabilos
 
normal  
Applied  
Accepted As Marked  
   
mirabilos
Positional Parameters
2312
74465-74471
See Note: 0005429
$0 ≠ ${00} ?
There’s a probable oversight in:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_05_01 [^]

   A positional parameter is a parameter denoted by the decimal value represented by one or more digits,
   other than the single digit 0. The digits denoting the positional parameters shall always be interpreted
   as a decimal value, even if there is a leading zero. When a positional parameter with more than one
   digit is specified, the application shall enclose the digits in braces (see [47]Parameter Expansion).

This reads as $0 or ${0} is not a positional parameter (single digit 0)
but ${00} is.
Change…

    other than the single digit 0

… to:

    other than the number 0
Notes
(0005418)
shware_systems   
2021-07-28 17:33   
I'd prefer "that evaluate to a positive value other than zero" instead. Then the ${00} case is the same, functionally, as ${0}, I'd think, for evaluation purposes. Use of "number" I see as leaving open implementations allowing other number formats as extensions.

Also, it is one or more leading zeros, not just a single leading one, that gets ignored for establishing the final value.
(0005419)
kre   
2021-07-28 22:26   
Why is this in "online pubs" which is mostly (I believe) intended for
errors in the conversion to HTML format?

While it would be tempting to define ${00} to mean the same as $0 (or ${0})
I am not sure we can, as currently you're right, that's not the way the spec
reads, and yash implemented things according to the spec... There's no way
to set the 0'th numeric parameter, so ${00} there is always empty, but it isn't
the same as $0.

I think that means that the updated spec needs to say that ${0...0} gives
unspecified results (any number of 0's > 1)- which is OK, as there's no
rational reason to ever write such an expansion.

So, perhaps what is needed is something a long the lines of

    If the value 0 is written with more than one 0 character, the
    results of the expansion are unspecified, this is neither a special
    parameter nor a numeric parameter (nor a variable).

Re Note: 0005418 - you're right, it is "one or more leading zeroes" that are
irrelevant (except for $0 of course) - but the way the spec is written now
is fine - the point there is that a leading 0 does not make the value be
interpreted as octal (or hex if it was 0x - not that that is possible since
only digits are allowed), and for that it is only the first leading zero that
is material, in any context either 00000013 is octal, or it is decimal,
in exactly the same circumstances as 013 is - the extra 0's are irrelevant.
(0005420)
Don Cragun   
2021-07-28 23:17   
Was filed against Online pubs with URL https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_05_01 [^] but this is not an issue about publication problems. Moved to Issue 8 drafts with page and lines numbers from Issue 8 draft 2.
(0005429)
Don Cragun   
2021-08-05 15:29   
On page 2312 change lines 74465-74466 from:

    
A positional parameter is a parameter denoted by the decimal value represented by one or more digits, other than the single digit 0. The digits denoting the positional parameters shall always be interpreted as a decimal value, even if there is a leading zero. When a positional parameter with more than one digit is specified, the application shall enclose the digits in braces (see Section 2.6.2). Positional parameters are initially assigned...


to:

    
A positional parameter is a parameter denoted by a decimal representation of a positive integer. The digits denoting the positional parameters shall always be interpreted as a decimal value, even if there is a leading zero. When a positional parameter with more than one digit is specified, the application shall enclose the digits in braces (see Section 2.6.2).


    Examples:


        
  • "$8", "${8}", "${08}", "${008}", etc. all expand to the value of the eighth positional parameter.


  •     
  • "${10}" expands to the value of the tenth positional parameter.


  •     
  • "$10" expands to the value of the first positional parameter followed by the character '0'.


  •     



    <small>Note: 0 is a special parameter, not a positional parameter, and therefore the results of expanding ${00} are unspecified.</small>


    Positional parameters are initially assigned...



On page 2318 change lines 74733-74736 from:

    
If the parameter is not enclosed in braces, and is a name, the expansion shall use the longest valid name (see XBD Section 3.207), whether or not the variable represented by that name exists. Otherwise, the parameter is a single-character symbol, and behavior is unspecified if that character is neither a digit nor one of the special parameters (see Section 2.5.2).


to:

    
For a parameter that is not enclosed in braces:

    


        
  • If the parameter is a name, the expansion shall use the longest valid name (see XBD Section 3.207), whether or not the variable denoted by that name exists.


  •     
  • Otherwise, the parameter is a single-character symbol, and behavior is unspecified if that character is neither a digit nor one of the special parameters (see Section 2.5.2).


  •     




Viewing Issue Advanced Details
1490 [1003.1(2016/18)/Issue7+TC2] System Interfaces Comment Omission 2021-07-13 14:41 2021-11-18 17:19
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
exit()
796
27080
---
warn app writers about flush errors not being detectable by exit()
In light of the recent discovery that many implementations of standard utilities have an ancient bug whereby they leave buffer flushing of stdout to be done by exit() when exiting with a status that indicates successful completion, and thus do not handle write errors properly, it would be a good idea to give some advice about this in the exit() APPLICATION USAGE section to try and help application writers avoid falling into the same trap.
Change APPLICATION USAGE from "None" to:
When a stream that has unwritten buffered data is flushed by exit() there is no way for the calling process to discover whether or not exit() successfully wrote the data to the underlying file descriptor. Therefore, it is strongly recommended that applications always ensure there is no unwritten buffered data in any stream when calling exit(), or returning from the initial call to main(), with a status value that indicates no errors occurred.

For example, the following code demonstrates one way to ensure that stdout has already been successfully flushed before calling exit() with status 0. If the flush fails, the file descriptor underlying stdout is closed so that exit() will not try to repeat the failed write operation. If the flush succeeds, a final check with ferror() is performed to ensure that there were no write errors during earlier flush operations (that were not handled at the time).
int status = 0;
if (fflush(stdout) != 0) {
    perror("appname: standard output");
    close(fileno(stdout));
    status = 1;
}
else if (ferror(stdout)) {
    fputs("appname: write error on standard output\n", stderr);
    status = 1;
}
exit(status);

Change EXAMPLES from "None" to:
See APPLICATION USAGE.

(This is before the APPLICATION USAGE section, so putting the example
code here would mean it is before the explanation.)
There are no notes attached to this issue.




Viewing Issue Advanced Details
1489 [Issue 8 drafts] System Interfaces Editorial Error 2021-07-12 16:09 2021-08-16 11:15
rhansen
 
normal  
Applied  
Accepted  
   
Richard Hansen
malloc RATIONALE
1272
42599
malloc RATIONALE awkward wording
I find "since when" a bit difficult to read, and "described there" to be a bit misleading.
On page 1272 lines 42599-42600 (malloc RATIONALE), change:
Section 2.3 (on page 471) permits this behavior, since when multiple error conditions described there are simultaneously true there is no precedence between them.
to:
Section 2.3 (on page 471) permits this behavior; when multiple error conditions are simultaneously true there is no precedence between them.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1488 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Error 2021-07-12 09:31 2021-12-02 16:04
geoffclare
 
normal  
Resolved  
Accepted As Marked  
   
Geoff Clare
The Open Group
pwd
3131
104830
---
See Note: 0005538.
pwd RATIONALE does not account for ENOSPC
The part of the pwd RATIONALE that talks about partial output does not take into account the possibility of an ENOSPC condition causing a partial write.

The normative text in CONSEQUENCES OF ERRORS would also benefit from a small change, as the current text can be interpreted as requiring pwd to travel back in time and change what it has already done. Since time travel isn't possible, such an interpretation is obviously incorrect, but it is worth updating the text to stop readers trying to interpret it that way.
On page 3131 line 104819 section pwd, change:
If an error is detected, output shall not be written to standard output, a diagnostic message shall be written to standard error, and the exit status is not zero.
to:
If an error is detected other than a write error when writing to standard output, no output shall be written to standard output, a diagnostic message shall be written to standard error, and the exit status shall be non-zero.

On page 3131 line 104830 section pwd, change:
In most utilities, if an error occurs, partial output may be written to standard output. This does not happen in historical implementations of pwd. Because pwd is frequently used in historical shell scripts without checking the exit status, it is important that the historical behavior is required here; therefore, the CONSEQUENCES OF ERRORS section specifically disallows any partial output being written to standard output.
to:
In most utilities, if an error occurs, partial output may be written to standard output. This does not happen in historical implementations of pwd (unless an [ENOSPC] condition causes a partial write). Because pwd is frequently used in historical shell scripts without checking the exit status, it is important that the historical behavior is required here; therefore, the CONSEQUENCES OF ERRORS section specifically disallows any partial output being written to standard output, except when a write error occurs when writing to standard output.

Notes
(0005537)
geoffclare   
2021-11-25 14:45   
Reopening, as it was pointed out on the mailing list that the text should not single out ENOSPC; there are other error conditions that can cause a partial write. New proposed text will follow.
(0005538)
geoffclare   
2021-11-25 14:47   
On page 3131 line 104819 section pwd, change:
If an error is detected, output shall not be written to standard output, a diagnostic message shall be written to standard error, and the exit status is not zero.
to:
If an error is detected other than a write error when writing to standard output, no output shall be written to standard output, a diagnostic message shall be written to standard error, and the exit status shall be non-zero.

On page 3131 line 104830 section pwd, change:
In most utilities, if an error occurs, partial output may be written to standard output. This does not happen in historical implementations of pwd. Because pwd is frequently used in historical shell scripts without checking the exit status, it is important that the historical behavior is required here; therefore, the CONSEQUENCES OF ERRORS section specifically disallows any partial output being written to standard output.
to:
In most utilities, if an error occurs, partial output may be written to standard output. This does not happen in historical implementations of pwd (unless an error condition causes a partial write). Because pwd is frequently used in historical shell scripts without checking the exit status, it is important that the historical behavior is required here; therefore, the CONSEQUENCES OF ERRORS section specifically disallows any partial output being written to standard output, except when a write error occurs when writing to standard output.




Viewing Issue Advanced Details
1487 [Issue 8 drafts] Shell and Utilities Objection Omission 2021-07-01 13:47 2021-07-20 09:34
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
rm
3113
105337
rm -v addition needs an extra change in EXIT STATUS
The addition of the rm -v option should be taken into account in the EXIT STATUS section, where it describes what needs to have succeeded in order for the exit status to be 0.
After:
Each directory entry was successfully removed, unless its removal was canceled by a non-affirmative response to a prompt for confirmation.
add:
In addition, if the -v option is specified, information about each removed directory entry was successfully written to standard output.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1485 [Issue 8 drafts] System Interfaces Editorial Enhancement Request 2021-06-12 14:08 2021-07-02 11:12
mikecrowe
 
normal  
Applied  
Accepted As Marked  
   
Mike Crowe
pthread_cond_clockwait
1563
51671-51684
See Note: 0005385
It may be clearer to describe pthread_cond_timedwait in terms of pthread_cond_clockwait rather than the other way round
When I wrote my proposed text for issue 1216, I described the new pthread_*_clock* functions in terms of their existing _timed* equivalents. During the drafting of issue 8, various further document changes were made that made the pthread_*_clock* functions the primary ones referenced from elsewhere. I agree with this change.

However, the actual description of the behaviour of pthread_cond_clockwait is still written in terms of pthread_cond_timedwait. I think it would be clearer to reverse this too. It's easier to explain the clock as a parameter to pthread_cond_clockwait and then go on to explain that the parameter is not present and where the clock comes from for pthread_cond_timedwait.
Perhaps the affected lines could be replaced with something like:

Replace sentences starting at line 51671 with:

The pthread_clock_clockwait() function shall be equivalent to pthread_cond_wait(), except that an error is returned i the absolute time specified by abstime as measured against the clock indicated clock_id passes (that is, the current time measured by that clock equals or exceeds abstime) before the condition cond is signaled or broadcasted, or if the absolute time specified by abstime has already been passed at the time of the call. Implementations shall support passing CLOCK_REALTIME and CLOCK_MONOTONIC to pthread_cond_clockwait() as the clock_id argument. When such timeouts occur...[keep final sentence from existing paragraph.] The pthread_cond_clockwait() function is also a cancellation point.

Remove paragraph starting at line 51677. Its information is now contained in the updated version of the previous and next paragraphs.

Replace paragraph starting at line 51680 with:

The pthread_cond_timedwait() function shall be equivalent to pthread_cond_clockwait(), except that it lacks the clock_id argument. The clock to measure abstime against instead comes from the condition variable's clock attribute as set by pthread_condattr_setclock prior to its creation.

(Perhaps even add words to the effect of "If no clock attribute has been set then the default is CLOCK_REALTIME", to avoid readers having to refer to pthread_condattr_setclock section.)
Notes
(0005385)
nick   
2021-06-17 15:57   

Replace paragraph on lines 51671-51676 with:

The pthread_cond_clockwait() function shall be equivalent to pthread_cond_wait(), except that an error is returned if the absolute time specified by abstime as measured against the clock indicated by clock_id passes (that is, the current time measured by that clock equals or exceeds abstime) before the condition cond is signaled or broadcasted, or if the absolute time specified by abstime has already been passed at the time of the call. Implementations shall support passing CLOCK_REALTIME and CLOCK_MONOTONIC to pthread_cond_clockwait() as the clock_id argument. When such timeouts occur...[keep final sentence from existing paragraph.]


Remove paragraph starting at line 51677.

Replace paragraph starting at line 51680 with:

The pthread_cond_timedwait() function shall be equivalent to pthread_cond_clockwait(), except that it lacks the clock_id argument. The clock to measure abstime against shall instead come from the condition variable's clock attribute which can be set by pthread_condattr_setclock() prior to the condition variable's creation. If no clock attribute has been set, the default shall be CLOCK_REALTIME.




Viewing Issue Advanced Details
1484 [Issue 8 drafts] Base Definitions and Headers Editorial Omission 2021-06-09 16:19 2021-06-22 08:59
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
<termios.h>
416
14345
<termios.h> SEE ALSO is missing tcgetwinsize and tcsetwinsize
The tcgetwinsize() and tcsetwinsize() functions are missing in the SEE ALSO section of <termios.h>.
On page 416 line 14345 add tcgetwinsize() and tcsetwinsize() to the SEE ALSO section of <termios.h>.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1483 [Issue 8 drafts] System Interfaces Editorial Enhancement Request 2021-06-03 14:59 2021-06-22 08:56
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
dup
742
25482
Flags for dup3 are now also defined in <unistd.h>
The dup3() DESCRIPTION mentions that the O_CLOEXEC and O_CLOFORK flags are defined in <fcntl.h>. That is correct but in Issue 8 they are now also defined <unistd.h> which is where dup3() is declared. Therefore this remark is no longer needed.
On page 742 line 25482 delete
(defined in <fcntl.h>)
There are no notes attached to this issue.




Viewing Issue Advanced Details
1482 [Issue 8 drafts] System Interfaces Editorial Enhancement Request 2021-06-03 14:54 2021-06-22 08:54
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
bind
605
21140-21141
bind EILSEQ error only applies to AF_UNIX sockets
The ERRORS section of bind() lists errors that are specific to AF_UNIX sockets separately from errors that can happen with any socket. However the EILSEQ error can only happen with AF_UNIX sockets but is listed with the generic errors instead of with the AF_UNIX specific errors.
On page 605 move lines 21140-21141
[EILSEQ]
The last pathname component is not a portable filename, and cannot be
created in the target directory.
to just before the EIO error on line 21156.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1481 [Issue 8 drafts] Base Definitions and Headers Editorial Error 2021-06-03 14:50 2021-06-14 10:05
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
<netinet/in.h>
299
10404-10425
<netinet/in.h> IP6 shadings ends too early
The IP6 shading in <netinet/in.h> ends mid-sentence several paragraphs too early.
On page 299 expand the IP6 shading that begins on line 10393 so that it ends on line 10425.
Notes
(0005377)
geoffclare   
2021-06-04 11:21   
I have tracked this down to a bug in the shading macros and have pushed a fix to the sus_tools project in gitlab.

However, to use the fixed macros to build draft 2.1 will mean creating a new docker image and there will be other macro changes needed after draft 2.1 (e.g. C17 alignment needs a new margin code), so it may be preferable to use a work-around in the netinet_in.h troff source for now.
(0005378)
geoffclare   
2021-06-14 10:05   
The docker image has been updated and the latest build in the Issue8 branch does not have the problem, so I'm resolving this bug as Applied.




Viewing Issue Advanced Details
1479 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-05-28 09:14 2021-11-18 17:12
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
make
2980
98946
---
Not clear what "achieve results" means for make defaults
The description of the make utility says "The default rules for make shall achieve results that are the same as if the following were used" and then goes on to specify, among other things, default values for some macros.

It is not clear what is intended by "achieve results". In particular if these results include the output written to standard output by make when it executes a shell command, then the only way to achieve the same results would be for the defaults to be exactly those that are specified in the standard. However, just excluding that output would not fix the problem because a command such as "echo $(CFLAGS) > outfile" would produce different contents in outfile if CFLAGS does not have exactly the specified default value.

A quick check of the Solaris and GNU make implementations reveals the following differences from the defaults specified in the standard:

Both have ARFLAGS=rv (instead of ARFLAGS=-rv)
Both have CC=cc (instead of CC=c99)
Both have CFLAGS= (empty instead of CFLAGS=-O 1)
GNU has SCCSGETFLAGS= (empty instead of SCCSGETFLAGS=-s)

The proposed change would allow the first three but not the GNU SCCSGETFLAGS value, which hopefully the GNU make maintainer will not object to changing.
Change:
The default rules for make shall achieve results that are the same as if the following were used.
to:
The default rules and macro values for make shall achieve results that are the same as if the following were used, except that where a result includes the literal value of a macro, this value may differ.

Notes
(0005366)
psmith   
2021-05-28 12:56   
I have no problem with changing SCCSGETFLAGS.

I would like to point out that "CFLAGS=-O 1" is (IMO) a bug in the standard and this should be "CFLAGS=-O1" (no space). Some compilers (GCC, for example) do not allow space between "-O" and its argument.

For portability, the standard should not require CFLAGS to be set to a non-portable value by default.

$ touch /tmp/foo.c
$ cc -O1 -c /tmp/foo.c
$ cc -O 1 -c /tmp/foo.c
cc: error: 1: No such file or directory
(0005367)
joerg   
2021-05-28 13:23   
(edited on: 2021-05-28 13:32)
ARFLAGS= without a leading dash is historic practice that still works
on all ar implementations I am aware of.

I am not shure whether any compiler supports -O 1 with a space.
This seems to be a definite text formatting bug in the standard.

Regarding CC=c99, if everybody believes this is what should be used,
given that SunPro Make in it's current configuration decides how to
behave at runtime, when called as /usr/xpg4/bin/make, it would be
possible to let it read /usr/share/lib/make/posix.make.rules instead of
/usr/share/lib/make/make.rules and change that file to define CC=c99.

Should this be done? Reading posix.make.rules in that case would be
three additional lines of code.

(0005368)
geoffclare   
2021-05-28 13:33   
I think the standard has -O 1 with a space because conforming applications are required to pass the -O and the 1 as separate arguments to c99 (see XBD 12.1 item 2a). The c99 utility is required to accept them either separately or together.

An implementation can of course set the default CFLAGS to -O1 without the space under the "shall achieve results that are the same" wording that is the subject of this bug.

Setting the default CC to cc instead of c99 is fine on a particular implementation provided a conforming application compiled with cc behaves the same as if it was compiled with c99. (Remember that a conforming application must define _POSIX_C_SOURCE=200809 or _XOPEN_SOURCE=700.)
(0005369)
psmith   
2021-05-28 13:53   
Actually, GNU make already has conforming values of CC, SCCSGETFLAGS, and ARFLAGS if you set .POSIX:

$ cat > Makefile
.POSIX:
all: ; @echo $(CC) / $(SCCSGETFLAGS) / $(ARFLAGS)

$ make
c99 / -s / -rv
(0005370)
joerg   
2021-05-28 13:57   
I just checked Sun Studio c99 and you seem to be correct. -O 1 is
accepted.
(0005371)
geoffclare   
2021-05-28 15:02   
Re Note: 0005369 My bad for doing my checks without a .POSIX target.

I have re-checked Solaris with .POSIX, and ARFLAGS changes to -rv but CC and CFLAGS are still the same.
(0005372)
joerg   
2021-05-28 15:14   
Yes, this is line 3029 from file main.cc...

I did not know that. It seems to overwrite things at runtime
and it may be the only way to do this since .POSIX may be seen
after /usr/share/lib/make/make.rules has been read.

That block could be enhanced to also overwrite CC and CFLAGS.
(0005373)
psmith   
2021-05-28 16:10   
FWIW, GNU make won't change CFLAGS to "-O 1" even in POSIX mode. GCC (run as c99) will fail if that is done and in this case the real world must prevail over small conformance issues :).

Someone can mention this (not accepting "-O 1") to the GCC folks but I would be surprised if they felt that conformance in this situation was worth the effort I suspect would be involved with making it work (but I could be wrong, it could be easy). Nevertheless there are enough c99's out there where it fails today, that it's not feasible to change it in GNU make.

I understand that this is not being asked.




Viewing Issue Advanced Details
1478 [Issue 8 drafts] Shell and Utilities Editorial Clarification Requested 2021-05-23 16:12 2021-07-20 09:33
stephane
 
normal  
Applied  
Accepted As Marked  
   
Stéphane Chazelas
2.6.2 Parameter Expansion
2318-2320
Note: 0005389
clarify ${@?error}, ${@+set}, "${@+set}", "${*+set}"... etc. expansions
While the spec does leave expansions of ${@#pattern} (and variants with ##, %, %%) unspecified, or that $@/$* expansion shall not cause the shell to exit when the nounset option is set (without going as far as saying that $@ / $* should be considered always set), it leaves it quite unclear what the expansion of

${@?error} "${@?error}", ${@+"$@" more}, "${*-foo}", "${@:+foo}", ${*+foo} should yield.

In practice, there is a lot of variation in behaviour between implementations, with some implementations considering $@/$* "set" always (like for set -u) or only if $# > 0. Or "$*" "non-empty" only if $1 is non empty or some only if "$*" would yield a non-empty string (with variations in behaviour if $IFS is empty), etc. For some ${@?error} is a syntax error, etc.

Some shells with array support try to be consistent with their handling of $@ compared to that of other arrays, but in different ways as their arrays take different shapes, etc.

The only portable variants I've managed to find were: "${*-foo}" and "${*+foo}" which expand to foo (resp. "$*") if and only if $# == 0, which is not among the most useful ones.
Leaving the expansion of all those - + ? :- :+ :? operators unspecified for the @ and * parameters seems like the least bad option.

Alternatives are likely to either make things more confusing or break some shells backward compatibility.

Users can always spell things out like:

if [ "$#" -eq 0 ]; then
  ...
else
  ...
fi


or if:

if [ -n "$*" ]; then
  ...
else
  ...
fi


Notes
(0005365)
stephane   
2021-05-23 16:16   
For the record, this came to my attention at

https://unix.stackexchange.com/questions/650208/issue-processing-multiple-files-counting-lines/650590#comment1220610_650590 [^]

Where someone posted this:

#!/bin/sh -

wc -l -- "${@?expected to get filename argument(s)}"


code which in practice is not portable, and for which I'm not sure what the POSIX spec currently says.
(0005389)
geoffclare   
2021-06-21 09:37   
(edited on: 2021-06-21 09:38)
Suggested change:

On page 2318 line 74742 section 2.6.2 append a sentence to the paragraph:
If parameter is '*' or '@', the result of the expansion is unspecified.






Viewing Issue Advanced Details
1477 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Error 2021-05-20 11:16 2021-11-18 17:07
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
7.1, 8.2
135, 176
3967, 5768
---
Consequences of specifying locale categories with different charsets
XBD 7.1 says:
If different character sets are used by the locale categories, the results achieved by an application utilizing these categories are undefined. Likewise, if different codesets are used for the data being processed by interfaces whose behavior is dependent on the current locale, or the codeset is different from the codeset assumed when the locale was created, the result is also undefined.

XBD 8.2 says:
If these variables specify locale categories that are not based upon the same underlying codeset, the results are unspecified.

There are several problems with these statements:

1. They say different things. I suggest the statement in 8.2 should be replaced by a cross-reference to 7.1, since the latter is more precise.

2. The second sentence of the 7.1 statement contradicts requirements elsewhere, e.g. it says the behaviour is undefined in situations where individual function descriptions require them to return EILSEQ errors. It's also not clear what the point of referring to "the codeset assumed when the locale was created" is.

3. The statement in 7.1 uses "current locale" but should be inclusive of the *_l() functions.

4. Both statements are overly restrictive. If I set:

LANG=en_US.utf8
LC_TIME=POSIX

on a system where the codeset for the POSIX locale is ASCII and the codeset for en_US.utf8 is UTF-8, all of the characters used in the LC_TIME locale data exist, with the same encoding, in the codeset used for LC_CTYPE (via LANG), so there is no reason for the behaviour to be unspecified/undefined.
On page 135 line 3967 section 7.1, change:
If different character sets are used by the locale categories, the results achieved by an application utilizing these categories are undefined. Likewise, if different codesets are used for the data being processed by interfaces whose behavior is dependent on the current locale, or the codeset is different from the codeset assumed when the locale was created, the result is also undefined.
to:
If incompatible character sets are used by the locale categories, the results achieved by an application utilizing these categories are undefined. Two locale categories have incompatible character sets if one of the categories is LC_CTYPE and the locale data associated with the other category includes at least one character that either is not in the character set used by LC_CTYPE or has a different encoding than the same character in the character set used by LC_CTYPE.

Likewise, unless specified otherwise, if different codesets are used by a particular category of the selected locale and by the data being processed by an interface whose behavior is dependent on that category of the selected locale, the results are undefined.

On page 176 line 5768 section 8.2, change:
If these variables specify locale categories that are not based upon the same underlying codeset, the results are unspecified.
to:
See [xref to Section 7.1] for the consequences of setting these variables to locales with different character sets.

Move the following paragraph from page 3537 line 119926 section A.8.2
to after page 3525 line 119377 section A.7.1:
The locale settings of individual categories cannot be truly independent and still guarantee correct results. For example, when collating two strings, characters must first be extracted from each string (governed by LC_CTYPE) before being mapped to collating elements (governed by LC_COLLATE) for comparison. That is, if LC_CTYPE is causing parsing according to the rules of a large, multi-byte code set (potentially returning 20 000 or more distinct character codeset values), but LC_COLLATE is set to handle only an 8-bit codeset with 256 distinct characters, meaningful results are obviously impossible.
and add the following new paragraph after it:
Earlier versions of this standard stated that if different character sets are used by the locale categories, the results achieved by an application utilizing these categories are undefined. This was felt to be overly restrictive. For example, when setting:
LANG=en_US.utf8
LC_TIME=POSIX
on a system where the codeset for the POSIX locale is ASCII and the codeset for en_US.utf8 is UTF-8, all of the characters used in the LC_TIME locale data exist, with the same encoding, in the codeset used for LC_CTYPE (via LANG), so there is no reason for the behavior to be undefined in this case. This standard now has more precise requirements in this area.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1476 [Issue 8 drafts] Shell and Utilities Editorial Error 2021-05-19 12:21 2021-07-02 12:18
kre
 
normal  
Applied  
Accepted As Marked  
   
Robert Elz
XCU 2.14 / printf
3086
104321
Note: 0005390
Missing '+' character in the second example explanation
printf(1) contains an example

    printf "%d\n" 3 +3 -3 \'3 \"+3 "'-3"

which for the first two lines output, is stated to produce

    3 Numeric value of constant 3
    3 Numeric value of constant 3

The second of those should be

    3 Numeric value of constant +3

to match the command line given.
In line 104321 on page 3086 (XCU 2.14, printf, EXAMPLES)
(in issue 8, draft 2) change:

Numeric value of constant 3


to:

Numeric value of constant +3



As an optional extra (not a bug) it might be worth adding one more
constant to the command line (or two) and showing the output from
it (or them) as well, where the added command line arg(s) would be
hex &/or octal representations (0x1C or 0254 or whatever you like).
Notes
(0005388)
geoffclare   
2021-06-21 09:26   
(edited on: 2021-06-21 09:27)
Suggested changes...

After line 104302 on page 3085 (XCU 2.14, printf, EXAMPLES), add:
The command:
printf '%d\n' 10 010 0x10
produces:
10
8
16

If the implementation supports floating-point conversions, the command:
printf '%.2f\n' 10 010 0x10 10.1e2 010.1e2 0x10.1p2
produces:
10.00
10.00
16.00
1010.00
1010.00
64.25

In line 104321 on page 3086 (XCU 2.14, printf, EXAMPLES), change:
Numeric value of constant 3
to:
Numeric value of constant +3


(0005390)
geoffclare   
2021-06-24 16:17   
(edited on: 2021-06-24 16:26)
After line 104302 on page 3085 (XCU 2.14, printf, EXAMPLES), add:
The command:
printf '%d\n' 10 010 0x10
produces:
<table>
<tr><th>Output Line</th><th>Explanation</th></tr>
<tr><td><tt>10</tt></td><td>Decimal representation of the numeric value of decimal integer constant 10</td></tr>
<tr><td><tt>8</tt></td><td>Decimal representation of the numeric value of octal integer constant 010</td></tr>
<tr><td><tt>16</tt></td><td>Decimal representation of the numeric value of hexadecimal integer constant 0x10</td></tr>
</table>

If the implementation supports floating-point conversions, the command:
LC_ALL=C printf '%.2f\n' 10 010 0x10 10.1e2 010.1e2 0x10.1p2
produces:
<table>
<tr><th>Output Line</th><th>Explanation</th></tr>
<tr><td><tt>10.00</tt></td><td>The string "10" interpreted as a decimal value, with 2 digits of precision.</td></tr>
<tr><td><tt>10.00</tt></td><td>The string "010" interpreted as a decimal (not octal) value, with 2 digits of precision.</td></tr>
<tr><td><tt>16.00</tt></td><td>The string "0x10" interpreted as a hexadecimal value, with 2 digits of precision.</td></tr>
<tr><td><tt>1010.00</tt></td><td>The string "10.1e2" interpreted as a decimal floating-point value, with 2 digits of precision.</td></tr>
<tr><td><tt>1010.00</tt></td><td>The string "010.1e2" interpreted as a decimal (not octal) floating-point value, with 2 digits of precision.</td></tr>
<tr><td><tt>64.25</tt></td><td>The string "0x10.1p2" interpreted as a hexadecimal floating-point value, with 2 digits of precision.</td></tr>
</table>

On page 3086 replace lines 104317-104325 with:
In a locale that uses a codeset based on ISO/IEC 646: 1991, the command:
printf "%d\n" 3 +3 -3 \'3 \"+3 "'-3"
produces:
<table>
<tr><th>Output Line</th><th>Explanation</th></tr>
<tr><td><tt>3</tt></td><td>Decimal representation of the numeric value 3</td></tr>
<tr><td><tt>3</tt></td><td>Decimal representation of the numeric value +3</td></tr>
<tr><td><tt>-3</tt></td><td>Decimal representation of the numeric value -3</td></tr>
<tr><td><tt>51</tt></td><td>Decimal representation of the numeric value of the character <tt>'3'</tt> in the ISO/IEC 646: 1991 standard codeset</td></tr>
<tr><td><tt>43</tt></td><td>Decimal representation of the numeric value of the character <tt>'+'</tt> in the ISO/IEC 646: 1991 standard codeset</td></tr>
<tr><td><tt>45</tt></td><td>Decimal representation of the numeric value of the character <tt>'-'</tt> in the ISO/IEC 646: 1991 standard codeset</td></tr>
</table>

In the paragraph at line 104326, move the second sentence ("Since the last two ...") to before the first sentence ("Note that in a locale ...").





Viewing Issue Advanced Details
1475 [Issue 8 drafts] System Interfaces Editorial Clarification Requested 2021-05-19 01:36 2021-06-22 08:52
cmsmith
 
normal  
Applied  
Accepted  
   
Curtis Smith
socket(), shutdown()
1898, 1953
61830-61837, 63555
socket() and shutdown() should refer to close()
The socket() function description does not mention that the file descriptor returned should eventually be closed with the close() function. The see-also section mentions several related functions such as shutdown() but omits close(), and a reader may wrongly conclude that the proper way to relinquish the resources obtained through socket() is to merely call shutdown() as the shutdown() description makes no mention of close() as well.
In the shutdown() Application Usage, line 61830, replace "None" with "The file descriptor remains open after shutdown() returns to the calling application."

In the "See Also" sections for shutdown() and socket(), add a reference to the close() function.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1474 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2021-05-18 14:17 2021-11-18 17:02
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
strsignal()
2069
66341
---
strsignal() NAME section is misleading
The NAME section on the strsignal() page says "get name of signal".

This is misleading, as what is returned is not what most readers would think of as the "name" of a signal (which is what the new sig2str() function in Issue 8 draft 2 returns). It should be changed to be consistent with sterror(), which has "get error message string". The use of "message" would also match the text in the strsignal() description that says it uses "the same set of messages as the psignal() function."
Change:
get name of signal
to:
get signal message string

There are no notes attached to this issue.




Viewing Issue Advanced Details
1473 [Issue 8 drafts] Base Definitions and Headers Comment Omission 2021-05-18 11:17 2021-06-22 08:50
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
<unistd.h>
440
15384
Add Issue 7 info to <unistd.h> APPLICATION USAGE
The <unistd.h> APPLICATION USAGE section has some historical information about _POSIX_VERSION values etc. for old versions. Information about Issue 7 should be added to it.

Also, the current text refers to old POSIX standards in two different ways: "POSIX.1-yyyy" and "ISO POSIX-1: yyyy". For consistency these should be changed to match. I have proposed using POSIX.1-yyyy as there was no ISO publication of POSIX.1-2017.
On page 440 line 15376 section <unistd.h>, change:
ISO POSIX-1: 1996 standard
to:
POSIX.1-1996 standard

On page 440 line 15382 section <unistd.h>, change:
ISO POSIX-1: 2001 and ...
to:
POSIX.1-2001 and ...

On page 440 line 15384 section <unistd.h>, add:
POSIX.1-2008, POSIX.1-2017, and Single UNIX Specification, Version 4
_POSIX_VERSION == 200809L, plus (if the XSI option is supported) _XOPEN_UNIX and _XOPEN_VERSION == 700
Note that _POSIX_VERSION did not change in POSIX.1-2017 as it was technically identical to POSIX.1-2008 with its two technical corrigenda applied.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1472 [Issue 8 drafts] System Interfaces Editorial Error 2021-05-18 09:39 2021-06-22 08:47
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
pthread_mutex_clocklock(), pthread_rwlock_clockrdlock(), pthread_rwlock_clockwrlock()
1608, 1653-1656
(several)
Some new *_clock*() pages need s/this/these/
When the *_clock*() functions were added, some pages went from having 1 function to more than 1, but still use "this function" in some places.
On page 1608 line 53159 section pthread_mutex_clocklock(),
page 1653 line 54525 section pthread_rwlock_clockrdlock(), and
page 1655 line 54595 section pthread_rwlock_clockwrlock(), change:
This function shall not return an error code of [EINTR].
to:
These functions shall not return an error code of [EINTR].

On page 1653 line 54509 section pthread_rwlock_clockrdlock(), and
page 1655 line 54581 section pthread_rwlock_clockwrlock(), change:
The results are undefined if this function is called ...
to:
The results are undefined if these functions are called ...

On page 1654 line 54529 section pthread_rwlock_clockrdlock(), and
page 1656 line 54599 section pthread_rwlock_clockwrlock(), change:
Applications using this function ...
to:
Applications using these functions ...

There are no notes attached to this issue.




Viewing Issue Advanced Details
1471 [Issue 8 drafts] Shell and Utilities Editorial Enhancement Request 2021-05-16 12:15 2021-08-16 11:13
joerg
 
normal  
Applied  
Accepted As Marked  
   
Jörg Schilling
make
2888-
97001-
See Note: 0005440.
Add an orthogonal interface for immediate macro expansion definitions to make
GNU make introduced a ::= macro assignment where the value is first expanded and then assigned.

This has pitfalls, as it at the same time introduces a non-predictive behavior of the (since 35 years existing) += operator.

In order to allow predictive behavior, two new operators should be introduced:

macro:::=value

and

macro +:=value

The first (:::=) causes an assignment with immediate expansion but is creating the same macro type as reated with macro=value. Macros created with macro:::=value are delayed appended with macro +=value as usual.

The second (+:=) causes an append assignment with immediate expansion, regardless of whether the macro has been created by macro=value, macro::=value or macro:::=value before.

This feature is implemented by SunPro Make and smake since a while, see the current SunPro Man Page at:

http://schilytools.sourceforge.net/man/man1/make.1s.html [^]

and the current smake man page at:

http://schilytools.sourceforge.net/man/man1/smake.1.html [^]
On page 2888 line 97001 change

macro=value
macro::=value

to

macro=value
macro::=value
macro:::=value

After page 2894 line 97263 add:

The following form defines a normal delayed-expansion macro (replacing any previous definition of the macro named by string1):

string1 :::= [string2]

by immediately expanding string2 before assigning the value.

After page 2895 line 97282 add:

The following form (the append form) appends additional text to the value of a macro:

string1 +:= [string2]

When using this append form:

- If the macro named by string1 does not exist, this form shall be equivalent to the immediate-expansion form

  string1 :::= [string2]

- If the macro named by string1 exists, then a <space> or <tab> character followed by the evaluation of string2 shall be appended to the value currently assigned to the macro named by string1.
Notes
(0005356)
shware_systems   
2021-05-16 14:35   
Personally, I'd rather see ":+=" than ":::=", as keeping operators under 4 chars long, harder to typo, and more mnemonic of being related to "+:=" in intent. While this would be a breaking change for the relatively few SunPro make files that use the 4 char form now, it'll be clearer for users of other makes that add this.
(0005357)
psmith   
2021-05-16 17:18   
I don't see a need for these new operators to be standardized. GNU make has implemented the immediate/delayed assignment operators more or less as they are described in the standard today for over 25 years and there haven't been any requests that I recall to add more capabilities such as the ones described here.
(0005358)
shware_systems   
2021-05-16 19:02   
On the phone calls where this was originally brought up cases were brought up where assignment expressions executed later could have different results depending on the initial assignment. This could break boilerplate makefiles user provided makefiles were expected to reference, resulting in a much larger documentation requirement for those files to let end users know those variables were sensitive. The addition of these allows such boilerplate to be insensitive to which type of original assignment happened, and this was considered of enough future usability Jeorg was asked to put this ER together.
(0005359)
kre   
2021-05-16 19:12   
Something existing only in SunPro make (and smake, which is more or less
a derivative) is clearly not the standard.

For any of this to be adopted, it would need to have been implemented by
at least a couple of other relatively major make implementations.

It doesn't matter in the slightest what problems it might solve, this group
is not a legislature, and should never be attempting to force implementations
into something that their users are not demanding.
(0005360)
shware_systems   
2021-05-16 19:49   
As something addressing possible portability issues I believe additions like this are in scope for the standard. If you can point to where they are explicitly excluded from consideration that I've missed somehow I'll be hapy to look at that. The criteria I'm aware of for such additions is that it has been implemented at least once as proof of concept, not that all known implementations have to have adopted it. These now have 2 implementations.
(0005361)
joerg   
2021-05-16 21:26   
(edited on: 2021-05-16 21:27)
Re: Note: 0005359

smake is much older than SunPro Make and I am very sure that
SunPro Make is not based on smake.

So we have two completely independent implementations that implement
the proposal with different implementation problems. Also note that
Mark already mentioned that a single implementation is sufficient to prevent
this proposal from being a POSIX invention. Finally the GNU make ::=
addition was implemented by one make variant only and is (in constrast
to this proposal) really hard to implement in other makes.

Adding this proposal to a cleanly written make takes approx. 2-4 hours,
so be sure whether you like to spend more time in arguing than it
would take to implement.

BTW: The number of halfway POSIX compliant make implementations is
very modest.

(0005363)
psmith   
2021-05-22 19:02   
Regardless of code provenance it's hard to consider SunPro make and smake "independent implementations" when one person is maintaining and updating both of them. These new operators have been available for about two months now: I don't see how it's possible that any users, other than the maintainers perhaps, have had a chance to use even the original ::= much less come to the conclusion that there's a problem and decide they need new alternatives.

I won't argue that "::=" was implemented by one make variant. However, the capability was available and proven in GNU make for decades, not months, and was heavily used in almost all GNU make-specific makefiles; additionally the concept of an immediately expanded variable is simple and powerful and something that many users want; that may be why it was adopted.

In contrast, these new assignment operators seem (to me) to be esoteric and rarely needed. As I mentioned above, I don't recall any requests for anything like these new operators from the GNU make userbase in the last 25+ years.

Regarding amount of work to implement these new operators: it might be true that in terms of the bare code it's not excessively difficult (I'm not sure but I have no reason to disagree offhand). However just because it's not a lot of work doesn't mean it belongs in the standard.

Also consider that in addition to the code changes there is more work that needs to be done:

There will have to be a suite of regression tests added to GNU make verify these new operators behave with all the existing operators and in all contexts.

These new operators have to be documented. GNU make provides a full user's manual, in addition to the man page, which is expected to include thorough details of their behavior, clear explanations of how they differ from the other operators, as well as practical examples of when and how these operators might be used. Simply adding the text above won't be sufficient.

Finally, GNU make has other facilities beyond POSIX, and these new operators will have to be tested with them as well; target- and pattern-specific variable assignments at least certainly need to be considered.


Fundamentally, I'm not convinced that there's any need for these operators to appear in the standard.
(0005393)
geoffclare   
2021-07-08 16:43   
During the July 8, 2021 teleconference Joerg claimed that the proposed behaviour for :::= is the same as the behaviour of := in BSD make. It is surprising that this has not been mentioned before in this bug, so can someone with access to BSD make confirm this?

I.e. does this:

printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' | make -f -

output 1, or does it output $(A) like GNU make?
(0005394)
hvd   
2021-07-08 16:57   
> I.e. does this:

> printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' | make -f -

> output 1, or does it output $(A) like GNU make?

With bmake (http://crufty.net/help/sjg/bmake.html, [^] a portable version of NetBSD make), this outputs 1.
(0005395)
steffen   
2021-07-08 19:30   
#?0|kent:toolbox.git$ printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' | make -f -
echo '$(A)'
$(A)
#?0|kent:toolbox.git$ printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' | smake -f -
smake: WARNING: Nonportable ':=' assignment found for macro 'B' in line 2 col 4 of 'Standard in'
...echo '1'
1
#?0|kent:toolbox.git$ printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' | bmake -f -
echo '1'
1
(0005396)
steffen   
2021-07-08 19:32   
P.S.: bmake is also used on FreeBSD. Output with the newest version under Linux.
(0005397)
kre   
2021-07-08 20:01   
On actual NetBSD, both an oldish NetBSD 8 (from Jan 2018)
and an (almost) current HEAD

jinx$ printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' | make -sf -
$(A)

netbsd# printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' | make -sf -
$(A)

(I added the -s to supress the echo command itself). I am not sure why
Simon's bmake appears to be different, I will ask him. But I suspect
the answer will be the same as the last time I asked something like this
(paraphrasing) "bmake doesn't get updated very often".

When I look on NetBSD 6 vintage systems (circa 2013) ...

munnari$ printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' | make -sf -
1

So this looks to be something that was changed sometime between 2013
and 2018.
(0005398)
steffen   
2021-07-08 20:21   
.. i had not seen hvd's posting at first.
But here, no, this is the very very current bmake that has been imported into FreeBSD about two weeks ago or so. I package it for a Linux distribution.

t$ prt-get info bmake
Name: bmake
Path: /usr/ports/contrib
Version: 20210621
Release: 1
Description: NetBSD make program
URL: http://www.crufty.net/help/sjg/bmake.html [^]
Maintainer: Steffen Nurpmeso, steffen at sdaoden dot eu

P.S.: the portable branch requires two tests to be disabled on Linux. Interesting problem, the standard is explicit and says signals which were not ignored upon program start _only_. Which i deem problematic for SIGINT for example, given that bmake at least supports interrupt handlers etc. Let's see what the future brings. The standard is like this, however.
(0005399)
steffen   
2021-07-08 20:34   
P.S.: to be explicit, the tests have to be disabled only if running in an unshare(1) based container "as in"

  /usr/bin/unshare --ipc --uts --pid --fork make

because that seems to ignore SIGINT++.
(0005400)
kre   
2021-07-08 21:15   
Simon says (just had to start like that!) :

  The answer is "it depends" ;-)

  Tranditionally it would have output 1
  and still does if .MAKE.SAVE_DOLLARS=no

  The behavior was changed a few years ago (to be more
  compatible with GNU make ;-), so now it outputs $(A) by
  default. I added .MAKE.SAVE_DOLLARS so that the traditional behavior
  (which my dirdeps.mk depends on) can be selected.

  In recent bmake it all depends on the value of .MAKE.SAVE_DOLLARS
  there are a couple of *.mk files in bmake that set .MAKE.SAVE_DOLLARS=no

He also said he would take a look at this issue (I sent him its URL) but
perhaps not for a few days.
(0005401)
steffen   
2021-07-08 22:33   
Yes.

$ printf 'A = 1\nB := $$(A)\nall:\n\techo '\''$(B)'\''\n' |
  bmake .MAKE.SAVE_DOLLARS=yes -f -
echo '$(A)'
$(A)
You have new mail in /var/mail/steffen

And it is nicely documented also

    .MAKE.SAVE_DOLLARS
                     value should be a boolean that controls whether ‘$$’ are
                     preserved when doing ‘:=’ assignments. The default is
                     false, for backwards compatibility. Set to true for com‐
                     patability with other makes. If set to false, ‘$$’
                     becomes ‘$’ per normal evaluation rules.

Well i did not know (read the manual!), but i also will not change the defaults for the package i maintain.
(0005402)
steffen   
2021-07-08 22:33   
Thanks, kre.
(0005403)
joerg   
2021-07-10 12:30   
Re: Note: 0005393

Geoff, your test script is unfortunately not sufficient to investigate
and understand the problem caused by the deviating behavior implemented
in GNU make.

This is because there are two deviations in GNU make compared to other
make implementations and your script only tests for one of them. To test
for the other deviation, you could use this script:

  printf 'A = immediate\nB := b\nB += $A\nA = delayed\nall:\n\techo '\''$(B)'\''\n' | $MAKE -s -f -

smake prints:
smake: WARNING: Nonportable ':=' assignment found for macro 'B' in line 2 col 4 of 'Standard in'
b delayed

bmake prints:
b delayed

and gmake prints:
b immediate

As you see, smake and bmake fully agree and the deviating behavior from
gmake never should have made it into the standard. In special: If the
gmake documentation would have clearly pointed to it's deviating
behavior, I would have protested against 0000330 When 0000330 was
discussed in the teleconference. During that teleconference, I was
assuming, gmake was aligned with the behavior of other make
implementations like smake and bmake.

Note that the change in bmake regarding the expansion of := macros
at use time (see your test script) has been introduced after 2015, which
is years after 0000330 was closed with a wrong decision.

Also note that the tool people from netbsd yesterday clearly judged that
the deviating gmake behavior for += tested with the script from this
bugnote is seen as a bug by them.
(0005404)
psmith   
2021-07-10 18:03   
I reject the assertion that GNU make's documentation doesn't clearly define the behavior of "+=" in the context of immediately expanded variables. There's an entire section of the manual devoted to this, with examples:

  https://www.gnu.org/software/make/manual/html_node/Appending.html [^]

I also don't agree that this behavior wasn't well-described by the request to add ::= and += to the standard. I can't speak to what was or wasn't discussed in the teleconference but the change text is clear:

> The following form (the "append" form) modifies a macro by appending additional text to it:
> string1 += [string2]
>
> After the append form:
> ...
> * If the macro exists and is an immediately-expanded macro, it appends a space and the
> evaluation of string2 to the value currently assigned to string1.
> ...

and this text survived more or less intact through to the final wording. There was nothing hidden about this.


Regarding this proposal, I am ambivalent on +:=. One reason I'm ambivalent is that in 25+ years of maintaining GNU make I don't ever recall anyone wanting the ability to add immediately expanded text to a deferred expansion variable. The other reason is that this same behavior can already be had, although it requires using two variables:

This:

  D = a
  D +:= $(VAR)

can be written:

  D = a
  I ::= $(VAR)
  D += $I

If it's really the case that you always want this behavior then that's annoying. But as I said, no one has ever asked for this so I question whether it's really something people need a lot. If people use it only occasionally then the two-variable version might even be preferred as more clearly understandable.

Another reason for my ambivalence is that these do exactly the same thing:

  A ::= b
  A += $(VAR)

and

  A ::= b
  A +:= $(VAR)

which makes the latter seem redundant to me.

The :::= operator seems wholly unnecessary. We have two types of variables and we have two assignment operators for them, which is sufficient. If there is a case to be made that putting an already-expanded value into a delayed expansion macro is useful then this:

  A :::= $(VAR)

can be replaced by:

  A =
  A +:= $(VAR)

or using the current POSIX standard with no changes:

  A =
  I ::= $(VAR)
  A += $I
(0005405)
mirabilos   
2021-07-10 21:26   
On MirMake (inherited from OpenBSD make), it outputs 1.
(0005406)
joerg   
2021-07-15 14:51   
(edited on: 2021-07-15 14:52)
In 37+ years of maintaining smake, I did never receive a request to implement
the := behavior from GNU make and no other make variant did implement that
behavior.

I therefore seriously doubt that there is a need for that GNU make feature.

(0005407)
joerg   
2021-07-17 10:35   
(edited on: 2021-07-22 15:13)
As a result from the last teleconference, I decided that things
could only be understood, if all aspects of the behavior are
tested and documented.

There are four aspects with the various := implementations. Only
three of them may be tested by a makefile:

The first row is related to the behavior of recent bmake versions
and lets you see whether $$ is "expanded" to $ at the right side
of a := assignment, or whether $$ is forwarded as $$.
WARNING: The observed behavior here may also be influenced from
whether the make macro is expanded at use time or not, see gmake
notes below. So this test is only significant while comparing
different bmake versions.

The second row checks whether := results in an immediate expansion
of the right side before assigning the result to the macro. All
make implementations behave the same here, so this test is only
present for completeness.

The third row checks whether += appends using immediate expansion
or whether the appended text is expanded delayed at use time.
This is where GNU make is incompatible with all other make
implementations, as gmake behaves different with += if the macro
has been initially created with :=.

printf 'A = immediate\nB := $$A $A\nB += $A\nA = delayed\nall:\n\techo '\''$(B)'\''\n' | smake -s -f -

prints:
smake: WARNING: Nonportable ':=' assignment found for macro 'B' in line 2 col 4 of 'Standard in'
delayed immediate delayed

printf 'A = immediate\nB := $$A $A\nB += $A\nA = delayed\nall:\n\techo '\''$(B)'\''\n' | bmake-20150505 -s -f -

prints:
delayed immediate delayed

printf 'A = immediate\nB := $$A $A\nB += $A\nA = delayed\nall:\n\techo '\''$(B)'\''\n' | bmake-2020-04-03 -s -f -

prints:
$A immediate delayed

printf 'A = immediate\nB := $$A $A\nB += $A\nA = delayed\nall:\n\techo '\''$(B)'\''\n' | bmake-2021-06-21 -s -f -

prints:
delayed immediate delayed

printf 'A = immediate\nB := $$A $A\nB += $A\nA = delayed\nall:\n\techo '\''$(B)'\''\n' | gmake -s -f -

prints:
$A immediate immediate

What is not testable by this test but by looking at the internal
values of the variable via -p is whether make macros created via
:= are expanded at use time. Only gmake does not expand such macros
at use time. All other make implementation from above expand such
macros at use time.

As you see, the recent change in bmake, related to $$ expansion,
does definitely not result in gmake compatibility.

But the most recent bmake is by default compatible to the bmake
from 2015 again and it seems that at least for the last 20 years,
smake and bmake did implement identical behavior.

(0005408)
psmith   
2021-07-18 03:22   
> In 37+ years of maintaining smake, I did never receive a request to implement the := behavior from GNU make

In that case we can conclude is that this difference in behavior is of little or no concern to users, since they have not requested that either version be changed to support the other. That certainly doesn't *strengthen* the argument that we need to modify the standard to provide both models.

> There are four aspects with the various := implementations.

I'm sorry but I didn't understand the goal of these examples. They certainly do show that the behavior of := is different among different implementations, which is no surprise (at least not to me): that's why the := operator was NOT added to the standard and ::= was added instead. If BSD make users want the BSD make behavior of :=, they have := and they can continue to use it. If they want to change it in some way, for some reason, with or without an option, again that's up to the BSD make maintainers. Why is that a discussion for the POSIX committee?

This issue is proposing two NEW make operators: why should these new operators be added to the standard? The difference in behavior from the current set of available operators is obscure (as we've heard, no user has requested that either version of the behavior be changed) and there are ways to achieve the behavior these new operators provide using features available in the current POSIX standard, as shown above.


I've also read through the etherpad notes from 2021-07-15 and there seems to be confusion and false information there. In particular, it is not true that GNU make is not conforming to POSIX with respect to variable expansion. If someone thinks otherwise, please provide a POSIX-conforming makefile which exhibits incorrect variable expansion behavior in GNU make.
(0005409)
geoffclare   
2021-07-19 08:43   
> As you see, the recent change in bmake, related to $$ expansion,
> does definitely not result in gmake compatibility.

The statement about gmake compatibility was made in the context of how $$ is expanded. Obviously it is intended to indicate only that they have made $$ handling (more) compatible, not macro expansion in general. And it certainly is more compatible in that respect, as $$$$ on the right hand side of := now ends up being expanded to $$ instead of $.
(0005410)
steffen   
2021-07-19 12:30   
..and especially there was no

> As you see, the recent change in bmake, related to $$ expansion,
> does definitely not result in gmake compatibility.

"recent change"; instead bmake added support for the notion, but did not make it its default, at least not in the set of makefiles that define the rules bmake uses by default in it portable incarnation.
(0005411)
joerg   
2021-07-19 13:41   
(edited on: 2021-07-19 14:01)
Well, the bmake from NeBSD pkgsrc with a Changelog from April 3 2020 by
default does not expand $$.

Which version did you test?

(0005412)
steffen   
2021-07-19 13:55   
I guess you were asking Geoff, but nonetheless ($PKG comes from "system"):

# Description: NetBSD make program
# URL: http://www.crufty.net/help/sjg/bmake.html [^]
# Maintainer: Steffen Nurpmeso, steffen at sdaoden dot eu

name=bmake
version=20210621
release=1
source=(http://www.crufty.net/ftp/pub/sjg/${name}-${version}.tar.gz) [^]

build() {
   cd ${name}

   sed -i'' -e '/cmd-interrupt$/d' -e '/var-op-shell$/d' unit-tests/Makefile

   mkdir -p build
   cd build

   sh ../boot-strap --with-default-sys-path=/usr/share/mk

   sh ../boot-strap --prefix=/usr \
      --with-mksrc=/usr/share/mk \
      --install-destdir="${PKG}" op=install

   chmod g-w "${PKG}"/usr/bin "${PKG}"/usr/share/mk
   rm -rf "${PKG}"/usr/share/man/cat1
   mkdir "${PKG}"/usr/share/man/man1
   cp ../bmake.1 "${PKG}"/usr/share/man/man1/
}
(0005413)
joerg   
2021-07-19 14:10   
Thank you!

So it seems that I should edit my Note: 0005407 that a recent bmake
is fully aligned with smake again and the version 2020-04-03
was an exception. That makes things much easier...
(0005414)
geoffclare   
2021-07-23 09:45   
(edited on: 2021-07-23 12:26)
In yesterday's call we discussed whether, if we standardise BSD/bmake := as :::=, we would do so with or without $$ being preserved. Previously I had thought that if we went with the non-preserving behaviour, we could add some APPLICATION USAGE advising users to put $$$$ in values assigned with :::= in order to yield a single $ when expanded at use time. However, I now see that this only works for the simple case where a macro is used directly in a rule. If there are any intermediate := assignments, they each add another level of re-evaluation:
$ printf 'A := $$$$$$$$\nB := $(A)\nall:\n\techo $(B)\n' | bmake -f -
echo $
$

$ printf 'A := $$$$$$$$$$$$$$$$\nB := $(A)\nC := $(B)\nall:\n\techo $(C)\n' | bmake -f -
echo $
$

Worse still, if there are two rules that end up using the macro via different numbers of intermediate := assignments, it is impossible to put a string of $'s in the original value that will end up as a single $ in both rules (unless a single $ in the original value wouldn't have been expanded anyway).

This problem is presumably why .MAKE.SAVE_DOLLARS was added. The behaviour with it true seems greatly preferable to me, and I think the standard should require it for :::= (if we add it), although without specifying .MAKE.SAVE_DOLLARS. This would leave BSD/bmake implementors with two choices in order to conform:

1. Have :::= not be identical to :=, in that it is not affected by .MAKE.SAVE_DOLLARS but instead always preserves $$.

2. Have :::= be identical to :=, with .MAKE.SAVE_DOLLARS=true as the default (at least for makefiles that contain a .POSIX target).

I would like to hear what BSD/bmake implementors and users think about this.

(0005415)
joerg   
2021-07-23 17:27   
The bmake Changelog mentions a modification that $$ was preserved
when using := starting from 2016-02-17 and the comment does not
include a hint that this was related to gmake.

Two days later, .MAKE.SAVE_DOLLARS was introduced and the default
was set to false to achieve compatibility with older versions.

Given that := did exist in smake and bmake since a longer time
without that $$ exception, it is obvious that not expanding $$
is only needed in very rare corner cases. The oldest version of
bsd make in the netbsd version control is from 1993 and already
supports the := assignment. Smake started to support := 10 years
later.
(0005421)
geoffclare   
2021-07-29 08:17   
Re: Note: 0005414 I agree that not expanding $$ is likely only needed in rare cases. However, I think the need to expand $$ is even rarer, and probably non-existent. The vast majority of uses of := work the same either way. Before the changes in 2016, the only reason anyone had to put $$$$ on the rhs of a := would be as a work-around for it being expanded twice (or more).

I assume the reason the default is not to preserve $$ is because there were existing makefiles where people has used this work-around, and defaulting to preserving $$ would break them. If $$ had always been preserved, those makefiles would not have needed to use $$$$ in the first place.

We should not make the same mistake with :::=

(0005423)
psmith   
2021-07-29 18:07   
Until someone can explain to me the advantage of adding Yet Another Assignment Operator to make, I'm not excited about spending effort adding it to GNU make. If there's behavior which is already implemented across all variants of make then that's obviously appropriate for standardization. If there's behavior which is used widely in makefiles, proving it has real utility for users, AND it provides behaviors which are not currently available in the standard, then that could be appropriate for standardization as well. Just because a facility _exists_ in a subset of variants of make, even if it has existed for a long time, is not by itself sufficient justification IMO.

I haven't seen evidence that the difference between the behavior of POSIX ::= and BSD make := is something that users care about and that makefiles want to take advantage of, such that it warrants a new operator. I've also pointed out above that the behavior of BSD make := can be obtained in terms of the existing POSIX ::= operator (although you do need to add an extra variable to do it).
(0005440)
Don Cragun   
2021-08-12 15:19   
(edited on: 2021-08-12 15:25)
On page 2888 line 97001 change:

macro=value

macro::=value



to

macro=value

macro::=value

macro:::=value



After page 2894 line 97263 add:

The following form defines a delayed-expansion macro (replacing any previous definition of the macro named by string1):


<tt>string1 :::= [string2]</tt>


by immediately expanding macros in string2, if any, before assigning the value.



On page 2895 line 97310, change:
    

The macro $$ shall be replaced by the single character '$'.



to:
    

The string "$$" shall be replaced by the single character '$', except during the immediate expansion performed for the :::= operator, where it shall be left unmodified.


    
On page 2910 line 97956, change:
    

Therefore, the standard developers opted to require a new operator "::=" with specific semantics; this operator has semantics closest to the GNU make implementation of ":=", where '$' occurring in the immediate expansion of string2 are not further expanded in subsequent use of the macro, and it was felt that other implementations could easily support the required semantics.



to:
    

Therefore, the standard developers opted to require two new operators, "::=" and ":::=", with specific semantics; the "::=" operator has semantics closest to the GNU make implementation of ":=", where '$' occurring in the immediate expansion of string2 are not further expanded in subsequent use of the macro, and the ":::=" operator has semantics closest to the BSD make and smake implementations of ":=", where immediate expansion is performed when assigning to a delayed-expansion macro and "$$" is preserved. It was felt that other implementations could easily support the required semantics.






Viewing Issue Advanced Details
1470 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2021-05-07 13:22 2021-11-18 17:00
dennisw
 
normal  
Resolved  
Accepted  
   
Dennis Wölfing
tdelete
2138-2139
68532-68556
---
tdelete example is incorrect
The example says that a comparison function that always returns 0 can be used with tdelete to delete the root node of a binary search tree. However the standard does not actually guarantee this. It only says that some node will be deleted in this case but nothing says that this will be the root node.

Perhaps a balancing search tree algorithm might need to look at multiple nodes to ensure balancing and might then end up with deleting a non-root node since all nodes are considered to be equal.
On page 2138 line 68532 change
tdelete((void *)elementptr, &root, delete_root);
to
tdelete((void *)elementptr, &root, node_compare);


On pages 2138-2139 lines 68547-68556 delete
/*
 * This comparison routine can be used with tdelete()
 * when explicitly deleting a root node, as no comparison
 * is necessary.
 */
int
delete_root(const void *node1, const void *node2)
{
    return 0;
}
There are no notes attached to this issue.




Viewing Issue Advanced Details
1469 [Issue 8 drafts] Base Definitions and Headers Editorial Error 2021-05-04 16:03 2021-05-11 09:35
dennisw
 
normal  
Applied  
Accepted  
   
Dennis Wölfing
<search.h>
316
11052-11055
Inconsistent return type for tfind and tsearch
0001011 added the posix_tnode type and changed the return types of tfind and tsearch to posix_tnode*. However in the current draft the return type is only changed in XSH but not in XBD <search.h>.
On page 316 line 11052-11055 change
void *tfind(const void *, posix_tnode *const *,
    int(*)(const void *, const void *));
void *tsearch(const void *, posix_tnode **,
    int(*)(const void *, const void *));
to
posix_tnode *tfind(const void *, posix_tnode *const *,
    int(*)(const void *, const void *));
posix_tnode *tsearch(const void *, posix_tnode **,
    int(*)(const void *, const void *));
Notes
(0005350)
geoffclare   
2021-05-06 09:30   
Thanks for spotting this. Clearly an oversight on my part when I applied bug 1011. I changed the function parameters in <search.h> but not the return types.




Viewing Issue Advanced Details
1468 [1003.1(2008)/Issue 7] Shell and Utilities Editorial Enhancement Request 2021-04-24 15:20 2021-11-18 16:55
mortoneccc
ajosey  
normal  
Resolved  
Accepted  
   
Ed Morton
awk
2493
80182-80184
---
awk FS definition not quite correct
(sorry, I don't see any page or line numbers in the online spec, hence the 1 and 1 used above).

In the definition of FS in the awk spec (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html) [^] it says:

-----
The following describes FS behavior:

   If FS is a null string, the behavior is unspecified.

   If FS is a single character:

       If FS is <space>, skip leading and trailing <blank> and <newline> characters; fields shall be delimited by sets of one or more <blank> or <newline> characters.

       Otherwise, if FS is any other character c, fields shall be delimited by each single occurrence of c.

       Otherwise, the string value of FS shall be considered to be an extended regular expression. Each occurrence of a sequence matching the extended regular expression shall delimit fields.
-----

but that final case isn't exactly correct because an ERE can match a null string while a FS can't. Try for example splitting a record on all non-commas:

$ echo 'x,y,z' | awk -F'[^,]*' '{for (i=1;i<=NF;i++) print i, "<"$i">"}'
1 <>
2 <,>
3 <,>
4 <>

which makes sense since there's a null string before the first non-comma (x), 2 commas around the 2nd non-comma (y) and a null string after the last non-comma (z). Now remove the "y" from the middle to get:

$ echo 'x,,z' | awk -F'[^,]*' '{for (i=1;i<=NF;i++) print i, "<"$i">"}'
1 <>
2 <,,>
3 <>

and note that the null string between the 2 commas which would match the regexp `[^,]*` isn't actually matched by the FS `[^,]*`.
Change the final paragraph of the FS definition mentioned above to say something like "Otherwise, the string value of FS shall be considered to be an extended regular expression such that each occurrence of a sequence **of one or more characters** matching the extended regular expression shall delimit fields."
There are no notes attached to this issue.




Viewing Issue Advanced Details
1467 [Issue 8 drafts] Shell and Utilities Editorial Enhancement Request 2021-04-16 22:37 2021-05-11 09:33
kre
 
normal  
Applied  
Accepted  
   
Robert Elz
XCU 2.14 Special Built-In Utilities, "colon"
2316
75174-5
Poor language in CHANGE HISTORY entry for Issue 8 for ':' command (defect 1272) changes.
The CHANGE HISTORY added to the colon special built-in shell command
includes the words:

   the null utility does not expand its arguments,

This was added because the earlier description of the command said:

   This utility shall only expand command arguments.

which was incorrect, the shell does the arg expanding, the utility
does nothing at all with its arguments.

However the choice of words "does not expand" could be misinterpreted to
indicate that the shell should also not expand the arguments to this utility.
That is not what is intended.

Please note that (by request) this bug is being submitted against
Issue 8, draft 1.1 (and the page and line numbers reflect that) but
bug 1272 (which caused this section to be inserted) is tagged for Issue7-tc3
so if there ever is a tc3 for issue 7 (I know, not currently planned, but...)
this bug should be handled when the CHANGE HISTORY for this issue is
modified from Issue 8 to whatever it will be in that case.
Change the clause (which spans lines 75174 and 75175) from

    the null utility does not expand its arguments,

to

    the null utility does not process its arguments,

which reinforces the args being ignored, which they are, while not
suggesting that the args would not be expanded before the utility is invoked.

Other wording which avoids the suggestion that the args do not get
expanded would be fine.
Notes
(0005320)
kre   
2021-04-16 22:48   
Ugh: When I added that "Please Note" paragraph in the Description
(the last thing I did before submitting) I though I was added it right
at the end of the description ... the paragraph that follows should
come before that one!

Apologies for not checking more thoroughly.
(0005321)
Don Cragun   
2021-04-17 00:33   
Description modified as suggested in Note: 0005320.
(0005322)
geoffclare   
2021-04-19 08:13   
Regarding getting this change into TC3 (if there is one), this bug should be tagged tc3-2008 when it is accepted, so that it will be applied in the Issue7TC3 branch in gitlab.
(0005324)
kre   
2021-04-19 17:09   
Re Note: 0005321 Thanks.




Viewing Issue Advanced Details
1466 [Issue 8 drafts] Base Definitions and Headers Editorial Enhancement Request 2021-04-16 15:22 2021-07-20 09:31
jauricchio
 
normal  
Applied  
Accepted As Marked  
   
Joe Auricchio
Apple Inc.
3.277 Pseudo-Terminal and others
68-2126
2129-68880
Note: 0005392
Pseudo-terminal devices should be renamed
At Apple, we’re working to remove and replace non-inclusive language across our developer ecosystem, including within Xcode, platform APIs, documentation, and open source projects [1]. In this effort, we are allied with individuals and organizations across the software industry. For example, in July 2020, the Linux kernel style guide was updated to recommend avoiding non-inclusive language [2], and in December 2020, IEEE committed to write standards to avoid non-inclusive and insensitive terminology [3]. As part of this effort, we would like the Austin Group’s help in updating the language used in the POSIX standard.

The two device ends of a pseudo-terminal are named the “master” and “slave” devices in previous and current versions of the standard. These should be renamed.

We invite the Austin Group community to propose new names, debate, and reach consensus. Since this has become an issue of public discourse, many suggestions have been made on forums across the internet, so there is no shortage of ideas out there.

[1] https://developer.apple.com/news/?id=1o9zxsxl [^]
[2] https://standards.ieee.org/about/sasb/resolutions.html [^] (December 2020)
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/coding-style.rst#n338 [^]
The words “master” or “slave” within the context of pseudo-terminal devices appear in the following places in Issue 8 Draft 1.1 (202x_d1.1.pdf). Please update all of these when new names are chosen.

Page 68, lines 2129-2133. 3.277 Pseudo-Terminal
Page 181, line 6422. 11.1.1 Opening a Terminal Device File
Page 659, lines 22861-22863. close() - Description
Page 663, line 23061. close() - Change History
Page 1086, lines 37188-37191. grantpt() - Description
Page 1086, line 37200-37202. grantpt() - Errors
Page 1375, line 46052. open() - Description
Page 1375, line 46070-46072. open() - Description
Page 1377, line 46146. open() - Errors
Page 1421, line 47580. posix_openpt() - Description
Page 1421-1422, lines 47604-47619. posix_openpt() - Examples
Page 1422, line 47622-47624. posix_openpt() - Application Usage
Page 1422, lines 47627-47632. posix_openpt() - Rationale
Page 1667, lines 54812-54817. ptsname() - Description.
Page 1667, line 54823-54824. ptsname() - Return Value
Page 1667, line 54841. ptsname() - Errors
Page 1668, line 54849. ptsname() - Rationale
Page 2061, lines 66908-66910. tcsetwinsize() - Application Usage
Page 2126, line 68870-68872. unlockpt() - Name
Page 2126, line 68865. unlockpt() - Description
Page 2126, line 68880. unlockpt() - Errors
Notes
(0005325)
enh   
2021-04-19 20:44   
since POSIX is all about "standardizing existing practice"... fwiw, in Android we already went with "pty"/"tty" for master/slave in APIs like openpty(). from a checkin comment (https://android-review.googlesource.com/c/platform/bionic/+/1374048): [^]

```
This was initially just to try on "pty" and "tty" for size, while
discussing with other projects to try to align on vocabulary, but -- in
the implemention and tests at least -- but these work out so well that
I'm tempted to go with them anyway if we can't come to a concensus. We
can always come back and change them again later.

What I really like is that you pass the pty to functions like ptsname()
and grantpt() with a 'p' in the name, and the tty to functions like
ttyname() and tcsetattr() with just 't's.

The use of "parent" and "child" in forkpty() seems helpful too.
```


the Linux man-pages project went with
```
    slave ==> "terminal device"
           (or "terminal end of the pseudoterminal device pair")
    
    master ==> "pseudoterminal device"
           (or "pseudoterminal end of the pseudoterminal device pair")
    
    pseudoterminal (device) ==> "pseudoterminal device pair"
```
see https://www.spinics.net/lists/linux-man/msg16641.html [^] for more (including a long discussion of alternatives, but also background links for some of the precedent we went with).


(as an aside, if there's someone from Apple who'd be interested in joining the libc-coord@lists.openwall.com list, that's a good place for libc owners to coordinate changes, and i think Apple's the biggest libc that's not yet represented there.)
(0005387)
geoffclare   
2021-06-18 14:38   
Re Note: 0005325 A drawback of both of those solutions is that neither has a satisfactory retrofit for the "s" in ptsname().

The mailing list discussion in Aug 2020 included suggestions of using "primary" for one side but without finding a good enough word for the other side ("ancilliary" came close). It has now occurred to me that "secondary" could be used if these words are used to indicate not the role that the devices have but the order in which they are opened. The primary device is opened first, and then the name of the secondary device is obtained from ptsname() and is opened second.

I think this could be a good solution if we include an explanation of the reason behind the naming somewhere.
(0005392)
geoffclare   
2021-07-08 16:03   
Change "master" to "manager" and "slave" to "subsidiary" in all of the places listed in the desired action.




Viewing Issue Advanced Details
1465 [Online Pubs] Shell and Utilities Editorial Error 2021-04-11 06:18 2021-11-22 17:55
mihai_moldovan
 
normal  
Applied  
Accepted  
   
Mihai Moldovan
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#trap [^]
trap
trap synopsis missing newline
The trap utility has two different forms, which are described in the text later on.

However, the synopsis currently lists just one weird form, since both are concatenated there.
Add newline between the two different forms of the trap utility.
Notes
(0005310)
Don Cragun   
2021-04-11 08:16   
It displays correctly in the PDF of the 2017 version of the standard (2008 + TC1 + TC2) on P2420 L77484-77485.
(0005311)
geoffclare   
2021-04-12 08:38   
This was previously reported in bug 0001285 and the root cause was identified as a problem in the troff source; a fix has been applied there. However, it would make sense to do a separate fix in the current HTML version.
(0005536)
agadmin   
2021-11-22 17:55   
This has been applied to the online html edition.




Viewing Issue Advanced Details
1463 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2021-04-02 22:44 2021-11-18 16:41
quinq
 
normal  
Resolved  
Accepted  
   
Quentin Rameau
gencat
2820
92835
---
Small discrepency between the description and the explanation
The description of the quote option is stated as:

$quote n

and in the explanation, the character is illustrated as c instead of n:

This line specifies an optional quote character c
Replace the character 'n' in

$quote n

with 'c', as

$quote c
There are no notes attached to this issue.




Viewing Issue Advanced Details
1462 [1003.1(2016/18)/Issue7+TC2] System Interfaces Comment Enhancement Request 2021-03-26 15:34 2021-11-18 16:38
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
time()
2149
68870
---
Require (at least) 64-bit time_t
The time() FUTURE DIRECTIONS section says:
In a future version of this volume of POSIX.1-2017, time_t is likely to be required to be capable of representing times far in the future. Whether this will be mandated as a 64-bit type or a requirement that a specific date in the future be representable (for example, 10000 AD) is not yet determined.
The time to do something about this is now, as the next major revision is likely to be at least ten years after this one, which will be uncomfortably close to 2038.

Note that if time_t is required to be 64-bit, implementations can still support legacy programming environments where time_t is 32-bit. In order to conform, there just needs to be at least one environment where time_t is 64-bit. (The suggested changes include additions advising about this.)
On page 2050 line 65758 section strftime() RATIONALE, change:
With tm_year being a signed 32 or more-bit int and with many current implementations supporting 64-bit time_t types in one or more programming environments, these assumptions are clearly wrong.
to:
Since tm_year is a signed int with a width of at least 32 bits and time_t is required to have a width of at least 64 bits (in conforming programming environments), these assumptions no longer hold.

On page 2149 line 68857 section time() RATIONALE, change:
Implementations in which time_t is a 32-bit signed integer (many historical implementations) fail in the year 2038. POSIX.1-2017 does not address this problem. However, the use of the time_t type is mandated in order to ease the eventual fix.
to:
Earlier versions of this standard allowed the width of time_t to be less than 64 bits. A 32-bit signed integer (as used in many historical implementations) fails in the year 2038, and although a 32-bit unsigned integer does not fail until 2106 the preferred solution is to make time_t wider rather than to make it unsigned.

On page 2149 line 68870 section time(), change FUTURE DIRECTIONS to:
None.

Cross-volume changes to XBD ...

On page 404 line 13726 section <sys/types.h>, change:
  • clock_t shall be an integer or real-floating type. [CX]time_t shall be an integer type.[/CX]
to:
  • clock_t shall be an integer or real-floating type.

  • [CX]time_t shall be an integer type with a width (see [xref to <stdint.h>]) of at least 64 bits.[/CX]

On page 450 line 15504 section <unistd.h> APPLICATION USAGE, add a new paragraph:
Implementations may support multiple programming environments with some of them conforming to this standard and some not conforming. The _POSIX_Vn_ILP* and _POSIX_Vn_LP* constants, and corresponding sysconf() and getconf calls, only indicate whether each programming environment is supported; they do not indicate anything about conformance of the environments that are supported. For example, an implementation may support the ILP32_OFF32 environment for legacy reasons with a 32-bit time_t, whereas in a conforming environment time_tis required to have a width of at least 64 bits. Application writers should consult an implementation's POSIX Conformance Document for information about the conformance of each supported programming environment.

Cross-volume changes to XCU ...

On page 2546 line 82421 section c99 EXTENDED DESCRIPTION, change:
Applications can use sysconf() or getconf to determine which programming environments are supported.
to:
Applications can use the _POSIX_Vn_ILP* and _POSIX_Vn_LP* constants in <unistd.h>, and corresponding sysconf() and getconf calls, to determine which programming environments are supported.

On page 2549 line 82527 section c99 APPLICATION USAGE, add a new paragraph:
Implementations may support multiple programming environments with some of them conforming to this standard and some not conforming. The _POSIX_Vn_ILP* and _POSIX_Vn_LP* constants in <unistd.h>, and corresponding sysconf() and getconf calls, only indicate whether each programming environment is supported; they do not indicate anything about conformance of the environments that are supported. For example, an implementation may support the ILP32_OFF32 environment for legacy reasons with a 32-bit time_t, whereas in a conforming environment time_tis required to have a width of at least 64 bits. Application writers should consult an implementation's POSIX Conformance Document for information about the conformance of each supported programming environment.

Cross-volume change to XRAT ...

On page 3519 line 119141 section A.4.16 Seconds Since the Epoch, change:
Implementations that implement time_t as a signed 32-bit integer will overflow in 2038. This standard requires that time_t be an integer type with implementation-defined size, but does not mandate a particular size.
to:
The number of seconds since the epoch overflows a signed 32-bit integer in 2038. This standard requires that time_t is an integer type with a width of at least 64 bits (in conforming programming environments).

There are no notes attached to this issue.




Viewing Issue Advanced Details
1461 [1003.1(2016/18)/Issue7+TC2] System Interfaces Comment Clarification Requested 2021-03-22 10:40 2021-11-18 16:36
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
tgamma()
2145
68720
---
tgamma() with negative integer argument
The tgamma() FUTURE DIRECTIONS section says:
It is possible that the error response for a negative integer argument may be changed to a pole error and a return value of ±Inf.

C17 allows a domain error or a pole error (with implementation-defined return value). However, when IEC 60559 Floating-Point is supported, an "invalid" floating-point exception (i.e. a domain error) and NaN return are required. This has not changed in the Dec 2020 C2x draft. So changing POSIX to require a pole error would introduce a conflict with the C standard.

I would suggest that we just change POSIX to match the C standard, i.e. allow either a domain error or a pole error, when IEC 60559 Floating-Point is not supported.
On page 2145 line 68720 section tgamma(), change:
If x is a negative integer, a [CX]domain[/CX] error may occur and either a NaN (if supported) or an implementation-defined value shall be returned.
to:
If x is a negative integer, either a domain error or a pole error may occur and either a NaN (if supported) or +/-Inf (if supported), respectively, or an implementation-defined value shall be returned.

On page 2146 line 68762 section tgamma(), change:
Pole Error The value of x is zero.
to:
Pole Error The value of x is zero or a negative integer.

On page 2146 line 68781 section tgamma(), change:
It is possible that the error response for a negative integer argument may be changed to a pole error and a return value of ±Inf.
to:
None.

There are no notes attached to this issue.




Viewing Issue Advanced Details
1460 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Clarification Requested 2021-03-16 16:53 2021-11-18 16:30
geoffclare
 
normal  
Resolved  
Accepted  
   
Geoff Clare
The Open Group
hash
2847
93806
---
hash implementations differ when a utility is not found
The behaviour of the hash builtin differs between shells as to whether they report an error when a utility specified as an operand is not found. For example, bash reports an error but ksh (88 and 93) does not.
After:
The name of a utility to be searched for and added to the list of remembered locations.
add a new sentence:
If the search does not find utility, it is unspecified whether or not this is treated as an error.

Notes
(0005277)
joerg   
2021-03-16 18:22   
Do you have other shells than ksh and mksh in mind?

There is a problem anyway since hash on these shells is a builtin alias instead of a builtin command.
(0005278)
shware_systems   
2021-03-16 19:01   
What type of error report is it, an exit code unequal to zero or just text to stdout/err? If the latter, and they still add remaining found operands, I'd think that's more an App Usage note, that some are silent, some not.
(0005279)
kre   
2021-03-17 03:05   
All shells I have tested do the "text to stderr" (not stdout) thing on
"hash garbage", all add "ls" in addition to that on "hash garbage ls".
The difference is the exit code, the various ksh versions and the NetBSD sh
exit 0 (always), bash, zsh, bosh, the FreeBSD sh, dash, yash all exit 1
when something failed to be added.
(0005280)
geoffclare   
2021-03-18 09:38   
Which version of ksh93 did you test? With 93u+ I get no message on stderr:

$ ksh -c 'echo ${.sh.version}; hash garbage; echo status $?'
Version AJM 93u+ 2012-08-01
status 0

Tested on Solaris 11.4 and Debian Buster. This matches how ksh88 (/usr/xpg4/bin/sh) behaves on 11.4

(I checked with "type garbage" that neither system has a "garbage" utility to find.)

If a later ksh93 has started writing a message to stderr without changing the exit status, that's a new non-conformance (violating 1.4 Utility Description Defaults under STDERR).
(0005284)
kre   
2021-03-18 18:29   
(edited on: 2021-03-18 18:32)
You're right, I didn't look closely enough at that, I got no message
from it (or the other ksh variants I tested), and just didn't notice...

Apologies.

(0005285)
kre   
2021-03-18 18:52   
(edited on: 2021-03-18 19:00)
I also just noticed, that in the Application Usage section of hash, it says
(Lines 92140-2 of Issue 8 draft 1.1 - it is also in older versions)

     The effects of hash -r can also be achieved portably by resetting
     the value of PATH; in the simplest form, this can be:

     PATH="$PATH"

That isn't true, at least of the NetBSD sh - for a very long time it has
optimised PATH setting wrt the hash table ... rather than invalidate the
whole thing, it looks to find the first changed element of the new PATH
compared to the old, and only invalidates hash table entries from that
point down, so if we had

    PATH=/a:/b:/c
and then changed it to
    PATH=/a:/x:/b

then anything found in /a that was in the hash table remains and
remains used, things from /c are removed (obviously) and sp are those
from /b. as we don't know that they aren't overridden by what's in /x

In particular that means that PATH=${PATH} is a no-op when it comes to
what happens to the hash table.

If one were to insist on manipulating PATH for this purpose, the way
to do it would be
      x=$PATH; PATH=; PATH=$x; unset x
(where 'x' is any variable name that isn't used anywhere else).

I see no point in this note in this section, hash -r is the way to
remove everything from the hash table, it is less costly than assigning
to PATH (even in the simple case given in the current text).

Can we just delete it?

(0005287)
geoffclare   
2021-03-19 10:57   
Re Note: 0005285, that statement about PATH is pointing out a normative requirement from XCU 2.9.1. In Issue 8 draft 1.1 it's on page 2292 line 74177:
Once a utility has been searched for and found (either as a result of this specific search or as part of an unspecified shell start-up activity), an implementation may remember its location and need not search for the utility again unless the PATH variable has been the subject of an assignment.

If you think this needs to change please submit a separate bug.

(0005292)
kre   
2021-03-19 16:03   
Re Note: 0005287: No I don't think 2.9.1 about this needs to change, but why
is not a topic for here, so I will discuss that on the mailing list.

However I still believe the relevant text in the hash command application usage
section, even if you were right and it follows directly from 2.9.1 (with which
I disagree) is inappropriate and should be deleted.

One would add an application usage hint like that if there were some
defect in the command being described, but an alternative, known working,
method was available. Eg: if it were unspecified whether "hash -r"
worked or not for clearing the hash table, but assigning to PATH always
did, then it would be reasonable to have a note like that. It would
also be reasonable if the alternative method were cumbersone, or could
have other side effects

But "hash -r" is not unspecified, is simple, it does work to clear the hash
table. But what is a new reader to believe when reading this Application Usage
section and seeing an alternative *portable* method to clear the hash table?
Perhaps "hash -r" is not portable? Better not use that then. The note,
even if it were correct, sends entirely the wrong message. It isn't needed.
Doing PATH=$PATH is the wrong way even if it worked (in addition to clearing
the hash table it also needs to expand $PATH and then do a variable assignment, which also means (as PATH is usually exported) perhaps rebuilding the
exported environment for no good reason. Just delete it.

On procedural issues, if you'd prefer a new bug, rather than a tack on to
this one, for this specific issue, I can do that, but in that case, please
indicate whether you would prefer it filed against 7 TC2 or 8 D1.1 (or
something else).
(0005296)
geoffclare   
2021-03-22 09:40   
Re Note: 0005292, since you emphasised *portable*, I think I now see how we arrived at the current situation. In Issue 6, hash was part of the XSI option. That text in APPLICATION USAGE was put there to advise users of shells that do not have a hash built-in how to achieve the same thing as hash -r without it. In Issue 7, hash was made mandatory and this advice became out-of-date and should have been modified at the time (as should 2.9.1 and/or the description of PATH to mention the hash table).

Since this issue is not related to the issue raised in this bug, it should be dealt with in a separate bug (against Issue7+TC2), alongside any related change to 2.9.1 and/or PATH that comes out of the mailing list discussion.




Viewing Issue Advanced Details
1459 [Online Pubs] System Interfaces Editorial Clarification Requested 2021-03-13 11:01 2021-11-22 17:48
andras_farkas
 
normal  
Applied  
Accepted  
   
Andras Farkas
https://pubs.opengroup.org/onlinepubs/9699919799/functions/signgam.html [^]
signgam
signgam.html differs from lgamma.html
The webpage for signgam
https://pubs.opengroup.org/onlinepubs/9699919799/functions/signgam.html [^]
differs from the webpage for lgamma
https://pubs.opengroup.org/onlinepubs/9699919799/functions/lgamma.html [^]
The lgamma.html, lgammaf.html, and lgammal.html pages are identical: this makes me assume that signgam.html should also be the same.
I'm not mathematically proficient enough to be a good judge of which one's better, but I have an instinctive feeling that lgamma.html is probably the "correct" version and signgam.html should be changed to match.

Also, neither formula image has alt text. (I don't know if it'd be easy to do for a math formula, though)

I wonder if there's an easier way to do things than to have multiple copies of these pages.
Change signgam.html so it's identical with lgamma.html
Notes
(0005270)
geoffclare   
2021-03-15 09:26   
In the PDF the signgam page is a pointer page to lgamma(). The HTML translator handles pointer pages by copying the referenced page, and in the original 2008 edition it appears that signgam.html was indeed a copy of lgamma.html:

https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/functions/lgamma.html [^]

https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/functions/signgam.html [^]

However, there have been subsequent changes made to lgamma.html that have not been duplicated in signgam.html.
(0005535)
agadmin   
2021-11-22 17:48   
Change applied to the online html edition (by copying
lgamma.html over signgam.html)




Viewing Issue Advanced Details
1458 [Online Pubs] System Interfaces Editorial Omission 2021-03-13 10:08 2021-11-12 17:20
andras_farkas
 
normal  
Applied  
Accepted  
   
Andras Farkas
https://pubs.opengroup.org/onlinepubs/9699919799/functions/semctl.html [^]
semctl
semctl missing first link to sys/sem.h
On
https://pubs.opengroup.org/onlinepubs/9699919799/functions/semctl.html [^]
The first link that would normally be included isn't there.
In the text
#include <sys/sem.h>
the text
sys/sem.h
should be a link to
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_sem.h.html [^]

The semget and semop pages are currently correct, only semctl has this omission of a link.

This omission also affects the previous standard's equivalent page, too:
https://pubs.opengroup.org/onlinepubs/009695399/functions/semctl.html [^]
Make the text "sys/sem.h" in "#include <sys/sem.h>" a link to
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_sem.h.html [^]
Notes
(0005269)
geoffclare   
2021-03-15 09:16   
Looks like the root cause of this is that the troff source has:
#include\ <sys/sem.h>

The HTML translator is presumably looking for something like "#include[[:blank:]]*<" and so the backslash throws it off.

I will remove the backslash so that when the translator is run for Issue 8 the problem is not repeated, but the problem should in the meantime be corrected by hand in the current online edition.
(0005524)
agadmin   
2021-11-12 17:19   
Updated the online copy and the gitlab copy.
(0005525)
agadmin   
2021-11-12 17:20   
The download bundles have also been updated




Viewing Issue Advanced Details
1457 [1003.1(2008)/Issue 7] Shell and Utilities Editorial Enhancement Request 2021-03-09 21:21 2021-03-09 21:21
steffen
ajosey  
normal  
Under Review  
Open  
   
Steffen Nurpmeso
Vol. 3: Shell and Utilities.
(page or range of pages)
(Line or range of lines)
---
Add readlink(1) utility
Some POSIX utilities (like find) offer operators to work with symbolic links.
It is not possible to read content of symbolic links by itself as far as i know, as is offered by readlink(1) which seems to be widely available, for a long time.
The current standard does not mention this utility as having been obsoleted in the past either, so i am asking for inclusion.
Add the readlink utility.
The most minimal form seems to be from OpenBSD, it states:


NAME

readlink — display target of symbolic link on standard output
SYNOPSIS
readlink [-fn] file
DESCRIPTION

The readlink utility when invoked with the pathname of a symbolic link as its argument dereferences the symbolic link and prints the name of target on standard output. If the -f option is not specified and readlink is invoked with an argument other than the pathname of a symbolic link, it exits with a nonzero exit code without printing anything.

The options are as follows:

-f
    Canonicalize by following every symlink in every component of the given path recursively. readlink will resolve both absolute and relative paths and return the absolute pathname corresponding to file. The argument does not need to be a symbolic link.
-n
    Do not print a trailing newline character.

EXIT STATUS

The readlink utility exits 0 on success, and >0 if an error occurs.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1456 [1003.1(2013)/Issue7+TC1] System Interfaces Objection Omission 2021-03-09 20:09 2021-03-09 20:09
eblake
 
normal  
New  
Open  
   
Eric Blake
Red Hat
XSH 2.2.2
477
16418
---
Permit implementations to add additional confstr()/pathconf()/sysconf() constants
XSH 2.2.2 states that implementations can generally add more constants that fit within approved namespace restrictions, and further states that any identifier beginning with underscore then upper-case is reserved for the implementation (line 16438), so an implementation can indeed add more confstr() (_CS_*), pathconf() (_PC_*) and sysconf() (_SC_*) symbols without violating the requirements. However, without explicit direction, it is not obvious that an application can test for the existence of which extensions an implementation supports, nor that an implementation should not blindly define macros in this namespace if they are not compatible with confstr/pathconf/sysconf semantics.
Before page 477 line 16418 (XSH 2.2.2), add a new paragraph:
Additional symbolic constants with the prefix _CS_, _PC_, and _SC_ may be defined by the inclusion of <unistd.h>, but as these are already reserved for the implementation, they are not included in the table above. Extensions with these prefixes should be compatible with use by confstr(), pathconf(), and sysconf(), respectively.


After page 443 line 15205 (XBD <unistd.h>), add a paragraph:
The implementation may define additional symbolic constants beginning with _CS_* for use by confstr().


After page 444 line 15235, add a paragraph:
The implementation may define additional symbolic constants beginning with _PC_* for use by pathconf().


After page 447 line 15362, add a paragraph:
The implementation may define additional symbolic constants beginning with _SC_* for use by sysconf().


Optionally, the Austin Group should decide whether to add an additional requirement that extensions of these forms must be suitable for use in #if preprocessor directives.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1455 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Clarification Requested 2021-03-02 14:16 2021-11-26 15:12
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
2.4.3
494
17135,17137
---
readv() and writev() are missing from the table of async-signal-safe functions
It appears that readv() and writev() were accidentally omitted from the table of async-signal-safe functions in 2.4.3 when SUSv2 and POSIX.1-1996 were merged in 2001.

The description of readv() begins with "The readv() function shall be equivalent to read(), except as described below", and nothing "below" says that one of the exceptions is that the async-signal-safe requirement for read() does not apply to readv().

Likewise for writev().
Add readv() and writev() to the table.
There are no notes attached to this issue.




Viewing Issue Advanced Details
1454 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Error 2021-02-16 16:29 2021-11-26 15:10
geoffclare
 
normal  
Applied  
Accepted As Marked  
   
Geoff Clare
The Open Group
2.9.4.3
2372
75780
---
See Note: 0005523.
Conflict between "case" description and grammar
A discussion in comp.unix.shell identified that the format summary in the description of "case" conflicts with the syntax defined in the formal shell grammar, as it doesn't allow a case statement with no patterns:

case foo in
esac

whereas the grammar has a line whose sole purpose is to allow it:

Case WORD linebreak in linebreak Esac

The grammar has precedence, so the description should be updated to match it.

All shells I tried accepted a case statement with no patterns, although ksh93 reports a syntax error if the newline is omitted (case foo in esac). However, this appears to be a bug introduced into ksh93, as ksh88 accepts it. It is clear from the grammar that having a newline there should make no difference, since "linebreak" is zero or more newlines.
On page 2372 line 74408 section 2.9.4.3, change:
the first one of several patterns
to:
the first one of zero or more patterns

On page 2372 line 75780 section 2.9.4.3, delete the line:
[(] pattern1 ) compound-list ;;

Note to the editor: in Issue 8 draft 1.1 the line to delete has "terminator" instead of ";;" because of the change to allow ";&".
Notes
(0005241)
kre   
2021-02-16 20:44   
I agree that shells (and my testing also says all but ksh93) work this
way, and have always done so, and consequently as the grammar appears to
allow, should continue working this way .. but I suspect the ksh93 variation
might just be deliberate, rather than simply a bug, in that they might be
attempting to follow the POSIX standard's rules (words), rather than the
actual standard (what everyone (else) implements).

The issue is that I see nothing in the standard currently which allows that
"esac" to be parsed as the Esac (reserved word) token, or not when it is
not following a \n or ; (etc), or a pattern. That is, reserved words
generally (with the specific exceptions called out by references to the
XCU 2.10.2 rules in the grammar) are generally recognised only when in
the command word position (if the thing parsed were to be a simple command).

In "case foo in x) stuff ..." "stuff" is in the command word position, so
a reserved word lookup happens.

On the other hand, in "case foo in stuff..." "stuff" is not in the command
word position (it will be taken as the pattern, and should be followed by
')' or '|') and so is not subject to reserved word lookup. This is what
makes "case word in for) command;; esac" legal, "for" there is not the
reserved word, just a string. All shells accept that (ksh93 included).

But if "for" there is not a reserved word, how could it be if spelled "esac"
instead? Shells do reject "case word in esac) command;; esac", that is,
except for ksh93, which permits it.

So, I while I believe that we should ask ksh93 to follow the real standard,
rather than what is currently in the POSIX text, I also believe that we need
to add some more magic to the grammar, and tokeniser rules, in order to
make this all legitimate.

It isn't as simple as just updating the description in 2.9.4.3.
(0005242)
kre   
2021-02-16 20:55   
(edited on: 2021-02-16 21:02)
Ignore most of that (Note: 0005241) - I missed rule 4.

However, rule 4 only applies when looking at a pattern. That is
in a case_list (or case_list_ns) as a case_item (or case_item_ns).

The grammar rule in question:
      Case WORD linebreak in linebreak Esac
contains no case_list, hence no patterns, hence rule 4 would seem
not to apply.

The other rules always require a case_list[_ns] which always requires
at least one case_item[_ns] and those things always require a ')'
(every single possibility).

So, I still believe that the grammar needs work, even if slightly different
work than I expected in Note: 0005241 . It may be as simple as adding
/* Apply rule 4 */ to the grammar rule line quoted earlier in this note,
but I haven't considered all the ramifications of that yet.

(0005243)
geoffclare   
2021-02-17 10:07   
I agree that we should add /* Apply rule 4 */ to that line in the grammar.
Or something technically equivalent - there would be an editorial problem with simply adding it (and removing redundant spaces) because:

Case WORD linebreak in linebreak Esac /* Apply rule 4 */

is two characters too long to fit on the line.

Perhaps we could change all occurrences of "Apply rule X" to just "Rule X"?
The comments only need to make clear which rule it is that applies; it is the wording in 2.10.1 about those comments that specifies how the indicated rules are applied. (The comment that says "Do not apply rule 4" would stay as-is.)
(0005244)
kre   
2021-02-17 13:18   
Unfortunately, upon reflection, it is not quite that simple (ignoring
temporarily the editorial issue, for which just using "Rule N" instead
of "Apply..." would be fine) as rule 4 says that if the word is "esac"
then the Esac token is returned. If that applies to the grammar production
in question, then
     case esac in esac
isn't going to parse correctly, as the first "esac" which should be WORD
would instead become Esac and so not match.

Perhaps instead, change all occurrences of :"Esac": (in all the productions)
to "case_end", and add a new production:

     case_end: Esac ; /* Apply Rule 4 */

(formatted however is appropriate) which also conveniently side-steps the
editorial issue.

But please consider this carefully, it is a spur of the moment suggestion,
I'm not sure if it might cause other issues.
(0005245)
geoffclare   
2021-02-17 14:26   
Re Note: 0005244 I wondered about "case esac in ..." as well when I was writing my previous note, but decided it's not a problem because of this text in 2.10.1:
Some of the productions in the grammar below are annotated with a rule number from the following list. When a TOKEN is seen where one of those annotated productions could be used to reduce the symbol, the applicable rule shall be applied to convert the token identifier type of the TOKEN to a token identifier acceptable at that point in the grammar.

So rule 4 only causes "esac" to be recognised as the token Esac when it appears in the position of an Esac in the grammar, and it therefore doesn't apply to "case esac in ..." because the "esac" there is in the position of WORD in the grammar, not Esac.
(0005246)
kre   
2021-02-18 10:32   
Re Note: 0005245

That raises an additional problem, as (not concerning the case productions
here) that isn't the way things actually work.

   "convert the token identifier type of the TOKEN to a token identifier
    acceptable at that point in the grammar."

isn't what (normally) happens. If it were the command

    do my work

would run the "do" command, as "do" (the reserved word) is not acceptable
at that point in tne grammar, so the TOKEN "do" should have been (according
to that text) turned into a WORD rather tha nthe keywoprd "do".

There isn't a shell around that behaves like that, it is in the command line
position, "do" matches the spelling of the reserved word, Rule 1 applies, the
reserved word is generated, despite not being "acceptable at that point in
the grammar":.

Once we have an explanation for why this analysis isn't right, or a fix for
that text in 2.10.1 we can revisit how to handle the recognition of "ecac"
in that peculiar case statement with no patterns.
(0005252)
geoffclare   
2021-02-25 09:41   
New proposed changes based on comments made here and on the mailing list ...

On page 2372 line 74408 section 2.9.4.3, change:
the first one of several patterns
to:
the first one of zero or more patterns

On page 2372 line 75780 section 2.9.4.3, delete the line:
[(] pattern1 ) compound-list ;;

Note to the editor: in Issue 8 draft 1.1 the line to delete has "terminator" instead of ";;" because of the change to allow ";&".

On page 2375 line 75888 section 2.10.1, change:
... convert the token identifier type of the TOKEN to a token identifier acceptable at that point in the grammar.
to:
... convert the token identifier type of the TOKEN to:
  • The token identifier of the recognized reserved word, for rule 1.

  • A token identifier acceptable at that point in the grammar, for all other rules.

On page 2379,2380 line 76041,76043,76084-76134 section 2.10.2, change:
/* Apply rule ...
to:
/* Rule ...

On page 2379 line 76058 section 2.10.2, change:
case_item_ns      :     pattern ')' linebreak
                  |     pattern ')' compound_list
                  | '(' pattern ')' linebreak
                  | '(' pattern ')' compound_list
                  ;
case_item         :     pattern ')' linebreak     DSEMI linebreak
                  |     pattern ')' compound_list DSEMI linebreak
                  | '(' pattern ')' linebreak     DSEMI linebreak
                  | '(' pattern ')' compound_list DSEMI linebreak
                  ;
pattern           :             WORD      /* Apply rule 4 */
                  | pattern '|' WORD      /* Do not apply rule 4 */
to:
case_item_ns      : pattern_list ')' linebreak
                  | pattern_list ')' compound_list
                  ;
case_item         : pattern_list ')' linebreak     DSEMI linebreak
                  | pattern_list ')' compound_list DSEMI linebreak
                  ;
pattern_list      :                  WORD /* Rule 4 */
                  |              '(' WORD /* Do not apply rule 4 */
                  | pattern_list '|' WORD /* Do not apply rule 4 */
                  ;
(0005523)
geoffclare   
2021-11-11 16:47   
On page 2372 line 75769 section 2.9.4.3, change:
the first one of several patterns (see Section 2.13) that is matched by the string resulting
to:
the first pattern (see Section 2.13), if any are present, that is matched by the string resulting

On page 2372 line 75780 section 2.9.4.3, delete the line:
[(] pattern1 ) compound-list ;;

Note to the editor: in Issue 8 draft 1.1 the line to delete has "terminator" instead of ";;" because of the change to allow ";&".

On page 2375 line 75888 section 2.10.1, change:
... convert the token identifier type of the TOKEN to a token identifier acceptable at that point in the grammar.
to:
... convert the token identifier type of the TOKEN to:
  • The token identifier of the recognized reserved word, for rule 1.

  • A token identifier acceptable at that point in the grammar, for all other rules.

On page 2379 line 76058 section 2.10.2, change:
case_item_ns      :     pattern ')' linebreak
                  |     pattern ')' compound_list
                  | '(' pattern ')' linebreak
                  | '(' pattern ')' compound_list
                  ;
case_item         :     pattern ')' linebreak     DSEMI linebreak
                  |     pattern ')' compound_list DSEMI linebreak
                  | '(' pattern ')' linebreak     DSEMI linebreak
                  | '(' pattern ')' compound_list DSEMI linebreak
                  ;
pattern           :             WORD      /* Apply rule 4 */
                  | pattern '|' WORD      /* Do not apply rule 4 */
to:
case_item_ns      : pattern_list ')' linebreak
                  | pattern_list ')' compound_list
                  ;
case_item         : pattern_list ')' linebreak     DSEMI linebreak
                  | pattern_list ')' compound_list DSEMI linebreak
                  ;
pattern_list      :                  WORD /* Apply rule 4 */
                  |              '(' WORD /* Do not apply rule 4 */
                  | pattern_list '|' WORD /* Do not apply rule 4 */
                  ;




Viewing Issue Advanced Details
1453 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-02-16 10:10 2021-11-26 15:05
geoffclare
 
normal  
Applied  
Accepted  
   
Geoff Clare
The Open Group
lex
2885
95160, 95165
---
lex LC_COLLATE/LC_CTYPE incorrect wording about unspecified behaviour
The descriptions of LC_COLLATE and LC_CTYPE in the lex ENVIRONMENT VARIABLES section say:
If this variable is not set to the POSIX locale, the results are unspecified.

Taken at face value, this means that applications have to set those variables exactly as stated in order to avoid unspecified behaviour, e.g.:
LC_COLLATE=POSIX LC_CTYPE=POSIX lex ...
This is obviously not what was intended, which is that if the combination of the LANG, LC_COLLATE, LC_CTYPE, and LC_ALL variables results in the locale in effect for the LC_CTYPE or LC_COLLATE category not being the POSIX locale, the behaviour is unspecified.
On page 2885 line 95160,95165 section lex, delete:
If this variable is not set to the POSIX locale, the results are unspecified.

On page 2885 line 95170 section lex, add a paragraph after the list:
If the values (if any) of the LANG, LC_COLLATE, LC_CTYPE, and LC_ALL variables result in the locale in effect for the LC_CTYPE or LC_COLLATE category not being the POSIX locale, the behavior is unspecified.
and move the following text from LANG on page 2884 line 95154 to become a second sentence in the added paragraph:
(See [xref to XBD 8.2] for the precedence of internationalization variables used to determine the values of locale categories.)

There are no notes attached to this issue.




Viewing Issue Advanced Details
1452 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2021-02-11 15:07 2021-11-04 18:00
geoffclare
 
normal  
Interpretation Required  
Accepted As Marked  
   
Geoff Clare
The Open Group
prs
3120
104398
Proposed
See Note: 0005518.
prs :KV: is mandated but most implementations do not support it
The prs page includes :KV: in the table of "SCCS File Data Keywords" that can be used with the -d option. However, the AIX, Solaris and CSSC prs utilities do not recognise it (Solaris and CSSC output :KV: unaltered; non-support on AIX is based on the online AIX man page). HP-UX is the only implementation I could find that supports it, and it appears that all it does is substitute :KV: with "none", so I suspect the only reason it recognises the value is because it is listed in SUS.

The VSC test for this keyword says that it has not been implemented pending interpretation VWG/065/011994. So it seems that this is a known problem that was first raised in 1994 but has persisted until now. Unfortunately that date (Jan 19, 1994) predates the start of the ogtgbase email archive, and consequently I have been unable to find how that interpretation was resolved. Being 27 years old it probably doesn't affect how this should be handled now anyway.
1. Issue a "defect case" interpretation.

2. Delete the :KV: row of the table:
| :KV: | Keyword validation string | " | text | S |

Notes
(0005518)
Don Cragun   
2021-11-04 16:28   
(edited on: 2021-11-04 16:31)
Interpretation response
------------------------
The standard states the data keywords that can be used with prs -d, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
Only one known implementation supports :KV: and it always returns the value "None" for this keyword.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Delete the :KV: row of the table:

    
| :KV: | Keyword validation string | " | text | S |


(0005519)
agadmin   
2021-11-04 18:00   
An interpretation is proposed: 4th November 2021




Viewing Issue Advanced Details
1451 [Issue 8 drafts] Shell and Utilities Comment Clarification Requested 2021-02-11 11:07 2021-05-11 09:38
geoffclare
 
normal  
Applied  
Accepted As Marked  
   
Geoff Clare
The Open Group
awk
2395
78089
Note: 0005351
awk ARGV description addition about -v is problematic
Bug 0000974 added notes to the descriptions of ARGC and ARGV saying it is unspecified whether alterations to ARGC and to elements of ARGV can be made using the -v option.

The one for ARGC is fine, but the one for ARGV is problematic. This is because the standard places a requirement on applications to ensure that the -v option-argument has a variable name to the left of the '='. Thus it cannot be used to assign values to array elements.

Note that since bug 974 was tagged tc3-2008, this bug should also be tagged tc3-2008 despite being reported against Issue 8 draft 1.1.
Change:
It is unspecified whether this alteration to ARGV can be made using the -v option.
to:
Note that this alteration cannot be made using either the assignment operand or the -v option, because an operand with a '[' before '=' is treated as a file operand, not an assignment operand, and applications are required to ensure that the -v option-argument has the same form as an assignment operand. (See the OPTIONS and OPERANDS sections.)

Notes
(0005231)
geoffclare   
2021-02-11 11:08   
Credit to Arnold Robbins for catching this.
(0005351)
rhansen   
2021-05-06 15:18   
Change:
It is unspecified whether this alteration to ARGV can be made using the -v option.
to:
Note that alterations to ARGV cannot be made using either the assignment operand or the -v option, because an operand with a '[' before '=' is treated as a file operand, not an assignment operand, and applications are required to ensure that the -v option-argument has the same form as an assignment operand. (See the OPTIONS and OPERANDS sections.)
(0005352)
nick   
2021-05-06 15:21   
Note tagged as issue 8 since 0000974 is an approved interpretation




Viewing Issue Advanced Details
1449 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Editorial Enhancement Request 2021-02-04 17:25 2021-11-25 16:17
rhansen
 
normal  
Applied  
Accepted  
   
Richard Hansen
3 Definitions
---
"Decimal-point character" vs. "radix character"
The standard sometimes uses "decimal-point character" or "decimal point character" where "radix character" would be more appropriate (because POSIX does not have a definition of "decimal-point character").

Note: The C standard uses "decimal-point character" for the same concept as POSIX's "radix character" (section 7.1.1).
On page 53 after line 1637 insert a new definition:
Decimal-Point Character
See Radix Character.

Change page 83 line 2369 from:
Radix Character
to:
Radix Character (or Decimal-Point Character)
There are no notes attached to this issue.




Viewing Issue Advanced Details
1448 [Issue 8 drafts] System Interfaces Objection Error 2021-02-02 22:12 2021-07-02 10:58
kre
 
normal  
Applied  
Accepted As Marked  
   
Robert Elz
XSH 3/poll()
1399-1400
46800-1. 46807-46813, 46820, 46824, 46867
Note: 0005386
Misleading text in description of poll()
This report is about wording, I do not believe I am reporting any
technical issues with the text, simply incorrect ways of saying what
it is attempting to say. If this should have been filed as "Editorial"
rather than "Objection" then please change - but note that the issues are
not (quite) simple typos, punctuation errors, or similar.

I am also filing this against issue 8 draft 1.1 as the text in poll() has
been updated from issue 7 (+TC2) but the updates do not affect the issues I see
(I believe) so it could just as easily been files against that one.

At lines 46800-1:
   The poll( ) function shall identify those file descriptors on which an
   application can read or write data, or [...]

That would imply that (given a POLLIN response in revents) that it is
possible to successfully read data, which is not true. Later the text
(correctly) says (line 46846)
    Regular files shall always poll TRUE for reading and writing.
and that's true even if the seek pointer (current offset into the file)
is at EOF, in which case there is no more data to read.

Line 46807 (and lines 46808, 46809, and 46810 which are similar)
say it better:
    POLLIN Data other than high-priority data may be read without blocking.

what's important about poll() (and select()) is that after being told
that an operation may be attempted, the application will not block when
performing the relevant operation, not that anything in particular will
be returned from the attempt.

Much the same applies to the write is OK bits in revents - but there
there's another problem, the bit set indicates that some data can be
written without blocking, but it does not promise how much. The
APPLICATION USAGE section should probably be changed from None (line 46867)
and say that applications wanting to avoid blocking when writing need
to do all of: use poll() (or select()), set non-blocking mode on the
file descriptor, and write relatively small chunks of data (where how
small will depend upon the kind of file the descriptor references) - as
no matter what poll() says, since non-blocking mode does not apply in any
useful way to regular file i/o, if an application attempts to write a
gigabyte to a file in one operation, it is going to block.

And while this (at line 46819) about POLLHUP might be true
    This event and POLLOUT are mutually-exclusive;
however the rationale given (line 46820) makes no sense:
    a stream can never be writable if a hangup has occurred.
as what poll() is indicating is that the application will not
block if it attempts to (in this case) write using the file descriptor,
which is true if a hanhup has occurred, the write simply returns immediately
with an error indication.

In line 46824
    POLLNVAL The specified fd value is invalid.

Is there a definition somewhere of what it means for a file descriptor
to be "invalid". Normally one might assume that this would (inter alia)
include negative values, as file descriptors are all >= 0, but lines 46828-9
say what happens in the case of a fd < 0, and setting POLLNVAL is not it.

So does this just mean a fd greater than the maximum permitted by the
implementation? Or does an "invalid" fd simply mean one that is not
open? (that is where a read/write (etc) would return EBADF, except in
the < 0 case). Or something different than that?

[Aside; XBD 3.136 *File Descriptor" does not define "invalid" ones, there
is no definition of "invalid" (alone) in XBD 3 that I can see, but that I
failed to find it doesn't mean that it doesn't exist somewhere.


It might also be worth bringing back the EXAMPLES section that has been
deleted for Issue 8 (which was all about STREAMS I/O, all obsolete,
and so quite rightly deleted) with an example (or two) using sockets, the
application field (along with terminal I/O) where poll() is most used.
But supplying text for that in this bug report is too much for now...
If that does happen, please do not attempt to copy the old EXAMPLES text
at all, all that nonsense about "permission to write" was simply bigus
(poll has nothing whatever to do with permissions, or any other kind of
authorisation).
In the settence beginning on line 46801, change the words
    on which an application can read or write data
to
    on which an application can make an attempt to read or write data
    without blocking

In line 46808 change
    POLLRDNORM Normal data may be read without blocking.
to
    POLLRDNORM An attempt to read normal data will not block.

and similar changes on lines 46807, 46809, and 46810).

On line 46811 change
    POLLOUT Normal data may be written without blocking.
to
    POLLOUT An attempt to write one byte of data will not block.
                 How much data may be written without blocking depends
                 upon the object underlying the file descriptor, and its
                 current state.

and a similar change on line 46813 (because of the way it is worded, lime
46812 needs no change). Note that line 46813 (alone of all of these lines)
does not mention "without blocking", it should.

On line 46820 delete the words (and punctuation)
    ; a stream can never be writable if a hangup has occurred

On line 46824 either add an xref to the definition of "invalid" as it
applies to file descriptors, or change the words
     is invalid
to
     {something, I have no idea what this is actually trying to say}
but just maybe to:
     does not represent an open file descriptor in the process [thread ??].
     
On line 46867 change
     None
to (something like):
     When poll() indicates a read may be attempted, a resulting read()
     call may return data, indicate EOF, or indicate an error, but shall
     not block. When poll() indicates that a write may be attempted,
     the application may write one or more bytes, however, when more than
     one byte is written, there is no guarantee that no block will occur.
     Applications which desire to write more than a single byte each time
     poll() indicates that a write may occur, and which want to avoid
     blocking need to take other measures to determine how much may be
     successfully written.

     Applications should note that repeating a poll() call which indicated
     that I/O was possible on one or more of the file descriptors given,
     without causing some change to the state, either by altering the fds[]
     array, or causing appropriate input or output to occur on at least one
     file descriptor indicated as ready will simply result in ppll() busy
     waiting - a subsequent call will always return immediately indicating
     the same (or perhaps more) revents as the previous one.

It would also be useful to have more information included on how the
various flags apply to terminals and sockets (rather than the undue overload
applied to FIFOs). What constitutes an error (POLLERR) or hangup (POLLHUP)
or a terminal fd? A hangup, a revoked fd, a parity error ??? And similarly
for sockets, a closed connection? Receipt of an ICMP message (on INET[46[
sockets), a TCP RST? a TCP FIN? Lack of response (broken network) (after
how long?)

Lines 46826-7 do say:
    The significance and semantics of normal, priority, and high-priority
    data are file and device-specific.
but there is nothing similar about HUP or ERR.
Notes
(0005386)
geoffclare   
2021-06-18 14:08   
(edited on: 2021-06-24 15:29)
In the June 17th teleconference we decided that we should align the wording for poll() with what is currently used for select(), i.e. talk about descriptors being "ready for reading/writing" (with an explanation of what that means).

I think the following changes are close to what we would have come up with if we hadn't run out of time. My aim in posting them here is that we should use this as the starting point on the June 24th call, but will be able to address any feedback in that call rather than waiting for a subsequent call.

Page and line numbers are for draft 1.1 but the changes account for the addition of ppoll() in draft 2.

In the sentence beginning on line 46801, change:
on which an application can read or write data
to:
on which an application can make an attempt to read or write data without blocking

At line 46807, change:
POLLIN Data other than high-priority data may be read without blocking.
to:
POLLIN The file descriptor is ready for reading data other than high-priority data.

In line 46808 change:
POLLRDNORM Normal data may be read without blocking.
to:
POLLRDNORM The file descriptor is ready for reading normal data.

In line 46809 change:
POLLRDBAND Priority data may be read without blocking.
to:
POLLRDBAND The file descriptor is ready for reading priority data.

In line 46810 change:
POLLPRI High-priority data may be read without blocking.
to:
POLLPRI The file descriptor is ready for reading high-priority data.

In line 46811 change:
POLLOUT Normal data may be written without blocking.
to:
POLLOUT The file descriptor is ready for writing normal data.

In line 46813 change:
POLLWRBAND Priority data may be written.
to:
POLLWRBAND The file descriptor is ready for writing priority data.

In line 46814 change:
An error has occurred on the device or stream.
to:
An error condition is present on the file descriptor. All error conditions that arise solely from the state of the object underlying the open file description and would be diagnosed by a return of -1 from a read() or write() call on the file descriptor shall be reported as a POLLERR event.

On line 46820 delete the words (and punctuation):
; a stream can never be writable if a hangup has occurred

On line 46824 change:
is invalid
to:
is not an open file descriptor

Before line 46826 insert a new paragraph:
A file descriptor shall be considered ready for reading when a call to an input function with O_NONBLOCK clear would not block, whether or not the function would transfer data successfully. (The function might return data, an end-of-file indication, or an error other than one indicating that it is blocked, and in each of these cases the descriptor is considered ready for reading.) A file descriptor shall be considered ready for writing when a call to an output function with O_NONBLOCK clear would not block, whether or not the function would transfer data successfully. How much data could be written without blocking depends upon the object underlying the open file description and its current state.

On line 46826-7 after:
The significance and semantics of normal, priority, and high-priority data are file and device-specific.
add:
The semantics of device disconnection are device-specific.

On line 46867 change APPLICATION USAGE from:
None
to:
When a poll() or ppoll() call indicates a file descriptor is ready for reading, this means that if an attempt to read data had been made at the time that the status of the file descriptor was checked, it would have returned at least one byte of data, an end-of-file indication, or an error, without blocking (even if O_NONBLOCK is clear). When a poll() or ppoll() call indicates that a file descriptor is ready for writing, this means that if an attempt to write one byte of data had been made at the time that the status of the file descriptor was checked, it would have written that byte or returned an error, without blocking. However, if an attempt to write more than one byte had been made, it might have blocked (if O_NONBLOCK is clear). In both cases, by the time the call returns and a subsequent I/O operation is attempted, the state of the file descriptor might have changed (for example, because another thread read or wrote some data) and, if O_NONBLOCK is clear, there is no guarantee that the operation will not block (unless it would not block for some other reason, such as setting MIN=0 and TIME=0 for a terminal in non-canonical mode). Therefore it is recommended that applications always set O_NONBLOCK on file descriptors whose readiness for I/O they query with poll() or ppoll().

The error conditions specified for read() and write() that are reported as POLLERR events are only those that arise solely from the state of the object underlying the open file description. They do not include, for example, [EAGAIN] as this relates to the state of the open file description not (solely) the object underlying it.

Application writers should note that repeating a poll() or ppoll() call which indicated that I/O was possible on one or more of the file descriptors given, without causing some change to the state, either by altering the fds array or causing appropriate input or output to occur on at least one file descriptor indicated as ready, will result in ``busy waiting'' - a subsequent call will always return immediately indicating the same (or perhaps more) events as the previous one.

On page 1480 line 49261 section pselect(), change APPLICATION USAGE from:
None
to:
When a pselect() or select() call indicates a file descriptor is ready for reading, this means that if an attempt to read data had been made at the time that the status of the file descriptor was checked, it would have returned at least one byte of data, an end-of-file indication, or an error, without blocking (even if O_NONBLOCK is clear). When a pselect() or select() call indicates a file descriptor is ready for writing, this means that if an attempt to write one byte of data had been made at the time that the status of the file descriptor was checked, it would have written that byte or returned an error, without blocking. However, if an attempt to write more than one byte had been made, it might have blocked (if O_NONBLOCK is clear). In both cases, by the time the call returns and a subsequent I/O operation is attempted, the state of the file descriptor might have changed (for example, because another thread read or wrote some data) and, if O_NONBLOCK is clear, there is no guarantee that the operation will not block (unless it would not block for some other reason, such as setting MIN=0 and TIME=0 for a terminal in non-canonical mode). Therefore it is recommended that applications always set O_NONBLOCK on file descriptors whose readiness for I/O they query with pselect() or select().






Viewing Issue Advanced Details
1446 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2021-01-25 18:17 2021-11-25 16:28
dalias
 
normal  
Applied  
Accepted As Marked  
   
Rich Felker
musl libc
posix_spawn_file_actions_addopen et al
unknown
unknown
---
See Note: 0005511
Does a shall-fail in terms of an undefined limits.h macro impose any requirement?
posix_spawn_file_actions_addopen has a "shall fail" error condition for "The value specified by fildes is negative or greater than or equal to {OPEN_MAX}" and similar conditions appear elsewhere. If the implementation does not define OPEN_MAX, does this impose any requirement on the implementation to fail when the value exceeds a dynamic sysconf(_SC_OPEN_MAX), or is the condition vacuously false?
Clarity whether {OPEN_MAX} refers to the actual macro value or some abstract limit by the name whose value may only be available at runtime, or just explicitly limit the condition to the case where it is defined, as in:

EBADF
    The value specified by fildes is negative or the implementation defines OPEN_MAX in limits.h and the value specified by filedes is greater than or equal to OPEN_MAX.
Notes
(0005209)
geoffclare   
2021-01-26 09:39   
The <limits.h> page contains these paragraphs before the list of limits that includes {OPEN_MAX}:
A definition of one of the symbolic constants in the following list shall be omitted from <limits.h> on specific implementations where the corresponding value is equal to or greater than the stated minimum, but is unspecified.

This indetermination might depend on the amount of available memory space on a specific instance of a specific implementation. The actual value supported by a specific instance shall be provided by the sysconf() function.

I would hope that from this it is sufficiently clear that any error condition worded in terms of {OPEN_MAX} is required to be checked against the runtime value obtained from sysconf() if there is no OPEN_MAX definition in <limits.h>. Likewise for the other limits in the same list.
(0005211)
dalias   
2021-01-26 17:27   
I don't see any way that this is "sufficiently clear". Either reading is plausible and defensible, and moreover semantically the check only makes sense if the limit is static, since otherwise it's an inherent TOCTOU error (limit at time of creating the file actions object may be different than limit at the time of posix_spawn).

If a particular reading is intended it should be documented explicitly. That could be general text in the limits.h specification, to the effect that when {x} appears in the description of an interface it should be read as the dynamic limit obtainable by sysconf if the static limit is not defined. However I wonder if there might be other places where such a requirement is unintended and/or harmful, and I'd like to have a thorough review of everything potentially affected.
(0005212)
shware_systems   
2021-01-26 18:05   
I believe the expectation is these values will be static, at least for the life of the process, so applications and implementation internals can rely on sysconf(). They certainly would be if the header provided them as #define's, after all. The only reason I see for returning a different value on a subsequent call would be to mirror calls by an application to other interfaces requesting a higher or lower value be applied. Maybe this needs to be stated better in <limts.h>, maybe not, that it is on implementations not to modify these arbitrarily without such requests, as a conformance requirement.
(0005213)
dalias   
2021-01-26 18:15   
In the case of sysconf(_SC_OPEN_MAX), historically it varies dynamically within the process lifetime with setrlimit (RLIMIT_NOFILE), and moreover it's very reasonable to want to change the limit for a child process (in fact various implementors have proposed extending posix_spawn to include setrlimit-like action). Maybe it's inappropriate for sysconf(_SC_OPEN_MAX) to represent the rlimit anyway and it should just be returning -1. But this is something of a mess at present, I think.
(0005214)
shware_systems   
2021-01-26 18:49   
That is the application explicitly requesting changes, though, via setrlimit(). It's on the developer to guard against TOCTOU for that circumstance, not the standard.

Where interfaces indicate a value that isn't necessarily constant is to be referenced the behavior should be as if sysconf() is called, in the description of the curly brace notation by default more than per interface, whether the header includes an entry for the constant or not.
(0005511)
nick   
2021-10-28 16:35   
(edited on: 2021-10-28 16:39)
In frontmatter on page xvi change item 4 in the list from:
    
Names surrounded by braces represent symbolic limits or configuration values which may be declared in appropriate headers by means of the C #define construct.

to:
    
Names surrounded by braces represent symbolic limits or configuration values which may be declared in appropriate headers by means of the C #define construct, or obtained at runtime from functions or utilities that return limit or configuration values.

Add a new element to the list on P270 after L8998:
   
  • The limit has been changed at runtime by an application (see xref to setrlimit()).


  • On P270 L8999 change:
        
    For these reasons, an application may use the fpathconf( ), pathconf( ), and sysconf( ) functions to determine the actual value of a limit at runtime.

    to:
        
    For these reasons, an application can use the fpathconf( ), getrlimit(), pathconf( ), and sysconf( ) functions to determine the actual value of a limit at runtime.

    (0005539)
    geoffclare   
    2021-11-25 16:28   
    The troff changes have been applied in gitlab. The frontmatter change has been applied in an updated version to be used for Issue 8 draft 3. In the unlikely event that we do an Issue 7 TC3, the frontmatter change will also be needed there.




    Viewing Issue Advanced Details
    1445 [Issue 8 drafts] Shell and Utilities Editorial Error 2021-01-21 18:11 2021-02-12 16:50
    rhansen
     
    normal  
    Applied  
    Accepted  
       
    Richard Hansen
    2.14
    2311
    74980-74981
    Update "special built-in" to match changes from bug #854
    Bug #854 updated many sections regarding PATH search for different types of utilities but missed a spot in XCU 2.14 (as mentioned in Note: 0003228).
    On page 2311 lines 74980-74981 (XCU 2.14 2nd paragraph) change:
    The term "built-in" implies that the shell can execute the utility directly and does not need to search for it.
    to:
    The term "built-in" implies that there is no need to execute a separate executable file because the utility is implemented in the shell itself.
    There are no notes attached to this issue.




    Viewing Issue Advanced Details
    1444 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Editorial Error 2021-01-19 10:42 2021-11-25 16:12
    geoffclare
     
    normal  
    Applied  
    Accepted  
       
    Geoff Clare
    The Open Group
    <wordexp.h>
    466
    16113
    ---
    Broken cross-references to wordexp()
    The SEE ALSO section on the <wordexp.h> page is supposed to have a cross-reference to the wordexp() function in XSH, but it instead points to XCU 2.6. The XSH pages for fnmatch() and glob() have the same problem.

    This is happening because the troff source uses the same tag "wordexp" for both of these cross-reference targets.
    Fix the broken wordexp() cross-references in SEE ALSO on the <wordexp.h>, fnmatch(), and glob() pages.

    Note to the editor: there are many more correct xrefs to XCU 2.6 than there are broken xrefs to wordexp(), so the least disruptive way to fix this is to change the xref tag for wordexp() to "wordexp_xsh".
    There are no notes attached to this issue.




    Viewing Issue Advanced Details
    1443 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Omission 2021-01-19 10:40 2021-11-25 16:10
    geoffclare
     
    normal  
    Applied  
    Accepted  
       
    Geoff Clare
    The Open Group
    3.274 Pattern
    77
    2224
    ---
    Definition of "pattern" omits some uses
    The definition of "pattern" talks about pathname expansion, but not pathname matching or other uses of shell patterns, such as case statements or the four varieties of parameter expansion that provide for substring processing.

    It should talk about shell pattern matching notation the same way it does about regular expression notation, including referring in the small-font note to the section that defines the notation (instead of the current reference to the section about pathname expansion).

    The problem extends to the text "as a means of selecting various character strings or pathnames", which only covers a subset of the uses of patterns. It would be best just to omit this part, as the references in the small-font note can be followed to find out more about what each type of pattern is used for. (Although an addition to XCU 2.13 along the lines suggested below would help in this regard.)
    On page 77 line 2224 section 3.274 Pattern, change:
    A sequence of characters used either with regular expression notation or for pathname expansion, as a means of selecting various character strings or pathnames, respectively.
    to:
    A sequence of characters used either with regular expression notation or with shell pattern matching notation.

    On page 77 line 2227 section 3.274 Pattern, change:
    See also [xref to XCU 2.6.6 Pathname Expansion].
    to:
    Shell pattern matching notation is defined in detail in [xref to XCU 2.13 Pattern Matching Notation].

    On page 2308 line 74866 section 2.13 Pattern Matching Notation, after:
    The pattern matching notation described in this section is used to specify patterns for matching strings in the shell.
    add a sentence:
    This notation is also used by some other utilities (find, pax, and optionally make) and by some system interfaces (fnmatch(), glob(), and wordexp()).

    There are no notes attached to this issue.




    Viewing Issue Advanced Details
    1442 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Enhancement Request 2021-01-10 13:10 2021-11-25 16:08
    andras_farkas
     
    normal  
    Applied  
    Accepted  
       
    Andras Farkas
    getopts
    2840
    93538 and 93541
    ---
    Improvement to getopts example, remove double negation
    On getopts's page
    https://pubs.opengroup.org/onlinepubs/9699919799/utilities/getopts.html [^]
    The example uses interesting double negation in the two tests:
    if [ ! -z "$aflag" ]; then
    if [ ! -z "$bflag" ]; then
    ! -z could be shortened to -n
    Replace
    if [ ! -z "$aflag" ]; then
    with
    if [ -n "$aflag" ]; then
    and
    if [ ! -z "$bflag" ]; then
    with
    if [ -n "$bflag" ]; then
    There are no notes attached to this issue.




    Viewing Issue Advanced Details
    1441 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2021-01-07 16:56 2021-11-25 16:07
    rhansen
     
    normal  
    Applied  
    Accepted  
       
    Richard Hansen
    2.5.3 Shell Variables, PS4
    2352
    74965
    ---
    PS4 should be printed for non-interactive shells as well
    The specification of PS4 only applies to interactive shells. It should also apply to non-interactive shells.
    On page 2352 line 74965 (XCU 2.5.3 PS4), change:
    When an execution trace (set −x) is being performed in an interactive shell,
    to:
    When an execution trace (set −x) is being performed,

    There are no notes attached to this issue.




    Viewing Issue Advanced Details
    1440 [1003.1(2016/18)/Issue7+TC2] System Interfaces Comment Omission 2021-01-04 15:49 2021-11-04 15:28
    ciprian_craciun
     
    normal  
    Interpretation Required  
    Accepted As Marked  
       
    Ciprian Dorin Craciun
    system(3)
    N/A
    N/A
    Proposed
    Note: 0005510
    Calling `system("-some-tool")` fails (although it is a valid `sh` command)
    This report is in relation to the following `glibc` and Linux man-pages issues:
    * https://sourceware.org/bugzilla/show_bug.cgi?id=27143 [^]
    * https://bugzilla.kernel.org/show_bug.cgi?id=211029 [^]

    However it seems that the root cause is the interaction mandated by the POSIX standard between `system(3)` and `sh(1)`.


    In simple terms, calling `system("-some-tool")` has an unexpected behaviour when delegating that to `sh`, as in `sh -c -some-tool`. (For how to replicate this just see the first link.)


    In more details:

    * the `system(3)` specification requires that:
    ~~~~
    # from: https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html [^]

    The system() function shall behave as if a child process were created using fork(), and the child process invoked the sh utility using execl() as follows:

    execl(<shell path>, "sh", "-c", command, (char *)0);
    ~~~~

    * however the `sh(1)` specification requires that:
    ~~~~
    # from https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html [^]

    sh -c [-abCefhimnuvx] [-o option]... [+abCefhimnuvx] [+o option]...
           command_string [command_name [argument...]]
    ~~~~

    I.e. `sh(1)` allows extra flags between `-c` and the `command_string`; however the `system(3)` doesn't account for a command that might start with a hyphen.

    Thus, depending on what the command might be, the outcome is either a missing command (if the given command resembles an option accepted by `sh`), or an unknown option (if the given command doesn't resemble an option accepted by `sh`).
    The "optimal" solution would be to require `system(3)` to call `sh(1)` with an extra `--` between `-c` and the `command`, as in:
    ~~~~
    execl(<shell path>, "sh", "-c", "--", command, (char *)0);
                                    ^^^^
    ~~~~

    Given that `sh(1)` already mandates a compliant shell to correctly handle `--`, this shouldn't have many unintended consequences.

    Alternatively add a warning in the specification stating that a command starting with `-` has an undefined behaviour and should be worked-around by preperding a space to any such command.
    Notes
    (0005191)
    Vincent Lefevre   
    2021-01-06 13:20   
    Since with -c, there must be at least a non-option argument, how about doing a correction/improvement in sh as follows? If -c is given and there is a single argument after -c, then this argument should be regarded as a command string.

    (This could be done in addition to adding "--" on the system() side.)
    (0005192)
    ciprian_craciun   
    2021-01-06 15:41   
    > Since with -c, there must be at least a non-option argument, how about doing a correction/improvement in sh as follows? If -c is given and there is a single argument after -c, then this argument should be regarded as a command string.


    I think adding such a requirement is not advisable for multiple reasons:

    * first, none of the existing `sh` implementations (as reported in one of the tickets none of `bash`, `dash` or `zsh`) exhibit such a behaviour, therefore this change will all of a sudden break their POSIX compliance;

    * second, although it solves the `system` corner-case, it opens other potential issues; imagine for example a script that calls `sh -c -x "${@}"`; now if the user doesn't provide an argument, then all of a sudden `sh` executes `-x` as a command, as opposed to until now when it didn't execute anything;


    Personally I don't know what is the best course of action here, besides updating `system` to use the `--` separator which is backward compatible and shouldn't break virtually anything.

    However, given how much "legacy" there is based on `sh`, I wouldn't change its behaviour, except perhaps giving a warning in the man page about the situation.
    (0005193)
    joerg   
    2021-01-06 16:12   
    Please note that this problem has been introduced by POSIX. It does not
    exist in pre-POSIX shells.

    In the classical Bourne Shell, -c wasn't even seen as a real option but
    as some kind of prefix to the command argument.

    Since the problem has been introduced by shells while implementing POSIX
    requirements, I am fully open to set up a new POSIX requirement for shell
    option parsing that would fix the parsing to make it compatible to the
    historic Bourne Shell again.

    This would however require to find a simple and still always working
    algorithm. Just counting the arguments after -c does not work.

    Note that with the classical Bourne Shell

        sh -x -c 'echo foo'

    does not work, but requiring it to work by POSIX seems to be a useful
    enhancement compared to the classical Bourne Shell and the fact that
    POSIX requires

        sh -c -x 'echo foo'

    to work also seems to be a useful extension.

    As a result, it may be hard to fix this problem in the shell...
    (0005194)
    kre   
    2021-01-06 16:13   
    (edited on: 2021-01-06 16:22)
    Re Note: 0005191 and Note: 0005192

    Not only is the suggestion in Note: 0005191 not the right thing to do,
    as showin in Note: 0005192 (it would also cause sh to not satisfy the
    utility guidelines in XBD 12.1)

    But beyond that, it would also be insufficient to solve the problem
    in general, as there can be more than one arg after -c, and the first
    of those might begin with a "-" - which means the shell would either
    fail just the same way (just not perhaps when called from system()) or
    would run the wrong thing as a command.

    This needs to be fixed in system(), or in the application, not in sh.

    That could be done in system() adding a -- in the arg list, perhaps only if
    the command string starts with a '-', or by prefixing a space to the arg
    (though that would mean copying it, so probably would not be the method of
    choice), or however else the system() function wants to do it.

    Or we could simply note the potential issue in the APPLICATION USAGE of
    system() and tell application authors not to attempt system() (or popen())
    with a string starting with a '-' - there are a zillion ways (including
    prependiong a space, or quoting the command name, or ...) the application
    can avoid that ever happening.

    Personally, I would put the onus on the application - the way system()
    works has neevr really changed, the way sh works has never changed, it
    has always been necessary for the application to deal with the combination
    that results from that if they're doing something unusual (like starting
    a command with a '-').

    (0005199)
    Vincent Lefevre   
    2021-01-13 10:50   
    And what about the invocation of $EDITOR and $VISUAL?

    A proposed manpages patch for Debian at https://bugs.debian.org/866060 [^] has:

     .BR EDITOR / VISUAL
     The user's preferred utility to edit text files.
    +Any string acceptable as a command_string operand to the
    +.I sh -c
    +command shall be valid.

    Executing the command with "sh -c" will fail if it starts with "-". But I wonder how this is handled in practice in various implementations and whether something is missing in the spec, which is not very explicit.
    (0005510)
    geoffclare   
    2021-10-28 15:35   
    (edited on: 2021-11-04 15:36)
    Interpretation response
    ------------------------
    The standard states that system() behaves as if it passes "-c" followed by the command string to execl() , and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

    Rationale:
    -------------
    There is nothing known that applications can usefully do if the "--" is omitted, therefore there is no reason that the standard should not require the "--".
    The standard warns applications not to use filenames that start with <hyphen-minus> ('-'). (See the definition of Portable Filename in XBD chapter 3). However, there is no equivalent warning for filenames beginning with <plus-sign> ('+'), and applications should be able to use system() to execute commands where the command word begins with '+' (without the need for special precautions such as adding a <space> at the beginning).

    Notes to the Editor (not part of this interpretation):
    -------------------------------------------------------

    On page 1437 line 47730 section popen(), change:
    execl(shell path, "sh", "-c", command, (char *)0);
    where shell path is ...
    to:
    execl(<shell path>, "sh", "-c", "--", command, (ch
    ar *)0);
    where <shell path> is ...

        
    On page 2107 line 67568 section system(), change:
    execl(<shell path>, "sh", "-c", command, (char *)0);

    to:
    execl(<shell path>, "sh", "-c", "--", command
    , (char *)0);


    (Note: unwanted line splitting in the above is a Mantis artifact.)

    Add on page 1439 after line 47807 and on page 2110 after line 67736:
    Earlier versions of this standard required the command string to be passed as the next argument after "-c" (omitting "--"). This meant that portable applications needed to take care not to pass a command string beginning with <hyphen-minus> ('-') or <plus-sign> ('+'), as it would then be interpreted as containing options. Now that implementations are required to pass the "--", applications no longer need to do this.


    (0005512)
    agadmin   
    2021-10-29 08:44   
    (edited on: 2021-11-04 17:56)
    An interpretation is proposed: 4th November 2021 (note the timer is restarted)





    Viewing Issue Advanced Details
    1438 [1003.1(2016/18)/Issue7+TC2] System Interfaces Editorial Clarification Requested 2020-12-17 19:19 2021-11-25 16:05
    steffen
     
    normal  
    Applied  
    Accepted  
       
    steffen
    iconv
    1123
    38001
    ---
    iconv: clarify which "codeset" is meant
    Nitpick that seems to sometimes cause issues in practice, as shown in thread on ML.
    On page 1123 lines 38001 ff. change

    If a sequence of input bytes does not form a valid character in the specified codeset, conversion shall stop after the previous successfully converted character.

    to

    If a sequence of input bytes does not form a valid character in the specified input codeset, conversion shall stop after the previous successfully converted character.
    There are no notes attached to this issue.




    Viewing Issue Advanced Details
    1437 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-12-15 21:02 2021-11-26 15:03
    steffen
     
    normal  
    Applied  
    Accepted As Marked  
       
    steffen
    Vol. 3: Shell and Utilities, Issue 7, make
    (2975) 2989
    (98708, 98742) 99308
    ---
    See Note: 0005489.
    make: (document .NOTPARALLEL and .WAIT special targets) in RATIONALE
    Parallel, even massively parallel processing has become the widely supported and
    used default, yet the standard make(1) does not document it.

    If the -j command line option as requested in issue #1436 is standardized,
    the standard should also mention ways to synchronize rule processing, in order
    to support more complicated build relationships seen in real-life projects.

    The suggestion given here can be used with the most widely used make(1)
    implementations (GNU make, BSD make) on any tested platform (Linux, *BSD, SunOS
    / Solaris).

    The make family rooted on the Sun side of likfe has has at least .WAIT: reserved
    for decades, and in practice

      .NOTPARALLEL:
      .WAIT: # Luckily BSD make supports specifying this as target, too

      tangerine: config .WAIT build .WAIT test .WAIT install
      citron: config .WAIT build .WAIT install
      all: config .WAIT build

    produces the desired outcome.
    (
    On page 2975, insert before line 98708

      .NOTPARALLEL
        When specified all rules specified in the given makefile are processed
        sequentially, as if -j has not been given.

    On page 2975, insert before line 98472

      .WAIT
        When .WAIT appears in a dependency line, prerequisites that appear after it
        are built no sooner until all preceeding prerequisites have been built
        successfully.

    On page 2975, change on line 98472 ff.

      The special targets .IGNORE, .POSIX, .PRECIOUS, .SILENT, and .SUFFIXES shall
      be specified without commands.

    to

      The special targets .IGNORE, .NOTPARALLEL, .POSIX, .PRECIOUS, .SILENT,
      .SUFFIXES and .WAIT shall be specified without commands.
    )

    On page 2989, insert before line 99308

      The special targets .NOTPARALLEL and .WAIT represent different, rather
      incompatible synchronisation concepts of parallel build environments as
      used by the diverse implementations. By using an indirection both approaches
      can be supported: for implementations which use .NOTPARALLEL in a makefile to
      ensure sequential rule processing in the given file the full power of parallel
      processing will be available in recursive make instances invoked on makefiles
      without the directive, implementations which synchronize on the .WAIT special
      target will synchronize on it, for example:

        .NOTPARALLEL: # Process all rules sequentially
        .WAIT: # Define to satisfy "the other implementation"

        install: config .WAIT all
          echo Installing..
          ..
        all: config .WAIT build
        build:
          make -f makefile.build
        config:
          ./configure
    make.diff (16 KB) 2021-08-28 19:11
    easier.diff (7 KB) 2021-09-02 23:20
    gmake-4.3.wait.diff (6 KB) 2021-09-03 13:33
    Notes
    (0005184)
    psmith   
    2020-12-17 17:10   
    Just to note, GNU make does not support the .WAIT special target.
    (0005185)
    steffen   
    2020-12-17 19:23   
    Ach! I wish it would, i think the .WAIT approach "is better" (as in explicit, and does not need an indirection).
    But you said on the ML that you do not plan to support it either.

    This is why the first part above is in parenthesis, and this issue actually requests some words in RATIONALE instead.

    The presented solution has the desired effect also in GNU make ("build" target uses parallel processing).
    (0005186)
    psmith   
    2020-12-17 21:16   
    I don't believe I said I didn't want GNU make to support it; I agree that it seems like a useful feature. I just said that given the way make's parallel support works I suspect implementation is not so easy and I didn't plan to work on it as I have other things on my (relatively small) plate.

    But, if someone else were to get motivated to try to get this working that would be fine with me :). I wonder if creating some kind of internal/implicit prerequisite relationship that wasn't visible to users would be a simpler way to implement this... or not. Well, something for someone to consider.
    (0005444)
    joerg   
    2021-08-14 12:19   
    Well, implementing .WAIT is not really hard.

    You need to stop creating new active process slots if you see a .WAIT
    in the dependency list but wait until all running jobs for that target
    are finished before starting new jobs from the list to the right of
    .WAIT.

    You also need to remove .WAIT from the list of dependencies, if that
    list is needed for things like e.g. $^ or $?.

    In cases of doubt, SunPro Make is OpenSource...
    (0005446)
    rhansen   
    2021-08-19 16:22   
    On page 2975, insert before line 98708:
    .NOTPARALLEL
    The application shall ensure that this special target is specified without prerequisites or commands. When specified, make shall update one target at a time, regardless of whether the -j maxjobs option is specified. If the -j maxjobs option is specified, the option shall continue to be passed unchanged to sub-make invocations via MAKEFLAGS.

    On page 2975, insert before line 98742:
    .WAIT
    The application shall ensure that this special target, if specified as a target, is specified without prerequisites or commands. When .WAIT appears as a target, it shall have no effect. When .WAIT appears in a target rule as a prerequisite, it shall not itself be treated as a prerequisite; however, prerequisites that appear after it shall not be brought up-to-date until after all preceding prerequisites for the same target have been brought up-to-date.

    On page 2975, change line 98742 from:
    The special targets .IGNORE, .POSIX, .PRECIOUS, .SILENT, and .SUFFIXES shall be specified without commands.
    to:
    The special targets .IGNORE, .NOTPARALLEL, .POSIX, .PRECIOUS, .SILENT, .SUFFIXES and .WAIT shall be specified without commands.

    On page 2975 lines 98744-98747, change:
    Targets with names consisting of a leading <period> followed by the uppercase letters "POSIX" and then any other characters are reserved for future standardization. Targets with names consisting of a leading <period> followed by one or more uppercase letters are reserved for implementation extensions.
    to:
    Targets and prerequisites consisting of a leading <period> followed by the uppercase letters "POSIX" and then any other characters are reserved for future standardization. Targets and prerequisites consisting of a leading <period> followed by one or more uppercase letters, that are not described above, are reserved for implementation extensions.


    (0005447)
    psmith   
    2021-08-19 18:39   
    I'm assuming the title of this issue should reference the RATIONALE section not the RATIONAL section.

    I don't have a copy of the document that includes these line numbers so I can't quite understand what is being proposed here. The original proposal was to add this to the RATIONALE section which means it would not be binding and required (as I understand things) but instead only a suggestion, and probably reserving these targets for future implementations.

    As far as I can tell, though, these changes are being made to the "Extended Description" section, not the RATIONALE section, which makes these new features that are required of all conforming implementations.

    Am I wrong about that?

    As I said above, GNU make doesn't support the .WAIT feature.

    As I said above, it is likely not actually trivial to implement .WAIT in GNU make. I don't know how Sun make implements parallelism but in GNU make we don't require an entire branch of the graph to complete before moving to the next sibling branch, if parallel builds are enabled. For example:

      all: ONE TWO

      ONE: THREE FOUR

      ONE TWO THREE FOUR: ; echo $@; sleep 2

    If we run this as "make -j4" then we'll get the following output:

      echo THREE; sleep 1
      echo FOUR; sleep 1
      echo TWO; sleep 1
      THREE
      FOUR
      TWO
      echo ONE; sleep 1
      ONE

    Note that GNU make runs TWO in parallel with THREE and FOUR, even though ONE has not completed yet. You can imagine that TWO itself is a complex dependency graph and the actual target built in parallel may well be a long ways away (in the graph). If we change this makefile to add the putative .WAIT:

      all: ONE TWO

      ONE: THREE .WAIT FOUR

      ONE TWO THREE FOUR: ; echo $@; sleep 2

    what we'd like to see (one assumes) is:

      echo THREE; sleep 1
      echo TWO; sleep 1
      THREE
      TWO
      echo FOUR; sleep 1
      FOUR
      echo ONE; sleep 1
      ONE

    so that we'd run TWO in parallel with THREE, rather than waiting to start TWO until after FOUR is complete.

    This means we need to be able to detect a .WAIT target and stop searching the DAG at that point, but rather than just waiting for the previous dependencies to complete we want to move on to other siblings of the current target and see if there are any more targets that might be buildable, then come back later after THREE has completed.
    (0005448)
    rhansen   
    2021-08-19 18:55   
    (edited on: 2021-08-19 19:04)
    > As far as I can tell, though, these changes are being made to the "Extended Description" section, not the RATIONALE section, which makes these new features that are required of all conforming implementations.

    Correct, that was our intention. I asked about making .WAIT non-normative during today's telecon (see line 68 of https://posix.rhansen.org/p/2021-08-19 [^] ), and the group decided that it was useful enough to include as a normative requirement.

    But thinking about this more, isn't:
    target: $(leftstuff) .WAIT $(rightstuff); do_thing
    equivalent to:
    $(rightstuff): $(leftstuff)
    target: $(leftstuff) $(rightstuff); do_thing
    ?

    If so, two questions:
    • Could it be easily implemented in GNU make as syntactic sugar?
    • Does .WAIT add enough value to warrant inclusion in the standard?


    (0005449)
    shware_systems   
    2021-08-19 19:27   
    If this was to be a TC type addition, then yes, keeping it as Rationale and Future Direction would be the way to go. However, as most modern CPUs do have multiple cores capable of parallelism, the consensus was more the future is now, rather than wait until Issue 9, to make them normative.

    Your concerns were part of the discussion, including a paraphrase of the example you provided, and the feeling was more non-trivial is not the same as non-doable too, it just may take longer for someone to do it, so was not a bar to adopting it.
    (0005450)
    psmith   
    2021-08-19 22:47   
    > But thinking about this more, isn't: ... equivalent to:

    Yes, basically: that's what I meant when I wrote in my comment last December, above:

    > I wonder if creating some kind of internal/implicit prerequisite relationship that wasn't visible to users would be a simpler way to implement this... or not.

    However your construction is not quite correct because it says that $(rightstuff) will be rebuilt if $(leftstuff) is rebuilt, which is wrong. If you use GNU make's "order-only prerequisite" feature, though, you could write it like this and it would work properly:

      target: $(leftstuff) $(rightstuff); do_thing
      $(rightstuff): | $(leftstuff)

    I think a real implementation of .WAIT is more effort than is implied by the term "syntactic sugar", but this seems to be a promising way to think about the problem.

    At this time I have no plans, or time, to implement either the new :::= operator, which I continue to maintain as per my comments in that issue is simply not useful, or certainly not useful enough to rise to the level of being added to the standard; or the .WAIT feature, which I agree is a good idea but which is a non-trivial amount of work.

    So, I guess GNU make will just become a non-conforming implementation until such time as someone puts forward the effort to change that.

    > this was to be a TC type addition, then yes, keeping it as Rationale and Future Direction would be the way to go. However, as most modern CPUs do have multiple cores capable of parallelism, the consensus was more the future is now, rather than wait until Issue 9, to make them normative.

    This seems like a strawman to me: I have not suggested that adding -j was a problem: indeed, I welcome the addition of -j and agree it allows better use of modern systems.

    However, people have been writing makefiles supporting parallelism using GNU make for 25+ years without the .WAIT capability. It definitely makes some (fairly rare) things easier, but there are other ways to manage those situations and not having .WAIT has not prevented massive build systems from being successful using parallel GNU make.

    > Your concerns were part of the discussion, including a paraphrase of the example you provided, and the feeling was more non-trivial is not the same as non-doable too, it just may take longer for someone to do it, so was not a bar to adopting it.

    Pretty much everything is a SMOP, in the end. If the only barrier to standardization is what is possible then indeed there's virtually no limit.
    (0005451)
    shware_systems   
    2021-08-19 23:09   
    Re: 5448
    No, I don't see them as equivalent. The version with .WAIT has the greater of left or right count pool removals starting in parallel. The replacement you suggest has potentially left count times right count starting that way.
    (0005452)
    psmith   
    2021-08-20 18:38   
    Sorry, I didn't understand the comment #5451: I don't know what a "left or right count pool removal" is, or why anything would be "left count times right count". The comment in #5448 IS an equivalent way to represent the .WAIT version with one significant difference, as I said in my comment: it declares that if any of the targets in $(leftstuff) are updated, then all the targets in $(rightstuff) will be considered out of date. Other than that it behaves identically.

    The extra dependency relationship is a fundamental change in the behavior, though, so that's not good. GNU make has a feature which avoids it. Nevertheless, the .WAIT version is easier to read and write so it's helpful. Just, not necessarily trivial to implement.
    (0005453)
    rhansen   
    2021-08-20 19:28   
    (edited on: 2021-08-20 19:43)
    > If you use GNU make's "order-only prerequisite" feature, though, you could write it like this and it would work properly:

    TIL about GNU make's order-only prerequisites, thank you. It's good to know that all of the make implementations provide a way to order prerequisites, just with different syntax (.WAIT vs. pipe) and slightly different semantics (but similar enough that one can fairly easily convert a makefile from one format to the other).

    Given that all makes provide a way to order prerequisites, I think that adding the feature to POSIX in some form is a worthwhile activity. If everyone agrees, the questions I think we should consider futher are:

    • Would it be better to standardize GNU make's behavior instead of .WAIT? I agree with your statement "the .WAIT version is easier to read and write", so I think it would be better to standardize .WAIT.

    • Should we standardize it now? GNU make is the dominant make implementation, and if it's not going to have .WAIT any time soon then standardizing .WAIT just makes POSIX weaker, not better. However, if patches adding .WAIT to GNU make are welcome, and there is a potentially straightforward implementation path based on the existing order-only prerequisite feature, the act of standardizing .WAIT might motivate someone to contribute support.



    (0005454)
    shware_systems   
    2021-08-20 19:39   
    The description of a pool is part of the resolution of Bug 1436, basically the number of data allocations necessary to manage parallelism when -j value greater than one. The count is the subset of that number actually needed to start the invocations the makefile specifies to bring targets or prereqs up to date.

    With:
    rightstuff=foo bar baz
    leftstuff=w x y z
    $(rightstuff):$(leftstuff)

    you can have 12 invokes started in parallel, with 3 each trying to update w, x, y, or z independently. If -j 10 specified 2 of those are waiting anyways so other executions complete. With the .WAIT version the max in parallel is four, the count of leftstuff. Then rightstuff can reuse the allocations leftstuff needed to be tracked as completed or not.
    (0005455)
    psmith   
    2021-08-20 20:20   
    Re 5454:

    Sorry but I don't see anything in issue #1436 which talks about "the number of data allocations" or anything that describes a "pool", and I still didn't really understand your comment.

    I think there's a fundamental confusion here. The makefile:

        foo bar baz: w x y z

    is 100% identical to writing:

        foo: w x y z
        bar: w x y z
        baz: w x y z

    That is, you're declaring 3 targets and giving them all the same 4 prerequisites. That's ALL it means. In fact once the makefile is parsed there's no way to differentiate the above two constructs based on the in-memory graph representation (at least, not in GNU make...) and that equivalency is required by POSIX.

    There's no change to this idea introduced by issue #1436 and -j.

    Given the above makefile then per invocation of make you may build a maximum of 7 targets (w, x, y, z, foo, bar, baz), and have a maximum of 4 jobs running in parallel (w, x, y, z). Never more than that. You will definitely never need to run 12 jobs, in parallel or not.

    If you add .WAIT, say between "w x" and "y z", you're only reducing the maximum number of possible parallel jobs because now make can't start y or z until after w and x are complete.

    But nothing about .WAIT or -j can ever increase the maximum number of targets that could be built.
    (0005456)
    psmith   
    2021-08-20 20:57   
    Regarding comment #5453:

    Just to point out that GNU make's order-only prerequisites is a more powerful construct than .WAIT. .WAIT is used when you can't allow two targets to run in parallel BUT at the same time you don't want them to depend on each other (if they could depend on each other you'd just use dependencies, not .WAIT). That's a different and fundamentally weaker guarantee than OO prerequisites provide: they give a true ordered relationship represented in the dependency graph, just with a differently colored edge that doesn't imply dependency.

    Consider this example:

      all: foo bar
      foo: one .WAIT two
      bar: two one

    Here we see that while we're building "foo", "two" won't be built in parallel with "one".

    However, while we're building "bar", "one" and "two" CAN be built in parallel because there's no .WAIT between them.

    If you set -j high enough (here, we only need -j2) then you'll still get "one" and "two" building at the same time; "one" is built as a dependency of "foo", then the updating of dependencies of "foo" is paused due to .WAIT so make continues looking for something else to build and finds "two" as a dependency of "bar"; there's no .WAIT here so make will go ahead and build "two", while "one" is still (potentially) running.

    Although easy to write, my opinion is that .WAIT is actually pretty hard to use in a reliable way due to this behavior. I consider it a kind of a hacky / quick-and-dirty shortcut that avoids implementing a robustly correct solution in your makefile.

    With order-only prerequisites you would write this makefile:

      all: foo bar
      foo: one
      one: | two
      bar: two one

    Here, "one" actually depends on "two" and that's represented in the dependency graph as an edge, so no matter where "one" appears it can never be updated before "two" is complete.


    However I was not really suggesting that OO prerequisites should be chosen instead of .WAIT: OO prerequisites were not actually created for this purpose (at least not directly). I'm just saying that their existence in GNU make allows people to implement sort-of-.WAIT-like features in GNU make makefiles, if they want to.

    And I'm also saying that, even though OO prequisites exist in GNU make, I'm not convinced it's completely straightforward to use it to implement .WAIT.

    However, I am open to patches implementing it (assuming the appropriate copyright assignment paperwork is completed etc.)
    (0005457)
    joerg   
    2021-08-21 17:06   
    (edited on: 2021-08-21 23:12)
    I am not sure whether the gmake method with the pipe symbol is
    able to be usable as a complete replacement for features offered
    by .WAIT.

    I also see no chance that this gmake specific method will ever be
    supported by other make implementations. So adding that method
    to POSIX would definitely weaken the POSIX standard.

    BTW: gmake still has various other bugs that prevent me personally
    from being able to use gmake in parallel mode. I am talking about a
    makefile system that offers portability to all currently available
    platforms (z/OS/EBCDIC just has been made working a month ago). The
    makefile system currently works without problems using smake and
    SunPro Make and (in serial mode) using gmake.

    The make implementation specific definitions are inside special files
    that are included based on the name of the make implementation in use.
    That concept of course requires to only have features that may be used
    in such a system without a need to have make implementation specific
    rules in the general makefiles. With these experiences in mind, I worked
    on the recent POSIX standardization to get something that matches the
    criteria for using different existing make implementations.

    With this makefile system and trying to use gmake in parallel mode,
    this are the current show stoppers:

    - gmakes calls readdir()/stat() at startup time and caches all timestamps.

      Unfortunately, gmake does never update these cached timestamps and as a
      result, it incorrectly claims that files are not present even though they
      just have been created from a rule that just has been executed - just
      because these files have been missing at startup time.

      The gmake bug tracking system claims that this bug has been fixed,
      but I cannot see a new version for testing, nor got a report from
      Paul that he did test whether that fix works for the schily makefile
      system.


    - missing include files with rules that allow to remake them in many
      cases do not work because gmake executes the rule commands in the
      inverse expected order and gmake does not offer a way to enforce the
      intended order. Paul has been given the chance to verify that gmake
      offers a way to deal with that problem by sending a patch to the
      affected makefiles that does not prevent other make programs to work
      with these makefiles, but he preferred to do nothing.

      The gmake bug tracking system claims that this bug has been fixed,
      but I cannot see a new version for testing, nor got a report from
      Paul that he did test whether that fix works for the schily makefile
      system.

    BTW: Other make implementations just work fine even though there is
    no .WAIT pseudo-target in use at the locations that currently fail
    with gmake.

    With the current version of gmake, I need to call gmake 5 times from the
    top level directory (starting from am fresh unpacked source) in order be
    able to finish with the compilation in parallem mode.

    If there is a fix for the above problems, it would really help to see
    a new gmake version now, since the last version is from January 2020.


    Finally, I remember that I had a discussion with Paul where he offered to
    implement .WAIT in case I implement .NOTPARALLEL into SunPro Make. I
    unfortunately cannot find that anymore.

    But given the fact that I spent 3 weeks for implementing things like ::=
    and .NOTPARALLEL into SunPro Make, even though these features are not
    needed because there are better existing methods, it would be a nice
    gesture from Paul to spend some time in making gmake more compatible
    to other make implementations. .WAIT is available in SunPro Make and
    BSD make since 30 years and that should be a verification for a useful
    feature.

    (0005458)
    rhansen   
    2021-08-23 06:41   
    (edited on: 2021-08-23 06:45)
    In response to Note: 0005456:
    > Although easy to write, my opinion is that .WAIT is actually pretty hard to use in a reliable way due to this behavior. I consider it a kind of a hacky / quick-and-dirty shortcut that avoids implementing a robustly correct solution in your makefile.
    OK, it seems like my understanding of how .WAIT works was wrong. If .WAIT implementations behave as you describe, then you have a very compelling point.

    Note: 0005446 was written under the assumption that .WAIT created an order-only prerequisite relationship: "prerequisites that appear after it shall not be brought up-to-date until after all preceding prerequisites for the same target have been brought up-to-date." If existing implementations behave as you describe, then none of them would be conforming.

    I'm now leaning against including a normative requirement for .WAIT, at least as it is currently written. (Something in "future directions" could still be useful.) If existing implementations change .WAIT so that it creates an ordered relationship in the dependency graph like GNU make's OO prerequisites feature, then I'd be more interested in adding .WAIT. And if that happened, then it would be easier to implement in GNU make, yes?


    In response to Note: 0005457:
    > I am not sure whether the gmake method with the pipe symbol is able to be usable as a complete replacement for features offered by .WAIT.
    I now see that there isn't a 100% semantic match, but why wouldn't it be usable as a replacement for .WAIT? I agree with Paul; it seems stronger than .WAIT to me.
    > BTW: gmake still has various other bugs that prevent me personally from being able to use gmake in parallel mode.
    Please keep the discussion on-topic. Your bugnote derails the conversation, which makes it harder to properly address this issue.
    > .WAIT is available in SunPro Make and BSD make since 30 years and that should be a verification for a useful feature.
    This argument is disingenuous. First, there are lots of old features that aren't standardized. Second, GNU make is the 800-pound gorilla here, and it became the 800-pound gorilla despite not having .WAIT. It is disrespectful to GNU make's sizeable userbase to not take Paul's input seriously; if he doesn't see a compelling reason to add .WAIT then we should take a moment to really consider its inclusion.

    Maybe the other make implementations should learn from GNU make's success and replace .WAIT with a clone of GNU make's OO prerequisites feature. (I'm kidding. Mostly.)

    (0005459)
    joerg   
    2021-08-23 12:36   
    Re: Note: 0005448

    There is a fundamental difference between

    target: $(leftstuff) .WAIT $(rightstuff); do_thing


    and

    $(rightstuff): $(leftstuff)


    The first one says "build leftstuff before rightstuff"

    The second one says "build rightstuff from leftstuff"

    That alone would make both variants non-equivalent.

    But even worse, if you have:

    a: l 
            echo a 
     
    a b c: x y z 
    


    This results amongst others in this rule:

    a:    l x y z 
            echo a
    


    which is what the standard requires but not what you expect in this case...
    as you may have unexpected rule commands involved.

    From a theoretical view, problems in a parallel build are missing
    dependencies that would need to be defined. From practical experience
    it is usually hard to find the missing dependency in special if the
    dependency graph is large already. .WAIT on the other side delivers
    the method that fits to the observed problems and does not create
    rules, you do not want to have.

    If you need more than one .WAIT in a dependency list, this could
    be (if at all) only written as many dependencies that at some point
    cannot be understood anymore and of course enhances the chance that
    they are unexpectedly added as dependencies to existing rule commands
    as in the example above.
    (0005460)
    psmith   
    2021-08-23 20:18   
    (edited on: 2021-08-23 21:07)
    In response to Note: 0005458

    My understanding of the text "prerequisites that appear after it shall not be brought up-to-date until after all preceding prerequisites for the same target have been brought up-to-date." in particular "for the same target", is that it's specifically intended to limit the effects of .WAIT to the prerequisites within a given target. If the same target appears as a prerequisite of a different target, then the .WAIT doesn't (necessarily) apply. However, I suppose this statement is open to interpretation and maybe this should be addressed in the standard, to reduce the chance of confusion.

    In response to Note: 0005457

    We have had this discussion before. If you do want to have it again we should restrict it to the mailing list (or better yet, the GNU make mailing lists) and not have it here. The statements are at least misleading. All I will say here is that, IF you state all prerequisites of your targets explicitly in your makefiles then GNU make will rebuild your targets 100% accurately 100% of the time, regardless of -j level. If you write your rules so that they have "undeclared side effects" that other targets rely on, then it's true that your targets may not build correctly in some situations in GNU make. I understand that your makefile systems rely heavily on this type of "undeclared side effect", so it is hard to port them to be usable by GNU make. I have described the changes you'd need to make to convert the "undeclared side effects" into declared explicit prerequisites and you have declined to make those changes in order to facilitate the portability, and instead decided to run make multiple times, which is perfectly fine.

    I have agreed to make some changes to GNU make to reduce this issue, and those changes are in fact checked into the Git source code control for the next release of GNU make. These changes are not very important to the current userbase of GNU make, however, because GNU make makefiles generally DO explicitly declare all their prerequisites and so don't run into these problems.

    In response to Note: 0005459

    > From a theoretical view, problems in a parallel build are missing dependencies that would need to be defined. From practical experience it is usually hard to find the missing dependency in special if the dependency graph is large already.

    I simply disagree with this. If you know what a target does, then it should be quite clear what its prerequisites are. I need to build a target FOO. I know what commands are needed to build FOO. I know what other targets in my makefile need to be available to run those commands. Therefore I know which targets need to be created before FOO's commands can be run. So I declare those targets as prerequisites of FOO. There's no magic or deep insight needed here.

    Maybe if you are limited to .WAIT, where you have to actually modify the prerequisites list of a specific target and list things in a specific order, and especially you have to add the .WAIT to a rule (or all rules) that itself lists FOO as a prerequisite rather than declaring the relationship on FOO itself, it can be more difficult. However, with order-only prerequisites you can define the prerequisite relationship directly between FOO and whatever targets it requires, ANYWHERE in the makefile, just as you can define normal prerequisite relationships anywhere. For example if it's easier you can define the relationship next to the prerequisite, rather than next to the target.

    (0005461)
    joerg   
    2021-08-23 20:39   
    Re: Note: 0005458

    First, the arguments against .WAIT are just wrong. In contrary, the gmake
    method is unreliable to use, see Note: 0005459 This is because it is hard to
    use a feature reliably that is hard to understand in case that the
    makefile, where it is used, already has a complex dependency graph. Most
    makefiles from today match that criteria.

    If a makefile fails in parallel mode, you know where it fails and thus
    you know where to add a .WAIT, but you usually do not know how/where to add
    more dependencies to satisfy make's wishes.

    Regarding .WAIT in combination with -j, the effects to parallel execution
    are the same as with the gnu only method. This is because the sum of all
    make instances that run in a common make pool share a common job pool and
    if one targes cannot start a new job for certain dependencies for a limited
    time, then other targets in the same make instance or other targets in other
    make instances in the same make pool will run other jobs from the available
    slots in the job pool.

    Finally, gmake definitely is not the gorilla that convinces because of it's
    features but it is rather used because a lot of vendors only offer/advertize
    gmake to their customers.

    Gmake offers a lot of features that do not give added value and deviates
    in it's behavior from other make implementations for features that did exist
    long before gmake was initially written.

    In case you did not get the background of the sum of all feature enhancements
    to "make" in POSIX that I worked on in the past years, this is only in order
    to avoid vendor lock in effects to specific make implementations and to foster
    more commonly usable features.

    If we (after adding .WAIT) add the simplified pattern rules I mentioned in
    Note: 0005126 from 0000513 and BSD make would implement that, we would have
    the minimum set of features that are needed for todays makefiles and SunPro
    Make, smake, gmake, bmake would offer a sufficient common set of features to
    finally come to the usual POSIX grant of portability for the make utility as
    well.
    (0005462)
    rhansen   
    2021-08-23 20:58   
    (edited on: 2021-08-23 21:10)
    Re: Note: 0005459:
    > There is a fundamental difference between
    target: $(leftstuff) .WAIT $(rightstuff); do_thing
    and
    $(rightstuff): $(leftstuff)
    I know; Paul already explained the difference in Note: 0005450. In Note: 0005453 I was proposing adopting GNU make's order-only prerequisites, not regular prerequisites.

    Re: Note: 0005460:
    > However, I suppose this statement is open to interpretation and maybe this should be addressed in the standard, to reduce the chance of confusion.
    Agreed. The intention (or at least *my* intention) was to establish an order-only prerequisite in the dep graph. Either way, the wording needs to be clarified, if it is to be kept at all.

    Do you think there would be more interest in implementing .WAIT in GNU make if it did establish an order-only prerequisite? For example, the following:
    $(a): $(b) .WAIT $(c) .WAIT $(d); do_thing
    would be 100% equivalent to:
    $(c): | $(b)
    $(d): | $(c)
    $(a): $(b) $(c) $(d); do_thing


    (0005464)
    shware_systems   
    2021-08-24 04:03   
    (edited on: 2021-08-24 04:05)
    Re: 5455

    With:
    rightstuff=foo bar baz
    leftstuff=w x y z
    $(rightstuff):$(leftstuff)
    target: $(rightstuff) // missing line
    // gets leftstuff up to date indirectly

    is where you get the 12, as the rightstuff as prereq of target starts 3 parallel invokes, and these are put on hold while the 4 prereqs of each invoke are evaluated, all in parallel. Before the initial ones go on hold all it cares about is all the prereqs for its particular target are out of date, they have no way to determine that the other targets also have the same prereqs, as if they were on separate lines like you expanded.

    (0005465)
    psmith   
    2021-08-25 12:53   
    Re: 5464

    I'm assuming here you meant to have the "target" rule come before the "$(rightstuff):$(leftstuff)" rule, or else invoke "make target".

    The method you suggest is not how make works. Make creates a directed graph and walks the graph. It doesn't try to build any target (node in the graph) until AFTER all that target's prerequisites are completely created.

    Here we have a graph with a root of "target" and 3 sub-nodes, "foo", "bar", and "baz". There are then 4 leaves, "w", "x", "y", and "z". Each of the three sub-nodes connect to each of the four leaves.

    When make runs it walks the graph from whatever starting point is chosen using a depth-first, right-to-left ordering among siblings. So if we start at "target" then first it will build "w", then "x", then "y", then "z", then "foo" (because now foo's prerequisites are complete), then "bar" (because "bar"'s prerequisites were already built, as they are also prerequisites of "foo"), then "baz", and finally "target" (the root of the graph).

    With -j the walk of the graph is identical as without -j, only instead of waiting for each node to complete before moving on to the next one we start N nodes (for -jN) in parallel before we are forced to wait (or, we wait when there are no nodes left that can be built until after already-in-process nodes are complete).
    (0005466)
    psmith   
    2021-08-25 14:19   
    Re Note: 5462

    I don't think it will work to declare that equivalency, because I don't think these things are equivalent. The problem is that order-only prerequisites create an explicit "always present" relationship in the graph, and .WAIT does not. Suppose we have this:

        all: one .WAIT two
        one two: ; @echo $@

    versus this:

        all: one two
        two: | one
        one two: ; @echo $@

    What happens if you run "make two"? In GNU make you have declared a dependency relationship here, so make will first build "one" (if needed) and then it will decide to build "two" or not, without considering whether "one" was updated (the relationship is "order-only" so it doesn't impact the "out of date" decision).

    In the .WAIT version, only "two" will be built. "one" will not be considered, because "one" is not a dependency of "two".

    However, I've downloaded FreeBSD make (the only make I could easily install that supports both -j and .WAIT) and it appears the behavior of .WAIT is different than I thought. I don't know whether this behavior is specific to the FreeBSD version or not.

    Let's consider this makefile:

        all: foo bar

        foo: one .WAIT two
        bar: one two

        foo bar one two: ; @echo $@; sleep 1

    Suppose we invoke this makefile with -j10 so that the -j value is not a limiting factor. What do we expect to build in parallel, and where will we wait? My original assumption was that the .WAIT is a feature of the prerequisite list of the target, so that we would try to build "foo", which would build "one", then it would not build "two" here because "one" was still running. Then make would try to build "bar", and see that "one" and "two" could be built, and since there's no .WAIT here it would build "two" immediately. So I assumed that the output would be this:

        one
        two
        bar
        <pause>
        foo

    However, that's not what FreeBSD make does. It seems that this .WAIT does indeed create some kind of globally visible relationship between "one" and "two", so what I see is:

        one
        <pause>
        two
        <pause>
        foo
        bar

    This relationship persists even if we don't build "foo" at all! If I run "make bar" here we still get:

        one
        <pause>
        two
        <pause>
        bar

    The implementation has some strange subtleties. If I change the graph so that "two" appears before "one" in the graph, FreeBSD make still runs "one" first and waits for it:

        all: bar foo
        bar: two
        foo: one .WAIT two
        one two foo bar: ; @echo $@; sleep 1

    In this graph, make will consider targets in the order "two", "bar", "one", "foo". But running this gives:

        one
        <pause>
        two
        <pause>
        bar
        foo

    So, somehow FreeBSD make realizes, when it wants to build "two", that sometime in the future it will also want to build "one" and that "one" should be completed before "two" starts.

    This is very mysterious to me. Certainly implementing something like that in GNU make will be very difficult, because GNU make doesn't "look ahead" while walking the graph. It walks the graph in order and simply goes from beginning to end; at the time that make wants to build "two" it doesn't know whether it will want to build "one" or not.

    And of course note that in GNU make, where we can have long chains of implicit rules which are not resolved until runtime, the full shape of the graph can't even be known until we actually try to resolve the implicit rules. That "one" dependency might be found through a long chain of implicit rules, and not be known until all that work is done.

    However maybe I'm missing something. In any event it's clear to me that the current wording proposed here certainly does not capture the actual behavior of FreeBSD make's .WAIT with anything close to specificity.

    I think that if we want to add .WAIT to the standard we will have to make it much LESS specific, so that all the subtleties are not actually required by POSIX and instead we create the simplest possible behavior.
    (0005468)
    joerg   
    2021-08-25 15:34   
    (edited on: 2021-08-25 15:49)
    Re: Note: 0005466

    Please note that SunPro Make is easier to compile/install than BSD make.

    Just fetch a recent schilytools from

    http://sourceforge.net/projects/schilytools/files/ [^]

    today, the most recent one is:

    https://sourceforge.net/projects/schilytools/files/schily-2021-08-14.tar.bz2 [^]

    unpack that archive chdir into the tree and call "make", then "make install"

    Then call either /opt/schily/bin/make or /opt/schily/bin/dmake

    "make" is the variant that is installed in the base OS and "dmake"
    is the variant that by default runs in parallel mode.

    BSD make is not fully POSIX compliant, SunPro Make of course is, as it has
    been certified with Solaris.


    BTW: where did you get a compilable FreeBSD make from?

    I am not aware of such a source....

    I verified the makefile from your example and both SunPro Make
    and bmake (The portable version from NetBSD make) behave as you
    expect. They print one and two with no delay and then wait,
    before printing bar and foo.

    (0005469)
    psmith   
    2021-08-25 17:52   
    I installed the pre-built freebsd-buildutils package onto my GNU/Linux system. It contains an "fmake" program.

    I installed the schily version. It didn't quite build so easily, but I was able to build it. However, there is no "dmake" or "make" in the bin directory (or anywhere else in /opt/schily) after I run install. I was able to find an uninstalled command ./sunpro/Make/bin/make/common/OBJ/x86_64-linux-gcc/make in the build directory and that one does support .WAIT (unlike the "smake" program). Although the output is a little bit weird (to me), I can see that it does indeed behave the way I expected in my note 5466, and not the way FreeBSD make works.
    (0005470)
    joerg   
    2021-08-25 18:03   
    (edited on: 2021-08-25 18:09)
    Well you did probably not run "make install" as root.

    If your Linux system contains unknown modifications that prevent
    compilation of single programs, I recommand to use "make -i" and
    to make a bug report.

    This is an actively maintained project that responds to bug reports.

    smake currently does not support parallel compilation and since .WAIT
    was added to POSIX recently, there is no support yet. This is why the
    schily makefile system uses $(WAIT) that may be empty.

    BTW: SunPro Make always collects the output from rule commands and
    prints serialized output, so output of course looks different from
    simple make implementaions.

    (0005471)
    psmith   
    2021-08-26 13:09   
    I never install software I download and build myself as root. I have a separate user account I use for all that.

    The problem I ran into is that I built with GNU make, then during it build it attempted to invoke the make that was built (I suppose smake), but there were GNU make options in MAKEFLAGS that weren't recognized by the make that was built and it failed. However that was my own fault because I have some MAKEFLAGS options I set in my environment before I invoke make.

    Re Note: 5461

    We'll just have to disagree about which feature is easier to use. Nothing said here convinces me that .WAIT is easy to use: on the contrary to me it seems very difficult to use RELIABLY, EXCEPT perhaps in the very limited context that the BSD build system uses it in (where it's used to mix in pause points into prerequisite lists contained in variables like SUBDIRS).

    > In contrary, the gmake method is unreliable to use, see Note: 0005459

    There are no examples in that note showing that OO prereqs are hard to use. In fact, there are no OO prereqs mentioned in that note at all.

    > This is because it is hard to use a feature reliably that is hard to understand in case that the
    makefile, where it is used, already has a complex dependency graph.

    There is nothing about OO prereqs that make them hard to understand in complex dependency graphs. On the contrary, since OO prereqs create an actual edge in the graph while .WAIT does not, the behavior of OO prereqs are easier to understand.

    > If a makefile fails in parallel mode, you know where it fails and thus you know where to add a .WAIT, but you usually do not know how/where to add more dependencies to satisfy make's wishes.

    This is exactly the reverse of reality.

    When a target fails, you know what target failed and you know why it failed (which prerequisite did not exist that should have existed).

    With GNU make, you merely add this to your makefile:

        target: | prereq

    and now you've added a new edge to your graph that ensures that prereq will be complete before the target is invoked. Note that unlike normal prerequisites, order-only prerequisites DO say only "build prereq BEFORE target" and they do NOT say "build prereq FROM target". That's what makes them different from "normal" prerequisites. But unlike .WAIT, this relationship is defined as an edge in the graph between the target and prerequisite so it always exists, regardless of where target is used in the makefile.

    I'm not exactly sure what you mean by "where to add more dependencies"; like any other dependency relationship it can be added ANYWHERE in the makefile.

    On the other hand, to solve this issue with .WAIT you must search all your makefiles and envision the entire dependency graph so you can be sure that you have found all the other dependency lists that contain "target", and which can be reached from a place where "prereq" might have been invoked first, so you can add a .WAIT somewhere between "target" and "prereq". This is because .WAIT is not associated with target and prereq, and it doesn't apply to other places where these targets might exist.

    You can see this issue even with my very simple example above: I didn't add the .WAIT between the "one" and "two" targets when they were prerequisites of "bar", and so they were built in parallel. With .WAIT I have to search the entire graph looking for these types of relationships (which, unlike my example, could be very far apart in the graph and hard to see in the makefiles).

    Alternatively, you could try to simplify things by just searching all makefiles to find all places where "target" appears as a prerequisite and change them all to ".WAIT target", which would solve the problem but could introduce many more wait points than are needed or optimal.

    And, of course, when we consider things like pattern rules etc. it might not always be obvious at all that a given prerequisite list will have "target" as a prerequisite.
    (0005472)
    joerg   
    2021-08-26 19:23   
    (edited on: 2021-08-26 19:24)
    Re: Note: 0005466

    I did test your makefiles on a FreeBSD instance using "make".

    I canot repeat the problem you describe, however both makefiles are written
    in a way that makes them instable as their behavior depends on chance when
    in parallel mode. On FreeBSD, I see a comparable behavior with SunPro Make
    and BSD make.

    Re: Note: 0005471

    When you do not use the official makefiles of a trustworthy project, it
    is up to you, to do the right things. This is something you cannot blame
    the project for. Even the Linux distros, that support binary packages
    based on schilytools use "make install" (see README.install) to create
    the prototype trees for their packages.

    If you are using a private environment MAKEFLAGS, you are doing something
    that is being warned for. If your MAKEFLAGS environment even contains
    non-standard content, you should never expect that to work.

    Smake implements workarounds that deal with the non-POSIX structure from
    gmake's MAKEFLAGS, but it does not support to work around non-standard
    gmake options.

    And BTW: you are right, if you call "make" at top level, this first
    compiles a bootstrap smake and then continues with that compiled smake
    in order to be able to compile schilytools with make implementations
    that do not support the minimum feature set for the makefile system.

    (0005473)
    rhansen   
    2021-08-26 20:22   
    There was some spirited discussion about this bug during today's telecon. For now, we're going to wait to see if Steffen (or anyone else) can assemble a rough proof of concept patch for GNU make that takes an approach acceptable to Paul. If so, then it seems much more likely to us that GNU make would eventually support .WAIT. That would alleviate concerns about indefinite non-conformance by a major implementation.
    (0005474)
    psmith   
    2021-08-26 20:40   
    (edited on: 2021-08-26 20:52)
    I'm not sure which "problem" you refer to for FreeBSD make. Are you saying your FreeBSD make doesn't put the "pauses" in the same place as mine?

    Let's be very clear. If I have this makefile:

       all: bar foo
       bar: two one
       foo: one .WAIT two
       one two foo bar: ; @echo $@; sleep 10

    where I've increased the sleep time to be more obvious even on slow or busy systems, and I run FreeBSD make via "time", I get this:

       $ time fmake -j10
       one
         <pause>
       two
         <pause>
       bar
       foo
         <pause>

       real 0m30.018s
       user 0m0.019s
       sys 0m0.005s

    It takes 30 seconds to run this makefile.

    If I do the same with the SunPro make from schily, I get this:

       $ time ./schily/schily-2021-08-14/sunpro/Make/bin/make/common/OBJ/x86_64-linux-gcc/make -j10
       make: Warning: Can't find `make.rules': No such file or directory
       dmake: defaulting to parallel mode.
       local --> 1 job
       local --> 2 jobs
         <pause>
       local --> Job output
       two
       local --> Job output
       one
       local --> 1 job
       local --> 2 jobs
         <pause>
       local --> Job output
       bar
       local --> Job output
       foo

       real 0m20.018s
       user 0m0.019s
       sys 0m0.008s

    and the total time taken is 20 seconds.

    Are you saying you get different behavior from your version of FreeBSD make? That it also takes 20 seconds, not 30 seconds?

    I don't care about the _order_ in which the output appears, that's irrelevant. The important thing for this experiment is how many pauses there are: what is run in parallel and what is run serially.

    > If your MAKEFLAGS environment even contains non-standard content, you should never expect that to work.

    Uh, and...?? I did say, "However that was my own fault because I have some MAKEFLAGS options ..."

    (0005475)
    psmith   
    2021-08-26 20:50   
    My main concern at this point is that the text added to the standard be very clear about what is required by the standard, and what is left unspecified. See my example above and how the two different make versions that already support .WAIT behave.

    I assume that the text means to say that .WAIT only must take effect between two sets of dependencies when they are processed as prerequisites of the specific target where the .WAIT appears in a prerequisite list. If those same two targets appear as dependencies of some other target where .WAIT is not present, then the standard makes no statement about whether they should be ordered or not.

    Let me offer up a potential implementation and you can say whether it's intended to be allowed by the standard:

    Suppose a version of make is created where, when it sees a .WAIT in a prerequisit list, it pauses and waits for ALL currently-running jobs to complete, then keeps going.

    Ignoring quality of implementation issues, would such an implementation be POSIX conforming under the text being proposed?
    (0005476)
    rhansen   
    2021-08-26 21:00   
    (edited on: 2021-08-26 21:08)
    Re: Note: 0005474
    > Are you saying you get different behavior from your version of FreeBSD make? That it also takes 20 seconds, not 30 seconds?
    Using the Debian bmake-20181221-2 package (supposedly copied from NetBSD):

    $ time bmake -f - -j10 <<\EOF
    all: bar foo
    bar: two one
    foo: one .WAIT two
    one two foo bar: ; @echo $@; sleep 10
    EOF
    --- two ---
    --- one ---
    --- two ---
    two
    --- one ---
    one
                # pauses here for 10s
    --- bar ---
    --- foo ---
    --- bar ---
    bar
    --- foo ---
    foo
                # pauses here for 10s
    
    real    0m20.032s
    user    0m0.018s
    sys     0m0.010s
    Maybe this is a bug in NetBSD's make?

    Re: Note: 0005466
    > [...] order-only prerequisites create an explicit "always present" relationship in the graph
    Once again my understanding of how implementations actually behave is upended. :)

    I was under the impression that "|" only affected when the stuff on the right was built, not whether it was built. For example, I had assumed that the following would not build 'one':
    $ make -f - -j10 two <<\EOF
    two: | one
    one two: ; @echo $@; sleep 2; echo done sleeping $@
    EOF
    Given that GNU make does build 'one', the name "order-only prerequisite" seems like a misnomer. Maybe "existence-only prerequisite" would be a better name.

    If GNU make did have a true order-only prerequisite feature, then I think that .WAIT could be implemented as syntactic sugar. The resulting behavior would be more conservative than BSD's .WAIT behavior (less parallelism), but that's OK.

    Re: Note: 0005475
    > My main concern at this point is that the text added to the standard be very clear about what is required by the standard, and what is left unspecified.
    Agreed. I think everyone understands that the wording in Note: 0005446 is problematic. Coming up with precise and correct wording might be tricky, but now that we collectively understand the problem better I am confident that we can come up with something suitable.

    > [...] would such an implementation be POSIX conforming under the text being proposed?
    Yes, POSIX should allow that behavior. Implementations should always be permitted to have less concurrency than is technically possible. For example, POSIX should allow an implementation to dynamically reduce parallelism if the current system load is too high.

    (0005477)
    joerg   
    2021-08-26 21:12   
    (edited on: 2021-08-26 21:17)
    Re: Note: 0005474

    Running the FreeBSD make on FreeBSD with your makefile indeed results
    in a total time of 20 seconds for the make run.

    This is the FreeBSD-13 beta from September 24 2020.

    --- two ---
    --- one ---
    --- two ---
    two
    --- one ---
    one
            delay...
    --- bar ---
    --- foo ---
    --- bar ---
    bar
    --- foo ---
    foo
            delay...

    Since even the output looks different from yours, you seem to use a
    different make implementation.

    P.S. The results have been verified to be identical to the results
    fetched with a FreeBSD installation from April 2021.

    (0005478)
    psmith   
    2021-08-28 12:42   
    RE Note: 5476

    > Given that GNU make does build 'one', the name "order-only prerequisite" seems like a misnomer. Maybe "existence-only prerequisite" would be a better name.

    It's possible I've been living with it too long (order-only prerequisites have existed in GNU make for almost 20 years, having been added in GNU make 3.80 in 2002) but I don't agree. The term "prerequisite" to me means that it must be brought up to date before "target" can be built. The "order-only" modifier means that only the order of targets, but not the out-of-date computation, is relevant for that edge in the graph.

    If a feature were created that told make that "targetB" should be built before "targetA" but if and only if "targetB" needed to be built for some other reason, that shouldn't be called a prerequisite at all IMO. You would need some other term that didn't contain "prerequisite".

    Such a feature wouldn't be so easy to create, but could be done. It would need to have some kind of "undefined state" edge in the graph which, when seen, would mean that the target node could not be built until that state was no longer undefined. The state would change to defined when either (a) the prerequisite node were considered (for some other reason than this undefined edge), or else (b) the entire graph has already been walked: at that time all "undefined" edges would revert to "considered with no changes" and we would have to build all the target nodes that could not previously be considered due to undefined states.

    Or, something like that.
    (0005479)
    joerg   
    2021-08-28 15:15   
    If that gmake feature has been added late, I did expect gmake to
    see beyond the edge of the plate and to check what other make
    implementations did offer as solution already instead of introducing
    something incompatible new.

    UNIX was a success story because implementors from vendor A did look
    at implementations from vendor B and reimplemented goot ideas. In the
    1980s a good idea from another vendor typically did take 2-4 years to
    appear everywhere. With that background, .WAIT should have been implemented
    in gmake no later than 1995.
    (0005480)
    psmith   
    2021-08-28 17:21   
    As I hope the discussion here makes obvious, .WAIT is not the same thing as an order-only prerequisite. As far as I'm aware there isn't anything similar to OO prerequisites in other versions of make so GNU make isn't implementing this feature in some incompatible way to other versions of make that already supported it.

    The question of whether or not GNU make supports .WAIT for compatibility with other versions of make is a completely separate issue.
    (0005481)
    steffen   
    2021-08-28 19:21   
    I added a Public Domain diff to make(1) that may be used freely shall it can be applied to the source of your program.

    It implements a true .WAIT that ensures the chosen ordering of prerequisites is stable across the entire makefile. For example, for the example above:

    <code>
    #?0|kent:tmp$ cat y/makefile
    all: bar foo
    bar: two one
    foo: one .WAIT two
    one two foo bar: ; @echo $@; sleep 3
    #?0|kent:tmp$ ~/src/.gmake.git/make -j4 -f y/makefile
    one
    two
    bar
    foo
    </code>

    or for the more sophisticated

    <code>
    #?0|kent:z$ cat makefile
    all: lib ham .WAIT bin bin2 ;@echo all command
    clean: ; rm -f bin/.stamp lib/.stamp
    lib: lib/.stamp; @echo lib command
    ham: ;@echo ham command
    lib/.stamp: ;cd lib && $(MAKE)
    bin: bin/.stamp; @echo bin command
    bin/.stamp: ;cd bin && $(MAKE)
    bin2: ; @echo bin2 command
    #?0|kent:z$ ~/src/.gmake.git/make -j4
    ham command
    cd lib && /home/steffen/src/.gmake.git/make
    make[1]: Entering directory '/tmp/z/lib'
    lib-x command

    lib-x after sleep
    lib/all command
    make[1]: Leaving directory '/tmp/z/lib'
    lib command
    cd bin && /home/steffen/src/.gmake.git/make
    bin2 command
    make[1]: Entering directory '/tmp/z/bin'
    bin-x command
    bin/all command
    make[1]: Leaving directory '/tmp/z/bin'
    bin command
    all command
    </code>

    It may be up to the core developers to leave the exact meaning of .WAIT an implementation issue, but i think this way of doing things -- extending a chosen barrier to be global -- is right.

    I want to note that this likely does not cause issues in practice, since Makefiles yet making use of .WAIT are likely ordered in a way that would avoid the race condition which does occur if make(1) does not take care for ensuring barrier validity gracefully.
    Since most use cases of .WAIT are in the BSD make system this is a non-issue, since there so-called "singleton"s (thanks Paul Smith) are used, and .WAIT thus occurs naturally only to guarantee correct ordering in this singleton.

    P.S.: survived -fsanitize=address testing.
    (0005482)
    joerg   
    2021-08-29 12:37   
    With the proposal from Steffen, we would need to add a hint to the
    standard that a makefile like this:

    all: bar foo
    bar: two one
    foo: one .WAIT two
    one two foo bar: ; @echo $@; sleep 3


    is non-conforming, since the target "bar" and it's prerequisite list is in
    conflict with the definitions for target "foo".

    Steffens code causes an implicit dependency and even though this may
    result in working code while using gmake, it will not work with
    SunPro Make and BSD Make in case that "all" is used as the main target
    to build. This is because SunPro Make and BSD make evaluate .WAIT at run
    time, based only on the list of dependencies for the current target.
    (0005484)
    rhansen   
    2021-09-02 06:14   
    (edited on: 2021-09-02 06:17)
    Re: Note: 0005478
    > You would need some other term that didn't contain "prerequisite".
    Good point.

    Re: Note: 0005482
    > [...] the target "bar" and it's prerequisite list is in conflict with the definitions for target "foo".
    How so? It looks OK to me.


    Some experiments with Steffen's patch with the following makefile:
    all: bar foo
    bar: two one
    foo: one .WAIT two
    one two foo bar: ; @echo $@; sleep 2
    Building 'all' works as expected:
    $ time ./install/bin/make -j10 -f - <<\EOF
    all: bar foo
    bar: two one
    foo: one .WAIT two
    one two foo bar: ; @echo $@; sleep 2 
    EOF
    one
            <2s pause here>
    two
            <2s pause here>
    bar
    foo
            <2s pause here>
    
    real    0m6.021s
    user    0m0.019s
    sys     0m0.004s
    Building just 'bar' causes 'one' to be built before 'two' even though the BSD makes build 'one' and 'two' in parallel. I personally prefer this behavior over the BSD make behavior, so I don't consider it to be a bug. Output:
    $ time ./install/bin/make -j10 -f - bar <<\EOF
    all: bar foo
    bar: two one
    foo: one .WAIT two
    one two foo bar: ; @echo $@; sleep 2 
    EOF
    one
            <2s pause here>
    two
            <2s pause here>
    bar
            <2s pause here>
    
    real    0m6.028s
    user    0m0.012s
    sys     0m0.016s
    Building just 'two' causes 'one' to be built. This feels like a bug to me, and it doesn't match the behavior of BSD makes. Output:
    $ time ./install/bin/make -j10 -f - two <<\EOF
    all: bar foo
    bar: two one
    foo: one .WAIT two
    one two foo bar: ; @echo $@; sleep 2
    EOF
    one
            <2s pause here>
    two
            <2s pause here>
    
    real    0m4.022s
    user    0m0.014s
    sys     0m0.008s
    Based on the above, it looks like the patch effectively converts .WAIT into order-only prerequisites.

    (0005485)
    psmith   
    2021-09-02 12:23   
    Re Note 5482:

    I don't think I'd like to say that a makefile like that is non-conforming. Rather I think we should say that .WAIT is only guaranteed to create a wait point in the prerequisites while building the target in which the .WAIT appears. It's undefined (by the standard) whether it will add a wait point between these prerequisites if they appear in other rules.

    Maybe that amounts to the same thing, more or less.
    (0005486)
    joerg   
    2021-09-02 13:22   
    (edited on: 2021-09-02 13:25)
    Re: Note: 0005484

    Your makefile (using dmake == SunPro Make) ==>

    mostly starts with two on Solaris

    mostly starts with one on FreeBSD on a RaspPi

    more or less alternatingly starts with one or the other on Linux


    This is because the "all: bar foo" dependency has no order and bar
    has no order as well. Another important reason for that behavior
    is the different fork() timing on the different operating systems.

    I believe it is important that if we allow implementations that use a
    prerequisite-like implementaion, the standard explicitely mentions that
    in such a makefile, the order is undefined.

    (0005487)
    shware_systems   
    2021-09-02 15:11   
    (edited on: 2021-09-02 15:26)
    Re: 5486

    This applies to non-parallel builds too, that I see, there's no requirement prereqs are evaluated left to right, just commands top to bottom. An order can be specified for both in the standard as to when they start, but a parallel build still has no control over when a prereq finishes updating.

    (0005488)
    psmith   
    2021-09-02 15:30   
    Re Note 5487:

    No, you're not correct here. The standard explicitly says:

    > The make utility shall treat all prerequisites as targets themselves and recursively ensure that they are up-to-date, processing them in the order in which they appear in the rule.

    Prerequisites MUST be considered in the order in which they are listed in the makefile.

    Also, I'm not sure I understood your second sentence but note make orders things by when jobs COMPLETE, not by when they start.
    (0005489)
    rhansen   
    2021-09-02 16:28   
    (edited on: 2021-09-02 16:32)
    Here's a new attempt:

    On page 2975, insert before line 98708:
    .NOTPARALLEL
    The application shall ensure that this special target is specified without prerequisites or commands. When specified, make shall update one target at a time, regardless of whether the -j maxjobs option is specified. If the -j maxjobs option is specified, the option shall continue to be passed unchanged to sub-make invocations via MAKEFLAGS.

    On page 2975, insert before line 98742:
    .WAIT
    The application shall ensure that this special target, if specified as a target, is specified without prerequisites or commands. When .WAIT appears as a target, it shall have no effect. When .WAIT appears in a target rule as a prerequisite, it shall not itself be treated as a prerequisite; however, make shall not recursively process the prerequisites to the right of the .WAIT until the prerequisites to the left of it have been brought up-to-date. Implementations may also enforce the same ordering between the affected prerequisites while processing other target rules that have some or all of the same affected prerequisites.

    On page 2975, change line 98742 from:
    The special targets .IGNORE, .POSIX, .PRECIOUS, .SILENT, and .SUFFIXES shall be specified without commands.
    to:
    The special targets .IGNORE, .NOTPARALLEL, .POSIX, .PRECIOUS, .SILENT, .SUFFIXES and .WAIT shall be specified without commands.

    On page 2975 lines 98744-98747, change:
    Targets with names consisting of a leading <period> followed by the uppercase letters "POSIX" and then any other characters are reserved for future standardization. Targets with names consisting of a leading <period> followed by one or more uppercase letters are reserved for implementation extensions.
    to:
    Targets and prerequisites consisting of a leading <period> followed by the uppercase letters "POSIX" and then any other characters are reserved for future standardization. Targets and prerequisites consisting of a leading <period> followed by one or more uppercase letters, that are not described above, are reserved for implementation extensions.

    On page 2985 after line 99167 (make EXAMPLES) append a new example:
    7. With the following makefile, <tt>make -j 10 all</tt> may bring <tt>one</tt> and <tt>two</tt> up-to-date in parallel despite the .WAIT in the prerequisite list for <tt>foo</tt>. This is because the .WAIT does not stop make from recursively processing <tt>bar</tt> and its prerequisites in parallel. However, if only <tt>foo</tt> is specified (<tt>make -j 10 foo</tt>), make will wait for <tt>one</tt> to be brought up-to-date before bringing <tt>two</tt> up-to-date. Note also that the .WAIT does not create a prerequisite relationship between <tt>one</tt> and <tt>two</tt>, so <tt>make -j 10 two</tt> will not build <tt>one</tt>.
    all: foo bar
    foo: one .WAIT two
    bar: one two
    foo bar one two: ; @echo $@


    (0005490)
    shware_systems   
    2021-09-02 16:36   
    (edited on: 2021-09-02 16:43)
    Re 5488:

    Yes, that was pointed out during today's phone call; I was looking at the Target Rule text more, skipped last part of that sentence.

    make has no control over, once it establishes what prereqs of a target are not up-to-date, how long the commands specified to bring them up-to-date will need to execute. An early prereq may be a compile that takes 10 minutes, the last one may simply be a touch that takes milliseconds. Processed as parallel targets that last prereq will finish first is what I was getting at.

    (0005492)
    steffen   
    2021-09-02 23:25   
    Re: 5484

    Well ok, then this is even easier. The patch easier.diff should do just that.

    With my example it still prints "lib command" before stepping over to "bin" (different to unpatched make), but the "two one" example happens in that order (but when used for "foo", but then race free).
    It surely could be optimized, and it likely is possible to extend this to global level nonetheless (with some effort).

    Re 5489:

    That example is false, then, it should be "bar: two one"

    Ciao!
    (0005493)
    steffen   
    2021-09-03 13:37   
    P.S.: i added yet another public domain file, gmake-4.3.wait.diff, which can be patch(1)ed onto the currently released version of Paul Smith's version of make without conflicts.
    It is not so easy to build and use development versions of software in today's software world, unfortunately.
    It is identical to easier.diff but for removal of useless cruft hunks.

    (Just flags targets which should be waited for, then later recurses into these to also mark all their dependencies like that, so that in effect they are waited for. Works for me!?! Ciao.)
    (0005494)
    psmith   
    2021-09-03 14:48   
    Re Note 5489: That text seems acceptable to me. I can't say for sure when this feature will be made available in GNU make however.




    Viewing Issue Advanced Details
    1436 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-12-15 21:00 2021-11-26 15:00
    steffen
     
    normal  
    Applied  
    Accepted As Marked  
       
    steffen
    Vol. 3: Shell and Utilities, Issue 7, make
    2969
    98473
    ---
    Note: 0005496
    make: add "-j max_jobs" option to support simultaneous rule processing
    Parallel, even massively parallel processing has become the widely supported and
    used default, yet the standard make(1) does not document it.

    P.S.:
    Even though on SunOS/Solaris the first approach taken (long ago) involved
    distribution of make jobs to different computers via a special "dmake" program,
    it seems even there a default make(1) could follow an adjusted standard by doing
    the equivalence of "exec dmake ARGUMENTS".
    On page 2969, insert before line 2969

      -j max_jobs
        Specifies the maximum number of rule-processing jobs to run simultaneously.
    Notes
    (0005299)
    psmith   
    2021-03-22 21:27   
    Defining -j in a single instance of make is not too hard to understand and most likely all versions of make, that currently support -j, do it essentially the same way.

    The question is, what happens when a parent make with -j invokes a command which is a sub-make. This is where it gets hard, and where there's probably not too much agreement on what the behavior should be between existing implementations.
    (0005327)
    rhansen   
    2021-04-22 15:34   
    At (TC2) page 2969 line 98438, make SYNOPSIS, add:
    [-j maxjobs]
    (SD shaded as the rest)


    After page 2969 line 98472, make OPTIONS, add:
    -j maxjobs Set the maximum number of targets that can be updated concurrently. If this option is specified multiple times, the last value of maxjobs specified shall take precedence. If this option is not specified, or if maxjobs is 1, only one target shall be updated at a time (no parallelization). If the value of maxjobs is non-positive, the behavior is unspecified. When this option is specified with maxjobs greater than 1 and a rule invokes make (perhaps via <tt>$(MAKE)</tt>), the parent make shall attempt to establish communication with the child make in an unspecified manner, possibly via an implementation-specific flag in MAKEFLAGS. If the -j option is passed to the child make via the MAKEFLAGS environment variable with the same maxjobs value as the parent and is not overridden by a maxjobs value from another source (even if it has the same value), the child make shall establish communication with the parent make before it attempts to update any targets. In all other cases it is unspecified whether the child make establishes communication with the parent make. The parent make and any children it is communicating with, recursively, shall between them update no more than maxjobs targets in parallel. Implementations are permitted to silently limit maxjobs to an unspecified positive value; if this limit is 1, make need not attempt to establish communication with a child make.


    At page 2972 line 98587, make EXTENDED DESCRIPTION, change:
    The make utility shall treat all prerequisites as targets themselves and recursively ensure that they are up-to-date, processing them in the order in which they appear in the rule. The make utility shall use the modification times of files to determine whether the corresponding targets are out-of-date.
    to:
    The make utility shall treat all prerequisites as targets themselves and recursively ensure that they are up-to-date, using the modification times of files to determine whether the corresponding targets are out-of-date. If the -j option is not specified, a rule's prerequisites shall be processed in the order in which they appear in the rule. If targets are being updated in parallel (see -j in OPTIONS), the order of processing of prerequisites is unspecified except the make utility shall ensure that all of a prerequisite's own prerequisites are up-to-date before the prerequisite itself is made up-to-date.


    At page 2986 line 99174, make RATIONALE, delete the bullet item:
    Syntax supporting parallel execution (such as from various multi-processor vendors, GNU, and others)


    After page 2987 line 99244, make RATIONALE, add a new paragraph:
    Some implementations of make allow omission of the option-argument for the -j option, although not in the manner described in item 2.b in [xref to XBD 12.1] (where the option-argument, if present, needs to be directly adjacent to the option in the same argument string). To allow the option-argument to follow -j as a separate argument, these implementations check whether the next argument begins with a digit. If it does, it is treated as an option-argument for -j; if it does not, the -j is processed as not having an option-argument and the next argument is processed as if it followed a -j option-argument. This behavior is not suitable for inclusion in this standard as it does not meet the syntax guidelines. However, it is an allowed extension since following -j with an argument that does not begin with a digit would otherwise be a syntax error. At least one implementation of make uses <tt>-j 0</tt> to mean "use a sensible value for the maximum number of targets that can be updated in parallel". If an implementation wants to add this feature, the standard developers suggest following this convention.


    At page 2988 line 99280, make RATIONALE, change:
    The make utilities in most historical implementations process the prerequisites of a target in left-to-right order, and the makefile format requires this. It supports the standard idiom used in many makefiles that produce yacc programs; for example:
    foo: y.tab.o lex.o main.o
        $(CC) $(CFLAGS) −o $@ t.tab.o lex.o main.o
    In this example, if make chose any arbitrary order, the lex.o might not be made with the correct y.tab.h. Although there may be better ways to express this relationship, it is widely used historically. Implementations that desire to updatem prerequisites in parallel should require an explicit extension to make or the makefile format to accomplish it, as described previously.
    to:
    When targets are not being updated in parallel (see -j in OPTIONS), make processes the prerequisites of a target in left-to-right order. This supports a common idiom used in many makefiles that produce yacc programs; for example:
    foo: y.tab.o lex.o main.o
        $(CC) $(CFLAGS) −o $@ y.tab.o lex.o main.o
    In this example, if make chose any arbitrary order, the lex.o might not be made with the correct y.tab.h. Although there may be better ways to express this relationship (that would be needed if -j is specified), it is widely used historically.
    (0005328)
    psmith   
    2021-04-22 18:36   
    Thanks for all the work on this! I will make an effort to review this from a GNU make perspective and provide comments, if any, before Monday.
    (0005329)
    psmith   
    2021-04-23 17:40   
    > When this option is specified with maxjobs greater than 1 and a rule invokes make (perhaps via <tt>$(MAKE)</tt>),

    I think that this isn't sufficient. It's not clear what make is intended to do here: how can make determine if a "rule invokes make"? Make doesn't have a shell parser so it cannot determine whether or not a command will actually invoke make or not.

    In GNU make, we require the recipe to contain a reference to the MAKE variable; the command string must contain either $(MAKE) or ${MAKE} somewhere. Or, it must be prefixed with "+". If either of those are true then make assumes that the child may or will invoke make, and it prepares to allow that process to participate in the parallel domain. If neither of those are true then make assumes that the child will not invoke make and it won't configure that child to be available in the parallel domain.
    (0005330)
    psmith   
    2021-04-23 17:47   
    (edited on: 2021-04-23 18:09)
    > shall between them update no more than maxjobs targets in parallel

    It's not clear from the text whether this set of targets includes make itself. In a recursive make scenario, you will be running sub-makes to create targets. In GNU make we do not count make itself as one of the targets to be updated and I recommend this behavior here as well.

    If you don't have this, then you can have a situation where you run with -jN and you invoke N instances of sub-makes, and now no sub-make can start a new target because all available jobs are used.

    Not counting make is sensible because while all jobs are running, make itself is not actually running: it's waiting for one of the jobs to complete.

    In GNU make, we basically say that each invocation of make is given one "free" jobserver token that it can use for starting one job at a time: if it ever wants to start more than one job it needs to get another jobserver token as expected. This way every instance of make can always make progress.

    (0005331)
    psmith   
    2021-04-23 18:28   
    I think the rest of the content here will work for GNU make. Thanks!
    (0005345)
    rhansen   
    2021-04-29 16:59   
    (edited on: 2021-04-29 17:00)
    Re: Note: 0005330:
    It's not clear from the text whether this set of targets includes make itself. In a recursive make scenario, you will be running sub-makes to create targets. In GNU make we do not count make itself as one of the targets to be updated and I recommend this behavior here as well.
    Out of curiosity, how is this implemented in GNU make? Does the act of establishing a connection to the parent make conceptually return a worker to the pool of available workers? Make doesn't parse shell code, so make can't know whether a rule spawns a submake even if the rule expands the MAKE macro (<tt>echo $(MAKE)</tt> expands the MAKE macro but it doesn't spawn a submake).

    (0005346)
    rhansen   
    2021-04-29 17:03   
    (edited on: 2021-04-29 17:18)
    > Out of curiosity, how is this implemented in GNU make?

    Nevermind, you already answered this in Note: 0005330:
    In GNU make, we basically say that each invocation of make is given one "free" jobserver token that it can use for starting one job at a time: if it ever wants to start more than one job it needs to get another jobserver token as expected.
    Apologies for not reading closely.

    (0005347)
    rhansen   
    2021-04-29 17:30   
    (edited on: 2021-04-29 17:34)
    Re: Note: 0005330:
    > In GNU make, we basically say that each invocation of make is given one "free" jobserver token that it can use for starting one job at a time: if it ever wants to start more than one job it needs to get another jobserver token as expected.

    If I understand GNU make's behavior correctly, a user could exceed N parallel jobs if they write a rule that starts multiple submakes in parallel. For example:
    all:
        i=0; subs=''; while [ $$i -lt 10 ]; do $(MAKE) -C subdir$$i & subs="$$subs $$!"; i
    =$$((i+1)); done; for sub in $$subs; do wait $$sub || exit 1; done

    The above could have up to 11 jobs running in parallel if <tt>-j2</tt> is passed to make. Correct? We'll want to make sure the standard permits that behavior, probably by saying that the behavior is unspecified if a rule invokes multiple submakes in parallel.

    (0005348)
    psmith   
    2021-04-29 20:19   
    Yes, that's correct: it is possible to circumvent this limit. It's unfortunate but alternative implementations which are completely reliable are proportionately more difficult, and I just figured if people want to write their makefiles that way and break the jobserver behavior, then, they get to keep both pieces :).
    (0005353)
    psmith   
    2021-05-14 14:12   
    I see this is still "Accepted". I hope it can be reopened and the issues above addressed before the text is applied. I don't think the current text is appropriate for being applied to the standard.
    (0005354)
    geoffclare   
    2021-05-14 14:30   
    Reopening.

    We had left it as resolved in the expectation that we would just need to make a small wording change to the note containing the final accepted text. However, we have since realised that a more extensive rewrite is needed.
    (0005355)
    psmith   
    2021-05-14 14:35   
    Thanks Geoff! Please let me know if I can help. Cheers!
    (0005362)
    rhansen   
    2021-05-20 17:08   
    We think we have achieved consensus on a rewrite of the description of the -j option; see "attempt #3" on line 65 of https://posix.rhansen.org/p/2021-05-20. [^] Feedback would be appreciated.
    (0005364)
    psmith   
    2021-05-22 19:19   
    Thanks for your work on this! I've reviewed the text in "attempt #3" and it seems acceptable to me as-is. I did have this thought which you may accept or ignore as you like :)

    > When make is bringing a target with commands up-to-date

    It might be more clear, although not meaningfully different, to say something like:

    When make is bringing one or more targets with commands up-to-date
    (0005496)
    nick   
    2021-09-09 15:35   
    (edited on: 2021-09-09 15:36)
    Updated changes after resolving 0001437

    After page 2969 line 98472, make OPTIONS, add:
    -j maxjobs Set the maximum number of targets that can be updated concurrently. If this option is specified multiple times, the last value of maxjobs specified shall take precedence. If this option is not specified, or if maxjobs is 1, only one target shall be updated at a time (no parallelization). If the value of maxjobs is non-positive, the behavior is unspecified. When maxjobs is greater than 1, make shall create a pool of up to maxjobs - 1 tokens. (Note that implementations are not required to create a pool of exactly maxjobs - 1 tokens. For example, an implementation could limit the pool size based on the number of processors available.) If the size of the token pool would be 0, make need not implement a token pool.

    When all of the following are true:

    • there is a target with commands that is not up-to-date

    • the target's prerequisites (if any) are up-to-date

    • make is not waiting to bring the target up-to-date (see .WAIT)

    • make is currently bringing a different target with commands up-to-date

    • make is not currently bringing maxjobs targets up-to-date in parallel

    • the special target .NOTPARALLEL is not specified

    • the token pool is not empty

    then make may attempt to remove one token from the pool. If a token is successfully removed, it shall attempt to bring this target up-to-date in parallel, and after this processing completes shall return the token to the pool. When make is bringing a target without commands up-to-date, it need not remove a token from the pool.

    If a rule invokes a sub-make either via the MAKE macro or via a command line that begins with '+', the sub-make is the same implementation as the make that invoked the sub-make, and the -j option is passed to the sub-make via the MAKEFLAGS environment variable with the same maxjobs value and is not overridden by a maxjobs value from another source (even if it has the same value), the sub-make shall use the same token pool as its invoking make rather than create a new token pool. Otherwise, it is unspecified whether the sub-make uses the same token pool as its invoking make or creates a new token pool. If a rule executes multiple sub-make processes asynchronously the behavior is unspecified.


    On page 2969 line 98476 change:
    However, lines with a <plus-sign> (<tt>'+'</tt>) prefix shall be executed.

    to:
    However, lines with a <plus-sign> (<tt>'+'</tt>) prefix and lines that expand the MAKE macro shall be executed.


    On page 2986 line 99178, delete the bullet item:
        
    Specifying that command lines containing the strings "${MAKE}" and "$(MAKE)" are executed when the −n option is specified (GNU and System V).


    On page 2987, delete lines 99229 - 99235 (the paragraph beginning "Early proposals ...").

    On page 2990, after line 99387 add the following new paragraphs:
    The standard specifies a way for portable applications to request parallel updating of targets with commands by using the -j maxjobs option. This feature is described in terms of a token pool initially containing up to maxjobs - 1 tokens. Note that this is not intended to prescribe a particular implementation design; the usual "as if" rule applies.

    Implementations are permitted to silently limit the pool size for a few reasons, including:
    Implementations that do not support parallelism can support the -j option by simply ignoring the option (other than passing it to sub-make invocations via the MAKEFLAGS environment variable). In effect, such an implementation silently restricts the size of the token pool to zero (and therefore need not create a token pool).
    Some historical implementations dynamically limit the token pool size based on current system load to avoid overloading the system.
    Implementations may want to limit the token pool size based on the number of processors available.
    Implementations may want to limit the token pool size based on resource limits.
    Limiting the pool size does not change the value of maxjobs that is passed to sub-make invocations via the MAKEFLAGS environment variable.

    When a different maxjobs value is passed to a sub-make, some historical make implementations created a separate pool of tokens while other historical make implementations continued to obtain tokens from the invoking make but limited the number of tokens held at a time to the new value of maxjobs - 1. Both behaviors are believed to have merit in different situations: The former gives a sub-make complete control the amount of parallelism while the latter allows the user to control overall system load. The standard permits either behavior.

    The standard calls for a token pool of size maxjobs - 1, and for removal from that pool only for the second and subsequent tasks in a set of parallel tasks. This design was chosen because this is effectively what existing implementations do, and also because the token consumed by a parallel task that invokes a sub-make is effectively lent to the sub-make. Lending the token to the sub-make has the following advantages:

    • It prevents the sub-make from being completely idle due to token starvation, allowing it to always make some progress regardless of how many tokens other sub-make invocations have consumed.

    • It prevents token pool exhaustion caused by a long chain of sub-make invocations. If the token consumed by the invoking rule was not effectively lent to the sub-make, then the pool would be exhausted by a chain of sub-make invocations that is maxjobs long. Such a chain would never accomplish any work, and would thus never complete.



    When a rule invokes multiple sub-make processes asynchronously (for example by using an asynchronous list in the shell), some implementations allow each sub-make to execute at least one rule even though this would cause the total number of parallel rule executions across all make instances to exceed maxjobs (after discounting the rules that execute sub-make processes). This behavior may not be ideal, but it is easier to implement and is unlikely to cause problems in practice because applications typically do not have any rules that invoke multiple sub-make processes asynchronously. For this reason the behavior is unspecified if a rule executes multiple sub-make processes asynchronously.

    When multiple sub-make processes are running in parallel there is no requirement placed on the ordering of output from these processes. Some implementations of make attempt to serialize output from each sub-<make>; others make no such attempt. If diagnostic messages from failed commands are intermixed, the usual way to deal with this is to repeat the make without -j (or with -j 1) so that intermixing will not occur.






    Viewing Issue Advanced Details
    1435 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-12-15 14:50 2021-08-16 10:47
    zackw
     
    normal  
    Applied  
    Accepted As Marked  
       
    Zack Weinberg
    GNU
    exec
    (unknown)
    (unknown)
    ---
    Note: 0005298
    execlp and execvp should not execute a command interpreter when other members of the exec family would fail with ENOEXEC
    When execve would fail for a particular process image file and set errno to [ENOEXEC], execlp and execvp are specified to retry execution of that file as-if it were a shell script (see the text quoted as to-be-deleted, under "Desired Action").

    This behavior exists only for historical reasons -- it predates the `#!` mechanism implemented by all current-generation Unixes -- and *may* constitute a security hole. On the computer where I'm typing this, the first eight bytes of a machine-code executable are consistently 7f 45 46 02 01 01 00. The behavior of `sh` when fed a file beginning with these bytes is not specified, as far as I can tell; I have observed at least one implementation which, when given an input file containing those eight bytes, attempts to execute a program whose name is "\177ELF\002\001\001" (C string notation). Suppose an unprivileged process that can create a program with that name in a location on a higher-privileged process's PATH. The higher-privileged process is believed only to run programs with known names, but one of the programs with a known name is corrupt and will be rejected by `execve` with ENOEXEC, causing `execvp` to attempt to run it as a shell script, and in turn to execution of the injected program named "\177ELF\002\001\001".

    Yes, this exploit chain involves a questionable implementation of `sh` *and* two different system misconfigurations, but I think we all know that vendors will try to argue that each of those three things is harmless and does not need fixing.
    From <https://pubs.opengroup.org/onlinepubs/9699919799/functions/exec.html>, [^] delete all of the following text from the DESCRIPTION section:

    ---
    There are two distinct ways in which the contents of the process image file may cause the execution to fail, distinguished by the setting of errno to either [ENOEXEC] or [EINVAL] (see the ERRORS section). In the cases where the other members of the exec family of functions would fail and set errno to [ENOEXEC], the execlp() and execvp() functions shall execute a command interpreter and the environment of the executed command shall be as if the process invoked the sh utility using execl() as follows:

    execl(<shell path>, arg0, file, arg1, ..., (char *)0);

    where <shell path> is an unspecified pathname for the sh utility, file is the process image file, and for execvp(), where arg0, arg1, and so on correspond to the values passed to execvp() in argv[0], argv[1], and so on.
    ---

    Also, in the ERRORS section, move the entry for [ENOEXEC] to the first group of errors ("The exec functions shall fail if: ...")
    Notes
    (0005173)
    geoffclare   
    2020-12-15 15:22   
    There is no point changing execlp() and execvp() as there are many other ways that a shell can be invoked to try to execute such a file (e.g. system(), popen(), or just interactive use of a shell). The right place to deal with the issue is in the shell, and this was done in bug 0001226.

    However, looking again at 1226 I see that there are two occurrences of the text that it fixes, and it only fixes one of them. Since that bug has already been applied, we should use this new bug as an opportunity to fix the other one.

    On page 2368 line 75615 section 2.9.1.1, change:
    If the executable file is not a text file, the shell may bypass this command execution. In this case, it shall write an error message, and shall return an exit status of 126.
    to:
    The shell may apply a heuristic check to determine if the file to be executed could be a script and may bypass this command execution if it determines that the file cannot be a script. In this case, it shall write an error message, and shall return an exit status of 126.
    <small>Note: A common heuristic for rejecting files that cannot be a script is locating a NUL byte prior to a <newline> byte within a fixed-length prefix of the file. Since sh is required to accept input files with unlimited line lengths, the heuristic check cannot be based on line length.</small>
    (0005174)
    zackw   
    2020-12-15 15:31   
    The other C-level APIs you mention, system and popen, are expected to invoke a shell. The exec family, on the other hand, is expected *not* to invoke a shell. I think it is worth changing execlp and execvp just to eliminate this corner case where a shell might be invoked, contrary to expectations.
    (0005175)
    geoffclare   
    2020-12-15 15:45   
    Re Note: 0005174

    No, only execl, execle, execv, and execve are expected not to invoke a shell. The execlp and execvp functions have always invoked a shell if the file cannot be executed directly and are very much expected to do so. Changing this would break a huge number of applications which rely on them executing a shell in cases where the shell script does not start with #! or the system does not support #!, including every strictly conforming application that includes a shell script to be executed using execlp or execvp (since strictly conforming applications cannot use #!)
    (0005176)
    zackw   
    2020-12-15 15:53   
    If you would prefer a more practical rationale for the proposed change, consider a program (a test suite driver, for instance) that wants to execute binaries that may have been cross-compiled and therefore may not be executable on the system where it's running, and distinguish ENOEXEC-type failure from other forms of failure. It runs on a wide variety of systems and therefore cannot rely on any binary-rejection heuristic being implemented by the shell. The only alternative is to avoid using execlp and execvp, but this means reimplementing PATH search by hand, and depending on the implementation language, it may not even be possible. (For instance, the Perl interpreter *only* exposes execvp and system to Perl programs.)
    (0005177)
    zackw   
    2020-12-15 15:54   
    On a closely related note, why on earth hasn't #! been standardized? I have *never* encountered a system that claims POSIX conformance but doesn't implement it, not even 25 years ago.
    (0005178)
    zackw   
    2020-12-15 15:56   
    > Changing this would break a huge number of applications

    Please give one example of such an application.
    (0005179)
    shware_systems   
    2020-12-15 16:11   
    Those interfaces support the idea a utility may be implemented as a binary or a script, with binaries having attempt priority. The #! concept is an adjunct to this, not a replacement. It is for hardware and configuration dependant reasons like in the description why #! is precluded by the standard, however. There was a request to add it, a few years ago, and the issues discussed for rejecting it include factors out of scope for the standard.

    It is also on the shell to provide a heuristic that recognizes sequences like in the description as being non-text, as the specified behavior, and reject processing it, per bug 1226. This is the same as the binary loader is expected to reject binaries for an incompatible processor. If it doesn't do this adequately it is a bug in that shell or loader, that I see, not a standard defect.
    (0005180)
    dannyniu   
    2020-12-16 02:08   
    (edited on: 2020-12-16 02:18)
    As a read-only member on the mailing list who currently supervises the security logic of the web app deployed on our Linux server, I totally agree this is a serious potential security exploit.

    I suggest a moderate modification. That is:

    Change:

    > execlp() and execvp() functions shall execute a command interpreter ...

    To

    > execlp() and execvp() functions may execute the "sh" command interpreter ...

    This would allow implementations with additional access control deny certain scripts with say "security flags" attached to them when they're fetched from network or scanned by anti-virus software.

    (0005294)
    kre   
    2021-03-19 17:51   
    We certainly cannot make the change as requested, which looks to me to be
    an attempt to force implementation change via the standards process, which
    is not the way that things should work.

    If implementations find that security is being compromised because of the
    way execlp() and execvp() work, then they will change the way those things
    work, or the applications that use them and become vulnerable. If the
    former were to happen, updating the standard might become reasonable.

    I actually doubt there is any real problem however, privileged applications
    shouldn't (while privileged) be running anything using execlp() or execvp(),
    there are way too many ways to compromise that ... doing so using the
    method described here would be way down the list of things to try - and one
    of earlier attempts would probably succeed. Avoiding using those functions
    in privileged applications is the correct solution (and doing manual PATH
    searches as a replacement is almost as bad, so they shouldn't do that either).

    Just reject this bug.
    (0005295)
    kre   
    2021-03-19 17:55   
    Re Note: 0005294

    Just in case I caused confusion, I have no problem using this bug as
    a mechanism to fix the issue noted in Note: 0005173

    Just not to do what is requested in the Desired Action.
    (0005298)
    geoffclare   
    2021-03-22 15:43   
    At page 784, line 26542 exec DESCRIPTION, change:
    The argument path points to a pathname that identifies the new process image file.
    to:
    For the execl(), execle(), execv(), and execve() functions, the argument path points to a pathname that identifies the new process image file.

    At page 784, line 26543 exec DESCRIPTION, change:
    The argument file is used to construct a pathname ...
    to:
    For the execlp() and execvp() functions, the argument file is used to construct a pathname ...

    On page 2368 line 75615 section 2.9.1.1, change:
    If the executable file is not a text file, the shell may bypass this command execution. In this case, it shall write an error message, and shall return an exit status of 126.
    to:
    The shell may apply a heuristic check to determine if the file to be executed could be a script and may bypass this command execution if it determines that the file cannot be a script. In this case, it shall write an error message, and shall return an exit status of 126.
    <small>Note: A common heuristic for rejecting files that cannot be a script is locating a NUL byte prior to a <newline> byte within a fixed-length prefix of the file. Since sh is required to accept input files with unlimited line lengths, the heuristic check cannot be based on line length.</small>




    Viewing Issue Advanced Details
    1434 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Comment Enhancement Request 2020-12-15 01:08 2021-04-01 09:40
    andras_farkas
     
    normal  
    Applied  
    Accepted As Marked  
       
    Andras Farkas
    pr
    3107
    103845-103847
    ---
    Note: 0005283
    Moving the -p option in the pr utility synopsis
    In
    https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pr.html [^]
    -f and -p are put together as [-fp] in the synopsis. As far as I can tell, if I understand it right, this is a historical artifact from them being added to POSIX from the old SUS standard at the same time. -f is XSI while -p is normal (required, unshaded) so I feel -p can be moved into [-adFmrt] for [-adFmprt] since those are all the other regular flags that don't take arguments.

    I originally noted this as a minor nit here:
    https://austingroupbugs.net/view.php?id=1432 [^]

    An additional thing that could be done might be to move -f too, in the sense of taking it from the very end of the synopsis, and putting it closer to the front, like so:
    [-adFmprt] [-f]
    Move -p into [-adFmrt] to make that section [-adFmprt]
    Notes
    (0005171)
    andras_farkas   
    2020-12-15 01:10   
    Or, even, since a single flag can be XSI shaded in a group of flags, maybe even...
    [-adfFmprt] or [-adFfmprt] (with XSI shading around the f)
    (0005172)
    andras_farkas   
    2020-12-15 01:10   
    Though if that should be a separate issue, I can make that. Or we can rename this issue to "moving multiple options in the pr synopsis" or something like that.
    (0005283)
    geoffclare   
    2021-03-18 15:52   
    Change:
    [−adFmrt]
    to:
    [−adfFmprt]
    with XSI shading on just the "f" character.

    Delete "[ −fp]"
    (0005293)
    andras_farkas   
    2021-03-19 16:03   
    That's even better! Thanks, geoffclare.




    Viewing Issue Advanced Details
    1433 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Clarification Requested 2020-12-12 00:16 2021-04-27 09:13
    andras_farkas
     
    normal  
    Applied  
    Accepted As Marked  
       
    Andras Farkas
    pr
    Approved
    Note: 0005282
    The use of carriage-return in pr
    On
    https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pr.html [^]
    I see that -p waits for <carriage-return>.

    Is this intentional?

    I see that OpenSolaris (System V), Unix 10th edition, FreeBSD, and NetBSD pr wait for <newline> ('\n') and not <carriage-return>. I did not find an implementation which waits for <carriage-return>. Also, in grepping the entire POSIX standard for carriage-return and \r I find very few uses of it, while uses of <newline> for user input are universal.
    Figure out whether pr should truly wait for <carriage-return> or not.
    Notes
    (0005282)
    Don Cragun   
    2021-03-18 15:41   
    Interpretation response
    ------------------------
    The standard states thar pr waits for <carriage-return>, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor.

    Rationale:
    -------------
    This is not the way any implementation works and conflicts with similar requirements for other utilities.

    Notes to the Editor (not part of this interpretation):
    -------------------------------------------------------

    On page 3108 line 103918 section pr, change:
    <carriage-return>
    to:
    <newline>
    (0005288)
    agadmin   
    2021-03-19 11:35   
    Interpretation proposed: 19 March 2021
    (0005326)
    agadmin   
    2021-04-22 08:42   
    Interpretation Approved: 22 April 2021




    Viewing Issue Advanced Details
    1431 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Error 2020-12-10 16:00 2021-04-01 09:38
    dirk_f
     
    normal  
    Applied  
    Accepted  
       
    Dirk Fieldhouse
    XCU/4 Utilities/command
    2596
    84261ff
    ---
    Incorrect xref for shell function lookup in specification of the 'command' utility
    > The command utility shall cause the shell to treat the arguments as a simple command,
    > suppressing the shell function lookup that is described in Section 2.9.1.1 (on page 2367), item 1b.

    Should be "item 1c".

    From Geoff Clare:
    "It seems that the fault lies with bug 935 which added a new item 1b (and was applied via TC2)."

    Change the text quoted in the Description above, as below:

    "
    The command utility shall cause the shell to treat the arguments as a simple command, suppressing the shell function lookup that is described in Section 2.9.1.1 (on page 2367), item 1c.
    "

    The issue continues in the latest Issue 8 draft, although the Section reference itself has been updated to 2.9.1.4. Is there an automatic process for forwarding this, or does it need a separate report?
    Notes
    (0005163)
    geoffclare   
    2020-12-10 17:40   
    Re "Is there an automatic process for forwarding this, or does it need a separate report?"

    It's automatic. Assuming this bug is resolved with a tc3-2008 tag, the fix will be applied in the Issue7TC3 branch in gitlab and then will automatically get merged into the Issue8 branch the next time I do a "git pull origin Issue7TC3" in that branch.




    Viewing Issue Advanced Details
    1430 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Error 2020-12-09 12:02 2021-04-01 09:36
    geoffclare
     
    normal  
    Applied  
    Accepted  
       
    Geoff Clare
    The Open Group
    write()
    2311, 2314
    73485, 73634
    ---
    Text about interleaving data should say thread not process
    Where the description of write() talks about data written to a pipe (or FIFO) being interleaved it refers to "other processes" but should say "other threads".

    Also, the text on line 73481 before the bullet list says "pipe or FIFO" but the bullet list items only say "pipe".
    On page 2311 line 73485 section write(), change:
    Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other processes doing writes on the same pipe. Writes of greater than {PIPE_BUF} bytes may have data interleaved, on arbitrary boundaries, with writes by other processes, whether or not the O_NONBLOCK flag of the file status flags is set.
    to:
    Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other threads performing write operations on the same pipe. Writes of greater than {PIPE_BUF} bytes may have data interleaved, on arbitrary boundaries, with write operations by other threads, whether or not the O_NONBLOCK flag of the file status flags is set.

    On page 2311 lines 73483-73500 section write(), change all occurrences of "pipe" to "pipe or FIFO".

    On page 2314 line 73634 section write(), change:
    interleaved with data from any other process
    to:
    interleaved with data from any other thread

    There are no notes attached to this issue.




    Viewing Issue Advanced Details
    1429 [1003.1(2016/18)/Issue7+TC2] System Interfaces Objection Omission 2020-12-02 16:57 2021-04-01 09:34
    Konrad_Schwarz
     
    normal  
    Applied  
    Accepted As Marked  
       
    Konrad Schwarz
    Siemens AG
    recvfrom timeout
    recvfrom
    1
    1
    ---
    Note: 0005281
    recvfrom does not specify the errno on timeout when the SO_RCVTIMEO option has been set
    POSIX specifies the SO_RCVTIMEO socket option, that causes a receiving call on a socket to timeout when an error occurs. However, the recvfrom() (and presumably the related calls recv(), read() etc.) do not specify what errno is set to in this case.
    According to the Linux socket(7) man page, EAGAIN/EWOULDBLOCK is returned in this case. (ETIMEDOUT would also kind of make sense and is what I assumed originally).

    Add to the description of EGAIN in ERRORS the possibility of a timeout due to SO_RCVTIMEO.
    Notes
    (0005148)
    geoffclare   
    2020-12-03 10:39   
    The required behaviour is specified in XSH 2.10.16 Use of Options:

    P530 L18640 (SO_RCVTIMEO):
    If a receive operation has blocked for this much time without receiving additional data, it shall return with a partial count or errno shall be set to [EAGAIN] or [EWOULDBLOCK] if no data were received.

    P531 L18665 (SO_SNDTIMEO):
    If a send operation has blocked for this much time, it shall return with a partial count or errno set to [EAGAIN] or [EWOULDBLOCK] if no data were sent.


    We could either repeat this detail on each affected function page or just add a reference to 2.10.16.
    (0005149)
    Konrad_Schwarz   
    2020-12-03 10:48   
    My apologies; I think I was a bit tired yesterday.

    I think the suggested reference would be fine.
    (0005150)
    shware_systems   
    2020-12-03 11:25   
    I'm more partial to requiring EAGAIN, not either, reserving EWOULDBLOCK for discovered conditions unrelated to timing such as hardware power failure on other end, if these are 2 different values. If they are the same value this would be a case for requiring them to be different.
    (0005151)
    kre   
    2020-12-03 13:59   
    Re Note: 0005150

    XBD 14 <errno.h> says:

        [EAGAIN] Resource unavailable, try again (may be the same value as
              [EWOULDBLOCK]).

        [EWOULDBLOCK] Operation would block (may be the same value as [EAGAIN]).

    In an I/O operation they mean the same thing, the two are very often the
    same errno number (which is why the standard says they may be).

    Apart from that (which I suppose could be undone, at the expense of breaking
    many systems, as changing these values totally destroys binary compat for
    the system) "reserving EWOULDBLOCK for discovered conditions unrelated to
    timing such as hardware power failure on other end" is definitely not
    EWOULDBLOCK - that's EIO.

    The only argument against EAGAIN/EWOULDBLOCK here (and it is a weak one) is
    that those generally occur immediately, rather than after a timeout period.
    ETIMEDOUT (if it had its description changed from "connection timed out" to
    "operation timed out" would probably fit a little better for an operation which
    tries for a while, then eventually gives up.

    But what really matters is not what anyone believes makes sense, but what
    the actual implementations return.
    (0005159)
    shware_systems   
    2020-12-04 18:02   
    I would expect EIO to be about the system calling the interface, not about conditions on the other end of a connection. This is because there are codes like EAGAIN that are explicitly about the other end of a connection. Maybe some implementations don't differentiate and just use EIO as a blanket report code, given the terseness of "I/O error" as its definition in <errno.h>; maybe some have EAGAIN and EWOULDBLOCK as the same value, and use that where some other implementation might use EIO, but other systems may have 3 values. It is a definite portability issue on these systems if some return EAGAIN, and others EWOULDBLOCK; it forces application code to test for both constants, bloating both code and exec time, and does not give an application sufficient info about whether to retry or abort the attempt.

    There's not a single one for indicating a timeout occurred but should complete eventually (nothing broke, latency is just higher than was expected), and a different one that says something happened where it can't tell if the operation might complete (something did break, but it wasn't corrected on other end before the timeout expired), and even a third for something broke on our end and any retry can't succeed until that's fixed.
    (0005281)
    geoffclare   
    2021-03-18 15:23   
    (edited on: 2021-03-18 15:24)
    On page 1793, libes 58071-58074 section recv(), change:
        
    If no messages are available at the socket and O_NONBLOCK is not set on the socket’s file descriptor, recv( ) shall block until a message arrives.

    to:
        
    If no messages are available at the socket and O_NONBLOCK is not set on the socket’s file descriptor, recv( ) shall block until a message arrives or a timeout occurs (see SO_RCVTIMEO in [xref to section 2.10.16]).


    On page 1794 line 58081 section recv() ERRORS, append:
    See also SO_RCVTIMEO in [xref to section 2.10.16].


    On page 1794 at line 58111 add setsockopt() to the "SEE ALSO" list.

    On page 1797 lines 58170-58173 section recvfrom(), change:
    If no messages are available at the socket and O_NONBLOCK is not set on the socket’s file descriptor, recvfrom( ) shall block until a message arrives.

    to:
    If no messages are available at the socket and O_NONBLOCK is not set on the socket’s file descriptor, recvfrom( ) shall block until a message arrives or a timeout occurs (see SO_RCVTIMEO in [xref to section 2.10.16]).


    On page 1797 line 58185 section recvfrom() ERRORS, append:
    See also SO_RCVTIMEO in [xref to section 2.10.16].


    On page 1798 at line 58208 add setsockopt() to the "SEE ALSO" list.

    On page 1799 lines 58251-58252 section recvmsg(), change:
    If no messages are available at the socket and O_NONBLOCK is not set on the socket’s file descriptor, recvmsg( ) shall block until a message arrives.

    to:
    If no messages are available at the socket and O_NONBLOCK is not set on the socket’s file descriptor, recvmsg( ) shall block until a message arrives or a timeout occurs (see SO_RCVTIMEO in [xref to section 2.10.16]).


    On page 1800 line 58284 section recvmsg() ERRORS, append:
    See also SO_RCVTIMEO in [xref to section 2.10.16].


    On page 1801 at line 58310 add setsockopt() to the "SEE ALSO" list.

    On page 1877 line 60823-60825 section send(), change:
    If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does not have O_NONBLOCK set, send( ) shall block until space is available.

    to:
    If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does not have O_NONBLOCK set, send( ) shall block until space is available or a timeout occurs (see SO_SNDTIMEO in [xref to section 2.10.16]).


    On page 1877 line 60837 section send() ERRORS, append:
    See also SO_SNDTIMEO in [xref to section 2.10.16].



    On page 1881 lines 60918-60920 section sendmsg(), change:
    If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does not have O_NONBLOCK set, sendmsg( ) shall block until space is available.

    to:
    If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does not have O_NONBLOCK set, sendmsg( ) shall block until space is available or a timeout occurs (see SO_SNDTIMEO in [xref to section 2.10.16]).


    On page 1881 line 60934 section sendmsg() ERRORS, append:
    See also SO_SNDTIMEO in [xref to section 2.10.16].


    On page 1884 line 61051-61054 section sendto(), change:
    If space is not available at the sending socket to hold the message to be transmitted and the socket file descriptor does not have O_NONBLOCK set, sendto( ) shall block until space is available.

    to:
    If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does not have O_NONBLOCK set, sendto( ) shall block until space is available or a timeout occurs (see SO_SNDTIMEO in [xref to section 2.10.16]).


    On page 1885 line 61066 section sendto() ERRORS, append:
    See also SO_SNDTIMEO in [xref to section 2.10.16].






    Viewing Issue Advanced Details
    1428 [1003.1(2016/18)/Issue7+TC2] Base Definitions and Headers Objection Omission 2020-11-27 14:56 2021-04-01 09:30
    geoffclare
     
    normal  
    Applied  
    Accepted  
       
    Geoff Clare
    The Open Group
    3
    71,79
    2069,2270
    ---
    Add definitions of "negative" and "positive"
    A discussion on the mailing list that arose from Note: 0005108 identified the need to add definitions of the terms "negative" and "positive".
    On page 71 line 2069 section 3, insert:

    3.239 Negative
    When describing a value (not a sign), less than zero. Note that in the phrase ``negative zero'' it describes a sign, and therefore negative zero (also represented as -0.0) is not a negative value.
    and renumber the later subsections.

    On page 79 line 2270 section 3, insert:

    3.285 Positive
    When describing a value (not a sign), greater than zero. Note that in the common phrase ``positive zero'' (which is not used in this standard, although the representation +0.0 is) it describes a sign, and therefore positive zero (+0.0) is not a positive value.
    and renumber the later subsections.
    There are no notes attached to this issue.



    Viewing Issue Advanced Details
    1427 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Objection Error 2020-11-26 15:21 2021-04-27 09:11
    geoffclare
     
    normal  
    Applied