Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0001437 [1003.1(2016/18)/Issue7+TC2] Shell and Utilities Editorial Enhancement Request 2020-12-15 21:02 2021-11-26 15:03
Reporter steffen View Status public  
Assigned To
Priority normal Resolution Accepted As Marked  
Status Applied  
Name steffen
Organization
User Reference
Section Vol. 3: Shell and Utilities, Issue 7, make
Page Number (2975) 2989
Line Number (98708, 98742) 99308
Interp Status ---
Final Accepted Text See Note: 0005489.
Summary 0001437: make: (document .NOTPARALLEL and .WAIT special targets) in RATIONALE
Description Parallel, even massively parallel processing has become the widely supported and
used default, yet the standard make(1) does not document it.

If the -j command line option as requested in issue #1436 is standardized,
the standard should also mention ways to synchronize rule processing, in order
to support more complicated build relationships seen in real-life projects.

The suggestion given here can be used with the most widely used make(1)
implementations (GNU make, BSD make) on any tested platform (Linux, *BSD, SunOS
/ Solaris).

The make family rooted on the Sun side of likfe has has at least .WAIT: reserved
for decades, and in practice

  .NOTPARALLEL:
  .WAIT: # Luckily BSD make supports specifying this as target, too

  tangerine: config .WAIT build .WAIT test .WAIT install
  citron: config .WAIT build .WAIT install
  all: config .WAIT build

produces the desired outcome.
Desired Action (
On page 2975, insert before line 98708

  .NOTPARALLEL
    When specified all rules specified in the given makefile are processed
    sequentially, as if -j has not been given.

On page 2975, insert before line 98472

  .WAIT
    When .WAIT appears in a dependency line, prerequisites that appear after it
    are built no sooner until all preceeding prerequisites have been built
    successfully.

On page 2975, change on line 98472 ff.

  The special targets .IGNORE, .POSIX, .PRECIOUS, .SILENT, and .SUFFIXES shall
  be specified without commands.

to

  The special targets .IGNORE, .NOTPARALLEL, .POSIX, .PRECIOUS, .SILENT,
  .SUFFIXES and .WAIT shall be specified without commands.
)

On page 2989, insert before line 99308

  The special targets .NOTPARALLEL and .WAIT represent different, rather
  incompatible synchronisation concepts of parallel build environments as
  used by the diverse implementations. By using an indirection both approaches
  can be supported: for implementations which use .NOTPARALLEL in a makefile to
  ensure sequential rule processing in the given file the full power of parallel
  processing will be available in recursive make instances invoked on makefiles
  without the directive, implementations which synchronize on the .WAIT special
  target will synchronize on it, for example:

    .NOTPARALLEL: # Process all rules sequentially
    .WAIT: # Define to satisfy "the other implementation"

    install: config .WAIT all
      echo Installing..
      ..
    all: config .WAIT build
    build:
      make -f makefile.build
    config:
      ./configure
Tags issue8
Attached Files ? file icon make.diff [^] (16,063 bytes) 2021-08-28 19:11
? file icon easier.diff [^] (7,414 bytes) 2021-09-02 23:20
? file icon gmake-4.3.wait.diff [^] (6,555 bytes) 2021-09-03 13:33

- Relationships
related to 0001436Applied make: add "-j max_jobs" option to support simultaneous rule processing 

-  Notes
(0005184)
psmith (developer)
2020-12-17 17:10

Just to note, GNU make does not support the .WAIT special target.
(0005185)
steffen (reporter)
2020-12-17 19:23

Ach! I wish it would, i think the .WAIT approach "is better" (as in explicit, and does not need an indirection).
But you said on the ML that you do not plan to support it either.

This is why the first part above is in parenthesis, and this issue actually requests some words in RATIONALE instead.

The presented solution has the desired effect also in GNU make ("build" target uses parallel processing).
(0005186)
psmith (developer)
2020-12-17 21:16

I don't believe I said I didn't want GNU make to support it; I agree that it seems like a useful feature. I just said that given the way make's parallel support works I suspect implementation is not so easy and I didn't plan to work on it as I have other things on my (relatively small) plate.

But, if someone else were to get motivated to try to get this working that would be fine with me :). I wonder if creating some kind of internal/implicit prerequisite relationship that wasn't visible to users would be a simpler way to implement this... or not. Well, something for someone to consider.
(0005444)
joerg (reporter)
2021-08-14 12:19

Well, implementing .WAIT is not really hard.

You need to stop creating new active process slots if you see a .WAIT
in the dependency list but wait until all running jobs for that target
are finished before starting new jobs from the list to the right of
.WAIT.

You also need to remove .WAIT from the list of dependencies, if that
list is needed for things like e.g. $^ or $?.

In cases of doubt, SunPro Make is OpenSource...
(0005446)
rhansen (manager)
2021-08-19 16:22
edited on: 2021-08-19 16:22

On page 2975, insert before line 98708:
.NOTPARALLEL
The application shall ensure that this special target is specified without prerequisites or commands. When specified, make shall update one target at a time, regardless of whether the -j maxjobs option is specified. If the -j maxjobs option is specified, the option shall continue to be passed unchanged to sub-make invocations via MAKEFLAGS.

On page 2975, insert before line 98742:
.WAIT
The application shall ensure that this special target, if specified as a target, is specified without prerequisites or commands. When .WAIT appears as a target, it shall have no effect. When .WAIT appears in a target rule as a prerequisite, it shall not itself be treated as a prerequisite; however, prerequisites that appear after it shall not be brought up-to-date until after all preceding prerequisites for the same target have been brought up-to-date.

On page 2975, change line 98742 from:
The special targets .IGNORE, .POSIX, .PRECIOUS, .SILENT, and .SUFFIXES shall be specified without commands.
to:
The special targets .IGNORE, .NOTPARALLEL, .POSIX, .PRECIOUS, .SILENT, .SUFFIXES and .WAIT shall be specified without commands.

On page 2975 lines 98744-98747, change:
Targets with names consisting of a leading <period> followed by the uppercase letters "POSIX" and then any other characters are reserved for future standardization. Targets with names consisting of a leading <period> followed by one or more uppercase letters are reserved for implementation extensions.
to:
Targets and prerequisites consisting of a leading <period> followed by the uppercase letters "POSIX" and then any other characters are reserved for future standardization. Targets and prerequisites consisting of a leading <period> followed by one or more uppercase letters, that are not described above, are reserved for implementation extensions.


(0005447)
psmith (developer)
2021-08-19 18:39

I'm assuming the title of this issue should reference the RATIONALE section not the RATIONAL section.

I don't have a copy of the document that includes these line numbers so I can't quite understand what is being proposed here. The original proposal was to add this to the RATIONALE section which means it would not be binding and required (as I understand things) but instead only a suggestion, and probably reserving these targets for future implementations.

As far as I can tell, though, these changes are being made to the "Extended Description" section, not the RATIONALE section, which makes these new features that are required of all conforming implementations.

Am I wrong about that?

As I said above, GNU make doesn't support the .WAIT feature.

As I said above, it is likely not actually trivial to implement .WAIT in GNU make. I don't know how Sun make implements parallelism but in GNU make we don't require an entire branch of the graph to complete before moving to the next sibling branch, if parallel builds are enabled. For example:

  all: ONE TWO

  ONE: THREE FOUR

  ONE TWO THREE FOUR: ; echo $@; sleep 2

If we run this as "make -j4" then we'll get the following output:

  echo THREE; sleep 1
  echo FOUR; sleep 1
  echo TWO; sleep 1
  THREE
  FOUR
  TWO
  echo ONE; sleep 1
  ONE

Note that GNU make runs TWO in parallel with THREE and FOUR, even though ONE has not completed yet. You can imagine that TWO itself is a complex dependency graph and the actual target built in parallel may well be a long ways away (in the graph). If we change this makefile to add the putative .WAIT:

  all: ONE TWO

  ONE: THREE .WAIT FOUR

  ONE TWO THREE FOUR: ; echo $@; sleep 2

what we'd like to see (one assumes) is:

  echo THREE; sleep 1
  echo TWO; sleep 1
  THREE
  TWO
  echo FOUR; sleep 1
  FOUR
  echo ONE; sleep 1
  ONE

so that we'd run TWO in parallel with THREE, rather than waiting to start TWO until after FOUR is complete.

This means we need to be able to detect a .WAIT target and stop searching the DAG at that point, but rather than just waiting for the previous dependencies to complete we want to move on to other siblings of the current target and see if there are any more targets that might be buildable, then come back later after THREE has completed.
(0005448)
rhansen (manager)
2021-08-19 18:55
edited on: 2021-08-19 19:04

> As far as I can tell, though, these changes are being made to the "Extended Description" section, not the RATIONALE section, which makes these new features that are required of all conforming implementations.

Correct, that was our intention. I asked about making .WAIT non-normative during today's telecon (see line 68 of https://posix.rhansen.org/p/2021-08-19 [^] ), and the group decided that it was useful enough to include as a normative requirement.

But thinking about this more, isn't:
target: $(leftstuff) .WAIT $(rightstuff); do_thing
equivalent to:
$(rightstuff): $(leftstuff)
target: $(leftstuff) $(rightstuff); do_thing
?

If so, two questions:
  • Could it be easily implemented in GNU make as syntactic sugar?
  • Does .WAIT add enough value to warrant inclusion in the standard?


(0005449)
shware_systems (reporter)
2021-08-19 19:27

If this was to be a TC type addition, then yes, keeping it as Rationale and Future Direction would be the way to go. However, as most modern CPUs do have multiple cores capable of parallelism, the consensus was more the future is now, rather than wait until Issue 9, to make them normative.

Your concerns were part of the discussion, including a paraphrase of the example you provided, and the feeling was more non-trivial is not the same as non-doable too, it just may take longer for someone to do it, so was not a bar to adopting it.
(0005450)
psmith (developer)
2021-08-19 22:47

> But thinking about this more, isn't: ... equivalent to:

Yes, basically: that's what I meant when I wrote in my comment last December, above:

> I wonder if creating some kind of internal/implicit prerequisite relationship that wasn't visible to users would be a simpler way to implement this... or not.

However your construction is not quite correct because it says that $(rightstuff) will be rebuilt if $(leftstuff) is rebuilt, which is wrong. If you use GNU make's "order-only prerequisite" feature, though, you could write it like this and it would work properly:

  target: $(leftstuff) $(rightstuff); do_thing
  $(rightstuff): | $(leftstuff)

I think a real implementation of .WAIT is more effort than is implied by the term "syntactic sugar", but this seems to be a promising way to think about the problem.

At this time I have no plans, or time, to implement either the new :::= operator, which I continue to maintain as per my comments in that issue is simply not useful, or certainly not useful enough to rise to the level of being added to the standard; or the .WAIT feature, which I agree is a good idea but which is a non-trivial amount of work.

So, I guess GNU make will just become a non-conforming implementation until such time as someone puts forward the effort to change that.

> this was to be a TC type addition, then yes, keeping it as Rationale and Future Direction would be the way to go. However, as most modern CPUs do have multiple cores capable of parallelism, the consensus was more the future is now, rather than wait until Issue 9, to make them normative.

This seems like a strawman to me: I have not suggested that adding -j was a problem: indeed, I welcome the addition of -j and agree it allows better use of modern systems.

However, people have been writing makefiles supporting parallelism using GNU make for 25+ years without the .WAIT capability. It definitely makes some (fairly rare) things easier, but there are other ways to manage those situations and not having .WAIT has not prevented massive build systems from being successful using parallel GNU make.

> Your concerns were part of the discussion, including a paraphrase of the example you provided, and the feeling was more non-trivial is not the same as non-doable too, it just may take longer for someone to do it, so was not a bar to adopting it.

Pretty much everything is a SMOP, in the end. If the only barrier to standardization is what is possible then indeed there's virtually no limit.
(0005451)
shware_systems (reporter)
2021-08-19 23:09

Re: 5448
No, I don't see them as equivalent. The version with .WAIT has the greater of left or right count pool removals starting in parallel. The replacement you suggest has potentially left count times right count starting that way.
(0005452)
psmith (developer)
2021-08-20 18:38

Sorry, I didn't understand the comment #5451: I don't know what a "left or right count pool removal" is, or why anything would be "left count times right count". The comment in #5448 IS an equivalent way to represent the .WAIT version with one significant difference, as I said in my comment: it declares that if any of the targets in $(leftstuff) are updated, then all the targets in $(rightstuff) will be considered out of date. Other than that it behaves identically.

The extra dependency relationship is a fundamental change in the behavior, though, so that's not good. GNU make has a feature which avoids it. Nevertheless, the .WAIT version is easier to read and write so it's helpful. Just, not necessarily trivial to implement.
(0005453)
rhansen (manager)
2021-08-20 19:28
edited on: 2021-08-20 19:43

> If you use GNU make's "order-only prerequisite" feature, though, you could write it like this and it would work properly:

TIL about GNU make's order-only prerequisites, thank you. It's good to know that all of the make implementations provide a way to order prerequisites, just with different syntax (.WAIT vs. pipe) and slightly different semantics (but similar enough that one can fairly easily convert a makefile from one format to the other).

Given that all makes provide a way to order prerequisites, I think that adding the feature to POSIX in some form is a worthwhile activity. If everyone agrees, the questions I think we should consider futher are:

  • Would it be better to standardize GNU make's behavior instead of .WAIT? I agree with your statement "the .WAIT version is easier to read and write", so I think it would be better to standardize .WAIT.

  • Should we standardize it now? GNU make is the dominant make implementation, and if it's not going to have .WAIT any time soon then standardizing .WAIT just makes POSIX weaker, not better. However, if patches adding .WAIT to GNU make are welcome, and there is a potentially straightforward implementation path based on the existing order-only prerequisite feature, the act of standardizing .WAIT might motivate someone to contribute support.



(0005454)
shware_systems (reporter)
2021-08-20 19:39

The description of a pool is part of the resolution of Bug 1436, basically the number of data allocations necessary to manage parallelism when -j value greater than one. The count is the subset of that number actually needed to start the invocations the makefile specifies to bring targets or prereqs up to date.

With:
rightstuff=foo bar baz
leftstuff=w x y z
$(rightstuff):$(leftstuff)

you can have 12 invokes started in parallel, with 3 each trying to update w, x, y, or z independently. If -j 10 specified 2 of those are waiting anyways so other executions complete. With the .WAIT version the max in parallel is four, the count of leftstuff. Then rightstuff can reuse the allocations leftstuff needed to be tracked as completed or not.
(0005455)
psmith (developer)
2021-08-20 20:20

Re 5454:

Sorry but I don't see anything in issue #1436 which talks about "the number of data allocations" or anything that describes a "pool", and I still didn't really understand your comment.

I think there's a fundamental confusion here. The makefile:

    foo bar baz: w x y z

is 100% identical to writing:

    foo: w x y z
    bar: w x y z
    baz: w x y z

That is, you're declaring 3 targets and giving them all the same 4 prerequisites. That's ALL it means. In fact once the makefile is parsed there's no way to differentiate the above two constructs based on the in-memory graph representation (at least, not in GNU make...) and that equivalency is required by POSIX.

There's no change to this idea introduced by issue #1436 and -j.

Given the above makefile then per invocation of make you may build a maximum of 7 targets (w, x, y, z, foo, bar, baz), and have a maximum of 4 jobs running in parallel (w, x, y, z). Never more than that. You will definitely never need to run 12 jobs, in parallel or not.

If you add .WAIT, say between "w x" and "y z", you're only reducing the maximum number of possible parallel jobs because now make can't start y or z until after w and x are complete.

But nothing about .WAIT or -j can ever increase the maximum number of targets that could be built.
(0005456)
psmith (developer)
2021-08-20 20:57

Regarding comment #5453:

Just to point out that GNU make's order-only prerequisites is a more powerful construct than .WAIT. .WAIT is used when you can't allow two targets to run in parallel BUT at the same time you don't want them to depend on each other (if they could depend on each other you'd just use dependencies, not .WAIT). That's a different and fundamentally weaker guarantee than OO prerequisites provide: they give a true ordered relationship represented in the dependency graph, just with a differently colored edge that doesn't imply dependency.

Consider this example:

  all: foo bar
  foo: one .WAIT two
  bar: two one

Here we see that while we're building "foo", "two" won't be built in parallel with "one".

However, while we're building "bar", "one" and "two" CAN be built in parallel because there's no .WAIT between them.

If you set -j high enough (here, we only need -j2) then you'll still get "one" and "two" building at the same time; "one" is built as a dependency of "foo", then the updating of dependencies of "foo" is paused due to .WAIT so make continues looking for something else to build and finds "two" as a dependency of "bar"; there's no .WAIT here so make will go ahead and build "two", while "one" is still (potentially) running.

Although easy to write, my opinion is that .WAIT is actually pretty hard to use in a reliable way due to this behavior. I consider it a kind of a hacky / quick-and-dirty shortcut that avoids implementing a robustly correct solution in your makefile.

With order-only prerequisites you would write this makefile:

  all: foo bar
  foo: one
  one: | two
  bar: two one

Here, "one" actually depends on "two" and that's represented in the dependency graph as an edge, so no matter where "one" appears it can never be updated before "two" is complete.


However I was not really suggesting that OO prerequisites should be chosen instead of .WAIT: OO prerequisites were not actually created for this purpose (at least not directly). I'm just saying that their existence in GNU make allows people to implement sort-of-.WAIT-like features in GNU make makefiles, if they want to.

And I'm also saying that, even though OO prequisites exist in GNU make, I'm not convinced it's completely straightforward to use it to implement .WAIT.

However, I am open to patches implementing it (assuming the appropriate copyright assignment paperwork is completed etc.)
(0005457)
joerg (reporter)
2021-08-21 17:06
edited on: 2021-08-21 23:12

I am not sure whether the gmake method with the pipe symbol is
able to be usable as a complete replacement for features offered
by .WAIT.

I also see no chance that this gmake specific method will ever be
supported by other make implementations. So adding that method
to POSIX would definitely weaken the POSIX standard.

BTW: gmake still has various other bugs that prevent me personally
from being able to use gmake in parallel mode. I am talking about a
makefile system that offers portability to all currently available
platforms (z/OS/EBCDIC just has been made working a month ago). The
makefile system currently works without problems using smake and
SunPro Make and (in serial mode) using gmake.

The make implementation specific definitions are inside special files
that are included based on the name of the make implementation in use.
That concept of course requires to only have features that may be used
in such a system without a need to have make implementation specific
rules in the general makefiles. With these experiences in mind, I worked
on the recent POSIX standardization to get something that matches the
criteria for using different existing make implementations.

With this makefile system and trying to use gmake in parallel mode,
this are the current show stoppers:

- gmakes calls readdir()/stat() at startup time and caches all timestamps.

  Unfortunately, gmake does never update these cached timestamps and as a
  result, it incorrectly claims that files are not present even though they
  just have been created from a rule that just has been executed - just
  because these files have been missing at startup time.

  The gmake bug tracking system claims that this bug has been fixed,
  but I cannot see a new version for testing, nor got a report from
  Paul that he did test whether that fix works for the schily makefile
  system.


- missing include files with rules that allow to remake them in many
  cases do not work because gmake executes the rule commands in the
  inverse expected order and gmake does not offer a way to enforce the
  intended order. Paul has been given the chance to verify that gmake
  offers a way to deal with that problem by sending a patch to the
  affected makefiles that does not prevent other make programs to work
  with these makefiles, but he preferred to do nothing.

  The gmake bug tracking system claims that this bug has been fixed,
  but I cannot see a new version for testing, nor got a report from
  Paul that he did test whether that fix works for the schily makefile
  system.

BTW: Other make implementations just work fine even though there is
no .WAIT pseudo-target in use at the locations that currently fail
with gmake.

With the current version of gmake, I need to call gmake 5 times from the
top level directory (starting from am fresh unpacked source) in order be
able to finish with the compilation in parallem mode.

If there is a fix for the above problems, it would really help to see
a new gmake version now, since the last version is from January 2020.


Finally, I remember that I had a discussion with Paul where he offered to
implement .WAIT in case I implement .NOTPARALLEL into SunPro Make. I
unfortunately cannot find that anymore.

But given the fact that I spent 3 weeks for implementing things like ::=
and .NOTPARALLEL into SunPro Make, even though these features are not
needed because there are better existing methods, it would be a nice
gesture from Paul to spend some time in making gmake more compatible
to other make implementations. .WAIT is available in SunPro Make and
BSD make since 30 years and that should be a verification for a useful
feature.

(0005458)
rhansen (manager)
2021-08-23 06:41
edited on: 2021-08-23 06:45

In response to Note: 0005456:
> Although easy to write, my opinion is that .WAIT is actually pretty hard to use in a reliable way due to this behavior. I consider it a kind of a hacky / quick-and-dirty shortcut that avoids implementing a robustly correct solution in your makefile.
OK, it seems like my understanding of how .WAIT works was wrong. If .WAIT implementations behave as you describe, then you have a very compelling point.

Note: 0005446 was written under the assumption that .WAIT created an order-only prerequisite relationship: "prerequisites that appear after it shall not be brought up-to-date until after all preceding prerequisites for the same target have been brought up-to-date." If existing implementations behave as you describe, then none of them would be conforming.

I'm now leaning against including a normative requirement for .WAIT, at least as it is currently written. (Something in "future directions" could still be useful.) If existing implementations change .WAIT so that it creates an ordered relationship in the dependency graph like GNU make's OO prerequisites feature, then I'd be more interested in adding .WAIT. And if that happened, then it would be easier to implement in GNU make, yes?


In response to Note: 0005457:
> I am not sure whether the gmake method with the pipe symbol is able to be usable as a complete replacement for features offered by .WAIT.
I now see that there isn't a 100% semantic match, but why wouldn't it be usable as a replacement for .WAIT? I agree with Paul; it seems stronger than .WAIT to me.
> BTW: gmake still has various other bugs that prevent me personally from being able to use gmake in parallel mode.
Please keep the discussion on-topic. Your bugnote derails the conversation, which makes it harder to properly address this issue.
> .WAIT is available in SunPro Make and BSD make since 30 years and that should be a verification for a useful feature.
This argument is disingenuous. First, there are lots of old features that aren't standardized. Second, GNU make is the 800-pound gorilla here, and it became the 800-pound gorilla despite not having .WAIT. It is disrespectful to GNU make's sizeable userbase to not take Paul's input seriously; if he doesn't see a compelling reason to add .WAIT then we should take a moment to really consider its inclusion.

Maybe the other make implementations should learn from GNU make's success and replace .WAIT with a clone of GNU make's OO prerequisites feature. (I'm kidding. Mostly.)

(0005459)
joerg (reporter)
2021-08-23 12:36

Re: Note: 0005448

There is a fundamental difference between

target: $(leftstuff) .WAIT $(rightstuff); do_thing


and

$(rightstuff): $(leftstuff)


The first one says "build leftstuff before rightstuff"

The second one says "build rightstuff from leftstuff"

That alone would make both variants non-equivalent.

But even worse, if you have:

a: l 
        echo a 
 
a b c: x y z 


This results amongst others in this rule:

a:    l x y z 
        echo a


which is what the standard requires but not what you expect in this case...
as you may have unexpected rule commands involved.

From a theoretical view, problems in a parallel build are missing
dependencies that would need to be defined. From practical experience
it is usually hard to find the missing dependency in special if the
dependency graph is large already. .WAIT on the other side delivers
the method that fits to the observed problems and does not create
rules, you do not want to have.

If you need more than one .WAIT in a dependency list, this could
be (if at all) only written as many dependencies that at some point
cannot be understood anymore and of course enhances the chance that
they are unexpectedly added as dependencies to existing rule commands
as in the example above.
(0005460)
psmith (developer)
2021-08-23 20:18
edited on: 2021-08-23 21:07

In response to Note: 0005458

My understanding of the text "prerequisites that appear after it shall not be brought up-to-date until after all preceding prerequisites for the same target have been brought up-to-date." in particular "for the same target", is that it's specifically intended to limit the effects of .WAIT to the prerequisites within a given target. If the same target appears as a prerequisite of a different target, then the .WAIT doesn't (necessarily) apply. However, I suppose this statement is open to interpretation and maybe this should be addressed in the standard, to reduce the chance of confusion.

In response to Note: 0005457

We have had this discussion before. If you do want to have it again we should restrict it to the mailing list (or better yet, the GNU make mailing lists) and not have it here. The statements are at least misleading. All I will say here is that, IF you state all prerequisites of your targets explicitly in your makefiles then GNU make will rebuild your targets 100% accurately 100% of the time, regardless of -j level. If you write your rules so that they have "undeclared side effects" that other targets rely on, then it's true that your targets may not build correctly in some situations in GNU make. I understand that your makefile systems rely heavily on this type of "undeclared side effect", so it is hard to port them to be usable by GNU make. I have described the changes you'd need to make to convert the "undeclared side effects" into declared explicit prerequisites and you have declined to make those changes in order to facilitate the portability, and instead decided to run make multiple times, which is perfectly fine.

I have agreed to make some changes to GNU make to reduce this issue, and those changes are in fact checked into the Git source code control for the next release of GNU make. These changes are not very important to the current userbase of GNU make, however, because GNU make makefiles generally DO explicitly declare all their prerequisites and so don't run into these problems.

In response to Note: 0005459

> From a theoretical view, problems in a parallel build are missing dependencies that would need to be defined. From practical experience it is usually hard to find the missing dependency in special if the dependency graph is large already.

I simply disagree with this. If you know what a target does, then it should be quite clear what its prerequisites are. I need to build a target FOO. I know what commands are needed to build FOO. I know what other targets in my makefile need to be available to run those commands. Therefore I know which targets need to be created before FOO's commands can be run. So I declare those targets as prerequisites of FOO. There's no magic or deep insight needed here.

Maybe if you are limited to .WAIT, where you have to actually modify the prerequisites list of a specific target and list things in a specific order, and especially you have to add the .WAIT to a rule (or all rules) that itself lists FOO as a prerequisite rather than declaring the relationship on FOO itself, it can be more difficult. However, with order-only prerequisites you can define the prerequisite relationship directly between FOO and whatever targets it requires, ANYWHERE in the makefile, just as you can define normal prerequisite relationships anywhere. For example if it's easier you can define the relationship next to the prerequisite, rather than next to the target.

(0005461)
joerg (reporter)
2021-08-23 20:39

Re: Note: 0005458

First, the arguments against .WAIT are just wrong. In contrary, the gmake
method is unreliable to use, see Note: 0005459 This is because it is hard to
use a feature reliably that is hard to understand in case that the
makefile, where it is used, already has a complex dependency graph. Most
makefiles from today match that criteria.

If a makefile fails in parallel mode, you know where it fails and thus
you know where to add a .WAIT, but you usually do not know how/where to add
more dependencies to satisfy make's wishes.

Regarding .WAIT in combination with -j, the effects to parallel execution
are the same as with the gnu only method. This is because the sum of all
make instances that run in a common make pool share a common job pool and
if one targes cannot start a new job for certain dependencies for a limited
time, then other targets in the same make instance or other targets in other
make instances in the same make pool will run other jobs from the available
slots in the job pool.

Finally, gmake definitely is not the gorilla that convinces because of it's
features but it is rather used because a lot of vendors only offer/advertize
gmake to their customers.

Gmake offers a lot of features that do not give added value and deviates
in it's behavior from other make implementations for features that did exist
long before gmake was initially written.

In case you did not get the background of the sum of all feature enhancements
to "make" in POSIX that I worked on in the past years, this is only in order
to avoid vendor lock in effects to specific make implementations and to foster
more commonly usable features.

If we (after adding .WAIT) add the simplified pattern rules I mentioned in
Note: 0005126 from 0000513 and BSD make would implement that, we would have
the minimum set of features that are needed for todays makefiles and SunPro
Make, smake, gmake, bmake would offer a sufficient common set of features to
finally come to the usual POSIX grant of portability for the make utility as
well.
(0005462)
rhansen (manager)
2021-08-23 20:58
edited on: 2021-08-23 21:10

Re: Note: 0005459:
> There is a fundamental difference between
target: $(leftstuff) .WAIT $(rightstuff); do_thing
and
$(rightstuff): $(leftstuff)
I know; Paul already explained the difference in Note: 0005450. In Note: 0005453 I was proposing adopting GNU make's order-only prerequisites, not regular prerequisites.

Re: Note: 0005460:
> However, I suppose this statement is open to interpretation and maybe this should be addressed in the standard, to reduce the chance of confusion.
Agreed. The intention (or at least *my* intention) was to establish an order-only prerequisite in the dep graph. Either way, the wording needs to be clarified, if it is to be kept at all.

Do you think there would be more interest in implementing .WAIT in GNU make if it did establish an order-only prerequisite? For example, the following:
$(a): $(b) .WAIT $(c) .WAIT $(d); do_thing
would be 100% equivalent to:
$(c): | $(b)
$(d): | $(c)
$(a): $(b) $(c) $(d); do_thing


(0005464)
shware_systems (reporter)
2021-08-24 04:03
edited on: 2021-08-24 04:05

Re: 5455

With:
rightstuff=foo bar baz
leftstuff=w x y z
$(rightstuff):$(leftstuff)
target: $(rightstuff) // missing line
// gets leftstuff up to date indirectly

is where you get the 12, as the rightstuff as prereq of target starts 3 parallel invokes, and these are put on hold while the 4 prereqs of each invoke are evaluated, all in parallel. Before the initial ones go on hold all it cares about is all the prereqs for its particular target are out of date, they have no way to determine that the other targets also have the same prereqs, as if they were on separate lines like you expanded.

(0005465)
psmith (developer)
2021-08-25 12:53

Re: 5464

I'm assuming here you meant to have the "target" rule come before the "$(rightstuff):$(leftstuff)" rule, or else invoke "make target".

The method you suggest is not how make works. Make creates a directed graph and walks the graph. It doesn't try to build any target (node in the graph) until AFTER all that target's prerequisites are completely created.

Here we have a graph with a root of "target" and 3 sub-nodes, "foo", "bar", and "baz". There are then 4 leaves, "w", "x", "y", and "z". Each of the three sub-nodes connect to each of the four leaves.

When make runs it walks the graph from whatever starting point is chosen using a depth-first, right-to-left ordering among siblings. So if we start at "target" then first it will build "w", then "x", then "y", then "z", then "foo" (because now foo's prerequisites are complete), then "bar" (because "bar"'s prerequisites were already built, as they are also prerequisites of "foo"), then "baz", and finally "target" (the root of the graph).

With -j the walk of the graph is identical as without -j, only instead of waiting for each node to complete before moving on to the next one we start N nodes (for -jN) in parallel before we are forced to wait (or, we wait when there are no nodes left that can be built until after already-in-process nodes are complete).
(0005466)
psmith (developer)
2021-08-25 14:19

Re Note: 5462

I don't think it will work to declare that equivalency, because I don't think these things are equivalent. The problem is that order-only prerequisites create an explicit "always present" relationship in the graph, and .WAIT does not. Suppose we have this:

    all: one .WAIT two
    one two: ; @echo $@

versus this:

    all: one two
    two: | one
    one two: ; @echo $@

What happens if you run "make two"? In GNU make you have declared a dependency relationship here, so make will first build "one" (if needed) and then it will decide to build "two" or not, without considering whether "one" was updated (the relationship is "order-only" so it doesn't impact the "out of date" decision).

In the .WAIT version, only "two" will be built. "one" will not be considered, because "one" is not a dependency of "two".

However, I've downloaded FreeBSD make (the only make I could easily install that supports both -j and .WAIT) and it appears the behavior of .WAIT is different than I thought. I don't know whether this behavior is specific to the FreeBSD version or not.

Let's consider this makefile:

    all: foo bar

    foo: one .WAIT two
    bar: one two

    foo bar one two: ; @echo $@; sleep 1

Suppose we invoke this makefile with -j10 so that the -j value is not a limiting factor. What do we expect to build in parallel, and where will we wait? My original assumption was that the .WAIT is a feature of the prerequisite list of the target, so that we would try to build "foo", which would build "one", then it would not build "two" here because "one" was still running. Then make would try to build "bar", and see that "one" and "two" could be built, and since there's no .WAIT here it would build "two" immediately. So I assumed that the output would be this:

    one
    two
    bar
    <pause>
    foo

However, that's not what FreeBSD make does. It seems that this .WAIT does indeed create some kind of globally visible relationship between "one" and "two", so what I see is:

    one
    <pause>
    two
    <pause>
    foo
    bar

This relationship persists even if we don't build "foo" at all! If I run "make bar" here we still get:

    one
    <pause>
    two
    <pause>
    bar

The implementation has some strange subtleties. If I change the graph so that "two" appears before "one" in the graph, FreeBSD make still runs "one" first and waits for it:

    all: bar foo
    bar: two
    foo: one .WAIT two
    one two foo bar: ; @echo $@; sleep 1

In this graph, make will consider targets in the order "two", "bar", "one", "foo". But running this gives:

    one
    <pause>
    two
    <pause>
    bar
    foo

So, somehow FreeBSD make realizes, when it wants to build "two", that sometime in the future it will also want to build "one" and that "one" should be completed before "two" starts.

This is very mysterious to me. Certainly implementing something like that in GNU make will be very difficult, because GNU make doesn't "look ahead" while walking the graph. It walks the graph in order and simply goes from beginning to end; at the time that make wants to build "two" it doesn't know whether it will want to build "one" or not.

And of course note that in GNU make, where we can have long chains of implicit rules which are not resolved until runtime, the full shape of the graph can't even be known until we actually try to resolve the implicit rules. That "one" dependency might be found through a long chain of implicit rules, and not be known until all that work is done.

However maybe I'm missing something. In any event it's clear to me that the current wording proposed here certainly does not capture the actual behavior of FreeBSD make's .WAIT with anything close to specificity.

I think that if we want to add .WAIT to the standard we will have to make it much LESS specific, so that all the subtleties are not actually required by POSIX and instead we create the simplest possible behavior.
(0005468)
joerg (reporter)
2021-08-25 15:34
edited on: 2021-08-25 15:49

Re: Note: 0005466

Please note that SunPro Make is easier to compile/install than BSD make.

Just fetch a recent schilytools from

http://sourceforge.net/projects/schilytools/files/ [^]

today, the most recent one is:

https://sourceforge.net/projects/schilytools/files/schily-2021-08-14.tar.bz2 [^]

unpack that archive chdir into the tree and call "make", then "make install"

Then call either /opt/schily/bin/make or /opt/schily/bin/dmake

"make" is the variant that is installed in the base OS and "dmake"
is the variant that by default runs in parallel mode.

BSD make is not fully POSIX compliant, SunPro Make of course is, as it has
been certified with Solaris.


BTW: where did you get a compilable FreeBSD make from?

I am not aware of such a source....

I verified the makefile from your example and both SunPro Make
and bmake (The portable version from NetBSD make) behave as you
expect. They print one and two with no delay and then wait,
before printing bar and foo.

(0005469)
psmith (developer)
2021-08-25 17:52

I installed the pre-built freebsd-buildutils package onto my GNU/Linux system. It contains an "fmake" program.

I installed the schily version. It didn't quite build so easily, but I was able to build it. However, there is no "dmake" or "make" in the bin directory (or anywhere else in /opt/schily) after I run install. I was able to find an uninstalled command ./sunpro/Make/bin/make/common/OBJ/x86_64-linux-gcc/make in the build directory and that one does support .WAIT (unlike the "smake" program). Although the output is a little bit weird (to me), I can see that it does indeed behave the way I expected in my note 5466, and not the way FreeBSD make works.
(0005470)
joerg (reporter)
2021-08-25 18:03
edited on: 2021-08-25 18:09

Well you did probably not run "make install" as root.

If your Linux system contains unknown modifications that prevent
compilation of single programs, I recommand to use "make -i" and
to make a bug report.

This is an actively maintained project that responds to bug reports.

smake currently does not support parallel compilation and since .WAIT
was added to POSIX recently, there is no support yet. This is why the
schily makefile system uses $(WAIT) that may be empty.

BTW: SunPro Make always collects the output from rule commands and
prints serialized output, so output of course looks different from
simple make implementaions.

(0005471)
psmith (developer)
2021-08-26 13:09

I never install software I download and build myself as root. I have a separate user account I use for all that.

The problem I ran into is that I built with GNU make, then during it build it attempted to invoke the make that was built (I suppose smake), but there were GNU make options in MAKEFLAGS that weren't recognized by the make that was built and it failed. However that was my own fault because I have some MAKEFLAGS options I set in my environment before I invoke make.

Re Note: 5461

We'll just have to disagree about which feature is easier to use. Nothing said here convinces me that .WAIT is easy to use: on the contrary to me it seems very difficult to use RELIABLY, EXCEPT perhaps in the very limited context that the BSD build system uses it in (where it's used to mix in pause points into prerequisite lists contained in variables like SUBDIRS).

> In contrary, the gmake method is unreliable to use, see Note: 0005459

There are no examples in that note showing that OO prereqs are hard to use. In fact, there are no OO prereqs mentioned in that note at all.

> This is because it is hard to use a feature reliably that is hard to understand in case that the
makefile, where it is used, already has a complex dependency graph.

There is nothing about OO prereqs that make them hard to understand in complex dependency graphs. On the contrary, since OO prereqs create an actual edge in the graph while .WAIT does not, the behavior of OO prereqs are easier to understand.

> If a makefile fails in parallel mode, you know where it fails and thus you know where to add a .WAIT, but you usually do not know how/where to add more dependencies to satisfy make's wishes.

This is exactly the reverse of reality.

When a target fails, you know what target failed and you know why it failed (which prerequisite did not exist that should have existed).

With GNU make, you merely add this to your makefile:

    target: | prereq

and now you've added a new edge to your graph that ensures that prereq will be complete before the target is invoked. Note that unlike normal prerequisites, order-only prerequisites DO say only "build prereq BEFORE target" and they do NOT say "build prereq FROM target". That's what makes them different from "normal" prerequisites. But unlike .WAIT, this relationship is defined as an edge in the graph between the target and prerequisite so it always exists, regardless of where target is used in the makefile.

I'm not exactly sure what you mean by "where to add more dependencies"; like any other dependency relationship it can be added ANYWHERE in the makefile.

On the other hand, to solve this issue with .WAIT you must search all your makefiles and envision the entire dependency graph so you can be sure that you have found all the other dependency lists that contain "target", and which can be reached from a place where "prereq" might have been invoked first, so you can add a .WAIT somewhere between "target" and "prereq". This is because .WAIT is not associated with target and prereq, and it doesn't apply to other places where these targets might exist.

You can see this issue even with my very simple example above: I didn't add the .WAIT between the "one" and "two" targets when they were prerequisites of "bar", and so they were built in parallel. With .WAIT I have to search the entire graph looking for these types of relationships (which, unlike my example, could be very far apart in the graph and hard to see in the makefiles).

Alternatively, you could try to simplify things by just searching all makefiles to find all places where "target" appears as a prerequisite and change them all to ".WAIT target", which would solve the problem but could introduce many more wait points than are needed or optimal.

And, of course, when we consider things like pattern rules etc. it might not always be obvious at all that a given prerequisite list will have "target" as a prerequisite.
(0005472)
joerg (reporter)
2021-08-26 19:23
edited on: 2021-08-26 19:24

Re: Note: 0005466

I did test your makefiles on a FreeBSD instance using "make".

I canot repeat the problem you describe, however both makefiles are written
in a way that makes them instable as their behavior depends on chance when
in parallel mode. On FreeBSD, I see a comparable behavior with SunPro Make
and BSD make.

Re: Note: 0005471

When you do not use the official makefiles of a trustworthy project, it
is up to you, to do the right things. This is something you cannot blame
the project for. Even the Linux distros, that support binary packages
based on schilytools use "make install" (see README.install) to create
the prototype trees for their packages.

If you are using a private environment MAKEFLAGS, you are doing something
that is being warned for. If your MAKEFLAGS environment even contains
non-standard content, you should never expect that to work.

Smake implements workarounds that deal with the non-POSIX structure from
gmake's MAKEFLAGS, but it does not support to work around non-standard
gmake options.

And BTW: you are right, if you call "make" at top level, this first
compiles a bootstrap smake and then continues with that compiled smake
in order to be able to compile schilytools with make implementations
that do not support the minimum feature set for the makefile system.

(0005473)
rhansen (manager)
2021-08-26 20:22

There was some spirited discussion about this bug during today's telecon. For now, we're going to wait to see if Steffen (or anyone else) can assemble a rough proof of concept patch for GNU make that takes an approach acceptable to Paul. If so, then it seems much more likely to us that GNU make would eventually support .WAIT. That would alleviate concerns about indefinite non-conformance by a major implementation.
(0005474)
psmith (developer)
2021-08-26 20:40
edited on: 2021-08-26 20:52

I'm not sure which "problem" you refer to for FreeBSD make. Are you saying your FreeBSD make doesn't put the "pauses" in the same place as mine?

Let's be very clear. If I have this makefile:

   all: bar foo
   bar: two one
   foo: one .WAIT two
   one two foo bar: ; @echo $@; sleep 10

where I've increased the sleep time to be more obvious even on slow or busy systems, and I run FreeBSD make via "time", I get this:

   $ time fmake -j10
   one
     <pause>
   two
     <pause>
   bar
   foo
     <pause>

   real 0m30.018s
   user 0m0.019s
   sys 0m0.005s

It takes 30 seconds to run this makefile.

If I do the same with the SunPro make from schily, I get this:

   $ time ./schily/schily-2021-08-14/sunpro/Make/bin/make/common/OBJ/x86_64-linux-gcc/make -j10
   make: Warning: Can't find `make.rules': No such file or directory
   dmake: defaulting to parallel mode.
   local --> 1 job
   local --> 2 jobs
     <pause>
   local --> Job output
   two
   local --> Job output
   one
   local --> 1 job
   local --> 2 jobs
     <pause>
   local --> Job output
   bar
   local --> Job output
   foo

   real 0m20.018s
   user 0m0.019s
   sys 0m0.008s

and the total time taken is 20 seconds.

Are you saying you get different behavior from your version of FreeBSD make? That it also takes 20 seconds, not 30 seconds?

I don't care about the _order_ in which the output appears, that's irrelevant. The important thing for this experiment is how many pauses there are: what is run in parallel and what is run serially.

> If your MAKEFLAGS environment even contains non-standard content, you should never expect that to work.

Uh, and...?? I did say, "However that was my own fault because I have some MAKEFLAGS options ..."

(0005475)
psmith (developer)
2021-08-26 20:50

My main concern at this point is that the text added to the standard be very clear about what is required by the standard, and what is left unspecified. See my example above and how the two different make versions that already support .WAIT behave.

I assume that the text means to say that .WAIT only must take effect between two sets of dependencies when they are processed as prerequisites of the specific target where the .WAIT appears in a prerequisite list. If those same two targets appear as dependencies of some other target where .WAIT is not present, then the standard makes no statement about whether they should be ordered or not.

Let me offer up a potential implementation and you can say whether it's intended to be allowed by the standard:

Suppose a version of make is created where, when it sees a .WAIT in a prerequisit list, it pauses and waits for ALL currently-running jobs to complete, then keeps going.

Ignoring quality of implementation issues, would such an implementation be POSIX conforming under the text being proposed?
(0005476)
rhansen (manager)
2021-08-26 21:00
edited on: 2021-08-26 21:08

Re: Note: 0005474
> Are you saying you get different behavior from your version of FreeBSD make? That it also takes 20 seconds, not 30 seconds?
Using the Debian bmake-20181221-2 package (supposedly copied from NetBSD):

$ time bmake -f - -j10 <<\EOF
all: bar foo
bar: two one
foo: one .WAIT two
one two foo bar: ; @echo $@; sleep 10
EOF
--- two ---
--- one ---
--- two ---
two
--- one ---
one
            # pauses here for 10s
--- bar ---
--- foo ---
--- bar ---
bar
--- foo ---
foo
            # pauses here for 10s

real    0m20.032s
user    0m0.018s
sys     0m0.010s
Maybe this is a bug in NetBSD's make?

Re: Note: 0005466
> [...] order-only prerequisites create an explicit "always present" relationship in the graph
Once again my understanding of how implementations actually behave is upended. :)

I was under the impression that "|" only affected when the stuff on the right was built, not whether it was built. For example, I had assumed that the following would not build 'one':
$ make -f - -j10 two <<\EOF
two: | one
one two: ; @echo $@; sleep 2; echo done sleeping $@
EOF
Given that GNU make does build 'one', the name "order-only prerequisite" seems like a misnomer. Maybe "existence-only prerequisite" would be a better name.

If GNU make did have a true order-only prerequisite feature, then I think that .WAIT could be implemented as syntactic sugar. The resulting behavior would be more conservative than BSD's .WAIT behavior (less parallelism), but that's OK.

Re: Note: 0005475
> My main concern at this point is that the text added to the standard be very clear about what is required by the standard, and what is left unspecified.
Agreed. I think everyone understands that the wording in Note: 0005446 is problematic. Coming up with precise and correct wording might be tricky, but now that we collectively understand the problem better I am confident that we can come up with something suitable.

> [...] would such an implementation be POSIX conforming under the text being proposed?
Yes, POSIX should allow that behavior. Implementations should always be permitted to have less concurrency than is technically possible. For example, POSIX should allow an implementation to dynamically reduce parallelism if the current system load is too high.

(0005477)
joerg (reporter)
2021-08-26 21:12
edited on: 2021-08-26 21:17

Re: Note: 0005474

Running the FreeBSD make on FreeBSD with your makefile indeed results
in a total time of 20 seconds for the make run.

This is the FreeBSD-13 beta from September 24 2020.

--- two ---
--- one ---
--- two ---
two
--- one ---
one
        delay...
--- bar ---
--- foo ---
--- bar ---
bar
--- foo ---
foo
        delay...

Since even the output looks different from yours, you seem to use a
different make implementation.

P.S. The results have been verified to be identical to the results
fetched with a FreeBSD installation from April 2021.

(0005478)
psmith (developer)
2021-08-28 12:42

RE Note: 5476

> Given that GNU make does build 'one', the name "order-only prerequisite" seems like a misnomer. Maybe "existence-only prerequisite" would be a better name.

It's possible I've been living with it too long (order-only prerequisites have existed in GNU make for almost 20 years, having been added in GNU make 3.80 in 2002) but I don't agree. The term "prerequisite" to me means that it must be brought up to date before "target" can be built. The "order-only" modifier means that only the order of targets, but not the out-of-date computation, is relevant for that edge in the graph.

If a feature were created that told make that "targetB" should be built before "targetA" but if and only if "targetB" needed to be built for some other reason, that shouldn't be called a prerequisite at all IMO. You would need some other term that didn't contain "prerequisite".

Such a feature wouldn't be so easy to create, but could be done. It would need to have some kind of "undefined state" edge in the graph which, when seen, would mean that the target node could not be built until that state was no longer undefined. The state would change to defined when either (a) the prerequisite node were considered (for some other reason than this undefined edge), or else (b) the entire graph has already been walked: at that time all "undefined" edges would revert to "considered with no changes" and we would have to build all the target nodes that could not previously be considered due to undefined states.

Or, something like that.
(0005479)
joerg (reporter)
2021-08-28 15:15

If that gmake feature has been added late, I did expect gmake to
see beyond the edge of the plate and to check what other make
implementations did offer as solution already instead of introducing
something incompatible new.

UNIX was a success story because implementors from vendor A did look
at implementations from vendor B and reimplemented goot ideas. In the
1980s a good idea from another vendor typically did take 2-4 years to
appear everywhere. With that background, .WAIT should have been implemented
in gmake no later than 1995.
(0005480)
psmith (developer)
2021-08-28 17:21

As I hope the discussion here makes obvious, .WAIT is not the same thing as an order-only prerequisite. As far as I'm aware there isn't anything similar to OO prerequisites in other versions of make so GNU make isn't implementing this feature in some incompatible way to other versions of make that already supported it.

The question of whether or not GNU make supports .WAIT for compatibility with other versions of make is a completely separate issue.
(0005481)
steffen (reporter)
2021-08-28 19:21

I added a Public Domain diff to make(1) that may be used freely shall it can be applied to the source of your program.

It implements a true .WAIT that ensures the chosen ordering of prerequisites is stable across the entire makefile. For example, for the example above:

<code>
#?0|kent:tmp$ cat y/makefile
all: bar foo
bar: two one
foo: one .WAIT two
one two foo bar: ; @echo $@; sleep 3
#?0|kent:tmp$ ~/src/.gmake.git/make -j4 -f y/makefile
one
two
bar
foo
</code>

or for the more sophisticated

<code>
#?0|kent:z$ cat makefile
all: lib ham .WAIT bin bin2 ;@echo all command
clean: ; rm -f bin/.stamp lib/.stamp
lib: lib/.stamp; @echo lib command
ham: ;@echo ham command
lib/.stamp: ;cd lib && $(MAKE)
bin: bin/.stamp; @echo bin command
bin/.stamp: ;cd bin && $(MAKE)
bin2: ; @echo bin2 command
#?0|kent:z$ ~/src/.gmake.git/make -j4
ham command
cd lib && /home/steffen/src/.gmake.git/make
make[1]: Entering directory '/tmp/z/lib'
lib-x command

lib-x after sleep
lib/all command
make[1]: Leaving directory '/tmp/z/lib'
lib command
cd bin && /home/steffen/src/.gmake.git/make
bin2 command
make[1]: Entering directory '/tmp/z/bin'
bin-x command
bin/all command
make[1]: Leaving directory '/tmp/z/bin'
bin command
all command
</code>

It may be up to the core developers to leave the exact meaning of .WAIT an implementation issue, but i think this way of doing things -- extending a chosen barrier to be global -- is right.

I want to note that this likely does not cause issues in practice, since Makefiles yet making use of .WAIT are likely ordered in a way that would avoid the race condition which does occur if make(1) does not take care for ensuring barrier validity gracefully.
Since most use cases of .WAIT are in the BSD make system this is a non-issue, since there so-called "singleton"s (thanks Paul Smith) are used, and .WAIT thus occurs naturally only to guarantee correct ordering in this singleton.

P.S.: survived -fsanitize=address testing.
(0005482)
joerg (reporter)
2021-08-29 12:37

With the proposal from Steffen, we would need to add a hint to the
standard that a makefile like this:

all: bar foo
bar: two one
foo: one .WAIT two
one two foo bar: ; @echo $@; sleep 3


is non-conforming, since the target "bar" and it's prerequisite list is in
conflict with the definitions for target "foo".

Steffens code causes an implicit dependency and even though this may
result in working code while using gmake, it will not work with
SunPro Make and BSD Make in case that "all" is used as the main target
to build. This is because SunPro Make and BSD make evaluate .WAIT at run
time, based only on the list of dependencies for the current target.
(0005484)
rhansen (manager)
2021-09-02 06:14
edited on: 2021-09-02 06:17

Re: Note: 0005478
> You would need some other term that didn't contain "prerequisite".
Good point.

Re: Note: 0005482
> [...] the target "bar" and it's prerequisite list is in conflict with the definitions for target "foo".
How so? It looks OK to me.


Some experiments with Steffen's patch with the following makefile:
all: bar foo
bar: two one
foo: one .WAIT two
one two foo bar: ; @echo $@; sleep 2
Building 'all' works as expected:
$ time ./install/bin/make -j10 -f - <<\EOF
all: bar foo
bar: two one
foo: one .WAIT two
one two foo bar: ; @echo $@; sleep 2 
EOF
one
        <2s pause here>
two
        <2s pause here>
bar
foo
        <2s pause here>

real    0m6.021s
user    0m0.019s
sys     0m0.004s
Building just 'bar' causes 'one' to be built before 'two' even though the BSD makes build 'one' and 'two' in parallel. I personally prefer this behavior over the BSD make behavior, so I don't consider it to be a bug. Output:
$ time ./install/bin/make -j10 -f - bar <<\EOF
all: bar foo
bar: two one
foo: one .WAIT two
one two foo bar: ; @echo $@; sleep 2 
EOF
one
        <2s pause here>
two
        <2s pause here>
bar
        <2s pause here>

real    0m6.028s
user    0m0.012s
sys     0m0.016s
Building just 'two' causes 'one' to be built. This feels like a bug to me, and it doesn't match the behavior of BSD makes. Output:
$ time ./install/bin/make -j10 -f - two <<\EOF
all: bar foo
bar: two one
foo: one .WAIT two
one two foo bar: ; @echo $@; sleep 2
EOF
one
        <2s pause here>
two
        <2s pause here>

real    0m4.022s
user    0m0.014s
sys     0m0.008s
Based on the above, it looks like the patch effectively converts .WAIT into order-only prerequisites.

(0005485)
psmith (developer)
2021-09-02 12:23

Re Note 5482:

I don't think I'd like to say that a makefile like that is non-conforming. Rather I think we should say that .WAIT is only guaranteed to create a wait point in the prerequisites while building the target in which the .WAIT appears. It's undefined (by the standard) whether it will add a wait point between these prerequisites if they appear in other rules.

Maybe that amounts to the same thing, more or less.
(0005486)
joerg (reporter)
2021-09-02 13:22
edited on: 2021-09-02 13:25

Re: Note: 0005484

Your makefile (using dmake == SunPro Make) ==>

mostly starts with two on Solaris

mostly starts with one on FreeBSD on a RaspPi

more or less alternatingly starts with one or the other on Linux


This is because the "all: bar foo" dependency has no order and bar
has no order as well. Another important reason for that behavior
is the different fork() timing on the different operating systems.

I believe it is important that if we allow implementations that use a
prerequisite-like implementaion, the standard explicitely mentions that
in such a makefile, the order is undefined.

(0005487)
shware_systems (reporter)
2021-09-02 15:11
edited on: 2021-09-02 15:26

Re: 5486

This applies to non-parallel builds too, that I see, there's no requirement prereqs are evaluated left to right, just commands top to bottom. An order can be specified for both in the standard as to when they start, but a parallel build still has no control over when a prereq finishes updating.

(0005488)
psmith (developer)
2021-09-02 15:30

Re Note 5487:

No, you're not correct here. The standard explicitly says:

> The make utility shall treat all prerequisites as targets themselves and recursively ensure that they are up-to-date, processing them in the order in which they appear in the rule.

Prerequisites MUST be considered in the order in which they are listed in the makefile.

Also, I'm not sure I understood your second sentence but note make orders things by when jobs COMPLETE, not by when they start.
(0005489)
rhansen (manager)
2021-09-02 16:28
edited on: 2021-09-02 16:32

Here's a new attempt:

On page 2975, insert before line 98708:
.NOTPARALLEL
The application shall ensure that this special target is specified without prerequisites or commands. When specified, make shall update one target at a time, regardless of whether the -j maxjobs option is specified. If the -j maxjobs option is specified, the option shall continue to be passed unchanged to sub-make invocations via MAKEFLAGS.

On page 2975, insert before line 98742:
.WAIT
The application shall ensure that this special target, if specified as a target, is specified without prerequisites or commands. When .WAIT appears as a target, it shall have no effect. When .WAIT appears in a target rule as a prerequisite, it shall not itself be treated as a prerequisite; however, make shall not recursively process the prerequisites to the right of the .WAIT until the prerequisites to the left of it have been brought up-to-date. Implementations may also enforce the same ordering between the affected prerequisites while processing other target rules that have some or all of the same affected prerequisites.

On page 2975, change line 98742 from:
The special targets .IGNORE, .POSIX, .PRECIOUS, .SILENT, and .SUFFIXES shall be specified without commands.
to:
The special targets .IGNORE, .NOTPARALLEL, .POSIX, .PRECIOUS, .SILENT, .SUFFIXES and .WAIT shall be specified without commands.

On page 2975 lines 98744-98747, change:
Targets with names consisting of a leading <period> followed by the uppercase letters "POSIX" and then any other characters are reserved for future standardization. Targets with names consisting of a leading <period> followed by one or more uppercase letters are reserved for implementation extensions.
to:
Targets and prerequisites consisting of a leading <period> followed by the uppercase letters "POSIX" and then any other characters are reserved for future standardization. Targets and prerequisites consisting of a leading <period> followed by one or more uppercase letters, that are not described above, are reserved for implementation extensions.

On page 2985 after line 99167 (make EXAMPLES) append a new example:
7. With the following makefile, <tt>make -j 10 all</tt> may bring <tt>one</tt> and <tt>two</tt> up-to-date in parallel despite the .WAIT in the prerequisite list for <tt>foo</tt>. This is because the .WAIT does not stop make from recursively processing <tt>bar</tt> and its prerequisites in parallel. However, if only <tt>foo</tt> is specified (<tt>make -j 10 foo</tt>), make will wait for <tt>one</tt> to be brought up-to-date before bringing <tt>two</tt> up-to-date. Note also that the .WAIT does not create a prerequisite relationship between <tt>one</tt> and <tt>two</tt>, so <tt>make -j 10 two</tt> will not build <tt>one</tt>.
all: foo bar
foo: one .WAIT two
bar: one two
foo bar one two: ; @echo $@


(0005490)
shware_systems (reporter)
2021-09-02 16:36
edited on: 2021-09-02 16:43

Re 5488:

Yes, that was pointed out during today's phone call; I was looking at the Target Rule text more, skipped last part of that sentence.

make has no control over, once it establishes what prereqs of a target are not up-to-date, how long the commands specified to bring them up-to-date will need to execute. An early prereq may be a compile that takes 10 minutes, the last one may simply be a touch that takes milliseconds. Processed as parallel targets that last prereq will finish first is what I was getting at.

(0005492)
steffen (reporter)
2021-09-02 23:25

Re: 5484

Well ok, then this is even easier. The patch easier.diff should do just that.

With my example it still prints "lib command" before stepping over to "bin" (different to unpatched make), but the "two one" example happens in that order (but when used for "foo", but then race free).
It surely could be optimized, and it likely is possible to extend this to global level nonetheless (with some effort).

Re 5489:

That example is false, then, it should be "bar: two one"

Ciao!
(0005493)
steffen (reporter)
2021-09-03 13:37

P.S.: i added yet another public domain file, gmake-4.3.wait.diff, which can be patch(1)ed onto the currently released version of Paul Smith's version of make without conflicts.
It is not so easy to build and use development versions of software in today's software world, unfortunately.
It is identical to easier.diff but for removal of useless cruft hunks.

(Just flags targets which should be waited for, then later recurses into these to also mark all their dependencies like that, so that in effect they are waited for. Works for me!?! Ciao.)
(0005494)
psmith (developer)
2021-09-03 14:48

Re Note 5489: That text seems acceptable to me. I can't say for sure when this feature will be made available in GNU make however.

- Issue History
Date Modified Username Field Change
2020-12-15 21:02 steffen New Issue
2020-12-15 21:02 steffen Name => steffen
2020-12-15 21:02 steffen Section => Vol. 3: Shell and Utilities, Issue 7, make
2020-12-15 21:02 steffen Page Number => (2975) 2989
2020-12-15 21:02 steffen Line Number => (98708, 98742) 99308
2020-12-17 17:10 psmith Note Added: 0005184
2020-12-17 19:23 steffen Note Added: 0005185
2020-12-17 21:16 psmith Note Added: 0005186
2021-08-14 12:19 joerg Note Added: 0005444
2021-08-19 16:22 rhansen Note Added: 0005446
2021-08-19 16:22 rhansen Note Edited: 0005446
2021-08-19 16:23 rhansen Interp Status => ---
2021-08-19 16:23 rhansen Final Accepted Text => Note: 0005446
2021-08-19 16:23 rhansen Status New => Resolved
2021-08-19 16:23 rhansen Resolution Open => Accepted As Marked
2021-08-19 16:24 rhansen Tag Attached: issue8
2021-08-19 16:59 rhansen Relationship added related to 0001436
2021-08-19 18:39 psmith Note Added: 0005447
2021-08-19 18:55 rhansen Note Added: 0005448
2021-08-19 19:00 rhansen Summary make: (document .NOTPARALLEL and .WAIT special targets) in RATIONAL => make: (document .NOTPARALLEL and .WAIT special targets) in RATIONALE
2021-08-19 19:03 rhansen Note Edited: 0005448
2021-08-19 19:04 rhansen Note Edited: 0005448
2021-08-19 19:04 rhansen Note Edited: 0005448
2021-08-19 19:27 shware_systems Note Added: 0005449
2021-08-19 22:47 psmith Note Added: 0005450
2021-08-19 23:09 shware_systems Note Added: 0005451
2021-08-20 18:38 psmith Note Added: 0005452
2021-08-20 19:28 rhansen Note Added: 0005453
2021-08-20 19:30 rhansen Note Edited: 0005453
2021-08-20 19:39 shware_systems Note Added: 0005454
2021-08-20 19:43 rhansen Note Edited: 0005453
2021-08-20 20:20 psmith Note Added: 0005455
2021-08-20 20:57 psmith Note Added: 0005456
2021-08-21 17:06 joerg Note Added: 0005457
2021-08-21 17:07 joerg Note Edited: 0005457
2021-08-21 17:08 joerg Note Edited: 0005457
2021-08-21 23:12 joerg Note Edited: 0005457
2021-08-23 06:41 rhansen Note Added: 0005458
2021-08-23 06:42 rhansen Note Edited: 0005458
2021-08-23 06:45 rhansen Note Edited: 0005458
2021-08-23 06:51 rhansen Status Resolved => New
2021-08-23 06:51 rhansen Resolution Accepted As Marked => Open
2021-08-23 12:36 joerg Note Added: 0005459
2021-08-23 20:18 psmith Note Added: 0005460
2021-08-23 20:39 joerg Note Added: 0005461
2021-08-23 20:58 rhansen Note Added: 0005462
2021-08-23 20:58 rhansen Note Edited: 0005462
2021-08-23 21:02 rhansen Note Edited: 0005462
2021-08-23 21:06 rhansen Note Edited: 0005462
2021-08-23 21:07 rhansen Note Edited: 0005460
2021-08-23 21:08 rhansen Note Edited: 0005462
2021-08-23 21:08 rhansen Note Edited: 0005462
2021-08-23 21:10 rhansen Note Edited: 0005462
2021-08-24 04:03 shware_systems Note Added: 0005464
2021-08-24 04:05 shware_systems Note Edited: 0005464
2021-08-25 12:53 psmith Note Added: 0005465
2021-08-25 14:19 psmith Note Added: 0005466
2021-08-25 14:19 psmith Note Added: 0005467
2021-08-25 14:19 psmith Note Deleted: 0005467
2021-08-25 15:34 joerg Note Added: 0005468
2021-08-25 15:35 joerg Note Edited: 0005468
2021-08-25 15:42 joerg Note Edited: 0005468
2021-08-25 15:46 joerg Note Edited: 0005468
2021-08-25 15:47 joerg Note Edited: 0005468
2021-08-25 15:47 joerg Note Edited: 0005468
2021-08-25 15:49 joerg Note Edited: 0005468
2021-08-25 17:52 psmith Note Added: 0005469
2021-08-25 18:03 joerg Note Added: 0005470
2021-08-25 18:05 joerg Note Edited: 0005470
2021-08-25 18:09 joerg Note Edited: 0005470
2021-08-26 13:09 psmith Note Added: 0005471
2021-08-26 19:23 joerg Note Added: 0005472
2021-08-26 19:24 joerg Note Edited: 0005472
2021-08-26 20:22 rhansen Note Added: 0005473
2021-08-26 20:40 psmith Note Added: 0005474
2021-08-26 20:50 psmith Note Added: 0005475
2021-08-26 20:52 psmith Note Edited: 0005474
2021-08-26 21:00 rhansen Note Added: 0005476
2021-08-26 21:01 rhansen Note Edited: 0005476
2021-08-26 21:01 rhansen Note Edited: 0005476
2021-08-26 21:08 rhansen Note Edited: 0005476
2021-08-26 21:12 joerg Note Added: 0005477
2021-08-26 21:16 joerg Note Edited: 0005477
2021-08-26 21:17 joerg Note Edited: 0005477
2021-08-28 12:42 psmith Note Added: 0005478
2021-08-28 15:15 joerg Note Added: 0005479
2021-08-28 17:21 psmith Note Added: 0005480
2021-08-28 19:11 steffen File Added: make.diff
2021-08-28 19:21 steffen Note Added: 0005481
2021-08-29 12:37 joerg Note Added: 0005482
2021-09-02 06:14 rhansen Note Added: 0005484
2021-09-02 06:15 rhansen Note Edited: 0005484
2021-09-02 06:15 rhansen Note Edited: 0005484
2021-09-02 06:17 rhansen Note Edited: 0005484
2021-09-02 12:23 psmith Note Added: 0005485
2021-09-02 13:22 joerg Note Added: 0005486
2021-09-02 13:22 joerg Note Edited: 0005486
2021-09-02 13:24 joerg Note Edited: 0005486
2021-09-02 13:25 joerg Note Edited: 0005486
2021-09-02 15:11 shware_systems Note Added: 0005487
2021-09-02 15:26 shware_systems Note Edited: 0005487
2021-09-02 15:26 shware_systems Note Edited: 0005487
2021-09-02 15:26 shware_systems Note Edited: 0005487
2021-09-02 15:26 shware_systems Note Edited: 0005487
2021-09-02 15:30 psmith Note Added: 0005488
2021-09-02 16:28 rhansen Note Added: 0005489
2021-09-02 16:29 rhansen Note Edited: 0005489
2021-09-02 16:31 rhansen Note Edited: 0005489
2021-09-02 16:32 rhansen Note Edited: 0005489
2021-09-02 16:36 shware_systems Note Added: 0005490
2021-09-02 16:36 shware_systems Note Added: 0005491
2021-09-02 16:36 shware_systems Note Deleted: 0005491
2021-09-02 16:37 rhansen Final Accepted Text Note: 0005446 =>
2021-09-02 16:43 Don Cragun Note Edited: 0005490
2021-09-02 23:20 steffen File Added: easier.diff
2021-09-02 23:25 steffen Note Added: 0005492
2021-09-03 13:33 steffen File Added: gmake-4.3.wait.diff
2021-09-03 13:37 steffen Note Added: 0005493
2021-09-03 14:48 psmith Note Added: 0005494
2021-09-09 15:13 Don Cragun Final Accepted Text => See Note: 0005489.
2021-09-09 15:13 Don Cragun Status New => Resolved
2021-09-09 15:13 Don Cragun Resolution Open => Accepted As Marked
2021-11-26 15:03 geoffclare Status Resolved => Applied


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker