Austin Group Defect Tracker

Aardvark Mark IV


Viewing Issue Simple Details Jump to Notes ] Issue History ] Print ]
ID Category Severity Type Date Submitted Last Update
0000513 [1003.1(2008)/Issue 7] Shell and Utilities Objection Enhancement Request 2011-11-17 00:02 2022-09-12 15:08
Reporter dwheeler View Status public  
Assigned To ajosey
Priority normal Resolution Open  
Status Under Review  
Name David A. Wheeler
Organization
User Reference
Section make
Page Number 2916
Line Number 95850
Interp Status ---
Final Accepted Text
Summary 0000513: Add pattern rules (metarules) to make
Description Pattern rules (aka "metarules") are implemented in several makes, including GNU make and Oracle/Sun make (http://download.oracle.com/docs/cd/E23824_01/html/821-1461/make-1s.html). [^] They were previously considered by the committee (lines 96347-96352). It was not added at that time, but as noted, the rules for target names have been set to reserve the '%' character, and that "Implementors are strongly encouraged to use these characters only for these purposes". This text sets up a situation where pattern rules can be added at some future time.

Here is an example of a pattern rule:
lib/%.o: lib/%.c
        $(CC) -fPIC -c $(CFLAGS) $(CPPFLAGS) $< -o $@
(Note: For this example to work, we also need to allow $< to be used in commands.)

I believe now is the time to add pattern rules to standard make:
* They are more flexible. In particular, they can properly handle prefixes (including directory prefixes) and filenames with multiple suffixes (like ".tar.gz"); basic inference rules cannot.
* They can support automatic chaining.
* They are clearer; using "%.o: %.c" to represent ".o depends on .c" is more consistent with the general target: precondition syntax than ".c.o".
* They are implemented in multiple implementations (as noted above).


Desired Action On page 2916, before the text on "Inference rules", add a new section called "Pattern rules".

If at least one target in a target rule includes the '%' character, it defines a "pattern rule". A pattern rule is used if no more specific target rule can be found to generate a target (if no pattern rule can be found, then an inference rule is used as described below where available).

The "%" matches 0 or more characters; it must be prefixed, suffixed, or both with other characters (which must match a filename where present). The characters that match the '%' character in each use of a pattern rule in a target are termed the "stem". A '%' character in the prerequisites, if any, must match the same stem.

If any prerequisites do not exist, make must determine if it can make those prerequisites (this is called "chaining"). The prerequisites must exist, or make must determine a chain of other rules that could make the prerequisites, for the pattern rule to apply. A rule whose prerequisites actually exist or are mentioned always takes priority over a rule with prerequisites that must be made by chaining other implicit rules.

If more than one rule can apply, the rule with the shortest stem is used. If there is more than one such rule, the one first mentioned in the makefile is used.

Tags No tags attached.
Attached Files

- Relationships

-  Notes
(0001023)
joerg (reporter)
2011-11-17 10:32

The proposed text is far from being a sufficient explanation.

See the SunOS make manual page (but this is unfortunately not complete). SunPro make introduced this feature in 1986 and there are some important constraints that need to be honored even in a simplified implenmentaion:

- Pattern matching rules must have a higher precedence than suffix rules. This is mentioned in the SunOS manual page.

- Pattern matching rules must have a defined precedence order. SunPro Make defines the order of evaluation == the order of definition of pattern rules. This is infortunately not mentioned in the SunOS man page.
(0001029)
Konrad_Schwarz (reporter)
2011-11-17 16:06
edited on: 2011-11-17 16:07

The pattern matching operator is insufficient to specify retrieval of SCCS or RCS files from directories other than the current one.

E.g., GNU Make has the following rules built in:
%:: RCS/%,v
%:: RCS/%
%:: s.%
%:: SCCS/s.%
This works only for files located in the current directory.

To eliminate recursive make, an extension is required.

For example, this could take the form (%D) and (%F),
allowing rules such as:
%:: (%D)/RCS/(%F),v
etc.

(0001032)
joerg (reporter)
2011-11-17 16:47

I am not sure whether it is apropriate to try to eliminate recursive make.

There are many other problems with non-resurcive make systems.

SunPro make introduced a technology that allows to deal with all needed dependencies in recursive build systems. This is done by not only auto-creating C-proprocessor dependencies (via environment SUNPRO_DEPENDENCIES= that is understood by Sun C and gcc). SunPro make in addition supports (together with the Solaris linker) via SGS_SUPPORT=libmakestate.so.1 to get a list of path names for all linked files (including libraries). This does not seem to leave any unclosed hole.
(0001033)
dwheeler (reporter)
2011-11-17 17:08

Regarding comment #1023:
The precedence rules are already there.
The first rule noted is embodied in this text:
"A pattern rule is used if no more specific target rule can be found to generate a target (if no pattern rule can be found, then an inference rule is used as described below where available)."
The second rule is also noted in a more complex way. The shortest stem is used first, and if that fails, THEN the first one defined is used. So the proposal adds another precedence rule (shortest stem), which is I think the right semantic (basically, "the most specific rule" is used).

Regarding comment #1029:
A bare "%" could be supported by changing the proposal slightly. Change:
  The "%" matches 0 or more characters; it must be prefixed, suffixed, or both with other characters (which must match a filename where present).
to:
  The "%" matches 0 or more characters; it may be prefixed, suffixed, or both with other characters (which must match a filename where present).

Regarding "(%D)" - I presume you mean "$(%D)" and "$(%F)", right? We could certainly add the various special variables from say GNU make, since those are already implemented (I much prefer stealing stuff from some existing make, instead of coming up with something new.) Let me know if there's some subset you think is especially important.

Regarding comment #1032:
I don't think we should try to PREVENT recursive make. But "make" should have enough functionality so that it is not considered NECESSARY if people don't want to do it. In any case, adding pattern rules to the standard helps both recursive and non-recursive make use cases.
(0001034)
joerg (reporter)
2011-11-17 17:26

Let me answer one by one. First avoiding recursion:

In order to avoid recursion, there is a need to be able to specify target specific macro definitions. The current POSIX make does not have this feature. SunPro make has it via

tgt list := macro = value

and

tgt list := macro += value

In addition, the current definitions for include statements is not sufficuent.
There is a need to define that and how rules are evaluated while trying to include non-existent or outdated include files and there is a need to define how/when make could enable re-evaluation of rules read from included files.

GNU make does not behave as expected in this area, smake implements something that has been verified as working as expected during the past 15 years.
(0001035)
psmith (developer)
2011-11-17 18:27

Make recursion is a tangent we don't need to follow here: it doesn't have anything to do with this proposal IMO. However, I'll say that I don't agree with comment #1034 about what is required for non-recursive implementations. There are lots of ways to manage macro namespaces, for example, in standard make without requiring target-specific macro definitions. That's a discussion best had elsewhere.

Regarding comments #1029 and #1033, first note that $(%D) etc. is NOT a standard GNU make macro, and in fact cannot be implemented as such because the standard requires target and prerequisite lists are expanded during makefile parsing, but the expansion of such a variable cannot be performed correctly until much later when make is walking the dependency graph and knows what the stem will be. Both Sun make and GNU make support a deferred expansion that helps with this (GNU make calls it "secondary expansion") but I don't think we want to make that a prerequisite for this proposal.

GNU make addresses the problem in comment #1029 for some situations by being clever about how it handles stems that contain slashes. See http://www.gnu.org/software/make/manual/html_node/Pattern-Match.html [^] This won't meet every possible need. Also it's a commonly misunderstood feature, which possibly says more about the quality of the documentation in this area but may also signal that the feature is too opaque to be a good candidate for standardization.

However I do agree that the proposal should say something about how file names containing slashes are handled... as worded here GNU make doesn't meet the proposal in this situation.
(0001036)
dwheeler (reporter)
2011-11-17 18:29

Replying to comment #1034:

A lot of the comment text seems to be an argument for a conditional capability in make. That should be a separate proposal. This proposal is about pattern rules, which are useful whether there are conditionals or not.

I agree that the standard make's include statements are not sufficient, but again, that is a separate issue. In fact, there are currently two other comments about include:
* One is to have a way to gracefully handle include files if they don't exist. A change has already been accepted to address this, in http://austingroupbugs.net/view.php?id=333 [^] - so I think that problem has been helped at least.
* Another is to automatically re-evaluate rules from included files, so that dependencies can be automatically determined and then applied. That was proposed in http://austingroupbugs.net/view.php?id=332 [^] and rejected, though the only reason stated was "no consensus". I have no idea WHY there was no consensus.
I'd like to see bug 332 re-opened, because this forces multiple invocations of make to do builds (one to figure out the dependencies, the other to actually use them), which is really unfortunate. Users should be able to just type "make" and be certain that everything necessary happen automatically; many current make implementations provide this, but it's not in the standard. But I digress; bug 513 isn't about that at all. If you want automatic rebuilding, I encourage you to revisit and argue for bug 332 (I'd love to see it!).
(0001037)
dwheeler (reporter)
2011-11-17 22:11

Regarding comment #1035:

I agree that deferred expansion is not necessary for this to be very useful, and it complicates things. I don't currently plan to add that to this proposal.

The "clever approach" GNU make takes to handling stems actually results in very intuitive results. Basically, if you don't include a "/" in the pattern, then the pattern will work in any directory at any level and produce the pathname with the right prefixed directories. I think this is important to include.

I think the challenge is that it's hard to express this idea precisely and clearly. Here's the GNU text: "When the target pattern does not contain a slash (and it usually does not), directory names in the file names are removed from the file name before it is compared with the target prefix and suffix. After the comparison of the file name to the target pattern, the directory names, along with the slash that ends them, are added on to the prerequisite file names generated from the pattern rule's prerequisite patterns and the file name. The directories are ignored only for the purpose of finding an implicit rule to use, not in the application of that rule. Thus, ‘e%t’ matches the file name src/eat, with ‘src/a’ as the stem. When prerequisites are turned into file names, the directories from the stem are added at the front, while the rest of the stem is substituted for the ‘%’. The stem ‘src/a’ with a prerequisite pattern ‘c%r’ gives the file name src/car."
(http://www.gnu.org/software/make/manual/html_node/Pattern-Match.html). [^]

Making this precise is tricky. Here's my try, please append this to the proposal:

If no target pattern in a given pattern rule contains a slash, then this rule is used to match targets and generate prerequisite pathnames:
* Only the target's basename compared with the target pattern (including its prefix and suffix).
* If there is a match and the target pathname includes slash, the stem begins with the target's dirname and ending slash. Then, whether there is a slash or not, the stem includes all of the target pathname that did not match any prefix or suffix in the target pattern.
* Any prerequisite with a '%' is formed in the following manner: If the stem includes slash, it begins with the stem's dirname and ending slash. This is followed by the prerequisite prefix (if any), the rest of the stem, and the prerequisite suffix (if any).
In contrast, if the target pattern does contain a slash, then the full pathname (including directory names if any) is compared to determine if there is a match.

There's probably a much simpler way to express this precisely.
(0001038)
joerg (reporter)
2011-11-18 10:46

Replying to comment #1036:

I have not been in the teleconference that decided "non consensus", but I know that gmake implements a method that does not evaluate include dependencies before doing the include and for this reason cannot include files that are created as a result from an existing make rule.

gmake instead evaluates these rules after doing all includes and then restarts itself. This is in conflict with the treatment of makefiles, where related rules are evaluated before a makefile is opened for reading.

I've implemented a working method in smake in 1997 and this method did not cause any problem since then. I am willing to write down the basics of this method.
(0001039)
joerg (reporter)
2011-11-18 11:00

Replying to comment #1035:

It is nice to see the explanation for gmake pattern rules, but please keep in mind that gmake tries to reimplement a feature that has been introduced in 1986 by SunPro make. In case of diverting behavior (I did not test so far whether there is one), I tend to standardize on the behavior of the initial implementation.
(0001040)
psmith (developer)
2011-11-18 13:37

Regarding comment #1038: that explanation of the GNU make behavior is confusing and misleading. However this isn't the place to discuss that issue: please use the comments of http://austingroupbugs.net/view.php?id=332 [^] for further discussion.

Regarding comment #1039: do we have access to Sunpro make? Unless it's generally available we'll rely on you or someone else with access to verify whether the proposals here would match its behavior and if not, describe the differences.
(0001041)
joerg (reporter)
2011-11-18 14:17

Regarding comment #1040:

The gmake behavior regarding includes is confusing and missleading. I am sure that for this reason the behavior has been accepted as a bug that needs a fix in 1998 by the gmake maintainer.

Bug #332 has been closed and does not seem to be apropriate as this would be a discussion about enhancing the include syntax in order to make include better suitable for portable makefiles. It seems that we need a way to migrate off-topic discussions to new bug numbers. Andrew, do you have a proposal how to do this best?

I have no idea who is ment by "we" but anybody who is interested can have access to SunPro Make. SunPro make replaced the AT&T make in 1986 on SunOS 3.x and it still is the standard make on Solaris/OpenSolaris. An enhanced version called "dmake" is part of the Sun compiler collection that even exists for Linux. The complete sources for "dmake" are not available but the sources for the basic software have been opensourced by Sun in December 2006 at my request.
(0001042)
psmith (developer)
2011-11-18 18:05

I will use the mailing list to discuss topics not related to this issue.

By "we" I mean anyone who is interested in investigating how the SunPro make implementation of pattern rules may or may not meet the description David proposes, and add that information here so that it can be considered when evaluating this issue.
(0005126)
joerg (reporter)
2020-12-01 17:34
edited on: 2020-12-01 17:36

This is an attempt to make a simplified proposal for adding pattern rules
to the POSIX make definition soon, to allow them to appear in the upcoming
issue 8 version of the standard. The simplification in this proposal is
identical to the simplification used by smake. So there is no need to
discuss whether this simplification describes a useful subset. This subset
has been verified to be useful since almost 25 years, so it disproves the
concerns currently written down on page 2909 in the lines 97901 ... 97906.

The page and line numbers match the 1.1 draft for issue 8.

On Page 2897 change line 97390 to:

Suffix based inference rules are formatted as follows:


On page 2897 after line 97401 add:

When no pattern matching inference rule applies, make shall check the
target name to see if it ends with a suffix in the known suffixes list.

On page 2898 after line 97438 add:

Pattern matching inference rules:

A target entry of the form:

       tp%ts: dp%ds
            command

or

       tp%ts:
            command

is a pattern matching inference rule, in which tp is a target prefix,
ts is a target suffix, dp is a dependency prefix, and ds
is a dependency suffix (any of which can be empty).

The % stands for a basename of zero or more characters that
shall be matched in the target, and shall be used to construct the name
of a dependency. When make encounters a match in its search
for an inference rule, it shall use the rule in that target entry
to build the target from the dependency file. Pattern-matching
inference rules typically make use of the $@ and $<
dynamic macros as placeholders for the target and dependency names.

Pattern Matching Inference Rules are searched in in the same
order as they have been defined.

Pattern Matching Inference Rules have precedence over Suffix Rules.
The builtin rules defined by make are typically defined
as Suffix Rules to allow them to be overwritten by Pattern Matching
Inference Rules in regular makefiles.

On page 2909 replace lines 97901 ... 97906 with:

SunPro Make supports pattern matching inference rules in the form:

      tp%ts: [dependency ...] dp%ds [dependency ...]
            command

or

       tp%ts: [dependency ...]
            command

as well. This feature has not been added to the standard, in order to make
it easier to implement the feature by make implementations that do not yet
support pattern matching inference rules. This feature shall be seen as a
valid enhancement that may become part of a future version of the standard.

GNU make permits to define pattern rules in the form:

       tp%ts:: dp%ds
            command

to mark them as termination rules that stop recursive searches for inference
rules. This could e.g. be used to define prefix based rules for SCCS. The
current standard does not include this feature, but future versions of the
standard may add it.

It has been discussed whether a pattern matching inference rule with no
command shall be used to remove an existing pattern matching inference rule
with the same pattern and whether the pattern matching inference rule:

     %: %

with no command shall remove all existing pattern matching inference
rules. This is helpful, since otherwise it would not be possible to
replace or overwrite previous pattern matching inference rule definitions.
This version of the standard does not require this behavior but
permits it in order to allow it to be added to a future version of the
standard.

(0005127)
psmith (developer)
2020-12-01 18:28

There are two issues that need to be addressed before this can be accepted (perhaps the only way to address them would be to make them unspecified which would be unfortunate...) I will create separate comments for them to try to focus discussion.

First, GNU make does not use pure definition order when searching for matching pattern rules. You can see the original proposal above for a description of GNU make's algorithm: it first chooses the "most specific" pattern match; by "most specific" it means the one with the smallest stem value. So if you have a target pattern "foo%.o" and another one "%.o" then a target "foobar.o" will use the first pattern rule even if it's defined later in the makefile, because the pattern match stem ("bar") is shorter than the other rule (stem is "foobar").

If there are multiple rules which have the same stem length("%.o" and "%.x") then the first one defined takes precedence.

One way to make this work is to ensure that your patterns are ordered properly in the makefile (shorter stems come first) then it works in either model. But I don't know how to write this in the standard.
(0005128)
psmith (developer)
2020-12-01 18:40

The second issue is what happens if the pattern doesn't contain a slash. GNU make has special handling of this situation to allow situations like this:

    f%.o : b%.c ; ...

to match a target like "dir/foo.o" and construct a prerequisite of "dir/boo.c".

This is really useful but may not be appropriate for standardization. But again I'm not sure how to write the standard text to manage this difference.

In both of these issues, I don't agree that when .POSIX is defined we would use a different algorithm. We'll need to find a way to allow these behaviors in makefiles with .POSIX set.
(0005130)
psmith (developer)
2020-12-01 18:54

The proposal also needs to consider how it will handle multiple definitions of the same pattern.

In GNU make if you redefine the same pattern rule (that means, both the target AND all the prerequisites must be the same, and in the same order) then the commands for that pattern rule are reset to the commands from the new instance of that pattern rule without complaint.

The passing reference to removing existing pattern rules by declaring a pattern rule with no commands is a useful extension to consider.

I personally don't see a lot of need for "% : %" deleting all pattern rules (at least given the above capabilities).
(0005131)
shware_systems (reporter)
2020-12-01 20:46
edited on: 2020-12-02 19:22

Re: Note: 0005127
I'd leave such out-of-ordering as an extension valid when .POSIX isn't specified, as it forces an implementation model of fully reading the makefile before it can determine if there is a better match or not.

Re: Note: 0005128
Such behavior is explicitly an allowed extension, with '/' being an additional character an implementation can permit in target or prereq names. The use of .POSIX is intended to disable such extensions, however, and different algorithms are expected. You don't get it both ways, that I see; you can get it that way invoking it as gmake, but not as make.

(0005132)
psmith (developer)
2020-12-01 21:17

A makefile always has to be fully read before any matching is done because you don't know whether an explicit rule will be defined later, and for other reasons such as .DEFAULT etc. It's always the case that ALL makefiles must be fully parsed before we can start processing even the first target (the mess around include file rebuilding we discussed last month is an exception to this which I'm ignoring).

I'm not willing to say that if you have a makefile with .POSIX defined which uses pattern rules and you run it with GNU make, that suddenly the algorithm we use for determining pattern rule match ordering is now different. So, we will have to find a way to allow both possible orderings to be used when .POSIX is defined.

It is definitely NOT true that the use of .POSIX is intended to disable extensions. I feel like this has come up before a number of times. The use of .POSIX is intended to say that IF your makefile uses only behaviors guaranteed by POSIX, then the implementation of make will process it as described by the standard.
(0005133)
shware_systems (reporter)
2020-12-01 21:26

Additionally, since the '%' char is potentially a literal some conforming file systems may permit in file names, I'd like to see a provision for escaping the '%' in the inference specs when .POSIX isn't used.
(0005134)
psmith (developer)
2020-12-01 21:48

In GNU make you can use backslash to escape %. Note that this is only needed in situations where it might be interpreted as a pattern when you don't want it to be; in other situations you don't use a backslash.
(0005135)
shware_systems (reporter)
2020-12-01 22:48

The definition of .POSIX has "If it appears as the first non-comment line in the makefile, make shall process the makefile as specified by this section; otherwise, the behavior of make is unspecified." Extensions, being unspecified or at most permitted, I see as precluded; their use is expected to abort the utility as a portability debugging aid, usually as invalid syntax reports, not give carte blanche to invoke implementation-specific functionality without any notice. That is what someone could reasonably expect of an implementation that does not try to add any extensions, anyways, making use of .POSIX superfluous. Maybe if that otherwise clause wasn't present your reasoning would have more merit, but it is there.
(0005136)
psmith (developer)
2020-12-02 01:03

> Extensions, being unspecified or at most permitted, I see as precluded;
> their use is expected to abort the utility as a portability debugging aid

That is not the case and in fact, no implementation of make works like that that I know of.

Yes, IF the standard defined some specific behavior for pattern rules, such as saying that the implementation was required to emit a diagnostic and stop processing, then IF you specified .POSIX in the makefile and used a pattern rule then the implementation would be required to emit a diagnostic and stop processing.

But, the standard doesn't say anything about what an implementation should do if it comes across a pattern rule, so there's no reason that the implementation has to do anything other than what GNU make (or smake or SunPro make) does when it sees pattern rules normally, just because .POSIX exists in the makefile.

We had this conversation on the conference call a few weeks ago. .POSIX does NOT mean "nothing but POSIX and everything else is an error". It means, everything POSIX defines must be handled the way POSIX defines it to be handled. If you write a strictly POSIX-conforming makefile then the implementation must process it correctly according to the standard. That's all it means.

This is true for all the different utilities that provide a way to enable a "POSIX mode", as far as I'm aware.
(0005138)
joerg (reporter)
2020-12-02 17:39

Re Note: 0005127

The original pattern rule implementation uses a first match algorithm to find
a matching rule. Let me check, whether gmake is compatible to the original behavior.

$ cat Makefile
a%b:
        echo one

aa%bb:
        echo two

all: aaxbb
        @:

# call dmake to make obvious we are using SunPro Make
# -m serial avoids parallel informational messages
#
$ dmake -m serial
echo one
one

$ smake
...echo one
one

$ gmake
echo one
one

If gmake did use a best match algorithm instead of a first match algorithm,
it should print two with this test.

So even though you suspect that gmake is using a "best match", it does
not seem to do so and rather seems to be compatible to the original definition.
(0005139)
psmith (developer)
2020-12-02 18:00

I can't explain your experience unless you're using a very old version of GNU make. I get:

    $ make
    echo two
    two
(0005141)
joerg (reporter)
2020-12-02 18:03

Re: Note: 0005128

If gmake implements this path handling carefully and in a way that is not in
conflict with the pattern rule definition, this does not seem to be a problem.

Smake has a probably similar feature since 40 years. It supports an object
directory where it automatically places its results. This feature must be
switched on via an .OBJDIR: name directive and did not cause problems since
smake has been enhanced to support pattern rules.

If gmake causes problems in this area, I would call this a bug that needs a fix.
(0005142)
psmith (developer)
2020-12-02 18:19
edited on: 2020-12-02 19:19

Re: Note: 0005141

I'm not sure what you mean by "implements this path handling carefully". I gave an example above of a pattern and how GNU make interprets it when the target is prefixed with a directory. I'll give a reproducible example:

  $ mkdir foo
  $ touch foo/boo.x

  $ cat Makefile
  f%.x : b%.x ; : $@ $<

  $ make foo/foo.x
  : foo/foo.x foo/boo.x

If you are saying that this behavior is acceptable to you then fine, we have no problems other than how to describe this in the text.

However, maybe this isn't really a problem because the syntax doesn't overlap. Maybe it works to just not define this behavior and leave GNU make's handling as an extension. That is, a conforming makefile couldn't rely on the above working but I don't think GNU make's behavior would cause a conforming makefile to NOT work. As long as the standard didn't require the above makefile / command to produce an error it may be OK.

> If gmake causes problems in this area, I would call this a bug that needs a fix.

I'm not going to consider functionality that has been available and documented in GNU make for 30 years to be a bug.

(0005143)
joerg (reporter)
2020-12-02 18:22

Re: Note: 0005130

Overwriting pattern rule commands is an incompatible extension to the
"always append" rule.

This is why I proposed the compatible extension to permit to remove
specific pattern rules by repeating the original pattern with no
rule command.

Given that gmake does not behave the way you described in your note,
gmake does not rewrite an existing pattern rule. Gmake rather removes
the existing rule and appends a new rule with the new command to the
end of the existing chain, so gmake does not seem to introduce an
incompatible behavior.
(0005144)
joerg (reporter)
2020-12-02 18:27

Re: Note: 0005139

It seems that the new gmake version you refer to introduced a bug by changing
its behavior in an incompatible way.

Since you do not seem to care about the gmake behavior, it seems to be not
a problem to change that back to a compatible behavior.
(0005145)
joerg (reporter)
2020-12-02 18:34

Re: Note: 0005142

That specific extension differs from the smake extension I had in mind from
reading your previous description.

The behavior you describe in Note: 0005142 appears to me as an acceptable
compatible extension to the official behavior.

Maybe this should be mentioned in the RATIONALE section.
(0005146)
psmith (developer)
2020-12-02 19:18
edited on: 2020-12-02 19:21

Re: Note: 0005130

An "always append" rule for patterns is not useful when the target and prerequisites are identical because second instance of that rule can never be reached: the first instance will always be chosen since it was defined first. This behavior is confusing and annoying.

When GNU make sees another instance of a pattern with the same target and prerequisites it overwrites the existing command list with the new command list. This behavior makes more sense and is useful.

In addition, when GNU make sees an instance of a pattern rule with a given target and prerequisites but *no* commands, it will stop using that pattern (e.g., the pattern is deleted). This is useful to allow already-existing patterns to be removed.

If you prefer to say that it's undefined by the standard what happens if you declare a second pattern rule with matching target and prerequisites, either with or without commands, that's fine with me.

(0005147)
psmith (developer)
2020-12-02 19:24
edited on: 2020-12-02 19:31

Re: Nbugnote:5144

The enhancement for selecting pattern rules via shortest stem was requested by many users (I didn't invent this idea: users asked for it) and is in use in many makefiles.

It's not a bug.

(0005152)
joerg (reporter)
2020-12-03 15:28

Re: Note: 0005147

You may call it whatever you like...

It is an incompatible change that has the potential of breaking
existing makefiles.

Whether your customers may have asked for that change or not does
still not make it a charter for an incompatible change.

As an alternative solution, you could have told your users to order
the pattern rules from the most specific to the less specific one,
or you could have introduced a method that does not break existing
behavior.

The original behavior is in SunPro Make since 40 years, in smake
since 25 years and even in gmake it has been for 24 years, before
you changed gmake, so this is a well established interface.

You would need to revert it, at least in case that .POSIX is present.
(0005153)
joerg (reporter)
2020-12-03 15:30

Re: Note: 0005146

I have no idea why you believe that gmake overwrites existing pattern rules,
A test verifies that gmake always appends a new rule.

cat Makefile
a%b:
        echo one

aa%bb:
        echo two

a%b:
        echo three

all: aaxbb
        @true

$ gmake
echo two
two


If gmake did overwrite the exiting rule, it should print "three".

BTW: "gmake -p" delivers an additional verification for the fact that
it always appends new entries to the list of rules.

The latter test can be verified with both old and new gmake versions,
the first test only works with the gmake from before introducing the
incompatible change.
(0005154)
kre (reporter)
2020-12-03 16:59

Re Note: 0005153

Joerg, that's the wrong test, there was no claim that a%b: would overwrite
aa%bb: just the earlier a%b:

If you test instead

a%b:
        echo one

a%b:
        echo three

all: aaxbb
        @true


jinx$ gmake -f /tmp/Makefile
echo three
three

And when I run:

jinx$ gmake -p -f /tmp/Makefile
[...]

# Implicit Rules

a%b:
# recipe to execute (from '/tmp/Makefile', line 5):
        echo three

[...]

that's the only a%b rule present.
(0005155)
kre (reporter)
2020-12-03 17:08

As an addition, when I use bmake (BSD make, from NetBSD), using the same
/tmp/Makefile as in Note: 0005154

jinx$ make -f /tmp/Makefile
make: "/tmp/Makefile" line 5: warning: duplicate script for target "a%b" ignored
make: "/tmp/Makefile" line 2: warning: using previous script for "a%b" defined here
echo one
one

This I believe adds support for the proposition that what happens with
duplicate rules should be left unspecified.

bmake does pick the first match though, not the most specific (shortest stem).

With /tmp/Makefile changed to be

a%b:
        echo one

aa%bb:
        echo two

all: aaxbb
        @true

then

jinx$ make -f /tmp/Makefile
echo one
one
(0005156)
psmith (developer)
2020-12-03 17:19

Re: Note: 0005152

Introducing shortest-stem patterns was unlikely to break makefiles that worked previously, in a practical sense. If a makefile using the first-match behavior had a shorter-stem pattern that appeared first then the behavior didn't change. If it had a shorter-stem pattern that appeared after a longer-stem pattern then the shorter-stem pattern would not be seen because the longer-stem pattern matched first. So, the only way this change could have impacted makefiles would be if had an unused shorter-stem pattern in their makefiles somewhere, which suddenly started working.

Changing back, however, would break a large number of existing environments.

> You would need to revert it, at least in case that .POSIX is present.

No. Any standardization of pattern rules will need to be worded in a way that doesn't require this.

Re: Note: 0005153

I see the distinction you're trying to make here. You are talking about rule ordering. When I say "replace" I was just talking about which set of commands will be used to build the target. Yes, it's true, if you want to be more specific what GNU make really does is remove the first instance of that pattern rule and add a new instance of the pattern rule, so the order of rule search may be different after the redefinition.
(0005158)
joerg (reporter)
2020-12-03 18:43

Re: Note: 0005154

Robert, your test does not check wether gmake overwrites a rule. Since
your test works on a rule chain with the length one, you cannot distinct
overwriting from what really happens in gmake:

  The old rule is removed by gmake and a new rule is appended to the
  list of then existing rules, that is of zero length in case of your
  test.

This is why I do not like to permit this way of "modifying" pattern
rules for the POSIX standard. If we ever add this feature to POSIX,
I would like to see the requirement to first remove an existing rule
by specifying a rule with the same pattern, but with no command
and then to add a new one with a command.

Re: Note: 0005155

Thanks for reminding that bmake also implements a pattern rule search that
is compatible to the original implementation from SunPro Make. So it is
obvious that the recent incompatible change in gmake is in conflict with
all other make implementations.

The fact that bmake prints a warning if you try to "overwrite" a rule
is another hint that my proposal to first require an explicit removal
with the same pattern and no command before permitting to redefine a
rule is the better way for a potential future.
(0005963)
nick (manager)
2022-09-12 15:08
edited on: 2022-09-12 15:08

This was discussed in the September 12th 2022 teleconference. We are leaning towards rejecting based on irreconcilable differences between existing implementations. However, the bug is being left open for now in case anyone wants to try to come up with a common subset of functionality that would be worth adding to the standard.


- Issue History
Date Modified Username Field Change
2011-11-17 00:02 dwheeler New Issue
2011-11-17 00:02 dwheeler Status New => Under Review
2011-11-17 00:02 dwheeler Assigned To => ajosey
2011-11-17 00:02 dwheeler Name => David A. Wheeler
2011-11-17 00:02 dwheeler Section => make
2011-11-17 00:02 dwheeler Page Number => 2916
2011-11-17 00:02 dwheeler Line Number => 95850
2011-11-17 10:32 joerg Note Added: 0001023
2011-11-17 16:06 Konrad_Schwarz Note Added: 0001029
2011-11-17 16:07 Konrad_Schwarz Note Edited: 0001029
2011-11-17 16:47 joerg Note Added: 0001032
2011-11-17 17:08 dwheeler Note Added: 0001033
2011-11-17 17:26 joerg Note Added: 0001034
2011-11-17 18:27 psmith Note Added: 0001035
2011-11-17 18:29 dwheeler Note Added: 0001036
2011-11-17 22:11 dwheeler Note Added: 0001037
2011-11-18 10:46 joerg Note Added: 0001038
2011-11-18 11:00 joerg Note Added: 0001039
2011-11-18 13:37 psmith Note Added: 0001040
2011-11-18 14:17 joerg Note Added: 0001041
2011-11-18 18:05 psmith Note Added: 0001042
2020-12-01 17:34 joerg Note Added: 0005126
2020-12-01 17:35 joerg Note Edited: 0005126
2020-12-01 17:36 joerg Note Edited: 0005126
2020-12-01 18:28 psmith Note Added: 0005127
2020-12-01 18:40 psmith Note Added: 0005128
2020-12-01 18:54 psmith Note Added: 0005130
2020-12-01 20:46 shware_systems Note Added: 0005131
2020-12-01 21:17 psmith Note Added: 0005132
2020-12-01 21:26 shware_systems Note Added: 0005133
2020-12-01 21:48 psmith Note Added: 0005134
2020-12-01 22:48 shware_systems Note Added: 0005135
2020-12-02 01:03 psmith Note Added: 0005136
2020-12-02 17:39 joerg Note Added: 0005138
2020-12-02 18:00 psmith Note Added: 0005139
2020-12-02 18:00 psmith Note Added: 0005140
2020-12-02 18:00 psmith Note Deleted: 0005140
2020-12-02 18:03 joerg Note Added: 0005141
2020-12-02 18:19 psmith Note Added: 0005142
2020-12-02 18:22 joerg Note Added: 0005143
2020-12-02 18:27 joerg Note Added: 0005144
2020-12-02 18:34 joerg Note Added: 0005145
2020-12-02 19:18 psmith Note Added: 0005146
2020-12-02 19:19 Don Cragun Note Edited: 0005142
2020-12-02 19:21 Don Cragun Note Edited: 0005146
2020-12-02 19:22 Don Cragun Note Edited: 0005131
2020-12-02 19:24 psmith Note Added: 0005147
2020-12-02 19:31 Don Cragun Note Edited: 0005147
2020-12-03 15:28 joerg Note Added: 0005152
2020-12-03 15:30 joerg Note Added: 0005153
2020-12-03 16:59 kre Note Added: 0005154
2020-12-03 17:08 kre Note Added: 0005155
2020-12-03 17:19 psmith Note Added: 0005156
2020-12-03 18:43 joerg Note Added: 0005158
2022-09-12 15:08 nick Note Added: 0005963
2022-09-12 15:08 nick Note Edited: 0005963


Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker