View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000857 | 1003.1(2013)/Issue7+TC1 | Shell and Utilities | public | 2014-07-08 10:14 | 2019-06-10 08:54 |
| Reporter | jammule | Assigned To | |||
| Priority | normal | Severity | Comment | Type | Omission |
| Status | Closed | Resolution | Accepted As Marked | ||
| Name | Jarmo Jaakkola | ||||
| Organization | |||||
| User Reference | |||||
| Section | make | ||||
| Page Number | 2937 | ||||
| Line Number | 97105-97113 | ||||
| Interp Status | --- | ||||
| Final Accepted Text | 0000857:0002393 | ||||
| Summary | 0000857: Make rules which do not create the target file or do unexpected things with its timestamp | ||||
| Description | I haven't been able to find clear answers from the standard to some behavior that seems to be taken for granted with make. These are all somewhat related, so I'll list them here instead of making a separate issue from each of them. Case 1: It is customary to use the following idiom to force a target's related commands to always run:
foo: FORCE
commands...
FORCE:
(This is what some implementations offer the ".PHONY: foo" special target as an extension for.) My reading of the standard does not come clear why, or if at all, this should work (assuming FORCE does not exist). This is the current formulation on page 2937, lines 97105-97113:
When "make foo" is issued this is what I think should happen:
Based on tests with NetBSD and GNU make the convention is to regard, for the current run, the target as having a timestamp that is newer than any target it is a prerequisite for (i.e. force them out-of-date). The requirement of a target entry with no commands being considered up-to-date hints at this also. Also, I think that in the sentence "If there are no commands listed for the target, [...]" the word "target" should be replaced with "target entry". As it now stands the sentence could be taken to mean "In the case there are no commands in the applicable rule or there is no applicable rule at all, the target shall be treated as up-to-date." This is because "target" is sometimes used for a single rule entry, sometimes for the collection of all matching rule entries and and sometimes for the target name. Case 2: Extension of the previous one:
foo: bar
commands...
bar:
commands_that_do_not_create_bar...
What now? Nowhere is there a requirement that the executed commands actually create the target. Nor should there be, because of the convention of meta-targets "all", "clean", etc. Based on my tests, this works exactly the same as the previous case. Of course, in both cases, if foo's commands actually try to use bar, it will fail and that is an error on the application's side. Case 3: Assuming bar exists and is older than foo:
foo: bar
commands...
bar:
commands_that_do_not_update_bar's_modification_time
Or even better, assuming bar is newer than foo:
foo: bar
commands...
bar:
make_bar's_modification_time_older_than_foo's...
It is not necessarily immediately obvious that the timestamps of a target and its prerequisites are to be checked after the prerequisites have been brought up-to-date. First it is said: "A target is considered out-of-date if it is older than any of its prerequisites or if it does not exist." Later: "After make has ensured that all of the prerequisites of a target are up-to-date and if the target is out-of-date, [...]" These do not in my opinion forbid a naive implementation that would check the out-of-dateness of a target before its prerequisites are updated. Case 4: It is not specified that it should be an error if there is no rule for target and there is no corresponding file. Neither is it specified that a target, for which no rule but a corresponding file exists, is effectively equivalent to target: | ||||
| Desired Action | If the observed implementation functionality is supposed to be standards compliant, the text on page 2937, lines 97105-97113 should be changed. Here is a suggestion, with the changes in bold:
| ||||
| Tags | tc2-2008 | ||||
|
|
see 0000523 for case 1 |
|
|
It strikes me that there are two key issues here: 1. The "make" section doesn't adequately define "out-of-date", "up-to-date", or "bring a target up-to-date". It gives a [questionable] definition for "out-of-date", and it's somewhat implied that "up-to-date" means "not out-of-date", and what it means to "bring a target up-to-date" is only explicitly described for the case where the target has no commands. 2. The wording "After make has ensured that all of the prerequisites of a target are up-to-date and if the target is out-of-date" implies the "out-of-date" check is done *after* recursing on the prerequisites, whereas I believe existing make implementations determine "out-of-date"ness before running any commands. Here's a quick test to try: $ cat Makefile .POSIX: one: two echo one two: echo two $ touch one $ make one Strictly following POSIX's definition of "out-of-date", target "two" here is "out-of-date" because it does not exist, but target "one" *does* exist and its not older than any of its prerequisites. It stands to reason that a POSIX-compliant make utility should only run "echo two". However, at least GNU make and NetBSD make (tested on Ubuntu 14.04) run both the "echo two" command and then also "echo one". I'd be interested in knowing if any other make implementations behave differently. I'd suggest addressing this issue along the following lines: 1. Define that "out-of-date" means the target does not exist, its file modification time is older than any of its prerequisites, or any of its prerequisites are "out-of-date". The determination of whether targets are "out-of-date" is made before "make" executes any commands. 2. Define that to "bring a target up-to-date" for an "out-of-date" target means to bring each of its prerequisites up-to-date (in order) and then to execute the target's commands (if any). If a target is not "out-of-date", then nothing needs to be done to bring it up-to-date. |
|
|
Mantis mangled my example Makefile, so here it is again (hopefully mangling free): .POSIX: one: two echo one two: echo two |
|
|
see 0000763 for case 4, first part |
|
|
Regarding mdempsky's comment in (0000857:0002336) I believe existing make implementations determine "out-of-date"ness before running any commands. This would be incorrect. Try running "make one" twice with this makefile:
one: two
touch one
two: three
[ -e two ] || touch two
three: FORCE
touch three
FORCE:
If make would check the out-of-dateness before running any commands, all targets should be created every time. This does not happen with at least NetBSD or GNU make. On the second run "one" is not touched, because "two" is not actually updated even if it was out-of-date compared to "three". This is in my opinion what is expected and is reasonable and also at least one of the reasons why the "-t" command line switch exists. Consider a case where a single file is used to generate multiple other files, which in turn are dependencies for other files. For example, some automatic code generation tool creates multiple header files for different modules. Just because the input for the code generator has changed does not mean that all or even any of the generated files have changed semantically. Running the commands to generate the headers cannot be avoided because their dependencies are newer, but those rules may well utilize temporary files to check if the generated file has actually changed and only update the target when that happens. This will prevent make (at least GNU and NetBSD) from unnecessarily updating any of the targets, which might be numerous, that depend on the header. Thus, I do not think it would be wise to add this requirement to the standard: The determination of whether targets are "out-of-date" is made before "make" executes any commands. |
|
|
Ah, I stand corrected. Thanks for that demonstration test case! |
|
|
On page 2937 line 97106 section make Change from: A target is considered out-of-date if it is older than any of its prerequisites or if it does not exist. to: A target is considered out-of-date when all of its prerequisites are up-to-date and it is older than any of them, or if it does not exist. A target may also be considered out-of-date when it has the same timestamp as any of its prerequisites. On page 2937 line 97112 section make Change from: If there are no commands listed for the target, the target shall be treated as up-to-date. to: If there are no commands listed for the target entry, the target shall be treated as up-to-date. If a file corresponding to the target does not exist after the target has been successfully brought up-to-date, the target shall be treated as being newer than any target it is a prerequisite for. When no rule matches a target for which a corresponding file exists, the target shall be treated as up-to-date. It shall be an error if there is no target rule for a prerequisite that does not have a corresponding file. On page 2948 line 97603 add a new paragraph to the APPLICATION USAGE: Shell command sequences like "make; cp original copy; make" may have problems on filesystems where the timestamp resolution is the minimum (1 second) required by the standard and where make considers identical timestamps to be up to date. Conversely, rules like "copy: original; cp -p original copy" will result in redundant work on make implementations that consider identical timestamps to be out of date. On page 2954 line 97847 add a new paragraph to the RATIONALE: On most historic systems, the make utility considered a target with a prerequisite that had an identical timestamp up-to-date. The HP-UX implementation of make treated it as out-of-date. The standard now allows either behavior, but implementations are encouraged to follow the example set by HP-UX. This is especially important on filesystems where the timestamp resolution is the minimum (1 second) required by the standard. All implementations of make should make full use of the finest timestamp resolution available on the filesystems holding targets and prerequisites to ensure that targets are up-to-date even for prerequisite files with timestamps that were updated within the same second. |
|
|
There's a misspelling "implementaions" in the accepted text. More importantly, the new paragraph of the rationale should not recommend the HP-UX behavior, as the traditional behavior is longstanding and has some performance advantages. For example, given the rule 'copy: original; cp -p original copy', if I run HP-UX 'make' twice, the second run will copy the file unnecessarily, but GNU 'make' will do the right thing. This comes up even more often on filesystems with coarse timestamps, because then even plain 'cp' can be affected. I suggest changing this: The standard now allows either behavior, but implementaions are encouraged to follow the example set by HP-UX. This is especially important on filesystems where the timestamp resolution is the minimum (1 second) required by the standard. to this: The standard now allows either behavior. I also suggest appending the following text: Shell command sequences like "make; cp original copy; make" may have problems on filesystems where the timestamp resolution is the minimum (1 second) required by the standard and where 'make' considers identical timestamps to be up to date. Conversely, rules like "copy: original; cp -p original copy" will result in redundant work on 'make' implementations that consider identical timestamps to be out of date. |
|
|
In the Sept 4 teleconference it was decided to include the final paragraph from 0000857:0002353 as application usage, and 0000857:0002351 has been updated accordingly. |
|
|
I think that the accepted change just made all implementations non-compliant... It shall be an error if there is no target rule for a prerequisite that does not have a corresponding file. This would mean that it should always be an error if the target of an inference rule did not exist before make was run. I did leave the bolded word out from my proposal for just that reason. I'd suggest changing that sentence like this: It shall be an error if there is no <del>target</del><ins>applicable</ins> rule for a prerequisite that does not have a corresponding file. (It seems that the del and ins elements are not supported. Please render them properly in your mind.) |
|
|
On page 2937 line 97106 section make, change from: A target is considered out-of-date if it is older than any of its prerequisites or if it does not exist. to: A target is considered out-of-date if it does not exist and has not already been made up-to-date by the current invocation of make, if it is older than any of its existing prerequisites, or if any of its existing prerequisites are out-of-date. A target may also be considered out-of-date when it has the same timestamp as any of its prerequisites that have not been made up-to-date by the current invocation of make. On page 2937 line 97112 section make, change from: If there are no commands listed for the target, the target shall be treated as up-to-date. to: If there are no commands listed for the target entry, the target shall be treated as up-to-date. If a file corresponding to the target does not exist after the target has been successfully brought up-to-date, the target shall be treated as being newer than any target for which it is a prerequisite. When no rule matches a target for which a corresponding file exists, the target shall be treated as up-to-date. It shall be an error if make attempts to bring a target up-to-date, the target does not exist, and there is neither a target rule nor an inference rule for that target. On page 2948 line 97603 add a new paragraph to the APPLICATION USAGE: Shell command sequences like "make; cp original copy; make" may have problems on filesystems where the timestamp resolution is the minimum (1 second) required by the standard and where make considers identical timestamps to be up to date. Conversely, rules like "copy: original; cp -p original copy" will result in redundant work on make implementations that consider identical timestamps to be out of date. On page 2954 line 97847 add a new paragraph to the RATIONALE: On most historic systems, the make utility considered a target with a prerequisite that had an identical timestamp up-to-date. The HP-UX implementation of make treated it as out-of-date. The standard now allows either behavior, but implementations are encouraged to follow the example set by HP-UX. This is especially important on filesystems where the timestamp resolution is the minimum (1 second) required by the standard. All implementations of make should make full use of the finest timestamp resolution available on the filesystems holding targets and prerequisites to ensure that targets are up-to-date even for prerequisite files with timestamps that were updated within the same second. |
|
|
The proposed change of Note 2376 for line 97106 has the problem that it does not cover the case that a prerequisite does not exist. This is easily rectified by using the following wording, as proposed in my E-mail from September 10, 2014. > A target is considered out-of-date if it does not exist, is older than > any of its existing prerequisites, or if any of its prerequisites are > out-of-date. A target may also be considered out-of-date if it is as old > as any of its existing prerequisites. Also the wording in Note 2376 assumes that make rechecks the time stamps of its targets during execution. My understanding has always been that Make operates in two passes: In the first pass, Make generates a dependency tree among its targets using the prerequisite information, and determines which targets are out-of-date. In the second pass, Make executes the command blocks for the out-of-date targets -- but only for those targets listed on the command line or the default target and their prerequisites. It is not an error for no command block to exist for such a target listed in a rule (but it is an error for a non-existing prerequisite to have no explicit rule and no applicable automatic rule). During this pass, file timestamps are not reexamined and the dependency tree is not modified. This two pass implementation manifests itself in the different ways macros are expanded in dependency lines and command blocks. In dependency lines, the current value of the macro is used, as defined just before reading the dependency line. This corresponds to how variables are used in conventional sequential programming semantics. For command blocks however, the _final_ value of the macro, as set after reading the entire Makefile and any included files, is used. It follows that the suggested change to line 97112 is superfluous if not incorrect. If anything, I would add to the original line 97112: It is an error if there is neither a target rule nor an inference rule for a non-existing target to be built by make. |
|
|
As Geoff Clare and Note 2340 point out, Make rechecks the time stamps of its targets during execution, in contrast to my assertion in the previous note. The seminal document (Make — A Program for Maintaining Computer Programs, S.I. Feldman) states this as follows: The basic operation of make is to update a target file by ensuring that all of the files on which it depends exist and are up to date, then creating the target if it has not been modified since its dependents were. So I think it is safe and proper to codify this behavior into the specification. |
|
|
I have some revisions to 0000857:0002376 in progress; I will post them when they're ready. |
|
|
On page 2937 lines 97106-97107 (make Extended Description), change from:A target is considered out-of-date if it is older than any of its prerequisites or if it does not exist. to: A target shall be considered up-to-date if it exists and is newer than all of its dependencies, or if it has already been made up-to-date by the current invocation of make (regardless of the target's existence or age). A target may also be considered up-to-date if it exists, is the same age as one or more of its prerequisites, and is newer than the remaining prerequisites (if any). On page 2937 lines 97111-97113 (make Extended Description), change from: After make has ensured that all of the prerequisites of a target are up-to-date and if the target is out-of-date, the commands associated with the target entry shall be executed. If there are no commands listed for the target, the target shall be treated as up-to-date. to: To ensure that a target is up-to-date, make shall ensure that all of the prerequisites of a target are up-to-date, then check to see if the target itself is up-to-date. If the target is not up-to-date, the target shall be made up-to-date by executing the rule's commands (if any). If the target does not exist after the target has been successfully made up-to-date, the target shall be treated as being newer than any target for which it is a prerequisite. After page 2948 line 97603 add a new paragraph to the APPLICATION USAGE: Shell command sequences like "make; cp original copy; make" may have problems on filesystems where the timestamp resolution is the minimum (1 second) required by the standard and where make considers identical timestamps to be up to date. Conversely, rules like "copy: original; cp -p original copy" will result in redundant work on make implementations that consider identical timestamps to be out of date. After page 2954 line 97847 add a new paragraph to the RATIONALE: On most historic systems, the make utility considered a target with a prerequisite that had an identical timestamp as up-to-date. The HP-UX implementation of make treated it as out-of-date. The standard now allows either behavior, but implementations are encouraged to follow the example set by HP-UX. This is especially important on filesystems where the timestamp resolution is the minimum (1 second) required by the standard. All implementations of make should make full use of the finest timestamp resolution available on the filesystems holding targets and prerequisites to ensure that targets are up-to-date even for prerequisite files with timestamps that were updated within the same second. However, if the timestamp resolutions of the filesystems containing a target and a prerequisite are different, the timestamp with the more precise resolution should be rounded down to the resolution of the less precise timestamp for the comparison. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2014-07-08 10:14 | jammule | New Issue | |
| 2014-07-08 10:14 | jammule | Name | => Jarmo Jaakkola |
| 2014-07-08 10:14 | jammule | Section | => make |
| 2014-07-08 10:14 | jammule | Page Number | => 2937 |
| 2014-07-08 10:14 | jammule | Line Number | => 97105-97113 |
| 2014-08-07 16:18 | eblake | Note Added: 0002335 | |
| 2014-08-07 16:18 | eblake | Relationship added | related to 0000523 |
| 2014-08-07 17:16 | mdempsky | Note Added: 0002336 | |
| 2014-08-07 17:17 | mdempsky | Note Added: 0002337 | |
| 2014-08-08 10:41 | antoinel | Note Added: 0002338 | |
| 2014-08-08 10:42 | antoinel | Note Edited: 0002338 | |
| 2014-08-08 10:42 | antoinel | Note Edited: 0002338 | |
| 2014-08-08 10:43 | antoinel | Note Edited: 0002338 | |
| 2014-08-08 10:44 | antoinel | Note Edited: 0002338 | |
| 2014-08-08 10:47 | antoinel | Note Edited: 0002338 | |
| 2014-08-08 10:47 | antoinel | Note Edited: 0002338 | |
| 2014-08-08 10:51 | antoinel | Note Edited: 0002338 | |
| 2014-08-08 11:44 | jammule | Note Added: 0002340 | |
| 2014-08-08 11:47 | jammule | Note Edited: 0002340 | |
| 2014-08-08 12:22 | eblake | Relationship added | related to 0000763 |
| 2014-08-08 17:10 | mdempsky | Note Added: 0002343 | |
| 2014-08-21 15:35 | geoffclare | Note Added: 0002351 | |
| 2014-08-21 15:39 | geoffclare | Interp Status | => --- |
| 2014-08-21 15:39 | geoffclare | Final Accepted Text | => 0000857:0002351 |
| 2014-08-21 15:39 | geoffclare | Status | New => Resolved |
| 2014-08-21 15:39 | geoffclare | Resolution | Open => Accepted As Marked |
| 2014-08-21 15:39 | geoffclare | Tag Attached: tc2-2008 | |
| 2014-08-21 16:13 | eggert | Note Added: 0002353 | |
| 2014-08-22 09:20 | geoffclare | Note Edited: 0002351 | |
| 2014-09-04 16:06 | geoffclare | Note Edited: 0002351 | |
| 2014-09-04 16:07 | geoffclare | Note Added: 0002372 | |
| 2014-09-05 08:33 | jammule | Note Added: 0002373 | |
| 2014-09-05 08:34 | jammule | Note Edited: 0002373 | |
| 2014-09-05 08:35 | jammule | Note Edited: 0002373 | |
| 2014-09-05 08:36 | jammule | Note Edited: 0002373 | |
| 2014-09-05 08:37 | jammule | Note Edited: 0002373 | |
| 2014-09-11 16:18 | geoffclare | Note Added: 0002376 | |
| 2014-09-11 16:20 | geoffclare | Final Accepted Text | 0000857:0002351 => 0000857:0002376 |
| 2014-09-11 16:37 | geoffclare | Note Edited: 0002376 | |
| 2014-09-12 08:55 | Konrad_Schwarz | Note Added: 0002383 | |
| 2014-09-15 10:24 | Konrad_Schwarz | Note Added: 0002388 | |
| 2014-09-18 15:04 | rhansen | Note Added: 0002390 | |
| 2014-09-18 15:05 | nick | Status | Resolved => Under Review |
| 2014-09-18 15:05 | nick | Resolution | Accepted As Marked => Reopened |
| 2014-09-18 16:59 | rhansen | Note Added: 0002393 | |
| 2014-09-18 17:01 | rhansen | Note Edited: 0002393 | |
| 2014-09-18 17:04 | rhansen | Note Edited: 0002393 | |
| 2014-09-18 17:05 | rhansen | Note Edited: 0002393 | |
| 2014-09-18 17:13 | rhansen | Note Edited: 0002393 | |
| 2014-09-18 17:14 | rhansen | Note Edited: 0002393 | |
| 2014-09-25 15:22 | rhansen | Note Edited: 0002393 | |
| 2014-09-25 15:35 | rhansen | Note Edited: 0002393 | |
| 2014-09-25 15:41 | rhansen | Note Edited: 0002393 | |
| 2014-09-25 15:42 | rhansen | Note Edited: 0002393 | |
| 2014-09-25 15:43 | geoffclare | Final Accepted Text | 0000857:0002376 => 0000857:0002393 |
| 2014-09-25 15:43 | geoffclare | Status | Under Review => Resolved |
| 2014-09-25 15:43 | geoffclare | Resolution | Reopened => Accepted As Marked |
| 2019-06-10 08:54 | agadmin | Status | Resolved => Closed |
