0000766: Add VPATH to make - Austin Group Defect Tracker

Notes
(0001872) joerg (reporter) 2013-10-10 16:38 edited on: 2013-10-10 16:59	Your description is ambiguous as it does not mention what should happen to the internal macros when VPATH is in use. AFAIR, different make implementations do not set up macros like $* $@ $< $? in a uniform way after a successful VPATH expansion did occur. There also should be a definition on whether a name that contains slashes (if this is not an absoloute path name) should be VATH expanded or not.

(0001874) dwheeler (reporter) 2013-10-10 17:30	Joerg: Great points! Thanks! It seems to me that internal macros ($, etc.) should be updated to use the VPATH entries, with the exception of a target being regenerated as I noted above. Different make implementation may not set them up uniformly today, but those look like bugs, not intended features. It's hard to imagine why "make" should find some prerequisites (for example) using VPATH, yet never update $?. Any name that's not absolute should be expanded, in my view. Here's a revised description, given those comments: In the EXTENDED DESCRIPTION, after "A target is considered out-of-date if it is older than any of its prerequisites or if it does not exist." add the following sentence: "If a target or prerequisite with a relative pathname does not exist, it is not listed in .PHONY, and VPATH is set, then the make utility will search for the prerequisite or target using the list of colon-separated directories listed in VPATH as if they were the current directory, and use that file if found." After the sentence, "After make has ensured that all of the prerequisites of a target are up-to-date and if the target is out-of-date, the commands associated with the target entry shall be executed." add the following sentence: "If the target is out-of-date but was found using VPATH, the target's pathname (as reported in $@ and $) should be given as if it was not found using VPATH, but the target MAY use the pathname as found via VPATH. The pathnames of prerequisites found using VPATH must prepend the VPATH directory chosen (as reported in $? and $<)" An interesting discussion about current VPATH differences is here: http://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/VPATH-and-Make.html#VPATH-and-Make [^] These variances are exactly the kind of problem that a standard should try to resolve :-).

(0001903) dwheeler (reporter) 2013-10-14 21:55	P.S.: Another "make" implementation with VPATH support is AT&T nmake, described here: http://web.archive.org/web/20130605191027/http://www2.research.att.com/~gsf/nmake/nmake.html [^]

(0001904) shware_systems (reporter) 2013-10-15 00:59	If I recall properly, it was noted in some discussion about VPATH that while its syntax and behavior in various environments might be well defined, makefiles could not presume that any fixed paths in one environment had the same purpose in another environment. So, all use of VPATH had to be considered implementation specific as an administrative matter beyond the scope of the standard. This is because there exists the possibility that a generated dependency or target might overwrite files from a separate package with the same names, especially where make is used to customize application scripts. This differs from INCLUDE, LIB, and PATH specs as they are used for finding files generally treated as marked read or execute only, not files expected to stay as writable. The current language is geared towards, as I see it, that make will write generated dependencies or targets either to the source code directory or the directory the makefile is in, or a sibling or sub-dir of those directories for out-of-tree type builds, and these have a common parent directory that doesn't conflict with any implementation specific dirs to avoid those kinds of accidental overwrites. Installation of final target files from there is also considered an administrative function that takes into account how the implementation wants things organized, not have the standard impose an organization on all implementations.

(0001910) dwheeler (reporter) 2013-10-15 18:28	Yes, the current language is focused more on dealing with dependencies and targets that are either in the source code directory or descended from the current directory. So the variances in paths are not a problem. Most of the VPATH builds I see are where there are multiple "build" subdirectories for different environments, and the source directory is some variation of "../". So in short, environment variation isn't a problem for standardization.

(0001916) joerg (reporter) 2013-10-16 10:50	This is why I mentioned that we first need to identify what is different with different make implementatiins. It does not help at all to standardize on VPATH and to do this in a way that does not allow to use this feature with different make implementation. When I last testes BSD make (aprox. in y2000), I could e.g. not use the make from FreeBSD for the schily makefile system because pattern macro expansions have not been implemented in a compatible way and because path expansions for VPATH have been done in an incompatible way to other make implementation. BTW: IIRC, gmake also has some minor deviations from UNIX make implementation with VPATH support here. BTW: smake implements VPATH as a compatibility courtesy only as there is a more powerful mechanism since 1982: .SEARCHLIST: holds a list of src-dir obbj-dir pairs that is used for searching. .OBJSEARCH: controls the search method in that list if present. Where value may be one of src, obj, all. causes smake to look for left-hand-sides of rules in only the source- directories, the object-directories or both. The default value for .OBJSEARCH: is: all. That causes smake to search for results in object-directories and source-directories. If no .SEARCHLIST target exists, but a VPATH= macro is found, smake transforms the content of the VPATH= macro into a form suitable for .SEARCHLIST:. This is done by putting each VPATH= entry twice into the .SEARCHLIST:.

(0001918) dwheeler (reporter) 2013-10-16 12:51	"It does not help at all to standardize on VPATH and to do this in a way that does not allow to use this feature with different make implementation." I agree. So what needs fixing? "BTW: smake implements VPATH as a compatibility courtesy only as there is a more powerful mechanism since 1982:..." That's my point. Lots of people implement VPATH, because users expect it. If there's a better mechanism that "make" implementers will agree to implement, that's fantastic. But if we can't gain agreement on a "better" mechanism, then let's get agreement on mechanisms that users use and implementations generally support. If you'd like to propose .SEARCHLIST and .OBJSEARCH list, that's fine. But as you noted, you had to also implement VPATH because users expect it. So, sounds like it's time to standardize it, so that users have SOME useful mechanism.

(0001926) joerg (reporter) 2013-10-17 10:36 edited on: 2013-10-17 10:37	In order to create a standard for this feature, we need to have a complete description of the way it should work. Before we have this, we should not standardize it. If you did ask me 10+ years before, I could answer these questions, but now I only remembered that gmake causes problems with the feature and BSD make is so different, that it is impossible to find a common denominator. What I have in mind: BSD make expands dynamic macros to some unexpected values if the target name contains slashes. gmake does not expand all dynamic macros the same way. For this reason, I believe that we need to define exactly how VPATH should work and how dynamic macros should be expanded in case the prerequisite search did use entries in VPATH. I am nowever not sure anymore and it may be that the problem was path expansion for named prerequisites that have been located via VPATH. It also may be that my problem was that there is no workaround that is usable for gmake and BSD make at the same time.

(0001927) dwheeler (reporter) 2013-10-17 14:49	Are you sure that those are the differences? Can you point us to any more info? I know there's a difference between BSD and GNU make's VPATH implementation in they determine what the target "really" is, if it was found using VPATH. Perhaps that's what you're remembering. It turns out that the GNU make documentation specifically notes this different algorithm and documents it: http://www.gnu.org/software/make/manual/make.html#Search-Algorithm [^] In typical use cases I've seen, the semantic differences between the BSD and GNU make approaches actually don't actually matter. Typically if you find a target using VPATH it would be up-to-date regardless. Even if it isn't, it doesn't matter for the build where the target is put, since that result will be used either way for that and all future builds. I think the GNU folks have a point that their algorithm is more likely to produce the "expected" results, but I don't see the need to mandate it. Thus, I think POSIX can leave that aspect unspecified and just move on. In my proposed action, I specifically stated that "the target MAY use the name as found via VPATH". That's the point of contention between the GNU and BSD makes, and my proposed text permits either behavior. At least, that was the intention :-).

(0001935) ranjit (reporter) 2013-10-19 12:52	I think the problem joerg is alluding to is that where there are two variant definitions, either the Committee picks the better one (such as sockets from BSD, and mmap from SVR4) or it defines a new function/interface where there is no clear consensus on what is best; examples would be anything starting posix_, pthread_ and so on; in this context ::= should make the situation clear. It is unusual for an operator to be defined, since most languages were defined with a fairly complete set. make is different: as its author said shortly after, he wrote it as he did because he hadn't learnt yacc at that point. It's not his fault no-one paid a blind bit of notice. In this context, AIUI GNU consider their algorithm superior since it enables read-only source trees to remain read-only as it never even attempts to write therein. This is particularly useful for nfs builds, as well as for centralised patching and developer/automation purposes. Unless the alternative has some compelling advantage no-one has written up yet, or I am simply ignorant thereto, that would seem to indicate a clear end-user benefit to me, and so gets my "vote" for plain VPATH until such time as said compelling advantage is presented, and deemed overwhelming. I do not agree that make is by definition constrained to any directory: however a portable application in this context is a set of sources delivered to be built via a [mM]akefile. Thus all paths within the source cpio/pax archive MUST be portable for conformant usage, afaict. As such, can we at least agree that wherever the sources are read from, the main makefile is typically used to build a tree of objects, starting from a parent which is the end-user's responsibility. Below that path, all paths SHOULD be portable, unless the user or implementation overrides them, in which case it's out of scope.

(0001969) msbrown (manager) 2013-11-07 17:13 edited on: 2013-11-07 17:15	Due to lack of consensus on the part of implementers on the particulars of how VPATH works, this change is rejected. If the developers of various versions of make agree on a new feature that will be implemented in all versions of make, we will certainly consider a proposal to standardize that new feature.

Issue History
Date Modified	Username	Field	Change
2013-10-10 16:16	dwheeler	New Issue
2013-10-10 16:16	dwheeler	Status	New => Under Review
2013-10-10 16:16	dwheeler	Assigned To	=> ajosey
2013-10-10 16:16	dwheeler	Name	=> David A. Wheeler
2013-10-10 16:16	dwheeler	Section	=> make
2013-10-10 16:16	dwheeler	Page Number	=> 2911
2013-10-10 16:16	dwheeler	Line Number	=> 95628-95636
2013-10-10 16:38	joerg	Note Added: 0001872
2013-10-10 16:59	joerg	Note Edited: 0001872
2013-10-10 17:30	dwheeler	Note Added: 0001874
2013-10-14 21:55	dwheeler	Note Added: 0001903
2013-10-15 00:59	shware_systems	Note Added: 0001904
2013-10-15 18:28	dwheeler	Note Added: 0001910
2013-10-16 10:50	joerg	Note Added: 0001916
2013-10-16 12:51	dwheeler	Note Added: 0001918
2013-10-17 10:36	joerg	Note Added: 0001926
2013-10-17 10:37	joerg	Note Edited: 0001926
2013-10-17 14:49	dwheeler	Note Added: 0001927
2013-10-19 12:52	ranjit	Note Added: 0001935
2013-11-07 17:13	msbrown	Interp Status	=> ---
2013-11-07 17:13	msbrown	Note Added: 0001969
2013-11-07 17:13	msbrown	Status	Under Review => Closed
2013-11-07 17:13	msbrown	Resolution	Open => Rejected
2013-11-07 17:15	msbrown	Note Edited: 0001969

Relationships

Aardvark Mark IV