0000336: Add recursive variable indirection in makefiles

Notes
(0000630) nick (manager) 2010-12-02 17:02 edited on: 2022-10-06 15:50	OLD Interpretation response ------------------------ The standard clearly states that the results are unspecified, and conforming applications must conform to this. Rationale: ------------- All known implementations have implemented the proposed action, and therefore this interpretation is being considered for TC1. Notes to the Editor (not part of this interpretation): ------------------------------------------------------- See desired action. If this interpretation is approved without objection, the editor is encouraged to include this action in the next published Technical Corrigendum.

(0000634) eblake (manager) 2010-12-09 17:01	During the 9 Dec 2010 meeting, the following points were raised: HP-UX make has a nesting limit of 100, IRIX make has a nesting limit of 1 (that is, IRIX did not support two levels of recursion). IRIX is no longer supported by SGI, so that does not influence the standard, but wording to this effect is needed: In a macro definition, behavior is unspecified if string2 recursively refers the expansion of string1, or if string2 has more than <X> levels of nested macros requiring expansion. where <X> needs some rationale and needs to be somewhere between 1 and 100. Also to be explored is whether all implementations support ${A} and $(A) synonymously (previous testing only focused on nesting via $(A)); and whether nesting via subst1 and subst2 in ${A:subst1=subst2} is universally supported.

(0000671) dwheeler (reporter) 2011-02-08 20:06	During the 9 Dec 2010 meeting, it was noted that HP-UX has a nesting limit of 100. No other nesting limits have been identified to date in widely-used implementations (there may be one, but I don't know of one, and no one else has identified any other than the now-abandoned IRIX). Thus, I propose adding this text to deal with limited levels of nesting, with a limit based on the largest limit of a currently-supported implementation: "In a macro definition, behavior is unspecified if string2 recursively refers the expansion of string1, or if string2 has more than 100 levels of nested macros requiring expansion." The previous note asked for some additional information. It said, "Also to be explored is whether all implementations support ${A} and $(A) synonymously (previous testing only focused on nesting via $(A)); and whether nesting via subst1 and subst2 in ${A:subst1=subst2} is universally supported". I don't know about "all implementations" and "universally supported", but I did test GNU make and FreeBSD's make, and in both cases (1) ${A} is synonymous with $(A), and (2) substitutions work as expected. So I don't think we need to say anything special about this at all. FYI, the test case I used was as follows: x=y y=z oldsuffix=.o newsuffix=.c mylist=a.o b.o whichlist=$(mylist) all: # should print 4 lines of "z": @echo ${${x}} @echo $($(x)) @echo ${$(x)} @echo $(${x}) # Check substitution, should print 2 lines of "a.c b.c" @echo ${whichlist:${oldsuffix}=${newsuffix}} @echo $(whichlist:$(oldsuffix)=$(newsuffix)) Again, this worked on both GNU make and FreeBSD make. That's not all "makes" but it's encouraging.

(0000999) eblake (manager) 2011-11-01 14:05	An interesting side note: NeXTSTEP 3.3 (now long obsolete) supports nesting only if the outer and inner macros alternate in bracketing operators (that is, ${$()} or $(${}) work, but $($()) fails): https://lists.gnu.org/archive/html/bug-automake/2011-11/msg00000.html [^] But since NeXTSTEP is no longer actively maintained and is not striving for POSIX compliance, this should not be used as a reason to block any proposed solution to this bug report.

(0001021) dwheeler (reporter) 2011-11-16 18:40	Given the comments above, here is the revised proposal: Lines 95801-95802 and 95807-95808 both currently contain: "If string1 in a macro expansion contains a macro expansion, the results are unspecified." In both cases, replace them with: "If string1 in a macro expansion contains a macro expansion, that inner macro expansion is recursively expanded before use. Systems shall support at least 100 levels of indirection." Rationale: This construct is useful in the same way that associative arrays are useful in many other languages (e.g., allowing for easy selection between sets of values, such as selecting different file sets for different platforms). It is also widely implemented. Regarding the limit of 100, the only system that might strive for future POSIX compliance, and has a limit at all, is HP-UX. Its maximum is 100. There is no particular rationale for a smaller number, and small values could impede portability in complex situations. Therefore, 100 is used here. The NeXTSTEP 3.3 and SGI systems will not meet this new requirement, but both are obsolete and not striving for POSIX compliance anyway. "100 ought to be enough for anyone." :-).

(0001873) dwheeler (reporter) 2013-10-10 16:40	Note that this construct can also implement simple portable conditionals for macro/variable values (e.g., see http://gallium.inria.fr/blog/portable-conditionals-in-makefiles/). [^] It can't replace conditionals for all cases, but for some cases it's adequate. Note that this is in addition to its utility in implementing associative arrays, and the fact that most makes already do this. So, that's yet another reason to add recursive variable indirection to the standard's specification of make.

(0004698) Love4Boobies (reporter) 2019-12-22 15:12	Is there an update on this ticket? It has been under review for almost 10 years. Comment #0001021 seems to provide a sensible solution.

(0004699) shware_systems (reporter) 2019-12-23 01:02	While Note 1021 provides a direction aimed at taking current practice into account, a limit that may vary between platforms, such as 100 or larger, requires additional support to be considered portable, that I see; and no current platform implements additions in any form, that might let an application adjust to a specific limit, so a proper proposal for Issue 8 can be written. Even with such a proposal, other platforms that would need modifying haven't indicated they'd be ok with adopting those changes. Unti this happens this will probably stay in limbo.

(0004700) Love4Boobies (reporter) 2019-12-23 12:38	It seems to me like leaving what happens past 100 levels of indirection as unspecified behavior is acceptable for two reasons: 1. A mechanism by which makefiles can detect the supported levels of indirection can be added at a later point, if required. 2. No one in the real world actually uses so many indirections. They might use a few. In the end, it looks like a non-problem blocking an actual problem from being solved.

(0004701) shware_systems (reporter) 2019-12-23 14:23	1. Various mechanisms already exist; this support simply hasn't been added to any of them by any platform. Because it is simple enough, to add it or invent something equally valid, it is not unreasonable to require it. 2. That's no one you know of actually uses that many. I'm not going to preclude use cases that might benefit from larger values, if the environment is capable of supporting them, just because I haven't thought of one yet. I can look up a number of reports where recursive algorithms fail because a compiler author presumed X amount of stack space was sufficient for all applications, and use cases happened where this presumption proved false.

(0004702) Love4Boobies (reporter) 2019-12-23 15:33	What would such a mechanism look like? An environment variable? It can't be something internal to make because, without adding a preprocessor to it (which is undoubtedly silly and overcomplicated), there is no way the makefile containing something like $($($(...))) could decide to do something else instead.

(0004706) psmith (developer) 2020-01-06 20:16	Note #1021 is fine and should be adopted. As mentioned it has virtually universal support among existing implementations. I disagree with both note #4699 and note #4701. First, the mere fact that "this support hasn't been added to any of them by any platform" is sufficient by itself to make any such extra feature ineligible for standardization and I see no point in delaying this capability until such a mechanism is invented and some number of existing implementations support it. Second, I don't see any point in such a mechanism: even if it were available who would use it, for what purpose, and how? There are no "test" operators in make, outside of those provided in a recipe. In what situations would someone want to write a recipe that would detect this limit? If you write a makefile with >100 recursive expansions and you run it on a system that doesn't support it, it will fail; how is that different than writing an explicit test? POSIX make doesn't have any way to dynamically decide how deep a recursion should go so there's no way to "work around" this limit in a compliant makefile.

(0005988) geoffclare (manager) 2022-10-06 15:20	On D2.1 page 2943 line 98700 section make, change: Macro expansions in string2 of macro definition lines shall be performed according to the form used to define the macro. Immediate-expansion forms (including appending to an existing immediate-expansion macro) shall be expanded in the macro definition line, and shall not be re-expanded when used in a rule or command. Delayed-expansion forms (including appending to an existing delayed-expansion macro, and conditional assignment to a macro not previously existing) shall be expanded when the macro identified by string1 is expanded in a rule or command, and the result of the expansion shall be scanned for further macros. to: Macro expansions in string2 of macro definition lines shall be performed according to the form of macro definition used. In immediate-expansion forms (including appending to an existing immediate-expansion macro), they shall be expanded in the macro definition line and the result of the expansion shall not be scanned for further macros when the macro identified by string1 is expanded. In delayed-expansion forms (including appending to an existing delayed-expansion macro, and conditional assignment to a macro not previously existing), they shall not be expanded in the macro definition line; they shall be expanded when the macro identified by string1 is expanded, and the result of the expansion shall be scanned for further macros. Implementations shall support at least 100 levels of indirection. On D2.1 page 2943 line 98720, 98726 section make, change: If string1 in a macro expansion contains a macro expansion, the results are unspecified. to: If string1 in a macro expansion contains a macro expansion, that inner macro expansion shall be performed as described above and the result substituted into string1 to produce the macro name used for the outer macro expansion.

Issue History
Date Modified	Username	Field	Change
2010-10-16 18:13	dwheeler	New Issue
2010-10-16 18:13	dwheeler	Status	New => Under Review
2010-10-16 18:13	dwheeler	Assigned To	=> ajosey
2010-10-16 18:13	dwheeler	Name	=> David A. Wheeler
2010-10-16 18:13	dwheeler	Section	=> make
2010-10-16 18:13	dwheeler	Page Number	=> 2915
2010-10-16 18:13	dwheeler	Line Number	=> 95801-95808
2010-12-02 17:02	nick	Note Added: 0000630
2010-12-02 17:11	nick	Interp Status	=> Pending
2010-12-02 17:11	nick	Final Accepted Text	=> See Note: 0000630
2010-12-02 17:11	nick	Status	Under Review => Interpretation Required
2010-12-02 17:11	nick	Resolution	Open => Accepted
2010-12-09 17:01	eblake	Note Added: 0000634
2010-12-09 17:03	eblake	Interp Status	Pending => ---
2010-12-09 17:03	eblake	Status	Interpretation Required => Under Review
2010-12-09 17:03	eblake	Resolution	Accepted => Reopened
2010-12-09 17:03	eblake	Final Accepted Text	See Note: 0000630 =>
2011-02-08 20:06	dwheeler	Note Added: 0000671
2011-11-01 14:05	eblake	Note Added: 0000999
2011-11-16 18:40	dwheeler	Note Added: 0001021
2013-10-10 16:40	dwheeler	Note Added: 0001873
2019-12-22 15:12	Love4Boobies	Note Added: 0004698
2019-12-23 01:02	shware_systems	Note Added: 0004699
2019-12-23 12:38	Love4Boobies	Note Added: 0004700
2019-12-23 14:23	shware_systems	Note Added: 0004701
2019-12-23 15:33	Love4Boobies	Note Added: 0004702
2020-01-06 20:16	psmith	Note Added: 0004706
2020-01-15 08:13	Love4Boobies	Issue Monitored: Love4Boobies
2022-10-06 15:20	geoffclare	Note Added: 0005988
2022-10-06 15:21	geoffclare	Final Accepted Text	=> Note: 0005988
2022-10-06 15:21	geoffclare	Status	Under Review => Resolved
2022-10-06 15:21	geoffclare	Resolution	Reopened => Accepted As Marked
2022-10-06 15:21	geoffclare	Tag Attached: issue8
2022-10-06 15:50	geoffclare	Note Edited: 0000630
2022-11-01 14:58	geoffclare	Status	Resolved => Applied
2024-06-11 08:53	agadmin	Status	Applied => Closed

Relationships

Aardvark Mark IV