I want to define a few similar rules, so I decided to try to define them inside of a function:
COMPILED_JS_FILES=$(COMPILED_JS_FILES) $(foreach m,$(notdir $(wildcard src/$(1)/*.$(2))),$(TARGET_DIR)/$(1)/$(m))
$(TARGET_DIR)/$(1)/%.$(2) : src/$(1)/%.$(2)
$(CAT) $< > $#
endef
$(eval $(call COMPILE_JS,modules,js))
$(eval $(call COMPILE_JS,modules,jsm))
$(eval $(call COMPILE_JS,models,js))
$(eval $(call COMPILE_JS,firefox,js))
$(eval $(call COMPILE_JS,helpers,js))
However, the $< and $# variables inside the rule evaluate to empty strings, presumably because they are being defined by the function that is running them, as opposed to being saved until the rule is evaluated.
I am curious about the answer to this question, but also into other reasonable solutions to this problem (other than restructuring the directory structure).
I actually figured out the answer (in this case, I don't know about in general) - $$< and $$#. Furthermore $$(CAT) will delay the expansion of $(CAT) until the evaluation of the rule.
This is an old problem with makefiles. There really isn't a good, general way to get around it within Make, because you are right: the $< and $# just are not available when you need them.
When I have a problem like this, I tend to write a support script outside Make. However, GNU Make does provide the .SECONDEXPANSION target, an obscure hack intended to make some tasks like yours possible to complete. You might investigate it, though my own experience is that .SECONDEXPANSION is not always a clean solution.
Related
I am trying to build a macro in Makefile which I can expand once but can still work as a macro after being expanded. This is useful to me as the first level expansion will fill in recursive parameters that won't last. Here's an example:
all: aperiod
TGT = hello
hello.TGT = world
world.TGT = period
define CREATE_TARGET
.SECONDARY: $(1)
$(3)$(1): $(4)$(2)
#echo $$$$(#)
$(foreach t,$($(1).TGT),$(call CREATE_TARGET,$(t),$(1),$$(1),$$(1)))
endef
define CREATE
$(call CREATE_TARGET,$(TGT),,$$(1),)
endef
CREATE_EXP := $(call CREATE)
TGT :=
$(eval $(call CREATE_EXP,a))
Error when running make:
make: *** No rule to make target aperiod', needed byall'. Stop.
TGT contains a changing set of values. I want CREATE_EXP to contain the full expanded creation method which accepts a parameter to give prefixes to the targets.
So optimally, I can call make aperiod and get hello world period, or call make bperiod after $(eval $(call CREATE_EXP,b)) and get the same thing
This is a highly reduced test case!
The value of CREATE_EXP is correct, but won't work for me as a macro anymore.
$(info $(value CREATE_EXP))
.SECONDARY: hello
$(1)hello:
#echo $$(#)
.SECONDARY: world
$(1)world: $(1)hello
#echo $$(#)
.SECONDARY: period
$(1)period: $(1)world
#echo $$(#)
I would like to know why Make behaves this way, as well as if there is a better way to accomplish the general gist of what I am trying to do.
EDIT: I found a solution to accomplish this, although I am still curious as to whether a call to $(call) can create a macro that still needs expansion.
define CREATE
define CREATE_EXP
$(call CREATE_TARGET,$(TGT),,$$(1),)
endef
endef
Use $(eval $(call CREATE))
The first time through, make will expand the variables inside. This allows for the recursive expansion as well as the creation of a function macro.
I would have to think more deeply about "a better way" and really understand what you're trying to do, but to answer "why make behaves this way": you are assigning CREATE_EXP as a simply-expanded variable, with :=:
CREATE_EXP := $(call CREATE)
That information is stored along with the variable and when make expands something like $(CREATE_EXP) it knows that the value of CREATE_EXP has already been expanded and it shouldn't be expanded again. That's the entire point, really, of using :=.
Here's an alternate model that might work for you:
$(foreach 1,a,$(eval $(CREATE_EXP)))
(I haven't tried this). The difference here is that first we set the variable 1 as the foreach variable, then we call eval in that context. Although the $(CREATE_EXP) expands to the value without further expansion, then eval will parse it as a makefile and expand it again, with 1=a set.
Just a note: this:
CREATE_EXP := $(call CREATE)
Is absolutely identical to this:
CREATE_EXP := $(CREATE)
If you pass no arguments to call it's the same as a simple macro expansion.
You might be interested to read the set of blog posts here: http://make.mad-scientist.net/category/metaprogramming/ (start from the oldest first).
The automatic variables created by GNU-Make are quite handy for some scenarios:
.SECONDEXPANSION:
%-processed.md: $$(wildcard $$*.m4) $(wildcard macros/*.m4) %.md
m4 $^ > $#
However this idilic world blows up when somebody tries to add on a dependency somewhere else in the makefile:
.PHONY: force
force: ;
mybook-processed.md: force
Now all of a sudden m4 complains that "force" isn't a valid input file. Which of course it's not, but its showing up in the $^ variable because it got appended to the list of prerequisites.
Is there a way to access only the prerequisites defined in the immediate recipe definition while ignoring other ones?
No it's not possible. That information is not even available in make's internal structures.
Your example solution is not one I'd use though. If your command only accepts certain types of files, I would use filter to ensure that it sees only those types of files:
.SECONDEXPANSION:
%-processed.md: $$(wildcard $$*.m4) $(wildcard macros/*.m4) %.md
m4 $(filter %.m4 %.md,$^) > $#
You could adopt a naming convention for pseudo-targets like force, say MAGIC/force. Then $(filter-out MAGIC/%,$^) -- still a little repetitive, but not that bad.
So far the only way I've found to do this is stop using automatic variables and compose my own. I'm really hoping somebody posites a better answer, but here's my current solution:
.SECONDEXPANSION:
process_prereqs = $(wildcard $1.m4) $(wildcarad macros/*.m4)
%-processed.md: %.md $$(call process_prereqs,$$*)
m4 $(call process_prereqs,$$*) $< > $#
This way only the more stable $< automatic variable is used and the other is more predicable, but the code is somewhat repetitive.
I am using makefiles to manage dependencies in computational experiments. It would very often be useful to have targets with multiple patterns.
For example, I may have some conditions, say A and B, and different folds (i.e. splits of data) of a cross-validation experiment, say 1 through 10.
Currently I just copy and paste the same commands for A and B, and pattern-match on the folds, but if there are more than two conditions it quickly becomes a maintenance problem.
output-%.A: input-%
run A input-$* output-$*.A
output-%.B: input-%
run B input-$* output-$*.B
Is there a better way of doing this in makefile?
If not, what other tool would solve this issue?
One drawback of Make is that it doesn't handle wildcards well. But there is a way to minimize the redundancy in this makefile.
First, let's use automatic variables to clean up the rules a little bit:
output-%.A: input-%
run A $< $#
output-%.B: input-%
run B $< $#
Then we define a rule template, use call to evaluate it for A and for B, and eval to interpret those things as actual rules:
define fold-rule
output-%.$(1): input-%
#echo run $(1) $$< $$# # <-- note the doubled '$'
endef
$(eval $(call fold-rule,A))
$(eval $(call fold-rule,B))
Finally we put the conditions in a variable, and iterate using foreach:
CONDITIONS := A B
define fold-rule
output-%.$(1): input-%
#echo run $(1) $$< $$#
endef
$(foreach x, $(CONDITIONS), $(eval $(call fold-rule,$(x))))
Now when you want to add a new condition, just modify the first line.
It isn't pretty, but it works.
Using a few features of makepp, like using variable as macro-functions, and early evalution with $[] it is easy:
define runner
output-%.$1: input-%
run $1 $$(input) $$(output)
enddef
$[runner A]
$[runner B]
$[runner C]
If you want to create a lot of rules, this may still be too verbose. Problem with the foreach function is that it joins the results with a space. So to separate the rules, you can put a dummy empty expression the last line:
define runner
output-%.$1: input-%
run $1 $$(input) $$(output)
$()
enddef
$[foreach type,D E F,$[runner $[type]]]
It appears that $$< in secondary prerequisite expansion for implicit rules, evaluates to the target, not, as I would expect from the manual, the first prerequisite already assigned. What am I missing?
> cat Makefile
.SECONDEXPANSION:
foobar%: prereq% $$<1
#echo prereqs: $^
touch $#
>touch foobar01 prereq0 prereq01
>make foobar0
prereqs: prereq0 foobar01
touch foobar0
From the manual, the prerequisites should be prereq0 prereq01
I believe this is a real bug... or something. See https://savannah.gnu.org/bugs/index.php?28456
This is tricky because at the time make is performing the second expansion it doesn't know what the value of $< is yet. It would need to go through the prerequisite list and, rather than expanding everything at once, do the expansion word by word to ensure that the automatic variables come into existence as soon as possible.
Do-able, but not trivial. Anyway. It doesn't work correctly now obviously.
I have to convert a set of file (let's say format fa) into another format (fb) by a command (fa2fb). Each target fb depends only on one fa file.
Data structure is in a format like this:
source:
./DATA/L1/fa/L1.fa
./DATA/L2/fa/L2.fa
...
./DATA/Ln/fa/Ln.fa
target:
./DATA/L1/fb/L1.fb
./DATA/L2/fb/L2.fb
...
./DATA/Ln/fb/Ln.fb
How can I implement it with make?
I have tried this but of course it did not work:
./DATA/%/fb/%.fb : ./DATA/%/fa/%.fb
#fa2fb $< $#
Is there any simple solution without changing the data directories?
Many thanks!
Use secondary expansion and the subst function to create a rule where the prerequisites are constructed as a more complex function of the target name:
.SECONDEXPANSION:
DATA/%.fb: $$(subst fb,fa,$$#)
#fa2fb $< $#
Note that this approach assumes that fb will not occur anywhere else in the filename (which holds true if all of your filenames are of the form DATA/Ln/fb/Ln.fb, for some integer n).
This may be the sloppiest makefile I have ever written.
define template
$(2) : $(1)
echo hi
endef
sources=DATA/L1/fa/L1.fa DATA/L2/fa/L2.fa
$(foreach source,$(sources),$(eval $(call template,$(source),$(subst /fa/,/fb/,$(subst .fa,.fb,$(source))))))
The idea is to define a macro to generate your rules, then use foreach and eval+call to invoke it once for each source. The source is the first argument to the call, so it becomes $(1) in the macro. The second argument is just the transformation from a source file name to a destination file name; it becomes $(2) in the macro.
Replace echo hi with your own rule and you should be good to go. And be sure to write a nice big clear comment or someday someone will surely show up at your door with a baseball bat.
This is basically the same as Nemo's answer. I just tried to make the foreach call a bit more readable, by creating a list of modules, containing simply L1 L2 ... Ln, instead of the list of full source names.
MODULES := $(notdir $(wildcard ./DATA/L*))
define rule
./DATA/$(1)/fb/$(1).fb: ./DATA/$(1)/fa/$(1).fa
#fa2fb $< $#
endef
$(foreach module, $(MODULES), $(eval $(call rule,$(module))))