I'm trying to improve my Makefile by using define as functions, but I have a little problem with Make functions in defines. Let me show you a demo
OBJ=obj/main.o \
obj/prompt.o
define buildMessage
src="$(patsubst $(OBJ_PREFIX)/%.o,%.c,$(1))"; \
obj="$(patsubst $(OBJ_PREFIX)/%.o,%.o,$(1))"; \
echo "$$src => $$obj"
endef
$(OBJ):
$(call buildMessage, "$#")
$(CC) -o $# -c $(patsubst $(OBJ_PREFIX)/%.o, $(SRC_PREFIX)/%.c, $#) $(CFLAGS)
So, yeah, I want the define to print the build step with fancy style, but it seems that the patsubst isn't applied in the define. It will output
obj/main.o => obj/main.o
obj/prompt.o => obj/prompt.o
It is exactly like if the patsubst was handled but without changing nor matching anything. Any suggestions?
Thank you for your time
Alex
I'm assuming that, although you didn't show it, the value of OBJ_PREFIX is obj.
Make doesn't care anything about quotes. When you write:
$(call buildMessage, "$#")
the value of $1 will be, literally, "obj/main.o" including the quotes. That string doesn't match the pattern obj/%.o and so the pattern doesn't apply, leaving the original string untouched.
You should write this:
$(call buildMessage,$#)
Related
I have a Makefile with user-specified input files in the variable INPUT_FILES.
For each input file, I need to create an input file prime.
Some notes:
Each input file can have an arbitrary file location
It is reasonable to assume there aren't duplicate filenames
Each output file needs to go into $(OUTPUT_DIR)
My basic strategy has been to generate the set of targets based INPUT_FILES and then try to determine which input file is the actual dependency of the target.
A few variations I've tried:
# Create a list of targets
OUTPUT_FILES = $(foreach file,$(notdir $(INPUT_FILES)),$(OUTPUT_DIR)/$(file))
# This doesn't work, because all input files are dependencies of each output file
$(OUTPUT_FILES): $(INPUT FILES)
program --input $^ --output $#
# This doesn't work because $# hasn't been resolved yet
$(OUTPUT_FILES): $(filter,$(notdir $#),$(INPUT FILES))
program --input $^ --output $#
# This doesn't work, I think because $# is evaluated too late
.SECONDEXPANSION:
$(OUTPUT_FILES): $(filter,$(notdir $$#),$(INPUT FILES))
program --input $^ --output $#
# This doesn't work either
.SECONDEXPANSION:
$(OUTPUT_FILES): $$(filter,$(notdir $#),$(INPUT FILES))
program --input $^ --output $#
I've looked into static pattern rules as well, but I'm not sure if it can help with what I need.
In your case .SECONDEXPANSION: works because you can use make functions (filter) to compute the prerequisite of each output file. In other circumstances it could be impossible. But there is another GNU make feature that can be used in cases like yours: if you use GNU make you can programmatically instantiate make statements using foreach-eval-call. Just remember that the macro that is used as the statements pattern gets expanded twice, reason why you must double some $ signs (more on this later):
OUTPUT_DIR := dir
OUTPUT_FILES := $(addprefix $(OUTPUT_DIR)/,$(notdir $(INPUT_FILES)))
.PHONY: all
all: $(OUTPUT_FILES)
# The macro used as statements pattern where $(1) is the input file
define MY_RULE
$(1)-output-file := $(OUTPUT_DIR)/$$(notdir $(1))
$$($(1)-output-file): $(1)
#echo program --input $$^ --output $$#
endef
$(foreach i,$(INPUT_FILES),$(eval $(call MY_RULE,$(i))))
Demo:
$ mkdir -p a/a b
$ touch a/a/a b/b c
$ make INPUT_FILES="a/a/a b/b c"
program --input a/a/a --output dir/a
program --input b/b --output dir/b
program --input c --output dir/c
Explanation:
When make parses the Makefile it expands $(foreach ...): it iterates over all words of $(INPUT_FILES), for each it assigns the word to variable i and expands $(eval $(call MY_RULE,$(i))) in this context. So for word foo/bar/baz it expands $(eval $(call MY_RULE,$(i))) with i = foo/bar/baz.
$(eval PARAMETER) expands PARAMETER and instantiates the result as new make statements. So, for foo/bar/baz, make expands $(call MY_RULE,$(i)) with i = foo/bar/baz and considers the result as regular make statements. The expansion of $(eval ...) has no other effect, the result is the empty string. This is why in our case $(foreach ...) expands as the empty string. But it does something: create new make statements dynamically for each input file.
$(call NAME,PARAMETER) expands PARAMETER, assigns it to temporary variable 1 and expands the value of make variable NAME in this context. So, $(call MY_RULE,$(i)) with i = foo/bar/baz expands as the expanded value of variable MY_RULE with $(1) = foo/bar/baz:
foo/bar/baz-output-file := dir/$(notdir foo/bar/baz)
$(foo/bar/baz-output-file): foo/bar/baz
#echo program --input $^ --output $#
which is what is instantiated by eval as new make statements. Note that we had a first expansion here and the $$ became $. Note also that call can have more parameters: $(call NAME,P1,P2) will do the same with $(1) = P1 and $(2) = P2.
When make parses these new statements (as any other statements) it expands them (second expansion) and finally adds the following to its list of variables:
foo/bar/baz-output-file := dir/baz
and the following to its list of rules:
dir/baz: foo/bar/baz
#echo program --input $^ --output $#
This may look complicated but it is not if you remember that the make statements added by eval are expanded twice. First when $(eval ...) is parsed and expanded by make, and a second time when make parses and expands the added statements. This is why you frequently need to escape the first of these two expansions in your macro definition by using $$ instead of $.
And it is so powerful that it is good to know.
When asking for help please provide some kind of actual example names so we can understand more clearly what you have. It also helps us use terminology which is not confusing.
You really want to use $< in your recipes, not $^, I expect.
IF your "input files" are truly input-only (that is, they are not themselves generated by other make rules) then you can easily solve this problem with VPATH.
Just use this:
VPATH := $(sort $(dir $(INPUT_FILES)))
$(OUTPUT_DIR)/% : %
program --input $< --output $#
I finally found a permutation that works - I think the problem was forgetting that filter requires a % for matching patterns. The rule is:
.SECONDEXPANSION:
$(OUTPUT_FILES): $$(filter %$$(#F),$(INPUT_FILES))
program --input $^ --output $#
I also realized I can use #F (equivalent to $$(notdir $$#)) for cleaner syntax.
The rule gets the target's filename on its second expansion ($$(#F)) and then gets the input file (with path) that corresponds to it on second expansion ($$(filter %$$(#F),$(INPUT_FILES))).
Of course, the rule only works if filenames are unique. If someone has a cleaner solution, feel free to post.
I have, for example, the following Makefile to generate PDF files from Markdown files in subdirectories:
FOLDERS = f1 f2 f3
.PHONY: $(FOLDERS)
f1: f1/f1.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
f2: f2/f2.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
f3: f3/f3.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
The expected result is that make f1 requires the existence of f1/f1.md, and generates the resulting PDF as f1/f1.pdf. The same for f2 and f3. This works, but the declarations seem unnecessarily repetitive.
Is there any way to combine these three rules into one, generic rule? That is, without needing to explicitly write out all of the paths to the PDF files or Markdown files, as I may be dynamically adding subfolders and I'd prefer to just change the definition of FOLDERS in the first line. I've googled around and tried a few things, but I feel like either I can't find the right incantation to use, or I'm missing a piece of knowledge about how Makefiles work. Could someone please point me in the right direction?
First, note that there's no good reason to use PHONY targets here, since these rules appear to be building files whose names are known beforehand. Targets like f1/f1.pdf would be much better.
Unfortunately we can't use a pattern rule when the stem (e.g. f1) is repeated in a prerequisite. But a "canned recipe" can do the trick:
define pdf_template
$(1): $(1)/$(1).md
cd $$# && pandoc $$(notdir $$^) -o $$(patsubst %.md,%.pdf,$$(notdir $$^))
endef
$(eval $(call pdf_template,f1))
$(eval $(call pdf_template,f2))
$(eval $(call pdf_template,f3))
(Note how you must escape the $ signs in the template.)
If those $(eval...) lines look too repetitive, you can replace them with a loop:
$(foreach folder,$(FOLDERS),$(eval $(call pdf_template,$(folder))))
EDIT: Come to think of it, there's another way. You can't construct a pattern rule that uses the stem more than once:
$(FOLDERS): %: %/%.md
cd $# && ... this won't work
And you can't use the automatic variables in the prerequisite list, because they aren't yet defined when they're needed:
$(FOLDERS): $#/$#.md
cd $# && ... this won't work either
But you can use them there if you use Secondary Expansion, which causes Make to expand the prereq list a second time:
.SECONDEXPANSION:
$(FOLDERS): $$#/$$#.md
cd $# && ... this works
Again, note the escaped $ symbols.
I have the following makefile I use to generate files from some templates, the generated files have two possible extensions:
%.tex: %.tex*_tpl
./generate $#_tpl -o $#
%.xml: %.xml*_tpl
./generate $#_tpl -o $#
The dependency list will here match things like a.tex_tpl, a.tex-subpart1_tpl, a.tex-subpart2_tpl.
While this works, is there a way to avoid repetition? For example by matching *.{tex,xml} in the rule name and use the whole matched name in the dependency list? Something that would look like that:
%.{tex,xml}: $#_tpl
./generate $< -o $#
(Though I know %.{tex,xml} is not a valid rule name and you can't use $# in the dependency list)
Or any other (cleaner?) way.
Seems to me this does what you are looking for:
#
# I've assumed that files of the form:
#
# a.xml_tpl
# b.tex_tpl
#
# determine what targets you want to build
#
TARGETS:=$(patsubst %_tpl,%,$(wildcard *.xml_tpl *.tex_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $^ -o $#
The key is to use .SECONDEXPANSION to allow $$(wildcard %*_tpl) to be evaluated in a second expansion phase. The double $ is not a typo, by the way; it protects the expression from being evaluated at the time of the first expansion.
If I populate a directory with these files:
a.tex-subpart1_tpl
a.tex_tpl
a.xml-subpart1_tpl
a.xml-subpart2_tpl
a.xml_tpl
and run make -n, I get this on the console:
./generate a.xml_tpl a.xml-subpart1_tpl a.xml-subpart2_tpl -o a.xml
./generate a.tex_tpl a.tex-subpart1_tpl -o a.tex
Why the Second Expansion?
Without the second expansion, you'd have to have $(wildcard %*_tpl) in the dependency because with the $$ the wildcard function would never execute. Instead, make would treat $$(wildcard..) literally as the dependency, which is obviously wrong.
Ok, so $(wildcard %*_tpl) would be evaluated at the time make first runs across that line (this is the "first expansion"). At that time % has no value yet so wildcard would roughly be doing something like what would be ls %*_tpl at the command line.
For reasons of speed, make does not by default give you the opportunity to do any evaluation later than during the first expansion. If you want a later opportunity you have to specify .SECONDEXPANSION, which turns on the second expansion processing. Make still performs the firts expansion as usual. This is why you need to have $$(wildcard: it is transformed to $(wildcard during the first expansion. At the time of the second expansion make sees $(wildcard %*_tpl), replaces % with the actual stem and then executes the wildcard function with the actual stem rather than with a literal %.
Why $(TARGETS) in the Pattern Rule?
The pattern rule could be written:
%: $$(wildcard %*_tpl)
./generate $^ -o $#
without $(TARGETS). However, this rule would do nothing, as it would be a "match-anything rule". Basically, if make took such a rule at face value, then the computation cost would be significant, and most likely it is not the case that the author of the Makefile really means to apply this rule to any file whatsoever. So such a rule comes with restrictions, which in the Makefile here make it useless.
Adding $(TARGETS) makes it into a static pattern rule, which is not a match-anything rule. The addition of $(TARGETS) in front of the target pattern tells make that the rule applies only to these targets, and nothing else.
I have a makefile with multiple targets that are generated by copying a file from outside the working directory.
a.tex : $(wildcard /foo/work1/a.tex)
cp -p $< $#
b.tex : $(wildcard /foo/work2/b.tex)
cp -p $< $#
I use $(wildcard) because sometimes I run Make on systems that do not have access to /foo.
What is the best way to avoid repeating the cp -p $< $# commands for every rule? Some options:
Setting up a %.tex : %.tex rule. This works, but it also applies to targets that aren't specifically indicated so I get lots of warnings like make: Circular a.tex <- a.tex dependency dropped.
Defining a sequence of commands with define. This seems pointless since the command is only one line. So instead of copying cp $< $# to every rule, I'd define a cp-dep sequence and copy cp-dep to every rule.
Defining the command as a variable so that I could do a.tex : $(wildcard /foo/work1/a.tex); $(CP-DEP)
Duplicating the target names as an additional rule. a.tex b.tex : ; cp -p $< $#. Error-prone.
Just copying and pasting. Clunky but effective and easy to understand.
I haven't tested it, but can't you just use a pattern rule without prerequisites, and specify the prerequisite for each target on a separate line?
a.tex: $(wildcard /foo/work1/a.tex)
b.tex: $(wildcard /foo/work2/b.tex)
%.tex:
cp -p $< $#
Btw. doesn't the wildcard function return the empty string when it doesn't find a match, so that $< is empty as well? Wouldn't that give a problem with cp?
I think your copyrule is overkill (and inflexible). If your objection to #eriktous's solution is that it will apply the rule to targets for which you haven't explicitly defined dependencies, that's easy to fix with a static pattern rule:
a.tex: $(wildcard /foo/work1/a.tex)
b.tex: $(wildcard /foo/work2/b.tex)
blue.tex: $(wildcard /some/other/path/green.tex)
TEXES = a.tex b.tex
$(TEXES): %.tex:
cp -p $< $#
(If this solves your problem you should accept eriktous's answer-- this is just a variation on it.)
I ended up doing this:
COPYFILES = /foo/work1/a.tex /foo/work2/b.tex
define copyrule
$(notdir $(1)): $$(wildcard $(1))
cp -p $$< $$#
endef
$(foreach file,$(COPYFILES),$(eval $(call copyrule,$(file))))
The advantage of this method is that I can easily add new files with a minimum of boilerplate text and I can easily copy the rule part of this to a new Makefile. The disadvantages are that I can no longer change the destination filename, and the implementation is rather opaque for people with less makefile experience.
I am trying to exclude main.cpp file from the list of files to be compiled defined by the rule below:
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
ifneq ($(notdir $<), main.cpp)
#$(COMPILE.cpp) $(OUTPUT_OPTION) $<
endif
This 'ifneq' condition always evaluates to true, which is bizarre. What am I doing wrong? Is there a better way to exlude one file from an explicit rule?
Why don't you try using the filter-out text function if you're using GNU Make.
Returns all whitespace-separated words in text that do not match any of the pattern words, removing the words that do match one or more. This is the exact opposite of the filter function.
For example, given:
objects=main1.o foo.o main2.o bar.o
mains=main1.o main2.o
the following generates a list which contains all the object files not in ‘mains’:
$(filter-out $(mains),$(objects))
That isn't the best way to do it, but if you do it along these lines, write it as a shell condition, not using GNU make conditionals:
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#if [ $(notdir $<) != main.cpp ]; \
then $(COMPILE.cpp) $(OUTPUT_OPTION) $<; \
fi
The continuation markers (backslashes) are needed. So are the semicolons. The values prefixed with $ will be expanded by make before the shell is invoked to interpret them. You probably don't want the echo where it is, either. You probably need:
$(TMPDIRPATH)%.o: %.cpp
#if [ $(notdir $<) != main.cpp ]; \
then echo compile $<; \
$(COMPILE.cpp) $(OUTPUT_OPTION) $<; \
fi
The way I would expect to do it is with a list of the files to be compiled. Using any wild card mechanism leads to problems when extra files are added - other tests, or stray files that aren't really part of the system.
The comment says "But the GNU Make Manual says ifneq should work".
The ifneq would work if it were positioned correctly, which means 'not indented as part of the commands associated with a rule'. You could, therefore, write something like (an appallingly bad example, but my brain's on the fritz):
ifneq (${CFLAGS}, -Wall)
CFLAGS += -Wall
endif
file1.o: file1.c
${CC} ${CFLAGS} -c $<
But when the ifneq is indented as in the question, it is just a command that actually isn't found on the system when the make runs the shell to process the command.
The ifneq line is evaluated only once, when make starts up and parses the makefile. In that context, $< is empty.
To get different behavior for each of the targets matched by your pattern rule, you could do something like
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#$(if $(filter main.cpp,$<),$(COMPILE.cpp) $(OUTPUT_OPTION) $<)
It might help you to think of the difference between ifneq and $(if) in a makefile as like the difference between #if and if() in C code.
Taking a step back, though: If you don't want main.cpp to be compiled by this rule, then you probably want to provide an explicit rule with $(TMPDIRPATH)main.o as its target, which will be preferred to the pattern rule always. Or, if you don't want$(TMPDIRPATH)main.o to get made at all, you should be looking for rules that have it on the right sight of the :, and removing it from there.
Make doesn't really have a good way to handle conditionals within a rule. You could put the conditional in the command, but in this case there's a much cleaner way:
$(TMPDIRPATH)main.o:
#echo compile $< (but not really)
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#$(COMPILE.cpp) $(OUTPUT_OPTION) $<
EDIT:
I didn't realize you didn't have a main.cpp. The solution is simple: remove main.cpp as the prerequisite of the main.o rule (I've removed it above). Now the makefile doesn't need it, and won't try to build it.
But you're still running the rule, which means that something is still trying to build main.o, as either an explicit target or a prerequisite of something else. That is a symptom of confusion, which this change to the makefile will not fix. If you tell us more about the situation, maybe we can propose a better solution. What calls for main.o? Do you have a main.o? What target do you specify when you call Make?