In the following makefile for loop, how can I edit the string, that the i variable represents, with a pattern substitution in the middle of the string? In my case, I wish to replace any / character in the string with a _ character.
for i in $(MODULES:%.cpp=%); do \
g++ -c Sources/$$i.cpp -o Build/$$i.o; \
done
For example if MODULES = Directory/File.cpp then the inner line should expand to
g++ -c Sources/Directory/File.cpp -o Build/Directory_File.o
This answer is valid only with GNU make and bash.
Simple bash substitution (${parameter/pattern/string}) in the context of a make recipe (double $):
for i in $(MODULES:%.cpp=%); do \
g++ -c Sources/$$i.cpp -o Build/$${i//\//_}.o; \
done
Warning: this works only if the shell used by make is bash. So, add maybe a:
SHELL := bash
at the beginning of your Makefile.
Explanation:
${i/X/_} expands as the value of variable i in which the first occurrence of X is replaced by _.
${i//X/_} expands as the value of variable i in which all occurrences of X are replaced by _.
In your case X is the / character and it must be escaped (\/): ${i//\//_}.
Note that there is probably a less bash and more make way to do the same. Something like:
SRCS := $(shell find Sources -type f -name *.cpp)
OBJS :=
define OBJ_rule
obj := Build/$$(subst /,_,$$(patsubst Sources/%.cpp,%,$(1))).o
OBJS += $$(obj)
$$(obj): $(1)
g++ -c $$< -o $$#
endef
$(foreach s,$(SRCS),$(eval $(call OBJ_rule,$(s))))
.PHONY: objs
objs: $(OBJS)
Which instantiates one rule per module and should do the same... with the significant advantage that, when you type make objs, only the outdated object files are rebuilt. But it's a bit more tricky.
Related
I have a Makefile with user-specified input files in the variable INPUT_FILES.
For each input file, I need to create an input file prime.
Some notes:
Each input file can have an arbitrary file location
It is reasonable to assume there aren't duplicate filenames
Each output file needs to go into $(OUTPUT_DIR)
My basic strategy has been to generate the set of targets based INPUT_FILES and then try to determine which input file is the actual dependency of the target.
A few variations I've tried:
# Create a list of targets
OUTPUT_FILES = $(foreach file,$(notdir $(INPUT_FILES)),$(OUTPUT_DIR)/$(file))
# This doesn't work, because all input files are dependencies of each output file
$(OUTPUT_FILES): $(INPUT FILES)
program --input $^ --output $#
# This doesn't work because $# hasn't been resolved yet
$(OUTPUT_FILES): $(filter,$(notdir $#),$(INPUT FILES))
program --input $^ --output $#
# This doesn't work, I think because $# is evaluated too late
.SECONDEXPANSION:
$(OUTPUT_FILES): $(filter,$(notdir $$#),$(INPUT FILES))
program --input $^ --output $#
# This doesn't work either
.SECONDEXPANSION:
$(OUTPUT_FILES): $$(filter,$(notdir $#),$(INPUT FILES))
program --input $^ --output $#
I've looked into static pattern rules as well, but I'm not sure if it can help with what I need.
In your case .SECONDEXPANSION: works because you can use make functions (filter) to compute the prerequisite of each output file. In other circumstances it could be impossible. But there is another GNU make feature that can be used in cases like yours: if you use GNU make you can programmatically instantiate make statements using foreach-eval-call. Just remember that the macro that is used as the statements pattern gets expanded twice, reason why you must double some $ signs (more on this later):
OUTPUT_DIR := dir
OUTPUT_FILES := $(addprefix $(OUTPUT_DIR)/,$(notdir $(INPUT_FILES)))
.PHONY: all
all: $(OUTPUT_FILES)
# The macro used as statements pattern where $(1) is the input file
define MY_RULE
$(1)-output-file := $(OUTPUT_DIR)/$$(notdir $(1))
$$($(1)-output-file): $(1)
#echo program --input $$^ --output $$#
endef
$(foreach i,$(INPUT_FILES),$(eval $(call MY_RULE,$(i))))
Demo:
$ mkdir -p a/a b
$ touch a/a/a b/b c
$ make INPUT_FILES="a/a/a b/b c"
program --input a/a/a --output dir/a
program --input b/b --output dir/b
program --input c --output dir/c
Explanation:
When make parses the Makefile it expands $(foreach ...): it iterates over all words of $(INPUT_FILES), for each it assigns the word to variable i and expands $(eval $(call MY_RULE,$(i))) in this context. So for word foo/bar/baz it expands $(eval $(call MY_RULE,$(i))) with i = foo/bar/baz.
$(eval PARAMETER) expands PARAMETER and instantiates the result as new make statements. So, for foo/bar/baz, make expands $(call MY_RULE,$(i)) with i = foo/bar/baz and considers the result as regular make statements. The expansion of $(eval ...) has no other effect, the result is the empty string. This is why in our case $(foreach ...) expands as the empty string. But it does something: create new make statements dynamically for each input file.
$(call NAME,PARAMETER) expands PARAMETER, assigns it to temporary variable 1 and expands the value of make variable NAME in this context. So, $(call MY_RULE,$(i)) with i = foo/bar/baz expands as the expanded value of variable MY_RULE with $(1) = foo/bar/baz:
foo/bar/baz-output-file := dir/$(notdir foo/bar/baz)
$(foo/bar/baz-output-file): foo/bar/baz
#echo program --input $^ --output $#
which is what is instantiated by eval as new make statements. Note that we had a first expansion here and the $$ became $. Note also that call can have more parameters: $(call NAME,P1,P2) will do the same with $(1) = P1 and $(2) = P2.
When make parses these new statements (as any other statements) it expands them (second expansion) and finally adds the following to its list of variables:
foo/bar/baz-output-file := dir/baz
and the following to its list of rules:
dir/baz: foo/bar/baz
#echo program --input $^ --output $#
This may look complicated but it is not if you remember that the make statements added by eval are expanded twice. First when $(eval ...) is parsed and expanded by make, and a second time when make parses and expands the added statements. This is why you frequently need to escape the first of these two expansions in your macro definition by using $$ instead of $.
And it is so powerful that it is good to know.
When asking for help please provide some kind of actual example names so we can understand more clearly what you have. It also helps us use terminology which is not confusing.
You really want to use $< in your recipes, not $^, I expect.
IF your "input files" are truly input-only (that is, they are not themselves generated by other make rules) then you can easily solve this problem with VPATH.
Just use this:
VPATH := $(sort $(dir $(INPUT_FILES)))
$(OUTPUT_DIR)/% : %
program --input $< --output $#
I finally found a permutation that works - I think the problem was forgetting that filter requires a % for matching patterns. The rule is:
.SECONDEXPANSION:
$(OUTPUT_FILES): $$(filter %$$(#F),$(INPUT_FILES))
program --input $^ --output $#
I also realized I can use #F (equivalent to $$(notdir $$#)) for cleaner syntax.
The rule gets the target's filename on its second expansion ($$(#F)) and then gets the input file (with path) that corresponds to it on second expansion ($$(filter %$$(#F),$(INPUT_FILES))).
Of course, the rule only works if filenames are unique. If someone has a cleaner solution, feel free to post.
Consider following makefile snippet:
CXX := g++
test := sort{1..4} # i.e. sort1 sort2 sort3 sort4
#all: $(test)
$(test): %: %.cpp
$(CXX) -std=c++17 -Wall -O3 -o $# $<
clean:
rm -f $(test)
where sort1, sort2, sort3, and sort4 are expected to be the output (executable) files. However, when executing in shell:make $test or make clean, the wildcard { } seem NOT to expand, and the command can't remove those four files. How can I expand wildcard { } in a makefile?
Additionally, I've noticed a interesting phenomena: first let's use the non-expanded version (before I figure out above problem), with test := sort1 sort2 sort3 sort4 instead of line 3. When there is no target using the variable test (i.e. comment line 5), then if we try: make $test we will memerly compile sort1. However, when we try to use a target with prerequisites on this variable, for example, uncomment line 5, then make $test is equivalent to make all, thus generating four output files. How to explain this?
Thanks in advance for taking time for this trivial question, but it would mean a lot to me ;)
Although make does not expand sequence expressions internally, a shell that does expand them could be run from make; try the examples below.
Running make $test from a shell likely finds the shell variable $test not set and shell variable substitution yields nothing, so the command-line is reduced to only make, which goes about building the first target. When the #all: line is commented-out, make finds sort1 as the first target (provided test is set as expected); after removing the comment from #all, the first target will be all, so then make would handle the provided prerequisites.
CXX := g++
# straight assignment
test := sort1 sort2 sort3 sort4
$(info plain: "$(test)")
# foreach built-in
nset := 1 2 3 4
test := $(foreach n,$(nset),sort$(n))
$(info foreach: "$(test)")
# run a shell for sequence expression
test := $(shell echo sort{1..4})
$(info shell: "$(test)")
.PHONY: all
all: $(test)
$(test): %: %.cpp
$(CXX) -std=c++17 -Wall -O3 -o $# $<
clean:
rm -f $(test)
I have, for example, the following Makefile to generate PDF files from Markdown files in subdirectories:
FOLDERS = f1 f2 f3
.PHONY: $(FOLDERS)
f1: f1/f1.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
f2: f2/f2.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
f3: f3/f3.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
The expected result is that make f1 requires the existence of f1/f1.md, and generates the resulting PDF as f1/f1.pdf. The same for f2 and f3. This works, but the declarations seem unnecessarily repetitive.
Is there any way to combine these three rules into one, generic rule? That is, without needing to explicitly write out all of the paths to the PDF files or Markdown files, as I may be dynamically adding subfolders and I'd prefer to just change the definition of FOLDERS in the first line. I've googled around and tried a few things, but I feel like either I can't find the right incantation to use, or I'm missing a piece of knowledge about how Makefiles work. Could someone please point me in the right direction?
First, note that there's no good reason to use PHONY targets here, since these rules appear to be building files whose names are known beforehand. Targets like f1/f1.pdf would be much better.
Unfortunately we can't use a pattern rule when the stem (e.g. f1) is repeated in a prerequisite. But a "canned recipe" can do the trick:
define pdf_template
$(1): $(1)/$(1).md
cd $$# && pandoc $$(notdir $$^) -o $$(patsubst %.md,%.pdf,$$(notdir $$^))
endef
$(eval $(call pdf_template,f1))
$(eval $(call pdf_template,f2))
$(eval $(call pdf_template,f3))
(Note how you must escape the $ signs in the template.)
If those $(eval...) lines look too repetitive, you can replace them with a loop:
$(foreach folder,$(FOLDERS),$(eval $(call pdf_template,$(folder))))
EDIT: Come to think of it, there's another way. You can't construct a pattern rule that uses the stem more than once:
$(FOLDERS): %: %/%.md
cd $# && ... this won't work
And you can't use the automatic variables in the prerequisite list, because they aren't yet defined when they're needed:
$(FOLDERS): $#/$#.md
cd $# && ... this won't work either
But you can use them there if you use Secondary Expansion, which causes Make to expand the prereq list a second time:
.SECONDEXPANSION:
$(FOLDERS): $$#/$$#.md
cd $# && ... this works
Again, note the escaped $ symbols.
Error : Makefile:12: *** recipe commences before first target. Stop.
My makefile:
objDir := obj
incDir := include
srcDir := src
binDir := bin
files := matrix palindrome encryption
define generateObject
#nasm -f elf32 -o $(objDir)/$(1).o $(srcDir)/$(1).asm
endef
object: $(addprefix $(srcDir)/,$(addsuffix .asm,$(files)))
#echo -n "Generating object files... "
$(foreach file,$(files),$(eval $(call generateObject,$(file))))
#echo "Done"
I read in a post that this could be due to unwanted whitespace/tab but i could not find any.
I tried cat -e -t -v Makefile and the output was :
objDir := obj$
incDir := include$
srcDir := src$
binDir := bin$
files := matrix palindrome encryption$
$
define generateObject$
^I#nasm -f elf32 -o $(objDir)/$(1).o $(srcDir)/$(1).asm$
endef$
$
object: $(addprefix $(srcDir)/,$(addsuffix .asm,$(files)))$
^I#echo -n "Generating object files... "$
^I$(foreach file,$(files),$(eval $(call generateObject,$(file))))$
^I#echo "Done"$
Your problem is use of the eval function. eval is used to parse make constructs, but you're passing it a shell command. Consider this line:
$(foreach file,$(files),$(eval $(call generateObject,$(file))))
Each time through the list you'll call generateObject with a filename. That will expand to a shell command; for example if file is matrix then call will expand to:
^I#nasm -f elf32 -o obj/matrix.o src/matrix.asm
Then you take that text string and pass it to eval which tries to read that as a makefile. Note that the text passed to eval must be a complete and valid makefile in itself; it's like you invoked make recursively and gave it this string as a makefile, except that the result of parsing are applied to the current makefile. You can't give eval just a part of a valid makefile (like one command line in a recipe) and have it insert that into the current makefile. Because that line by itself isn't valid, you get this error.
Instead of running eval on the results you want to concatenate them into one shell command. Try this:
define generateObject
nasm -f elf32 -o $(objDir)/$(1).o $(srcDir)/$(1).asm
endef
object: $(addprefix $(srcDir)/,$(addsuffix .asm,$(files)))
#echo -n "Generating object files... "
#$(foreach file,$(files),$(call generateObject,$(file)) && ) true
#echo "Done"
However, that's really not "the make way". You don't want to build multiple targets within a single rule: that defeats the main point of make which is that it only rebuilds the files that are out of date.
You should write your makefile like this:
object: $(files:%=$(objDir)/%.o)
$(objDir)/%.o : $(srcDir)/%.asm
#nasm -f elf32 -o $# $<
You don't need the generateObject variable, or call, or eval, or even foreach.
Is there a way to pick up the target name using automatic variable.
SOURCES = $(wildcard *.c)
dummytgt: $(OBJ)/tier.o
$(GCC) $(CFLAGS) -c $(filter $#,$(SOURCES)).c -o $#
I do not want to mention the filename as input but would want to use the filter function to get the .c file which is same as target name. make throws an error no input files
It's helpful to have a look at how make parses this:
SOURCES = $(wildcard *.c)
dummytgt: $(OBJ)/tier.o
$(GCC) $(CFLAGS) -c $(filter $#,$(SOURCES)).c -o $#
First off,
it will read the makefile, defining and expanding macros as it goes.
SOURCES = $(wildcard *.c) means that ${SOURCES} is a lazy variable with definition $(wildcard *.c).
Lazy? Yes, these recursive variables (as the make manual has it) only expand their right-hand side when they are themselves expanded.
Make needs the dependencies as it reads the file, so $(OBJ) is expanded.
Let's assume that the expansion of ${OBJ} is objs (say).
The shell command block remains as a single lazy variable.
It's important to note that these are not expanded until make decides that it wants to build dmmytgt.
You could have written this to exactly the same effect:
dummytgt: objs/tier.o
$(GCC) $(CFLAGS) -c $(filter $#,$(wildcard *.c)).c -o $#
For this fragment to work,
the file objs/tier.o must already exist.
Let's assume it does.
Make now has all it needs to build dummytgt (according to your makefile),
so now it expands the command block.
$(wildcard *.c) expands to 1.c 2.c (say).
$# is dummytgt
$(filter dummytgt,1.c 2.c) is of course empty (and always will be!)
${GCC} is gcc (say)
${CFLAGS} is empty (say)
Thus the shell gets
gcc -c .c -o dummytgt
Presumably gcc complains that there is no file called .c.
The resulting error stops make's execution.
A few thing not to like here:
$(wildcard ) is only for hacky one-liner makefiles IMHO.
dummytgt requires objs/tier.o, but its build instructions never reference it.
Your $(filter ) always expands to nothing.
$(filter $#.c,$(SOURCES))
But I don't see why you don't use
$#.c
Or better still, make it a prerequisite.