I often find myself wanting to refer to the stem not in the recipe but int the prerequisites themselves.
For example here I was playing around with some python code that scans the .cpp and .hpp files of the executable source code, in a recursive fashion, to detect what objects it depends from. The script itself works pretty well but I can't figure out how to connect it with the makefile since the input varies.
$(TESTS): bin/tests/%_a : bin/obj/%.o $(foreach var, $(shell python3 ./autoInc.py ./src/lib/%.cpp), bin/obj/$(var).o)
#echo "#---------------------------"
#echo "# Linking $# "
$(CC) -o $# $^
(Here the makefile executes ./autoInc.py ./src/lib/%.cpp without substitution)
This is the form:
.SECONDEXPANSION:
$(TESTS): %_a : $$(foreach var, $$(shell whatever $$*.cpp), $$(var).o)
...
I advise you to get it working with a very simple toy rule, before trying to incorporate your python.
Related
makefile noob here, my makefile always executes every recipe even though the files are up to date. Here is my code:
vpath *.pdf ../../../Figures/Arrowshape/ChemicalNoise
.PHONY : all clean
all : Fig_VP-CN-Revols_MeanfromDist_Dac0.0_F0.0-4.0_0to2.pdf\
Fig_VP-CN-Revols_MeanfromDist_Dac0.0_F0.0-4.0_2to4.pdf\
Fig_VP-CN-Revols_MeanfromDistImshow_Dac0.0_F0.0-4.0.pdf
Fig_%.pdf : %.py
$(warning Building $# )
python $<
Fig_%_2to4.pdf : %.py
$(warning Building $# )
python $<
Fig_%_0to2.pdf : %.py
$(warning Building $# )
python $<
clean:
rm all
I checked that the pdf files are put in the correct folder and the names match. What is wrong with my syntax?
Also, I know that my clean does not work, how would I make it work though?
When you say "put in the correct folder", which folder is that?
It's clearly not the local directory, because if it were your makefile would work.
The first thing wrong is you have the wrong syntax for vpath. See the manual; vpath takes a makefile pattern (that is a string with zero or one % character); it doesn't support shell globbing like *.h. This should be written:
vpath %.pdf ../../../Figures/Arrowshape/ChemicalNoise
However, even with that fix your makefile won't work as you hope, because vpath is not intended to find targets. It's intended to find source files (that is, files that are not created by make).
If you want to understand this deeply you can read http://make.mad-scientist.net/papers/how-not-to-use-vpath/
To get your makefile to work as you want you'll have to add in paths, like this:
OUTDIR = ../../../Figures/Arrowshape/ChemicalNoise
all : $(OUTDIR)/Fig_VP-CN-Revols_MeanfromDist_Dac0.0_F0.0-4.0_0to2.pdf\
$(OUTDIR)/Fig_VP-CN-Revols_MeanfromDist_Dac0.0_F0.0-4.0_2to4.pdf\
$(OUTDIR)/Fig_VP-CN-Revols_MeanfromDistImshow_Dac0.0_F0.0-4.0.pdf
$(OUTDIR)/Fig_%.pdf : %.py
$(warning Building $# )
python $<
$(OUTDIR)/Fig_%_2to4.pdf : %.py
$(warning Building $# )
python $<
$(OUTDIR)/Fig_%_0to2.pdf : %.py
$(warning Building $# )
python $<
I'm extremely new to makefiles and just spent a full day trying to figure out how to automate my testing. For this project I have one program, main.c and it accepts as parameters an input and an output file. The input file is of the form "test-{filler}.txt" so an example file could be "test-invalid-opcode.txt". The output file would be of the form "out-{filler}.txt", so it would be of the form "out-invalid-opcode.txt". I would then want to compare the output with the correct output saved in "correct-{filler}.txt", so this would be "correct-invalid-opcode.txt". Each test would therefore have an input, output, and correct output. I would want to check if the output and correct output have any differences and I would want this to run on every test file with the prefix "test-". I read through a lot of the makefile spec and a lot of different examples, but I'm really confused as to how to handle this. Any help would be really appreciated.
Here is what I have to run one test:
CC=gcc
CFLAGS=-o
main.o: main.c
$(CC) $(CFLAGS) main.o $<
.PHONY: test
test: main.o test-provided.txt test-out.cs
./program test-provided.txt test-out.txt
diff -q test-out.txt out-provided.txt
The layout of what I want the automatic test to be is something like
.PHONY: autotest
autotest: program
$(foreach <file with "test-" prefix> run:
main.o <test-name> <out-name>
diff -q <correct-name> <out-name>
But I really am lost on how to go about implementing that.
Thank you and sorry I couldn't post more code. I tried a lot of different things, but none of them were even close enough to be worth posting.
Edit to show working final version:
program: main.c
$(CC) $(CFLAGS) $a $<
TEST_INPUTS := $(wildcard test-*.txt)
.PHONY: autotest $(TEST_INPUTS)
autotest: $(TEST_INPUTS)
$(TEST_INPUTS): test-%.txt: program
echo '' > out-$*.txt
./$< $# out-$*.txt
diff -q correct-$*.txt out-$*.txt
In general using loops with make is not very "make-ish". A makefile is an entire language fundamentally based on iteration and recursion so trying to do "extra" iteration inside a recipe is often redundant.
If you need to iterate over something, most especially when that something is files, you should attempt to work with make by taking advantage of its target/prerequisite organization. Above you give an algorithm:
$(foreach <file with "test-" prefix> run:
main.o <test-name> <out-name>
diff -q <correct-name> <out-name>
which is perfectly suited to make's default behavior. To translate this into a makefile you write a rule which invokes a single iteration of the loop, then use prerequisites to run it for the files you want. Something like this:
TEST_INPUTS := $(wildcard test-*.txt)
.PHONY: autotest $(TEST_INPUTS)
autotest: $(TEST_INPUTS)
$(TEST_INPUTS): test-%.txt: program
$< $# test-out.txt
diff -q test-out.txt out-$*.txt
Not only is this much more make-like but it has other advantages: for example you can run make test-provided.txt and it will run just that one test instead of all the tests.
You have to use static pattern rules here not normal pattern rules because .PHONY targets can't work with pattern rules.
SOURCE=a.c b.c c.c
ASM=$(patsubst %.c,%.s, $(SOURCE))
all:%.o
gcc -o test $^
$(ASM):%.c
gcc -S -o $# $<
%.o:%.s
gcc -c -o$# $<
I want to generate assembly code (.s) first, then compile the assembly code to object (.o), then link them.
But it seems above makefile code does not work. What is the correct code?
When asking questions, does not work is never very useful... if it worked you probably wouldn't be asking a question! :-) Instead you should always show the command you ran and the output you received (or at least the failing part of the output if it's long). Please cut and paste the actual text rather than paraphrasing messages. Also, including the version of the make program you're using (make --version) and the platform you're running on is often helpful.
Luckily this time we can figure out the problem without this information:
This:
$(ASM):%.c
gcc -S -o $# $<
where ASM is a.s b.s c.s, is not a pattern rule because the targets don't contain a pattern character %. That means the prerequisite %.c is not treated as a pattern, but as an actual file name, literally %.c which obviously doesn't exist.
Similarly, this:
all: %.o
has the same problem: all is a target, so this depends on the literal file named %.o which doesn't exist, and can't be created.
Also as a general rule every recipe that creates a target must create the actual target you told make it would, so this all rule is wrong because the target name is all but the recipe creates the target test.
Finally, it's a very bad idea to name your program test because test is a common UNIX program and a shell built-in, so if you run test it won't do the right thing (if you run ./test it will work).
You want to have all depend on the program you want to build, say mytest, and mytest should depend on the actual .o files:
all: mytest
mytest: $(SOURCE:.c=.o)
gcc -o $# $^
Next, you need to define a pattern rule that knows how to create an assembly file from a source file:
%.s : %.c
gcc -S -o $# $<
That, along with your other pattern rules, is all you need: make will figure it all out from that.
Finally, make has a built-in rule that tells it how to build object files directly from source files. It's best to get rid of this to force make to use your rules; add this to your makefile to delete it:
%.o : %.c
I'm sure this is a totally normal thing to do, but I can't figure out how to get make to do this.
I have a compiler that generates make dependencies of the usual form:
M/A.o : M/A.hs
M/B.o : M/A.o
So I write a rule to compile %.hs into %.o, add a rule to link the binary, include the dependencies file, and all is well. But I want to have several binary targets with different flags. E.g. I want build/test built with -DTESTING and build/profile built with -prof. So I need to keep the .o files in a separate tree, where they will be compiled with special flags.
The straightforward way I can think of would be to have dependencies that look something like this:
build/test/M/A.o : M/A.hs
build/test/M/B.o : build/test/M/A.o
build/profile/M/A.o : M/A.hs
... etc.
And then rules so that %.hs to build/test/%.o compiles with -DTESTING, etc. I think this would work, but it's clumsy, means preprocessing the deps file to add all that build/whatever/ prefix stuff, and would multiply its size by however many kinds of builds.
VPATH appears to be designed for this sort of thing and my idea was that I could set the VPATH and compiler flags depending on the target, and it almost works, but:
%.o: %.hs
#mkdir -p build/M
cp $< build/$#
VPATH = build
main: M/A.o M/B.o
cat $^ >$#
M/A.o : M/A.hs
M/B.o : M/B.hs
The first time the main target wants to run 'cat M/A.o M/B.o >main' which seems contrary to the gnu make documentation that says $^ should include the include the VPATH directory in which the dependency was found. Curiously, if I remove 'main' and make again, this time it uses the correct path. This is GNU make, 3.81.
What's going on here? Is there a better way to build with different flags? VPATH seems like a clumsy tool, surely there is a better way?
Make is working correctly. It tries cat M/A.o M/B.o >main the first time because it can't find the prerequisites it needs, but it knows a rule for M/A.o' andM/B.o(<em>not</em>build/M/A.o' and build/M/B.o) and expects that that is what the rule will produce. If you remove main and try again, it will find build/M/A.o' andbuild/M/B.o` via VPATH.
Let's modify this makefile in stages. First we change the VPATH so that it can find the .hs files (Make is good at using things there to build things here, not vise-versa, and that's what VPATH is good for), and change the rules slightly:
build/%.o: %.hs
cp $< $#
VPATH = M
main: build/A.o build/B.o
cat $^ > $#
Now for the different object directories.
build/test/%.o build/project/%.o: %.hs
cp $< $#
VPATH = M
test: build/test/A.o build/test/B.o
cat $^ > $#
project: build/project/A.o build/project/B.o
cat $^ > $#
Then we simplify those last two rules, so that it's easy to add more object files and binary targets:
OBJECTS = A.o B.o
test: $(addprefix build/test/,$(OBJECTS))
project: $(addprefix build/project/,$(OBJECTS))
test project:
cat $^ > $#
Now for the different compiler flags:
build/test/%.o: FLAGS += test_flags
build/project/%.o: FLAGS += proj_flags
build/test/%.o build/project/%.o: %.hs
#echo building $# from $^ using flags $(FLAGS)
cp $< $#
Finally the dependencies. This is a little tricky. Suppose you want the dependency B.o : A.hs to apply to however many object you have. This is one approach:
OBJECT_PATHS = build/test/ build/project/
# The following is from the included file generated by the compiler
$(addsuffix B.o,$(OBJECT_PATHS)) : A.hs
To generate lines like that, I'd pipe the raw lines (e.g. B.o: A.hs) through sed 's/\(.*\):\(.*\)/\1:\2/', and note that if you want to put this in a makefile command, don't forget to double the $ signs to preserve them for the shell.
I know that's a lot to absorb. Take it one step at a time and let us know how it works out.
If you haven't solved your problem by now or are experiencing further problems, best give the autotools (automake and autoconf) a chance. They'll quickly build you a Makefile that supports more configurable and flexible out-of-tree builds.
I am trying to exclude main.cpp file from the list of files to be compiled defined by the rule below:
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
ifneq ($(notdir $<), main.cpp)
#$(COMPILE.cpp) $(OUTPUT_OPTION) $<
endif
This 'ifneq' condition always evaluates to true, which is bizarre. What am I doing wrong? Is there a better way to exlude one file from an explicit rule?
Why don't you try using the filter-out text function if you're using GNU Make.
Returns all whitespace-separated words in text that do not match any of the pattern words, removing the words that do match one or more. This is the exact opposite of the filter function.
For example, given:
objects=main1.o foo.o main2.o bar.o
mains=main1.o main2.o
the following generates a list which contains all the object files not in ‘mains’:
$(filter-out $(mains),$(objects))
That isn't the best way to do it, but if you do it along these lines, write it as a shell condition, not using GNU make conditionals:
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#if [ $(notdir $<) != main.cpp ]; \
then $(COMPILE.cpp) $(OUTPUT_OPTION) $<; \
fi
The continuation markers (backslashes) are needed. So are the semicolons. The values prefixed with $ will be expanded by make before the shell is invoked to interpret them. You probably don't want the echo where it is, either. You probably need:
$(TMPDIRPATH)%.o: %.cpp
#if [ $(notdir $<) != main.cpp ]; \
then echo compile $<; \
$(COMPILE.cpp) $(OUTPUT_OPTION) $<; \
fi
The way I would expect to do it is with a list of the files to be compiled. Using any wild card mechanism leads to problems when extra files are added - other tests, or stray files that aren't really part of the system.
The comment says "But the GNU Make Manual says ifneq should work".
The ifneq would work if it were positioned correctly, which means 'not indented as part of the commands associated with a rule'. You could, therefore, write something like (an appallingly bad example, but my brain's on the fritz):
ifneq (${CFLAGS}, -Wall)
CFLAGS += -Wall
endif
file1.o: file1.c
${CC} ${CFLAGS} -c $<
But when the ifneq is indented as in the question, it is just a command that actually isn't found on the system when the make runs the shell to process the command.
The ifneq line is evaluated only once, when make starts up and parses the makefile. In that context, $< is empty.
To get different behavior for each of the targets matched by your pattern rule, you could do something like
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#$(if $(filter main.cpp,$<),$(COMPILE.cpp) $(OUTPUT_OPTION) $<)
It might help you to think of the difference between ifneq and $(if) in a makefile as like the difference between #if and if() in C code.
Taking a step back, though: If you don't want main.cpp to be compiled by this rule, then you probably want to provide an explicit rule with $(TMPDIRPATH)main.o as its target, which will be preferred to the pattern rule always. Or, if you don't want$(TMPDIRPATH)main.o to get made at all, you should be looking for rules that have it on the right sight of the :, and removing it from there.
Make doesn't really have a good way to handle conditionals within a rule. You could put the conditional in the command, but in this case there's a much cleaner way:
$(TMPDIRPATH)main.o:
#echo compile $< (but not really)
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#$(COMPILE.cpp) $(OUTPUT_OPTION) $<
EDIT:
I didn't realize you didn't have a main.cpp. The solution is simple: remove main.cpp as the prerequisite of the main.o rule (I've removed it above). Now the makefile doesn't need it, and won't try to build it.
But you're still running the rule, which means that something is still trying to build main.o, as either an explicit target or a prerequisite of something else. That is a symptom of confusion, which this change to the makefile will not fix. If you tell us more about the situation, maybe we can propose a better solution. What calls for main.o? Do you have a main.o? What target do you specify when you call Make?