I'm trying to create a makefile where the target is used to search for dependencies in .d
Thanks to the answers here and here I have discovered that .SECONDEXPANSION: does the job. However I'm running into an issue nesting functions which is seemingly bizarre. The offending rule looks like this:
.SECONDEXPANSION:
$(APPS): %: $$(patsubst %.c,%.o,$$(wildcard $$#.d/*.c)) $$(INC_OBJS)
$(CC) $(CFLAGS) $^ -o $#
instead of:
gcc -Wall -std=c99 unittest.d/unittest.o common/cards.o -o unittest
which is what I want, I get:
gcc -Wall -std=c99 unittest.d/unittest.c common/cards.o -o unittest
which for some reason doesn't have the .o substituted for the .c. So it appears I'm almost there, I just need to get the substitution working. All help appreciated and if you think this is a poor way to organize a makefile, critizism on that front welcome.
The problem is that you're trying to use % as two different wildcards at once, the static pattern stem and the patsubst wildcard. So Make makes the stem replacement and gets this:
$(patsubst unittest.c,unittest.o,$(wildcard $#.d/*.c))
and then patsubst does nothing, because it finds no "unittest.c" to replace. (Note that the "unittest.c" in "unittest.d/unittest.c" doesn't match, because without a wildcard, patsubst looks for a perfect match.)
There's no need for this to be a static pattern rule (you never use that functionality), so you can just eliminate that part and the rest should work:
.SECONDEXPANSION:
$(APPS): $$(patsubst %.c,%.o,$$(wildcard $$#.d/*.c)) $$(INC_OBJS)
$(CC) $(CFLAGS) $^ -o $#
Related
I've just started trying to use static pattern rules and for loops together within makefiles, I'm still relatively new to using makefiles so please forgive me if I've missed something obvious.
In the code below I have tried to use a for loop to create 6 executables, two for each unique file.
Here is the makefile:
vpath %.h ../headers/
CXX := g++
CXXFLAGS := -std=c++11 -I../headers/
LDFLAGS :=
SUFFIX := fileA fileB fileC
memory-%.exe: primary-%.o memory.cpp
$(CXX) $(CXXFLAGS) $^ -o $#
timing-%.exe: primary-%.o timing.cpp
$(CXX) $(CXXFLAGS) $^ -o $#
all: for i in $(SUFFIX); \
do \
testing-$$i.exe: primary-$$i.o; \
memory-$$i.exe: primary-$$i.o; \
done
I am met with the error:
\bin\sh: 3: memory-fileA.exe:: not found
\bin\sh: 4: timing-fileA.exe:: not found
\bin\sh: 3: memory-fileB.exe:: not found
\bin\sh: 4: timing-fileB.exe:: not found
\bin\sh: 3: memory-fileC.exe:: not found
\bin\sh: 4: timing-fileC.exe:: not found
make: *** [all] Error 127
Is this even possible in the first place? I was just wondering if it were possible to be efficient using this method.
Any help is appreciated as I'd like to know more about the possibilities that makefiles allow.
Thank you.
You are mixing shell and make constructs. As tripleee pointed the recipes of make rules are shell scripts, not other make rules.
Moreover, there are a few issues with your Makefile:
You explain that you want to use static pattern rules but what you wrote is "simple" pattern rules.
You do not need to quote your suffixes. And you should not, make is not the shell, it preserves them. You will get errors because of this.
Your use of the standard CXXFLAGS make variable is extremely unusual. Traditionally it is limited to the compiler's flags, not the compiler itself for which CXX is used.
You are compiling source files and linking simultaneously. This too is not that usual. It causes useless re-compilations.
The c++11 option of g++ is new to me. Are you sure it is not -std=c++11?
The vpath directive is useless because you do not express dependencies on the header files. But let's keep it, I guess you do not show everything.
All-in-all, you can probably achieve want you want with:
vpath %.h ../headers/
CXX := g++
CXXFLAGS := -std=c++11 -I../headers/
LDFLAGS :=
SUFFIX := fileA fileB fileC
TESTING := $(patsubst %,testing-%.exe,$(SUFFIX))
MEMORY := $(patsubst %,memory-%.exe,$(SUFFIX))
.PHONY: all
all: $(TESTING) $(MEMORY)
%.o: %.cpp
$(CXX) $(CXXFLAGS) -c $^ -o $#
$(TESTING): testing-%.exe: primary-%.o memory.o
$(CXX) $(LDFLAGS) $^ -o $#
$(MEMORY): memory-%.exe: primary-%.o timing.o
$(CXX) $(LDFLAGS) $^ -o $#
The %.o: %.cpp... rule is a pattern rule. It tells make how to produce any object file from the corresponding C++ source file. The two last rules are really static pattern rules. The first of the two, for instance, declares that each target testing-<suffix>.exe listed in $(TESTING) depends on the corresponding primary-<suffix>.o and on memory.o. This single static pattern rule is thus equivalent to these 3 simple rules:
testing-fileA.exe: primary-fileA.o memory.o
g++ primary-fileA.o memory.o -o testing-fileA.exe
testing-fileB.exe: primary-fileB.o memory.o
g++ primary-fileB.o memory.o -o testing-fileB.exe
testing-fileC.exe: primary-fileC.o memory.o
g++ primary-fileC.o memory.o -o testing-fileC.exe
No need for loops. Note that, if you correctly use the standard make variables CXX and CXXFLAGS, you can drop the pattern rule (%.o: %.cpp...), it is one of the many implicit rules that make knows already.
I want to allow pattern rules to generate the correct header dependencies for me. I've attempted do this by, but it's failing
For example, if I have files foo.cpp, foo.h, bar.cpp, bar.h and foo.h includes bar.h. with Makefile:
foo_H=foo.h $(bar_H)
bar_H=bar.h
%.o: %.cpp $(%_H)
$(CPP) $(CPPFLAGS) $< -o $#
but make will not update when foo.h or bar.h is changed.
why does this fail and how can it be fixed?
If you really want to do that you can use secondary expansion for it:
foo_H = foo.h $(bar_H)
bar_H = bar.h
.SECONDEXPANSION:
%.o: %.cpp $$($$*_H)
$(CPP) $(CPPFLAGS) $< -o $#
That's not the way gnu make works.
the variable bar_H is undefined when assigning foo_H. So foo_H will just have the value foo.h.
the pattern expansion will not work inside $(...). It just will look up the variable %_H which does not exists, i.e. is empty.
%< and %# is wrong. You probably intended to write $< and $#.
You makefile needs at least one non target. A patterned rule is not sufficient.
A patterned rule will not be used unless all dependencies exist. This might not be intended here.
Your patterned rule will not apply because it searches for a foo.c rather than foo.cpp.
The intended behavior is achieved by
foo.o : foo.h bar.h
bar.o : bar.h
%.o: %.cpp
$(CPP) $(CPPFLAGS) $< -o $#
Note that the dependencies are specified separately from the executable commands.
Another note: if you want to get rid of the include dependency hell you might want to have a look at cmake.
$(%_H) could not expand because as stated in
https://www.gnu.org/software/make/manual/html_node/Pattern-Rules.html
Note that expansion using ‘%’ in pattern rules occurs after any variable or function expansions, which take place when the makefile is read
so it seems like using patterns to achieve this kind of logic is a dead end
As an alternative, I used foreach and include as follows:
makedep.mk
$(CUR_OBJ): $(CUR_OBJ:.o=.cpp) $($(CUR_OBJ:.o=_H))
$(CXX) $(CXXFLAGS) $(CPPFLAGS) $< -o $#
Makefile
foo_H=foo.h $(bar_H)
bar_H=bar.h
SRCS=foo.cpp bar.cpp
OBJS=$(SRCS:.cpp=.o)
$(foreach obj,$(OBJS),$(eval CUR_OBJ:=$(obj)) $(eval include makedep.mk))
since there are only make variables and no pattern matching % everything can expand properly
A part of my Makefile:
CFLAGS = -I ../headers -Wall
EXECUTABLES = testPattern testPatterns
$(EXECUTABLES): %Pattern: %.c pattern.o
g++ $(CFLAGS) $#.c pattern.o -o $#
I didn't include the object compilation because it's irrelevant.
The problem I have with this code is that the pattern matches only the first executable because it ends with 'Pattern', the second executable has an additional 's' at the end which kills the script. Is there any way I can make it work without changing the name of the second executable?
Thanks
There's no point in using a static pattern rule here, since the pattern doesn't appear in the prerequisites list. However, I assume you also wanted to include the .c file as a prerequisite here.
Why do you include the Pattern in the pattern match?
You can just write:
CFLAGS = -I ../headers -Wall
EXECUTABLES = testPattern testPatterns
$(EXECUTABLES): % : %.c pattern.o
g++ $(CFLAGS) $^ -o $#
ETA
It appears a better example that would show the real issues you face would be something like this:
EXECUTABLES = someThing somePattern morePatterns
where you want a static pattern rule that matches the two binaries containing Pattern but not the others.
As I said in my comment below, you cannot do this with a single pattern which means you can't do it in the target part of a static pattern rule.
However, you could do it like this:
$(foreach E,$(EXECUTABLES),$(if $(findstring Pattern,$E),$E)): % : %.c pattern.o
g++ $(CFLAGS) $^ -o $#
This basically loops through each entry in EXECUTABLES and tests to see if it contains the string Pattern, and if so expands to that string else expands to nothing.
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=main.cpp hello.cpp factorial.cpp
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=hello
all: $(SOURCES) $(EXECUTABLE)
$(EXECUTABLE): $(OBJECTS)
$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
$(CC) $(CFLAGS) $< -o $#
What do the $# and $< do exactly?
$# is the name of the target being generated, and $< the first prerequisite (usually a source file). You can find a list of all these special variables in the GNU Make manual.
For example, consider the following declaration:
all: library.cpp main.cpp
In this case:
$# evaluates to all
$< evaluates to library.cpp
$^ evaluates to library.cpp main.cpp
From Managing Projects with GNU Make, 3rd Edition, p. 16 (it's under GNU Free Documentation License):
Automatic variables are set by make after a rule is matched. They
provide access to elements from the target and prerequisite lists so
you don’t have to explicitly specify any filenames. They are very
useful for avoiding code duplication, but are critical when defining
more general pattern rules.
There are seven “core” automatic variables:
$#: The filename representing the target.
$%: The filename element of an archive member specification.
$<: The filename of the first prerequisite.
$?: The names of all prerequisites that are newer than the target,
separated by spaces.
$^: The filenames of all the prerequisites, separated by spaces. This
list has duplicate filenames removed since for most uses, such as
compiling, copying, etc., duplicates are not wanted.
$+: Similar to $^, this is the names of all the prerequisites separated
by spaces, except that $+ includes duplicates. This variable was
created for specific situations such as arguments to linkers where
duplicate values have meaning.
$*: The stem of the target filename. A stem is typically a filename
without its suffix. Its use outside of pattern rules is
discouraged.
In addition, each of the above variables has two variants for
compatibility with other makes. One variant returns only the directory
portion of the value. This is indicated by appending a “D” to the
symbol, $(#D), $(<D), etc. The other variant returns only the file
portion of the value. This is indicated by appending an “F” to the
symbol, $(#F), $(<F), etc. Note that these variant names are more than
one character long and so must be enclosed in parentheses. GNU make
provides a more readable alternative with the dir and notdir
functions.
The $# and $< are called automatic variables. The variable $# represents the name of the target and $< represents the first prerequisite required to create the output file.
For example:
hello.o: hello.c hello.h
gcc -c $< -o $#
Here, hello.o is the output file. This is what $# expands to. The first dependency is hello.c. That's what $< expands to.
The -c flag generates the .o file; see man gcc for a more detailed explanation. The -o specifies the output file to create.
For further details, you can read this article on linoxide about Linux Makefiles.
Also, you can check the GNU make manuals. It will make it easier to make Makefiles and to debug them.
If you run this command, it will output the makefile database:
make -p
The $# and $< are special macros.
Where:
$# is the file name of the target.
$< is the name of the first dependency.
The Makefile builds the hello executable if any one of main.cpp, hello.cpp, factorial.cpp changed. The smallest possible Makefile to achieve that specification could have been:
hello: main.cpp hello.cpp factorial.cpp
g++ -o hello main.cpp hello.cpp factorial.cpp
pro: very easy to read
con: maintenance nightmare, duplication of the C++ dependencies
con: efficiency problem, we recompile all C++ even if only one was changed
To improve on the above, we only compile those C++ files that were edited. Then, we just link the resultant object files together.
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
main.o: main.cpp
g++ -c main.cpp
hello.o: hello.cpp
g++ -c hello.cpp
factorial.o: factorial.cpp
g++ -c factorial.cpp
pro: fixes efficiency issue
con: new maintenance nightmare, potential typo on object files rules
To improve on this, we can replace all object file rules with a single .cpp.o rule:
OBJECTS=main.o hello.o factorial.o
hello: $(OBJECTS)
g++ -o hello $(OBJECTS)
.cpp.o:
g++ -c $< -o $#
pro: back to having a short makefile, somewhat easy to read
Here the .cpp.o rule defines how to build anyfile.o from anyfile.cpp.
$< matches to first dependency, in this case, anyfile.cpp
$# matches the target, in this case, anyfile.o.
The other changes present in the Makefile are:
Making it easier to changes compilers from g++ to any C++ compiler.
Making it easier to change the compiler options.
Making it easier to change the linker options.
Making it easier to change the C++ source files and output.
Added a default rule 'all' which acts as a quick check to ensure all your source files are present before an attempt to build your application is made.
in exemple if you want to compile sources but have objects in an different directory :
You need to do :
gcc -c -o <obj/1.o> <srcs/1.c> <obj/2.o> <srcs/2.c> ...
but with most of macros the result will be all objects followed by all sources, like :
gcc -c -o <all OBJ path> <all SRC path>
so this will not compile anything ^^ and you will not be able to put your objects files in a different dir :(
the solution is to use these special macros
$# $<
this will generate a .o file (obj/file.o) for each .c file in SRC (src/file.c)
$(OBJ):$(SRC)
gcc -c -o $# $< $(HEADERS) $(FLAGS)
it means :
$# = $(OBJ)
$< = $(SRC)
but lines by lines INSTEAD of all lines of OBJ followed by all lines of SRC
I am trying to exclude main.cpp file from the list of files to be compiled defined by the rule below:
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
ifneq ($(notdir $<), main.cpp)
#$(COMPILE.cpp) $(OUTPUT_OPTION) $<
endif
This 'ifneq' condition always evaluates to true, which is bizarre. What am I doing wrong? Is there a better way to exlude one file from an explicit rule?
Why don't you try using the filter-out text function if you're using GNU Make.
Returns all whitespace-separated words in text that do not match any of the pattern words, removing the words that do match one or more. This is the exact opposite of the filter function.
For example, given:
objects=main1.o foo.o main2.o bar.o
mains=main1.o main2.o
the following generates a list which contains all the object files not in ‘mains’:
$(filter-out $(mains),$(objects))
That isn't the best way to do it, but if you do it along these lines, write it as a shell condition, not using GNU make conditionals:
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#if [ $(notdir $<) != main.cpp ]; \
then $(COMPILE.cpp) $(OUTPUT_OPTION) $<; \
fi
The continuation markers (backslashes) are needed. So are the semicolons. The values prefixed with $ will be expanded by make before the shell is invoked to interpret them. You probably don't want the echo where it is, either. You probably need:
$(TMPDIRPATH)%.o: %.cpp
#if [ $(notdir $<) != main.cpp ]; \
then echo compile $<; \
$(COMPILE.cpp) $(OUTPUT_OPTION) $<; \
fi
The way I would expect to do it is with a list of the files to be compiled. Using any wild card mechanism leads to problems when extra files are added - other tests, or stray files that aren't really part of the system.
The comment says "But the GNU Make Manual says ifneq should work".
The ifneq would work if it were positioned correctly, which means 'not indented as part of the commands associated with a rule'. You could, therefore, write something like (an appallingly bad example, but my brain's on the fritz):
ifneq (${CFLAGS}, -Wall)
CFLAGS += -Wall
endif
file1.o: file1.c
${CC} ${CFLAGS} -c $<
But when the ifneq is indented as in the question, it is just a command that actually isn't found on the system when the make runs the shell to process the command.
The ifneq line is evaluated only once, when make starts up and parses the makefile. In that context, $< is empty.
To get different behavior for each of the targets matched by your pattern rule, you could do something like
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#$(if $(filter main.cpp,$<),$(COMPILE.cpp) $(OUTPUT_OPTION) $<)
It might help you to think of the difference between ifneq and $(if) in a makefile as like the difference between #if and if() in C code.
Taking a step back, though: If you don't want main.cpp to be compiled by this rule, then you probably want to provide an explicit rule with $(TMPDIRPATH)main.o as its target, which will be preferred to the pattern rule always. Or, if you don't want$(TMPDIRPATH)main.o to get made at all, you should be looking for rules that have it on the right sight of the :, and removing it from there.
Make doesn't really have a good way to handle conditionals within a rule. You could put the conditional in the command, but in this case there's a much cleaner way:
$(TMPDIRPATH)main.o:
#echo compile $< (but not really)
$(TMPDIRPATH)%.o: %.cpp
#echo compile $<
#$(COMPILE.cpp) $(OUTPUT_OPTION) $<
EDIT:
I didn't realize you didn't have a main.cpp. The solution is simple: remove main.cpp as the prerequisite of the main.o rule (I've removed it above). Now the makefile doesn't need it, and won't try to build it.
But you're still running the rule, which means that something is still trying to build main.o, as either an explicit target or a prerequisite of something else. That is a symptom of confusion, which this change to the makefile will not fix. If you tell us more about the situation, maybe we can propose a better solution. What calls for main.o? Do you have a main.o? What target do you specify when you call Make?