Given a Makefile:
all: build/a build/b build/c # need to change this to all: build/*
build/a:
...
build/b:
...
build/c:
...
I would like to change the all target to automatically build all targets that match build/*. This question seems to be very similar, except that it prints the result rather than acts on it. Also, I am not sure if that answer will work on both Linux & Mac.
Make has no such functionality built-in. And, in fact, keeping the list of the targets up-to-date manually is much cleaner and easier than any alternative solution. Personally, I'd probably start with something like this:
# second expansion is needed to get the value of
# $(targets) after the whole file was preprocessed
.SECONDEXPANSION:
all: $$(targets)
targets += build/a
build/a:
...
targets += build/b
build/b:
...
targets += build/c
build/c:
...
I don't believe that the requirement to add a single line per target (targets+=xxx) could be so annoying for you.
However, this is how we can "preprocess" Makefile ourselves:
# assume all targets are explicitly defined in Makefile
targets != grep -o '^ *build/\w* *:' Makefile | sed 's/^ *//;s/ *:$$//'
all: $(targets)
build/a:
...
build/b:
...
build/c:
...
Of course, this would fail if targets are in the included files, or contain substitutions, etc. However, it works for "simple cases".
Related
I have a makefile that looks something like this:
include anotherFile.mk
all:
someStuff
The file anotherFile.mk is like this:
include yetAnotherFile.mk
export SOME_VAR = 93
The problem is that anotherFile.mk and yetAnotherFile.mk are in a different directory from my Makefile. So my makefile can't just be changed to this:
include $(OTHER_PROJECT_PATH)/anotherFile.mk
all:
someStuff
The problem with this approach is that the include statement in anotherFile.mk will fail because it will be searching in the current directory.
A partial solution that I found is to pass the --include-dir=$OTHER_PROJECT_PATH flag to the invocation of make, but that's a bit user-unfriendly.
So my question is: Is there something I can put inside my makefile that will add to the directories that make searches for when executing an include? Something like MAKE_INCLUDE_DIRS += $(OTHER_PROJECT_PATH)
Surprisingly there doesn't seem to be a good answer to that question. Forcing .INCLUDE_DIR doesn't help and there doesn't seem to be any way around invoking make with --include-dir=$OTHER_PROJECT_PATH.
It is however possible to put the appropriate recursive make invocation inside the makefile but, in order to get it to work for all reasonable cases it quickly becomes too complicated to be worth it. In summary it requires:
a top level condition to check if the OTHER_PROJECT_PATH is in .INCLUDE_DIR
the appropriate target with the recipe invoking make recursively
possibly additional targets if there are multiple command goals
the real make file enclosed in the else part of the conditional
You Makefile would look like this:
OTHER_PROJECT_PATH := other
ifeq (,$(filter $(OTHER_PROJECT_PATH), $(.INCLUDE_DIRS)))
# this is the mechanism to add the include dir in a recursive make
$(or $(firstword $(MAKECMDGOALS)),all):
$(MAKE) -I$(OTHER_PROJECT_PATH) $(MAKECMDGOALS)
# add empty targets for additional goals if needed
ifneq (,$(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS)))
$(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS)):
endif
else
# this is where the real makefile starts
all more:
echo $#: $< $^
include a.mak
endif
It still does not seem possible from a makefile, but if you have a script that sets up environment variables, you can use MAKEFLAGS (e.g. export MAKEFLAGS=I/your/path ordentlich on Linux, or SET on Windows)
Using make's ''Remaking Makefiles'' feature I am generating parts of my makefile with include directives (see Makefile: defining rules and prerequisites in recipes). Now I'm stuck with being unable to see how I can express dependencies between included makefiles. They seem to be all evaluated at once.
Consider the following minimal makefile that illustrates my problem:
all:
-include foo.make
-include bar.make
foo.make: Makefile
echo FOO:=blub bla baz > foo.make
bar.make: Makefile foo.make
echo BAR:=$(FOO) > bar.make
If I now run make I will get:
$ cat foo.make
FOO:=blub bla baz
$ cat bar.make
BAR:=
Why? Since bar.make depends on foo.make, shouldn't the evaluation of bar.make wait until it successfully included foo.make?
And how do I fix this problem and make sure that bar.make is either re-evaluated later or only evaluated once foo.make exists, is included and can define the variable BAR?
The reason I cannot combine foo.make and bar.make into a single makefile and rule is two-fold:
Firstly, in my real setup, bar.make depends on more intermediate targets which in turn transitively depend on foo.make. So at the time foo.make can be created, the content of bar.make cannot yet be made.
Secondly, in my real setup, foo.make and bar.make do not just define variables but also eval() define/endef blocks. So I have to write:
-include makefile_with_prerequisite_variables
define MYDEF
sometarget-$1: $(TARGET_$1_PREREQUISITES)
[...]
endf
-include makefile_with_eval_call_statements
The content of makefile_with_prerequisite_variables and makefile_with_eval_call_statements cannot go into a single makefile snippet:
If I would put makefile_with_eval_call_statements above MYDEF together with makefile_with_prerequisite_variables then the $eval( $call( MYDEF)) statements in it would not work because MYDEF is only declared afterward.
If I would put makefile_with_prerequisite_variables below MYDEF together with makefile_with_eval_call_statements then the recipes defined in MYDEF would not have proper prerequisits because the $(TARGET_$1_PREREQUISITES) variables would then be declared afterward by makefile_with_prerequisite_variables.
In summary, I need to include two different makefiles where one depends upon the other. I do not know how I can express this relationship such that the content of one makefile would only be created after the other makefile is up-to-date and included into the main makefile.
First, your makefile creation in this simple example can easily be fixed by escaping the value of $(FOO) so that it's not expanded when bar.make is created but rather deferred until it's read in. So:
bar.make: Makefile foo.make
echo 'BAR:=$$(FOO)' > $#
However, that might not be sufficient in your more complex real-life makefiles.
GNU make works like this: first parse all the makefiles. Then for every included makefile, treat it as a goal and try to build it (e.g., act as if the user invoked make include1.mk include2.mk include3.mk ...). Then at the end of that, if any of the included makefiles was rebuilt, re-exec ourselves and start the entire process over from scratch.
GNU make does NOT work like this: parse makefiles, try to rebuild the first included makefile and if it's rebuilt, re-exec; if it's not rebuilt go on to the next included makefile, etc.
A simple trick you can use if you have to have this type of order is to put the include of bar.make into foo.make:
all:
-include foo.make
foo.make: Makefile
printf -- '-include bar.make' > $#
echo FOO:=blub bla baz >> $#
bar.make: Makefile foo.make
echo 'BAR:=$$(FOO)' > $#
By doing this you ensure that if foo.make doesn't exist, make can't see the include of bar.make and so it won't try to build it. Only after the first re-exec will make see the include of bar.make and try to build it.
One thing: if you get the latest version of GNU make you no longer need to use the -include trick. You can just use include even with generated makefiles.
Before I start, I'll mention that I'm not using GNU Make in this case for building a C/C++ project.
Makefile:
DEST_DIR = build/
SRC_DIR = src/
$(SRC_DIR)a/ : $(SOMETHING_ELSE)
$(DO_SOMETHING_TO_GENERATE_A_DIR)
$(DEST_DIR)% : $(SRC_DIR)%
cp -r $^ $#
ALL_DEPS += <SOMETHING>
... more code which appends to ALL_DEPS ...
.PHONY: all
all : $(ALL_DEPS)
I've got some files not generated via Make rules in $(SRC_DIR). (For the sake of this example, let's say there's a directory $(SRC_DIR)b/ and a file $(SRC_DIR)c .)
I want to append to ALL_DEPS all targets which represent files or directories in $(DEST_DIR) so that "make all" will run all of the available $(DEST_DIR)% rules.
I thought to do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(wildcard $(SRC_DIR)*)))
But of course, that doesn't catch anything that hasn't yet been made. (i.e. it doesn't append $(DEST_DIR)a/ to the list because $(SRC_DIR)a/ doesn't yet exist when the $(wildcard ...) invocation is evaluated and the shell doesn't include it in the results returned by the $(wildcard ...) invocation.)
So, rather than a function which finds all (currently-existing) files matching a pattern, I need one which finds all targets matching a pattern. Then, I could do something like this:
ALL_DEPS += $(addprefix $(DEST_DIR),$(notdir $(targetwildcard $(SRC_DIR)*)))
If it matters any, I've got much of the GNU Make code split across multiple files and included by a "master" Makefile. The ALL_DEPS variable is appended to in any of these files which has something to add to it. This is in an attempt to keep the build process modular as opposed to dropping it all in one monster Makefile.
I'm definitely still learning GNU Make, so it's not unlikely that I'm missing something fairly obvious. If I'm just going about this all wrong, please let me know.
Thanks!
It is simply not possible to do what you're trying to do; you're trying to get make to recognise something that doesn't exist.
This is part of the reason why, in general, wildcards are bad (the other being that you can end up including stuff you didn't mean to). The right thing to do here is to explicitly create a list of source files (ls -1 | sed -e 's/\(.*\)/sources+=\1/' > dir.mk) and perform the patsubst transformation on that list.
If you have additional files that are generate as part of the build, then you can append them to that list and their rules will be found as you'd expect.
Assume I have a make rule:
.PHONY:gen
gen: auto.template
generate-sources auto.template
that creates a bunch of files, for example auto1.src, auto2.src, auto3.src and so on.
If I now have rules to build targets from *.src files, like this:
$(patsubst %.src,%.target,$(wildcard *.src)): %.target: %.src
build $< > $#
How can I tell make to first execute the gen rule and then expand the preconditions for the second rule template? GNU extensions are welcome.
Note: I would like to keep it in one make invocation; A trivial solution to this would be to put the second rule in a secondary Makefile.secondrun and call $(MAKE) -f Makefile.secondrun after gen was processed. But I was wondering if there is a better option.
Building off Beta's answer, here's how you can do it using makefile remaking in GNU make, which is not the same thing as recursive make. Rather, it updates an included makefile using a rule in the main makefile, then restarts the original make instance. This is how *.d dependency files are typically generated and used.
# Get the list of auto-generated sources. If this file doesn't exist, or if it is older
# than auto.template, it will get built using the rule defined below, according to the
# standard behavior of GNU make. If autosrcs.mk is rebuilt, GNU make will automatically
# restart itself after autosrcs.mk is updated.
include autosrcs.mk
# Once we have the list of auto-generated sources, getting the list of targets to build
# from them is a simple pattern substitution.
TARGETS=$(patsubst %.src,%.target,$(AUTO_SRCS))
all: $(TARGETS)
# Rule describing how to build autosrcs.mk. This generates the sources, then computes
# the list of autogenerated sources and writes that to autosrcs.mk in the form of a
# make variable. Note that we use *shell* constructs to get the list of sources, not
# make constructs like $(wildcard), which could be expanded at the wrong time relative
# to when the source files are actually created.
autosrcs.mk: auto.template
./generate-sources auto.template
echo "AUTO_SRCS=`echo *.src`" > autosrcs.mk
# How to build *.target files from *.src files.
%.target: %.src
#echo 'build $< > $#'
Short answer: you can't. Make determines all of the rules it will have to execute before it executes any rule.
Longer answer: maybe you can. As you say, you can use recursive Make explicitly, or surreptitiously by, say, building a file which your makefile will include (I'm looking at you, Jack Kelly). Or if you could somehow obtain a list of the files which gen will build, you could write a rule around that. Or you could take a leap of faith like this:
%.target: %.src
build $< > $#
%.src: gen;
At work we use a common makefile that other makefiles include (via the include statement) and it has a generic "clean" target that kills some common files. I want to add on to that target in my new makefile so I can delete some specific files, but if I add a clean target in my makefile, it just overrides the old one.
I know I can just make a new target with a new name and have it call clean, and then do other stuff, but for sake of consistency I'd like to be able to just call make clean and have it do everything.
Is that possible?
I've seen this done at several shops. The most common approach is to use double-colon rules, assuming you're using something like GNU make. In your common makefile you would have something like this:
clean::
# standard cleanup, like remove all .o's:
rm -f *.o
Note that there are two colons following clean, not just one!
In your other makefile you just declare clean again, as a double-colon rule:
clean::
# custom cleanup, like remove my special generated files:
rm -f *.h.gen
When you invoke make clean, GNU make will automagically run both of these "branches" of the clean rule:
% make clean
rm -f *.o
rm -f *.h.gen
It's simple to set up and it composes quite neatly I think. Note that specifically because it is a double-colon rule, you don't get the "overriding commands" errors you normally get when you define two rules for the same target. That's sort of the point of double-colon rules.
You can write your own clean and make it a preq of the common clean.
clean: myclean
myclean:
rm whatever
Yours will run first. If for some reason you want the common clean to run first then the solution will be more complicated.
EDIT:
Here is the best solution I can see which runs the common rule before the local one:
include Makefile.common
clean:
$(MAKE) -f Makefile.common $#
rm whatever additional things
The include directive is necessary because the local makefile relies on the common one for things other than clean. The local clean rule overrides the common clean rule, but invokes the common clean rule before doing the additional work. (This overriding will cause some warnings, which is a nuisance; I don't know a good way to silence them.)
Use implicit rules:
existing-target: my-extention
my-extention:
echo running command 1
echo running command 2
Very simple make tutorial to ramp up.
When using :: you can run into issues since make complains when you mix single colon : and double colon :: rules:
a:
echo a
a::
echo aa
will result in:
. . .
*** target file `a' has both : and :: entries. Stop.
It seems like the common makefile's rule should be called something like common-clean. Then each main makefile would declare their clean rule as
clean: common-clean
and you're set.
If that isn't an option, you could take a look at double colon rules, but those introduce a whole other set of issues to consider.
Adding another possible solution I've seen for posterity... I know the OP was wary about changing the common makefile, but something like this works and involves minimal changes.
local makefile 1:
CLEAN=MyExe1 MyExe2
....
include /my/common/makefile
local makefile 2:
CLEAN=MyExe3 MyExe4
....
include /my/common/makefile
common makefile:
clean:
rm -f *.dep *.o *.a $(CLEAN)
Basically the idea is to define some variable (in this case CLEAN) in each local makefile with all the specific items you want to delete. Then the common makefile runs rm -f on all the common file types to delete, plus whatever was specifically flagged for deletion in each local makefile via the CLEAN variable. If there's nothing specific to delete, simply omit the variable declaration or leave it empty (CLEAN=)
So now if we run make clean for local makefile 1, it executes
rm -f *.dep *.o *.a MyExe1 MyExe2
And if we run make clean for local makefile 2, it executes
rm -f *.dep *.o *.a MyExe3 MyExe4
I've found a better solution:
.PHONY: my-extra-clean
clean: my-extra-clean
my-extra-clean:
rm <whatever-you-want>
include Makefile.common
The key line is clean: my-extra-clean. Ie, you can add dependencies in separate stanzas in different makefiles to add behaviour. my-extra-clean is run as a dependency of the root clean target.
For ours, we define a variable, EXTRAFILESTOCLEAN, then when the clean rule runs, it has a step to remove anything specified in the EXTRAFILESTOCLEAN variable
clean:
rm -f *.o
ifdef $(EXTRAFILESTOCLEAN)
rm -f $(EXTRAFILESTOCLEAN)
endif
That can cause unexpected problems if you set that variable to weird values, but you could guard against those by adding prefixes or other tests.
It's in the docs: https://www.gnu.org/software/make/manual/html_node/Overriding-Makefiles.html
So instead of include Makefile you use a wildcard target and forward it to the base Makefile:
# -include base.Makefile <--- not this
%:
#$(MAKE) -f base.Makefile $#