According to the make documentation, pattern rules with multiple outputs run only once while normal rules may run multiple times. I'd like to run pattern rules multiple times to share a recipe. I know I can share them by using a define but that comes with a different set of problems.
I am using gnu make version 3.8.1
This is the makefile I'd like to use. Is there a way to have it run the pattern rule four times?
build: test.a.x test.a.y test.b.x test.b.y
clean:
rm -f test.a.x test.a.y test.b.x test.b.y
%.a.x: optAB=a
%.a.x: optXY=x
%.a.y: optAB=a
%.a.y: optXY=y
%.b.x: optAB=b
%.b.x: optXY=x
%.b.y: optAB=b
%.b.y: optXY=y
%.a.x %.a.y %.b.x %.b.y: %.v
#echo optAB: $(optAB) optXY: $(optXY) $# $<
touch $#
The relevant line in the documentation is
If a pattern rule has multiple targets, make knows that the rule's recipe is responsible for making all of the targets.
So one way to get around this would be to define multiple targets. One way to achieve this is to use foreach/eval/call:
build: test.a.x test.a.y test.b.x test.b.y
clean:
rm -f test.a.x test.a.y test.b.x test.b.y
%.a.x: optAB=a
%.a.x: optXY=x
%.a.y: optAB=a
%.a.y: optXY=y
%.b.x: optAB=b
%.b.x: optXY=x
%.b.y: optAB=b
%.b.y: optXY=y
optset1 = a b
optset2 = x y
define target_template
%.$(1).$(2): %.v
#echo optAB: $$(optAB) optXY: $$(optXY) $$# $$<
touch $$#
endef
$(foreach opt1,$(optset1),$(foreach opt2,$(optset2),$(eval $(call target_template,$(opt1),$(opt2)))))
We define a template for the rule, and then call/eval that template for each of the options, to generate multiple rules, one for each target. Note the $$ in the template definition are to ensure the variables are not expanded when the template is defined, but only when it is evaluated to generate each target.
Related
I have inherited a large branched project? that requires a volatile set of .a archives $(LIB_FILES) to be included into link target, located in some directories $(LIB_DIRS). I can write an expression like this:
LIBDEP = $(foreach ldir, $(LIB_DIRS), \
$(filter $(addprefix %/, $(LIB_FILES)), $(wildcard $(ldir)/* )))
The problem is that they might not exist at moment of make's invocation and would be built by invoking $(MAKE) inside of another target's rule, which is a prerequisite to the link step.
The problem is actual list of files that should be created varies on external factors determined at their build steps, that I can't hard-code it properly, without turning makefile into a spaghetti mess and said variable is not re-evaluated at the moment of link command invocation.
I have suspicion that $(eval ) function can be used somehow, but manual is not very forthcoming as well as I didn't found examples of its use in this way.
Toolchain: GCC and binutils, make 3.81
Another solution is to create an explicit dependency of your make script on the output of the step which currently creates the variable $(LIB_FILES). This is what the manual is dealing with in the chapter How makefiles are remade and it aims at the technique which make is best at, namely deriving dependencies from the existence and timestamp of files (instead of variables). The following hopefully depicts your situation with the process of deducing a new set of libraries simulated by the two variables $(LIBS_THIS_TIME) and $(LIB_CONFIG_SET).
LIBS_THIS_TIME = foo.a:baz.a:bar.a
LIB_CONFIG_SET = $(subst :,_,$(LIBS_THIS_TIME))
include libdeps.d
linkstep:
#echo I am linking $^ now
touch $#
libdeps.d: $(LIB_CONFIG_SET)
-rm libdeps.d
$(foreach lib,$(subst :, ,$(LIBS_THIS_TIME)),echo linkstep: $(lib) >> libdeps.d;)
$(LIB_CONFIG_SET):
touch $#
If make finds that libdeps.d is not up to date to your current library configuration it is remade before make executes any other rule, although it is not the first target in the makefile. This way, if your build process creates a new or different set of libraries, libdeps.d would be remade first and only then make would carry on with the other targets in your top makefile, now with the correct dependecy information.
It sometimes happens that you need to invoke make several times in succession. One possibility to do this is to use conditionals:
ifeq ($(STEP),)
all:
<do-first-step>
$(MAKE) STEP=2 $#
else ifeq ($(STEP),2)
all:
<do-second-step>
$(MAKE) STEP=3 $#
else ifeq ($(STEP),3)
all:
<do-third-step>
endif
In each step you can generate new files and have them existing for the next step.
I've been looking through makefile syntax manuals and haven't found anything that really helps the usage case I'm trying to enact here.
What I have is a list of source files with varying directories under a common directory, like so:
src/a.h
src/b.h
src/dir/c.h
src/dir/dir/d.h
and would like make to use these individually as a dependency for a rule that ultimately creates:
build/a.h
build/b.h
build/c.h
build/d.h
which then are used as dependencies individually for more rules.
What I have so far:
LIST := src/a.h src/b.h src/dir/c.h src/dir/d.h
all : $(addprefix build/,$(notdir ${LIST}))
#echo 'All rule invoked'
What doesn't work:
$(LIST) : build/$(notdir %).h : %.h
#echo 'dst $* dat $# din $<'
target 'item' doesn't match the target pattern
build/%.h: %.h
no rule to make target 'build/a.h' needed by 'all'.
I'm guessing make got mad at me at this point, as the errors started telling me to stop.
Basically, I am reading in a list of files with a path prefix that is relevant for the search path and dependency, and want to dump each individual one only when the source file is updated. After this, these files in that single directory are used as dependencies for another batch of rules. How can I accomplish this?
Note: I've gotten it done by ignoring the dependency chain, but that's not going to work. I can also use make to run scripts that generate an explicit makefile that can do it properly, but that feels like overkill and a waste of resources, and make ought to be able to create a rule that does that by itself, as powerful as it is. I just don't know how to create generic rules that focus on the dependency variable for its text matching, rather than the target.
There's no good way of using a pattern rule here, as all the headers are (potentially) in different directories and you want to move them out to a common directory. If you're using GNU make, you can write a macro rule that expands to all the rules you need:
define copy_header_rule
build/$(notdir $(1)): $(1)
cp $$< $$#
endef
$(foreach hdr,$(LIST),$(eval $(call copy_header_rule,$(hdr))))
This goes through each of the headers in your $(LIST) a creates a rule to copy it to the build directory
You can make things pretty simple with vpath:
TARGS:= $(addprefix build/, $(notdir $(LIST)))
vpath %.h $(dir $(LIST))
all: $(TARGS)
build/%.h: %.h
#echo building $# from $<
...
I'm not asking if it is possible to call Makefile from another Makefile.
Suppose I have a rule for generating an executable which looks like this:
my-prog: some.o local.o dependencies.o
Note that I'm exploiting built-in rules here.
Now suppose I start using a third-party library. I'd like to keep this built-in syntax and just add the external rule to the dependency list:
my-prog: some.o local.o dependencies.o somelib/libsomelib.a
But that won't work:
No rule to make target 'somelib/libsomelib.a', needed by 'my-prog'.
I know that I can solve this issue by calling explicitly the other Makefile:
my-prog: some.o local.o dependencies.o
$(MAKE) -C somelib/ libsomelib.a
$(CC) $(LDFLAGS) -o $# $^ somelib/libsomelib.a
But that's what I'm trying to avoid. Any ideas?
In select cases it might be possible to just include the other Makefile, but in those cases they could likely have been written as one in the first place, so...failing that, the best you can do to make the dependency tracking work is to extend the recursive make approach -- your own makefile can't track the dependencies of somelib/libsomelib.a, so you will have to ask the other Makefile to do it for you every time. I'm afraid there's no way around that.
You can, however, enable yourself to keep using the implicit rules and shift the dependency tracking of foreign libs to the other makefile. I'm thinking along the lines of phony targets for these foreign builds like so:
somelib/libsomelib.a:
$(MAKE) -C somelib/ libsomelib.a
# This target needs to be phony so it is run every time because only the other
# makefile can determine that there's nothing to be done.
.PHONY: somelib/libsomelib.a
# then you can use it as a dependency just like locally built targets
my-prog: some.o local.o dependencies.o somelib/libsomelib.a
This can be extended to multiple foreign targets like this:
# list foreign targets here
FOREIGN_TARGETS = \
somelib/libsomelib.a \
foo/libfoo.a \
bar/libbar.a
$(FOREIGN_TARGETS):
# split the target into directory and file path. This assumes that all
# targets directory/filename are built with $(MAKE) -C directory filename
$(MAKE) -C $(dir $#) $(notdir $#)
.PHONY: $(FOREIGN_TARGETS)
What is a best practice to writing reusable code in Makefiles?
Suppose I have a Makefile:
.PHONY: all task01-all task01-clean task01-run task02-all task02-clean task02-run
all: task01-all task02-all
###############################################################################
task01-all: task01-clean task01 task01-run
task01-clean:
rm task01 task01.{exi,o} -f
task01:
compiler task01.ext -O2 --make
task01-run:
./task01
###############################################################################
task02-all: task02-clean task02 task02-run
task02-clean:
rm task02 task02.{exi,o} -f
task02:
compiler task02.ext -O2 --make
task02-run:
./task02
Now I want to add new family of tasks (task03), and I need to copypaste whole section, make s/02/03/ for it and add them to .PHONY section - it's noisy, disgusting and just not right.
How can I avoid that? Could I redefine all tasks with templates somehow to have nice mechanism for adding new task group in one line?
Since the question is about writing re-usable code in Makefiles, I'll give an example of how to use pattern rules in GNU Make (it looks like that's what you're using since you mention the .PHONY target). However, if you're not using any of Make's dependency checking, it may be simpler to do this with a shell script--something like:
#!/bin/sh
TASKS="task01 task02 task03"
for i in $TASKS; do
rm $i $i.ext $i.o -f;
compiler $i.ext -O2 --make;
./$i;
done
But, to expand the principle to Make, we have another issue to tackle. Lines of the form:
task01-all: task01-clean task01 task01-run
don't tell Make in what order to build the pre-requisites--just that they all need to be done before task01-all gets built. Instead, each step to run should depend on the step before it. Something like this:
TASKS=task01-run task02-run task03-run
.PHONY: all $(TASKS) $(TASKS:run=clean)
all: $(TASKS)
$(TASKS:run=clean): %-clean:
rm $* $*.ext $*.o -f
%: %.ext | %-clean
compiler $< -O2 --make
$(TASKS): %-run: %
./$<
The rules with % are called "pattern rules", and they're a great tool to avoid re-writing the same rule multiple times for different targets. One caveat is that Make doesn't normally check pattern rules for a .PHONY target; we tell Make to do this explicitly by prefixing those rules with the list of targets and a second colon (e.g., $(TASKS):).
Since task01-run needs task01 in order to work, we make %-run targets depend on %. Also, your Makefile shows that you want to run clean every time, so we make % depend on %-clean. Since %-clean doesn't actually produce any output, we make this an "order only" dependency--Make won't look for a time-stamp or anything, it will just run the %-clean rule first any time it needs to run the % rule. "Order only" dependencies are placed after a |:
%: %.ext | %-clean
It's worth mentioning that one of Make's greatest strengths is that it can save time by not repeating work that doesn't need to be repeated--i.e., it only runs a rule if the dependencies are newer than the target. So, you could leave off the dependency on %-clean, which would cause make to only run compiler $< -O2 --make if %.ext is newer than %:
%: %.ext
compiler $< -O2 --make
You could then add a rule just to run all of the %-clean targets:
.PHONY: all $(TASKS) $(TASKS:run=clean) clean
clean: $(TASKS:run=clean)
Last thing: I use some special variables in the recipes. $# stands for the target being built. $< stands for the first dependency. $* stands for the "stem" of a pattern rule (i.e., the part matched by the %).
Looks like what I am looking for:
ALLS=task01-all task02-all
BUILDS=${ALLS:-all=-build}
CLEANS=${ALLS:-all=-clean}
RUNS=${ALLS:-all=-run}
.PHONY: all $(ALLS) $(CLEANS) $(BUILDS) $(RUNS)
all: $(ALLS)
###############################################################################
$(ALLS): $(CLEANS) $(BUILDS) $(RUNS)
$(CLEANS):
rm ${#:-clean=} ${#:-clean=}.{ext,o} -f
$(BUILDS):
compiler ${#:-build=}.ext -O2 --make
$(RUNS):
./${#:-run=}
At work we use a common makefile that other makefiles include (via the include statement) and it has a generic "clean" target that kills some common files. I want to add on to that target in my new makefile so I can delete some specific files, but if I add a clean target in my makefile, it just overrides the old one.
I know I can just make a new target with a new name and have it call clean, and then do other stuff, but for sake of consistency I'd like to be able to just call make clean and have it do everything.
Is that possible?
I've seen this done at several shops. The most common approach is to use double-colon rules, assuming you're using something like GNU make. In your common makefile you would have something like this:
clean::
# standard cleanup, like remove all .o's:
rm -f *.o
Note that there are two colons following clean, not just one!
In your other makefile you just declare clean again, as a double-colon rule:
clean::
# custom cleanup, like remove my special generated files:
rm -f *.h.gen
When you invoke make clean, GNU make will automagically run both of these "branches" of the clean rule:
% make clean
rm -f *.o
rm -f *.h.gen
It's simple to set up and it composes quite neatly I think. Note that specifically because it is a double-colon rule, you don't get the "overriding commands" errors you normally get when you define two rules for the same target. That's sort of the point of double-colon rules.
You can write your own clean and make it a preq of the common clean.
clean: myclean
myclean:
rm whatever
Yours will run first. If for some reason you want the common clean to run first then the solution will be more complicated.
EDIT:
Here is the best solution I can see which runs the common rule before the local one:
include Makefile.common
clean:
$(MAKE) -f Makefile.common $#
rm whatever additional things
The include directive is necessary because the local makefile relies on the common one for things other than clean. The local clean rule overrides the common clean rule, but invokes the common clean rule before doing the additional work. (This overriding will cause some warnings, which is a nuisance; I don't know a good way to silence them.)
Use implicit rules:
existing-target: my-extention
my-extention:
echo running command 1
echo running command 2
Very simple make tutorial to ramp up.
When using :: you can run into issues since make complains when you mix single colon : and double colon :: rules:
a:
echo a
a::
echo aa
will result in:
. . .
*** target file `a' has both : and :: entries. Stop.
It seems like the common makefile's rule should be called something like common-clean. Then each main makefile would declare their clean rule as
clean: common-clean
and you're set.
If that isn't an option, you could take a look at double colon rules, but those introduce a whole other set of issues to consider.
Adding another possible solution I've seen for posterity... I know the OP was wary about changing the common makefile, but something like this works and involves minimal changes.
local makefile 1:
CLEAN=MyExe1 MyExe2
....
include /my/common/makefile
local makefile 2:
CLEAN=MyExe3 MyExe4
....
include /my/common/makefile
common makefile:
clean:
rm -f *.dep *.o *.a $(CLEAN)
Basically the idea is to define some variable (in this case CLEAN) in each local makefile with all the specific items you want to delete. Then the common makefile runs rm -f on all the common file types to delete, plus whatever was specifically flagged for deletion in each local makefile via the CLEAN variable. If there's nothing specific to delete, simply omit the variable declaration or leave it empty (CLEAN=)
So now if we run make clean for local makefile 1, it executes
rm -f *.dep *.o *.a MyExe1 MyExe2
And if we run make clean for local makefile 2, it executes
rm -f *.dep *.o *.a MyExe3 MyExe4
I've found a better solution:
.PHONY: my-extra-clean
clean: my-extra-clean
my-extra-clean:
rm <whatever-you-want>
include Makefile.common
The key line is clean: my-extra-clean. Ie, you can add dependencies in separate stanzas in different makefiles to add behaviour. my-extra-clean is run as a dependency of the root clean target.
For ours, we define a variable, EXTRAFILESTOCLEAN, then when the clean rule runs, it has a step to remove anything specified in the EXTRAFILESTOCLEAN variable
clean:
rm -f *.o
ifdef $(EXTRAFILESTOCLEAN)
rm -f $(EXTRAFILESTOCLEAN)
endif
That can cause unexpected problems if you set that variable to weird values, but you could guard against those by adding prefixes or other tests.
It's in the docs: https://www.gnu.org/software/make/manual/html_node/Overriding-Makefiles.html
So instead of include Makefile you use a wildcard target and forward it to the base Makefile:
# -include base.Makefile <--- not this
%:
#$(MAKE) -f base.Makefile $#