What is a best practice to writing reusable code in Makefiles?
Suppose I have a Makefile:
.PHONY: all task01-all task01-clean task01-run task02-all task02-clean task02-run
all: task01-all task02-all
###############################################################################
task01-all: task01-clean task01 task01-run
task01-clean:
rm task01 task01.{exi,o} -f
task01:
compiler task01.ext -O2 --make
task01-run:
./task01
###############################################################################
task02-all: task02-clean task02 task02-run
task02-clean:
rm task02 task02.{exi,o} -f
task02:
compiler task02.ext -O2 --make
task02-run:
./task02
Now I want to add new family of tasks (task03), and I need to copypaste whole section, make s/02/03/ for it and add them to .PHONY section - it's noisy, disgusting and just not right.
How can I avoid that? Could I redefine all tasks with templates somehow to have nice mechanism for adding new task group in one line?
Since the question is about writing re-usable code in Makefiles, I'll give an example of how to use pattern rules in GNU Make (it looks like that's what you're using since you mention the .PHONY target). However, if you're not using any of Make's dependency checking, it may be simpler to do this with a shell script--something like:
#!/bin/sh
TASKS="task01 task02 task03"
for i in $TASKS; do
rm $i $i.ext $i.o -f;
compiler $i.ext -O2 --make;
./$i;
done
But, to expand the principle to Make, we have another issue to tackle. Lines of the form:
task01-all: task01-clean task01 task01-run
don't tell Make in what order to build the pre-requisites--just that they all need to be done before task01-all gets built. Instead, each step to run should depend on the step before it. Something like this:
TASKS=task01-run task02-run task03-run
.PHONY: all $(TASKS) $(TASKS:run=clean)
all: $(TASKS)
$(TASKS:run=clean): %-clean:
rm $* $*.ext $*.o -f
%: %.ext | %-clean
compiler $< -O2 --make
$(TASKS): %-run: %
./$<
The rules with % are called "pattern rules", and they're a great tool to avoid re-writing the same rule multiple times for different targets. One caveat is that Make doesn't normally check pattern rules for a .PHONY target; we tell Make to do this explicitly by prefixing those rules with the list of targets and a second colon (e.g., $(TASKS):).
Since task01-run needs task01 in order to work, we make %-run targets depend on %. Also, your Makefile shows that you want to run clean every time, so we make % depend on %-clean. Since %-clean doesn't actually produce any output, we make this an "order only" dependency--Make won't look for a time-stamp or anything, it will just run the %-clean rule first any time it needs to run the % rule. "Order only" dependencies are placed after a |:
%: %.ext | %-clean
It's worth mentioning that one of Make's greatest strengths is that it can save time by not repeating work that doesn't need to be repeated--i.e., it only runs a rule if the dependencies are newer than the target. So, you could leave off the dependency on %-clean, which would cause make to only run compiler $< -O2 --make if %.ext is newer than %:
%: %.ext
compiler $< -O2 --make
You could then add a rule just to run all of the %-clean targets:
.PHONY: all $(TASKS) $(TASKS:run=clean) clean
clean: $(TASKS:run=clean)
Last thing: I use some special variables in the recipes. $# stands for the target being built. $< stands for the first dependency. $* stands for the "stem" of a pattern rule (i.e., the part matched by the %).
Looks like what I am looking for:
ALLS=task01-all task02-all
BUILDS=${ALLS:-all=-build}
CLEANS=${ALLS:-all=-clean}
RUNS=${ALLS:-all=-run}
.PHONY: all $(ALLS) $(CLEANS) $(BUILDS) $(RUNS)
all: $(ALLS)
###############################################################################
$(ALLS): $(CLEANS) $(BUILDS) $(RUNS)
$(CLEANS):
rm ${#:-clean=} ${#:-clean=}.{ext,o} -f
$(BUILDS):
compiler ${#:-build=}.ext -O2 --make
$(RUNS):
./${#:-run=}
Related
In embedded projects you typically convert an ELF output from gcc into a raw binary form with objcopy. I tried to write a pattern rule for that: %.bin: % (executable output from gcc doesn't have a suffix, at least on Linux). This fails with make: *** No rule to make target ...
Here is a complete test case:
test.c:
int main(int argc, char *argv[]) {
return 0;
}
Makefile:
%.bin: %
objcopy -O binary $< $#
This works by saying make test test.bin, that is, by explicitly asking it to build test executable first. It does not work by saying make test.bin. Somehow in that case make does not realize it can make prerequisite with a built in rule.
I know I can fix this by writing the rule as test.bin: test but I'd like to know, why this pattern rule does not work and can I make it work somehow.
Furthermore, I think the obvious and practical ways to make it work breaks the nice and elegant built in rule system. It could be rescued if I could put an option into e.g. LDFLAGS to make linker output the executable with some suffix e.g. .elf (do I still have to rewrite the built in rule for linking? Oh well...).
I admit, this is purely academic problem born in pedantic mind.
Your idea is correct, but you hit a caveat of make -- make will not chain pure % targets with other implicit rules --
https://www.gnu.org/software/make/manual/html_node/Chained-Rules.html#Chained-Rules
... for performance reasons make will not consider non-terminal
match-anything rules (i.e., ‘%:’) when searching for a rule to build a
prerequisite of an implicit rule (see Match-Anything Rules).
You can generate an executable output with a suffix in gcc by passing the output -o option with the name and extension.
Example:
# Give your elf file name
ELF_FILE= test.elf
$(ELF_FILE): $(OBJ_FILES)
echo "Creating elf file"
gcc.exe -o $(ELF_FILE) $(OBJ_FILES)
%.bin: %.elf
objcopy -O binary $< $#
My example works when running make like this: make test test.bin because, IIUC, test is built first and then the prerequisite is already there for building test.bin. Exploiting that, I modified the Makefile like this:
all: $(subst .bin, , $(filter %.bin, $(MAKECMDGOALS))) $(MAKECMDGOALS)
%.bin: %
objcopy -O binary $< $#
Now I can run make test.bin and make builds first test and then test.bin. Or I can run make foo bar.bin and make tries to build foo, bar and bar.bin. This is the closest thing I can come up with. On the positive side is that the Makefile does not have to reference explicitly any file names and no built in rules are rewritten.
I am doing a nasm project, and I need to execute the ej and use as a parameter the ex.asm . I tried searching through GNU how can I pick one by one the parameter. My solution has been writing ex1_ and ex2_, but I want to put those inside the $(ex) dependency, so I don't have to replicate multiple times the same code. Is there any way?
Thank you in advance
The code:
ej = ej1_gen ej2_gen
ex = ex1 ex2
# -----------------------------------------------
all: $(ej) $(ex)
exs: ex1_ ex2_
# -----------------------------------------------
$(ex): exs
nasm -g -o $#.o -f elf32 $#.asm
$(CC) $(FLAGS) -m32 -o $# $#.o alfalib.o
ex1_:
./ej1_gen ex1.asm
ex2_:
./ej2_gen ex2.asm
As I read the question, you have programs or scripts ej1_gen and ej2_jen in the project, serving to generate the wanted assembly sources. They each take the name of the output file as a command-line argument. Parts of this answer would need to be adjusted if that's a misinterpretation.
Rules to describe how to build the assembly files should designate the resulting assembly file(s) as the target. Also, supposing that the code-generator programs are part of the project, they should be designated as prerequisites, since changing those could cause them to produce different outputs. Any configuration files or similar that they read to inform their results should also be named as prerequisites (not shown). That leads to rules something like this:
ex1.asm: ej1_gen
./ej1_gen $#
ex2.asm: ej2_gen
./ej2_gen $#
It sounds like you may be asking for a way to express that via just one rule covering both, but I would not do so in this case. I don't think you get any clearer than the above, even if there are more than two assembly files to generate. It might be different if the same code generator program were being used, with different options, to generate all the assembly files, or perhaps if the generator name could be derived more directly from the target name.
With those rules in place, you can write a generic suffix rule or pattern rule to assemble the resulting files. Since you tag [gnu], I'll assume that a pattern rule is acceptable:
%.o: %.asm
nasm -g -o $# -f elf32 $<
And you can take a similar approach to expressing a link rule:
%: %.o alfalib.o
$(CC) $(FLAGS) -m32 -o $# $^
With that, you should be able to get rid of the ej variable and the exs target, too, leaving
all: $(ex)
as the only other rule (and it should still appear first in the file, as it does now).
I have these recipes in my Makefile. They generate cross-compiled objects for ARM architecture and link them into an elf binary:
%.ao: %.c
$(ARM_CC) $(ARM_CPPFLAGS) $(ARM_FLAGS) $(CFLAGS) -c -o $# $<
%.elf: %.ao startup_stm32f0xx.ao system_stm32f0xx.ao
$(ARM_CC) $(ARM_FLAGS) $other_arguments -o $# $^
This works fine from a clean build.
Contrary to my expectation, if I then say touch foo.c; make foo.elf, gmake responds with
make: 'foo.elf' is up to date.
If I try to make foo.ao, gmake says that it, too , is up to date.
What am I missing?
Edit after reading the comments:
TLDR: I did have multiple rules matching the same target, as John Bollinger alluded and HardcoreHenry said specifically.
In addition to the rules above, there's a rule for assembly sources so I can use those vendor files:
%.ao: %.s
$(ARM_CC) $(ARM_CPPFLAGS) $(ARM_FLAGS) $(CFLAGS) -c -o $# $<
I had been debugging some macros, and used -save-temps to look at preprocessor output. This option also writes .s files. So after I'd run make foo.elf, I'd have the following in my directory:
foo.c
foo.i
foo.s
foo.ao
foo.elf
I can touch foo.c, but make sees that there's a foo.s which is older than foo.ao, and produces the output that it does. On a clean build, there is no foo.s, so make finds the %.c:%.ao rule and the build proceeds from foo.c.
(BTW, .ao stands for ARM object. In addition to cross-compiling for AMR, I compile many of the sources to run unit tests on the host, using the built-in .o:.c rule)
I'm not a fan of pattern rules.
Make can make very strange decisions on which rules apply depending on whatever is lying around on your hard disks.
It's all a bit arbitrary.
Much better IMHO to tell make exactly what files you need for a target.
It's pretty easy too.
Just prefix your pattern rule with the list of targets you actually want it to apply to.
This makes it a Static Pattern Rule.
objects := main.ao tools.ao devices.ao# etc
${objects}: %.ao: %.c
$(ARM_CC) $(ARM_CPPFLAGS) $(ARM_FLAGS) $(CFLAGS) -c -o $# $<
%.elf: ${objects} startup_stm32f0xx.ao system_stm32f0xx.ao
$(ARM_CC) $(ARM_FLAGS) $other_arguments -o $# $^
As an added bonus, make now won't try to create the pre-existing startup_stm32f0xx.ao and system_stm32f0xx.ao.
Usually I find it nicer to list the source files, but YMMV:
sources := main.c tools.c devices.c
objects := $(patsubst $.c,%.ao,${sources})
(P.S. Using a Static Pattern Rule doesn't really give you any advantage over a normal rule in this noddy case. I just wanted to show a small tweak that would make your makefiles much more consistent in their behaviour.)
I know it's bad form to use an answer to respond to another answer, but I ran out of space in a comment to #bobbogo's answer.
Sorry but I can't agree with your assessment of pattern rules. It's not true that you will get "strange decisions" based on "whatever is lying around on your harddisks", and it's certainly not arbitrary.
There is one advantage of static pattern rules over pattern rules, and that is also its downside: a static pattern rule is a shorthand for creating an explicit rule, so that rule will always be used to build that target. A pattern rule, on the other hand, is just one possible way to build a target: if the prerequisites of a pattern rule don't exist and can't be made, then make keeps going and looks for other pattern rules that might be able to build that target.
So if you have multiple possible ways you can build a target then an explicit rule cannot be used for that.
The problem with pattern rules is that if NO pattern rule applies then make just assumes there is no rule to build that target. If the target exists then make simply says "up to date" (as we see in the question) since there's no rule to build it. That can be confusing to users.
If you use an explicit rule (including a static pattern rule) and some prerequisite doesn't exist and can't be created, then make will exit with an error, which can make it easier to figure out what went wrong.
Note: using MinGW's make (should be GNU make)
i have a couple of -include statements in my makefile to import dependencies which were generated using g++ -MM. However I would like to only do this when necessary. I have several different build targets and I don't want all of their respective dependency files to be included since this takes a while (suppose I'm running make clean: no need to include them in this case)
Here's the format of my makefile.
DEPS_debug = $(patsubst %.cpp,build_debug/%.d,$(SRC))
OBJ_debug = $(patsubst %.cpp,build_debug/%.o,$(SRC))
all: program_debug
-include $(DEPS_debug) #make: include: Command not found
program_debug: $(OBJ_debug)
$(CC) $(CFLAGS) $(OBJ_debug) -o $#
If you really don't want to include those files needlessly, you have a couple of options:
You can put in a conditional as Diego Sevilla suggests (but I would recommend using MAKECMDGOALS so that you can write a more flexible version, specific to targets, e.g. you'll include foo.d if and only if you're making foo.o).
You can use make recursively (heresy!), invoking $(MAKE) for each target object, using a makefile that includes that target's dependencies.
But actually including the file takes negligible time, it's the rebuilding of the file (automatic for any included file that's out of date) that takes time.
If needless rebuilding is what you want to avoid, you can use a very clever trick. When must foo.d be rebuilt? Only when something about foo has changed. But in that case foo.o must also be rebuilt. So don't have a seperate rule for foo.d, just rebuild it as a side effect of making foo.o. That way you can include all dependency files and not waste time rebuilding them if they aren't needed.
EDIT:
I'm astounded that merely including these files can add 2-3 seconds to make clean. My last paragraph is off the mark, so let me expand on the first two options.
If all is the only target for which these files should be included, and you make all from the command line (and not e.g. make all tests tarball install kitchenSink), then this will do it:
ifeq ($(MAKECMDGOALS),all)
-include $(DEPS_debug)
endif
Note that this will not include foo.d if you make foo.o. You can write a more sophisticated conditional, something like
$(foreach targ,$(MAKECMDGOALS),$(eval $(call include_deps $(targ)))...
but that's pretty advanced, so let's get a simple version working first.
If you'd rather avoid the conditional and use recursive Make, the simplest way is to split the makefile in two:
makefile:
all:
$(MAKE) -f makefile.all
clean:
rm whatever
...other rules
makefile.all:
DEPS_debug = $(patsubst %.cpp,build_debug/%.d,$(SRC))
OBJ_debug = $(patsubst %.cpp,build_debug/%.o,$(SRC))
-include $(DEPS_debug)
all: program_debug
program_debug: $(OBJ_debug)
$(CC) $(CFLAGS) $(OBJ_debug) -o $#
Indenting a line by a TAB makes make think it's a command to be passed to the shell (as you found out). It doesn't work that way.
The - in front of include suppresses errors that might result from DEPS_debug not existing (e.g. when running clean or release without having had a dependency-file-generating call first). Since DEPS_debug is not a dependency of those rules (clean / release), your dependency files do not get generated when you call them, and everything is fine. I don't really see the problem you're having - you don't have to make the include conditional.
Perhaps you'd like to change your approach, though. Instead of having a seperate *.d target, with a seperate -M preprocessor pass, you might want to try something like -MMD -MP which generates the dependency files inline during code generation, in your standard *.c -> *.o pass.
(I know this sounds completely wrong at first, but when you think about it, it makes sense. Makefile logic is a bit backwards that way, unless you're familiar with functional programming.)
includes are independent of the rules, as they are makefile indications, not compilation indications. You can, however, use makefile conditionals based on special makefile variables such as MAKECMDGOALS, that is set to the default goal:
ifeq ($(MAKECMDGOALS),all)
-include whatever
endif
This is included when no default goal is specified. You can change the condition to specify the exact goal you want to check to include other sub-makefiles.
Let's say you have a Makefile with two pseudo-targets, 'all' and 'debug'. The 'debug' target is meant to build the same project as 'all', except with some different compile switches (like -ggdb, for example). Since the targets use different compile switches, you obviously need to rebuild the entire project if you switch between the two. But GNUmake doesn't naturally recognize this.
So if you type make all you'll get
Building ...
...
Then if you type make debug, you get
make: Nothing to be done for `debug'.
So my question is: how do you implement a clean solution in the Makefile to notice that the last build used a different pseudo-target, or different compile switches, than the one you want currently? If they are different, the Makefile would rebuild everything.
Put the build products into different directory trees (whilst keeping one copy of the source of course). That way you are always just a short compile from an up-to-date build, be it debug or release (or even others). No possibility of confusion either.
EDIT
Sketch of the above.
src := 1.c 2.c 3.c
bare-objs := ${src:%.c=%.o}
release-objs := ${bare-objs:%=Release/%}
debug-objs := ${bare-objs:%=Debug/%}
Release/prog: ${release-objs}
Debug/prog: ${debug-objs}
${release-objs}: Release/%.o: %.c # You gotta lurve static pattern rules
gcc -c $< -o $#
${debug-objs}: Debug/%.o: %.c
gcc -c $< -o $#
Release/prog Debug/prog:
gcc $^ -o $#
.PHONY: all
all: Release/prog ; echo $# Success
.PHONY: debug
debug: Debug/prog ; echo $# Success
(Disclaimer: not tested, nor even run through make.)
There you go. It's even -j safe so you can do make -j5 all debug. There is a lot of obvious boiler plate just crying out for tidying up.
Keeping variant sets of object files (as in bobbogo's solution) is probably the best way, but if for some reason you don't want to do that, you can use empty files as markers, to indicate which way you last built the executable:
%-marker:
#rm -f $(OBJECTS) *-marker
#touch $#
debug: GCCFLAGS += -ggdb
debug: SOMEOTHERFLAG = WHATEVER
all debug: % : %-marker
#echo making $#
#$(MAKE) -S GCCFLAGS='$(GCCFLAGS)' SOMEOTHERFLAG='$(SOMEOTHERFLAG)' main
There are other variants on this idea; you could have a small file containing the flag settings, which the makefile would build and include. That would be clever, but not really any cleaner than this.
The only clean solution is to incorporate the difference into the target names.
E.g. you can define a variable $(DEBUG) and consistently use it in all targets that depend on the compile step.