I'm building my project for multiple platforms and architectures, and I'd like to be able to define a single Make rule that describes how to build them. Here's a pseudo-Makefile of what I'm trying to accomplish:
RELEASEDIR := bin/release
PLATFORMS := darwin linux
ARCHES := amd64 arm arm64
$(RELEASEDIR)/myproject-$(forall $(PLATFORMS))-$(forall $(ARCHES)): $(GO_FILES)
mkdir -p $(RELEASEDIR)
GOOS=$(PLATFORM) GOARCH=$(ARCH) go build -o $# -ldflags "-s -w" ./cmd/myproject
I would like to be able to call make bin/release/myproject-darwin-amd64 (or any other combination of my two lists) and have it build for that platform and arch.
What's the correct way to accomplish this in Make?
You have a couple of options to do this. The first is to use some for loops with an eval, which will autogenerate all of the rules:
define makerule
$$(RELEASEDIR)/myproject-$(1)-$(2): $$(GO_FILES)
mkdir -p $$(RELEASEDIR)
GOOS=$(1) GOARCH=$(2) go build -o $$# -ldflags "-s -w"
endef
$(foreach platform,$(PLATFORMS),$(foreach arch,$(ARCHES),\
$(eval $(call makerule,$(platform),$(arch))))
I'm not a big fan of this approach as it is a bit harder to maintain (there's a bunch of sharp sticks with $(eval), and many people don't know what it does)
A slightly better approach might be to create a list of targets, and then parse the target names to get the arch and platform:
ALL_TARGETS := $(foreach plat,$(PLATFORMS),$(foreach arch,$(ARCHES),\
$(RELEASEDIR)/myproject-$(plat)-$(arch)))
$(ALL_TARGETS): $(GO_FILES)
#mkdir -p $(RELEASEDIR)
GOOS=$(word 2,$(subst -, ,$#)) GOARCH=$(word 3,$(subst -, ,$#)) go build -o $# -ldflags "-s -w"
Here, you take the target name, replace all the -'s with spaces, and then get the second/third words respectively to get your platform and arch name.
Related
I have a collection of Markdown files, videos, graphviz dot files, etc. and I'd like to use make to process the files as necessary, and do this without having to repeat myself any more than necessary.
In the Makefile, I'm writing build instructions for each individual file because a) there aren't so many, and b) each file could theoretically require a slightly different process to build. The standard %.html: %.md type of rules won't be useful because not all html files are build using the same commands, for example.
However, I can't figure out how to abbreviate the prerequisite section of the rule syntax. Each target file depends on a file in the source directory with the same name. How do I avoid typing the name twice in a basic rule such as the following? I feel like I should be able to just specify the stem of the filename, the target suffix, and the source suffix, and that's it. This seems like it should be a very simple thing, so I'm probably just using the wrong terms to describe it and search for the solution.
./dist/00-introductions/overview.html: ./src/00-introductions/overview.md
$(PANDOC) $(PANDOC_OPTIONS) $(PANDOC_REVEALJS) -o $# $<
In case it matters,
$ make -v
GNU Make 4.1
Built for x86_64-pc-linux-gnu
You could use separate static pattern rules for each target
./dist/00-introductions/overview.html: ./dist%.html: ./src%.md
$(PANDOC) $(PANDOC_OPTIONS) $(PANDOC_REVEALJS) -o $# $<
A better solution would be to group targets under multiple generic rules, and use target-specific variables to tweak the recipes:
targets1 := ./dist/00-introductions/overview.html
targets2 := ./dist/00-introductions/foobar.html
$(targets1): ./dist%.html: ./src%.md
$(PANDOC) $(PANDOC_OPTIONS) $(PANDOC_REVEALJS) -o $# $<
$(targets2): ./dist%.html: ./src%.md
$(FOOBAR) $(FOOBAR_OPTIONS) -o $# $<
./dist/00-introductions/overview.html: PANDOC_OPTIONS := -foo
./dist/00-introductions/foobar.html: FOOBAR_OPTIONS := -bar
I am aware of tools like CMake and GNU Autotools but I'm trying to write a universal build system myself, to use for my C and C++ projects. I'll briefly explain how it works and hopefully, someone can suggest either improvements or a better design altogether.
The build system proper lives in one of the project's subdirectories (I import it as a Git submodule). The project's root directory has a wrapper makefile that defines a couple of macros and includes the main makefile from said subdirectory. That does most of the work: it follows the directory organization scheme (i.e., it outputs libraries in lib, binaries in bin, etc.), it handles automatic dependencies for the source code and the DocBook documentation, and provides the de facto standard targets: all, test, clean, install, as well as others.
Here's what a wrapper makefile that builds two binaries, foo and bar, might look like:
# foo-specific macros
FOO_SRC_FILES = foo1.c foo2.c foo3.c
FOO_OBJ_FILES = $(FOO_SRC_FILES:.c=.o)
FOO_BIN_FILE = foo
# bar-specific macros
BAR_SRC_FILES = bar1.c bar2.c
BAR_OBJ_FILES = $(BAR_SRC_FILES:.c=.o)
BAR_BIN_FILE = bar
# Inform the build system about them
SRC_FILES = $(FOO_SRC_FILES) $(BAR_SRC_FILES)
OBJ_FILES = R(BAR_OBJ_FILES) $(BAR_OBJ_FILES)
BIN_FILES = $(FOO_BIN_FILE) $(BAR_BIN_FILE)
# Only install the binaries. If I were building a library, I would instead
# select the "lib" and perhaps "include" directories.
INSTALL = bin
INSTALL_DIR = /usr/share
# Use the build system
include build/build.mk
Now here's the problem. While build.mk can use pattern rules to create dependency and object files, there's only one OBJ_FILES and only one BIN_FILES. So if I put a pattern rule like the following in the build system that looks like this:
$(BIN_DIR)/$(BIN_FILES): $(OBJ_FILES:%=$(OBJ_DIR)/%) $(LIB_FILES:%=$(LIB_DIR)/%) | $(BIN_DIR)
$(CC) $(LDFLAGS) -o $# $(OBJ_FILES:%=$(OBJ_DIR)/%) -L $(LIB_DIR) $(LIB_FILES:lib%.a=-l %)
then foo would depend on and link with everything that bar does and vice versa. So what I end up doing is asking the user to put these rules in the wrapper makefile, even though they feel like they belong in build.mk:
$(BIN_DIR)/$(FOO_BIN_FILE): $(FOO_OBJ_FILES:%=$(OBJ_DIR)/%) $(FOO_LIB_FILES:%=$(LIB_DIR)/%) | $(BIN_DIR)
$(CC) $(LDFLAGS) -o $# $(FOO_OBJ_FILES:%=$(OBJ_DIR)/%) -L $(LIB_DIR) $(FOO_LIB_FILES:lib%.a=-l %)
$(BIN_DIR)/$(BAR_BIN_FILE): $(BAR_OBJ_FILES:%=$(OBJ_DIR)/%) $(BAR_LIB_FILES:%=$(LIB_DIR)/%) | $(BIN_DIR)
$(CC) $(LDFLAGS) -o $# $(BAR_OBJ_FILES:%=$(OBJ_DIR)/%) -L $(LIB_DIR) $(BAR_LIB_FILES:lib%.a=-l %)
The same issue applies to libraries as well, of course. The upside is that these rules can be copied and pasted almost verbatim; only the prefixes need to be changed (e.g., FOO or BAR).
Ideas to fix this include:
Asking the user to have separate wrapper makefiles for separate things (e.g., one for foo and another for bar) but that is just terrible.
Changing things up a little bit and then using m4 to do some preprocessing but I don't want to go through that unless a more elegant solution doesn't exist.
I would really appreciate some ideas.
PS: I know that the pattern matching expressions in the last two code samples can be replaced with text functions but those are GNU Make-specific. The style I used is more portable and is in fact on the list of additions for the next version of the POSIX standard.
I have begin to develop a similar system for my own C projects, but the logic I use does rely on some features which I believe are specific to GNU Make.
The main idea is to use a combinaison of $(eval) and $(call), by defining the logic of the build system, and then applying to the project tree.
To do so, I have in each of my directories and subdirectories a piece of Makefile of the following form, which I name Srcs.mk:
SRC := foo.c foo_bar.c bar.c
TARGET := foo_bar
SRC_DIR := src
OBJ_DIR := obj
I define a variable, which is in fact a macro, which is expanded with $(call) and then passed to $(eval). It's defined this way:
define get_local_variables
include Srcs.mk
$1SRC := $(SRC)
$1SRC_DIR := $(SRC_DIR)
$1OBJ_DIR := $(OBJ_DIR)
$1TARGET := $(TARGET)
TARGET :=
SRC :=
SRC_DIR :=
OBJ_DIR :=
$(call get_local_variables, $(DIR)) will expand to the above, with $1 replaced by the content of $(DIR). Then it will be treated as a Makefile fragment by $(eval)
This way, I fill per-directory variables for each of my directory.
I have then a handful or other rules which use this variables, using the same principles.
### Macros ###
obj = $(patsubst %.c,$($1OBJ_DIR)/%.o,$($1SRC))
define standard_rules
$($1TARGET): $(obj)
$$(LINK)
$(obj): $($1OBJ_DIR)/%.o:$($1SRC_DIR)/%.c | $($1OBJ_DIR)
$$(COMPILE)
endef
The variable are computed $(call), then expanded and read as makefile fragments by $(eval).
(I use static pattern rules but that it not intrinsic to the idea).
The whole idea is basically to define directories as a kind of namespace, with data attached to them, and then run function over them.
My actual system is a bit more complicated, but that the whole idea.
If you have a way to emulate $(eval) and $(call) (I think these are specific to GNU make, but not sure), you could try that approach.
You can also implement non recursive make this way, by adding a SUBDIRS variables in each directory and running recursively the same macro which is run on the current one. But it should been done carefully, not to mess it up with the order of expansion and evaluation in make.
So get_local_variables need to be evaluated before the rest of the macros are expanded.
(My project is visible on my Github account if you want to take a look, under make-build-system. But it is far from be complete enough^).
Be aware, though, that this is quite painful to debug when things go wrong. Make (at least, GNU) basically catch the error (when there is one) on the higher $(call) or $(eval) expansion.
I have developed my own non-recursive build system for GNU make, called prorab, where I solved the problem you described as follows.
The approach to solve your problem is somewhat similar to what #VannTen described in his answer, except that I use a macro to clean all state variables before defining build rules for the next binary.
For example, a makefile which builds two binaries could look like this:
include prorab.mk
this_name := AppName
this_ldlibs += -lsomelib1
this_cxxflags += -I../src -DDEBUG
this_srcs := main1.cpp MyClass1.cpp
$(eval $(prorab-build-app))
$(eval $(prorab-clear-this-vars))
this_name := AnotherppName
this_ldlibs += -lsomelib1
this_cxxflags += -I../src -DDEBUG
this_srcs := main2.cpp MyClass2.cpp
$(eval $(prorab-build-app))
So, in this example it will build two binaries: AppName and AnotherppName.
As you can see the build is configured by setting a number of this_-prefixed variables and the calling the $(eval $(prorab-build-app)) which expands to defining all the build, install, clean etc. rules.
Then a call to $(eval $(prorab-clear-this-vars)) clears all this_-prefixed variables, so that those can be defined again from scratch for the next binary, and so on.
Also, the very first line which includes the prorab.mk also cleans all this_-prefixed variables of course, so that the makefiles can be safely included into each other.
You can read more about that build system concepts here https://github.com/cppfw/prorab/blob/master/wiki/HomePage.adoc
I'm trying to get a top-level makefile to call make in a number of subfolders. The top-level has several targets and the important bit is shown below:
MAKE_DIRS := $(dir $(wildcard apps/**/Makefile))
.PHONY: clean_apps apps $(MAKE_DIRS)
clean_apps: TARGET_INFO := clean
apps clean_aps: $(MAKE_DIRS)
$(MAKE_DIRS):
$(MAKE) -C $# $(TARGET_INFO)
Now this works fine when I call the targets independently:
make apps; make clean_apps
However if I call them on the same commandline with:
make clean_apps apps
Then the apps target justs say nothing to do. I guess it's something to do with the dependency on the directories not having changed between invocations, but I thought the .PHONY command would avoid that problem...
I'm happy to know if there's a better way to deal with this.
Thanks,
bob
It is something much more simpler :
SUBDIRS := $(dir $(shell find apps -name "Makefile"))
.PHONY: all clean
all clean:
$(foreach DIR, $(SUBDIRS), $(MAKE) $(MAKEFLAGS) -C $(DIR) $#;)
What is a best practice to writing reusable code in Makefiles?
Suppose I have a Makefile:
.PHONY: all task01-all task01-clean task01-run task02-all task02-clean task02-run
all: task01-all task02-all
###############################################################################
task01-all: task01-clean task01 task01-run
task01-clean:
rm task01 task01.{exi,o} -f
task01:
compiler task01.ext -O2 --make
task01-run:
./task01
###############################################################################
task02-all: task02-clean task02 task02-run
task02-clean:
rm task02 task02.{exi,o} -f
task02:
compiler task02.ext -O2 --make
task02-run:
./task02
Now I want to add new family of tasks (task03), and I need to copypaste whole section, make s/02/03/ for it and add them to .PHONY section - it's noisy, disgusting and just not right.
How can I avoid that? Could I redefine all tasks with templates somehow to have nice mechanism for adding new task group in one line?
Since the question is about writing re-usable code in Makefiles, I'll give an example of how to use pattern rules in GNU Make (it looks like that's what you're using since you mention the .PHONY target). However, if you're not using any of Make's dependency checking, it may be simpler to do this with a shell script--something like:
#!/bin/sh
TASKS="task01 task02 task03"
for i in $TASKS; do
rm $i $i.ext $i.o -f;
compiler $i.ext -O2 --make;
./$i;
done
But, to expand the principle to Make, we have another issue to tackle. Lines of the form:
task01-all: task01-clean task01 task01-run
don't tell Make in what order to build the pre-requisites--just that they all need to be done before task01-all gets built. Instead, each step to run should depend on the step before it. Something like this:
TASKS=task01-run task02-run task03-run
.PHONY: all $(TASKS) $(TASKS:run=clean)
all: $(TASKS)
$(TASKS:run=clean): %-clean:
rm $* $*.ext $*.o -f
%: %.ext | %-clean
compiler $< -O2 --make
$(TASKS): %-run: %
./$<
The rules with % are called "pattern rules", and they're a great tool to avoid re-writing the same rule multiple times for different targets. One caveat is that Make doesn't normally check pattern rules for a .PHONY target; we tell Make to do this explicitly by prefixing those rules with the list of targets and a second colon (e.g., $(TASKS):).
Since task01-run needs task01 in order to work, we make %-run targets depend on %. Also, your Makefile shows that you want to run clean every time, so we make % depend on %-clean. Since %-clean doesn't actually produce any output, we make this an "order only" dependency--Make won't look for a time-stamp or anything, it will just run the %-clean rule first any time it needs to run the % rule. "Order only" dependencies are placed after a |:
%: %.ext | %-clean
It's worth mentioning that one of Make's greatest strengths is that it can save time by not repeating work that doesn't need to be repeated--i.e., it only runs a rule if the dependencies are newer than the target. So, you could leave off the dependency on %-clean, which would cause make to only run compiler $< -O2 --make if %.ext is newer than %:
%: %.ext
compiler $< -O2 --make
You could then add a rule just to run all of the %-clean targets:
.PHONY: all $(TASKS) $(TASKS:run=clean) clean
clean: $(TASKS:run=clean)
Last thing: I use some special variables in the recipes. $# stands for the target being built. $< stands for the first dependency. $* stands for the "stem" of a pattern rule (i.e., the part matched by the %).
Looks like what I am looking for:
ALLS=task01-all task02-all
BUILDS=${ALLS:-all=-build}
CLEANS=${ALLS:-all=-clean}
RUNS=${ALLS:-all=-run}
.PHONY: all $(ALLS) $(CLEANS) $(BUILDS) $(RUNS)
all: $(ALLS)
###############################################################################
$(ALLS): $(CLEANS) $(BUILDS) $(RUNS)
$(CLEANS):
rm ${#:-clean=} ${#:-clean=}.{ext,o} -f
$(BUILDS):
compiler ${#:-build=}.ext -O2 --make
$(RUNS):
./${#:-run=}
Let's say you have a Makefile with two pseudo-targets, 'all' and 'debug'. The 'debug' target is meant to build the same project as 'all', except with some different compile switches (like -ggdb, for example). Since the targets use different compile switches, you obviously need to rebuild the entire project if you switch between the two. But GNUmake doesn't naturally recognize this.
So if you type make all you'll get
Building ...
...
Then if you type make debug, you get
make: Nothing to be done for `debug'.
So my question is: how do you implement a clean solution in the Makefile to notice that the last build used a different pseudo-target, or different compile switches, than the one you want currently? If they are different, the Makefile would rebuild everything.
Put the build products into different directory trees (whilst keeping one copy of the source of course). That way you are always just a short compile from an up-to-date build, be it debug or release (or even others). No possibility of confusion either.
EDIT
Sketch of the above.
src := 1.c 2.c 3.c
bare-objs := ${src:%.c=%.o}
release-objs := ${bare-objs:%=Release/%}
debug-objs := ${bare-objs:%=Debug/%}
Release/prog: ${release-objs}
Debug/prog: ${debug-objs}
${release-objs}: Release/%.o: %.c # You gotta lurve static pattern rules
gcc -c $< -o $#
${debug-objs}: Debug/%.o: %.c
gcc -c $< -o $#
Release/prog Debug/prog:
gcc $^ -o $#
.PHONY: all
all: Release/prog ; echo $# Success
.PHONY: debug
debug: Debug/prog ; echo $# Success
(Disclaimer: not tested, nor even run through make.)
There you go. It's even -j safe so you can do make -j5 all debug. There is a lot of obvious boiler plate just crying out for tidying up.
Keeping variant sets of object files (as in bobbogo's solution) is probably the best way, but if for some reason you don't want to do that, you can use empty files as markers, to indicate which way you last built the executable:
%-marker:
#rm -f $(OBJECTS) *-marker
#touch $#
debug: GCCFLAGS += -ggdb
debug: SOMEOTHERFLAG = WHATEVER
all debug: % : %-marker
#echo making $#
#$(MAKE) -S GCCFLAGS='$(GCCFLAGS)' SOMEOTHERFLAG='$(SOMEOTHERFLAG)' main
There are other variants on this idea; you could have a small file containing the flag settings, which the makefile would build and include. That would be clever, but not really any cleaner than this.
The only clean solution is to incorporate the difference into the target names.
E.g. you can define a variable $(DEBUG) and consistently use it in all targets that depend on the compile step.