In the current system, there was a ghetto hack to initiate a parallel build for the system. For instance, to call a parallel make required make JOBS=8 instead of make -j8. I have since fixed the makefile, however there are a lot of previous dependencies on the JOBS flags within scripts that call make. I was hoping to recursively call make as a workaround as such:
ifdef JOBS
%:
$(MAKE) $(MAKECMDGOALS) JOBS= -j$(JOBS)
endif
This has some odd behavior though. It will first call make JOBS= -j8 as it should, but after this rule is completed it seems to go on and rebuild everything again that is in $(MAKECMDGOALS).
#Example
TARGETS = lib0 lib1 lib2
ifdef JOBS
%:
$(MAKE) $(MAKECMDGOALS) -j$(JOBS) JOBS=
endif
all: $(TARGETS)
lib%:
#echo g++ $#
Cmd: make JOBS=8.
This is fine for most rules, however it keeps wasting time rebuilding PHONY rules, so this should be fixed. Is there a way to completely reroute a make JOBS=N command to make -j$(JOBS) without executing any other rules redundantly in either scope? (meaning nothing executes except a submake in make JOBS=N, and the submake is only called once with all $(MAKECMDGOALS) at once.)
EDIT: I would also like to avoid large ifdef; else; endif statements, at the very most putting a self contained one at the top of the file. I'm starting to think the solution may require something like that though:
TARGETS = lib0 lib1 lib2
ifdef JOBS
%: unique_make
#echo $# built > /dev/null
.PHONY: unique_make
unique_make:
$(MAKE) $(MAKECMDGOALS) -j$(JOBS) JOBS=
else
all: $(TARGETS)
lib%:
#echo g++ $#
endif
In GNU make, you can modify the make flags within a Makefile. Why not try something like this:
ifdef JOBS
MAKEFLAGS+=-j$(JOBS)
endif
Related
I have inherited a large branched project? that requires a volatile set of .a archives $(LIB_FILES) to be included into link target, located in some directories $(LIB_DIRS). I can write an expression like this:
LIBDEP = $(foreach ldir, $(LIB_DIRS), \
$(filter $(addprefix %/, $(LIB_FILES)), $(wildcard $(ldir)/* )))
The problem is that they might not exist at moment of make's invocation and would be built by invoking $(MAKE) inside of another target's rule, which is a prerequisite to the link step.
The problem is actual list of files that should be created varies on external factors determined at their build steps, that I can't hard-code it properly, without turning makefile into a spaghetti mess and said variable is not re-evaluated at the moment of link command invocation.
I have suspicion that $(eval ) function can be used somehow, but manual is not very forthcoming as well as I didn't found examples of its use in this way.
Toolchain: GCC and binutils, make 3.81
Another solution is to create an explicit dependency of your make script on the output of the step which currently creates the variable $(LIB_FILES). This is what the manual is dealing with in the chapter How makefiles are remade and it aims at the technique which make is best at, namely deriving dependencies from the existence and timestamp of files (instead of variables). The following hopefully depicts your situation with the process of deducing a new set of libraries simulated by the two variables $(LIBS_THIS_TIME) and $(LIB_CONFIG_SET).
LIBS_THIS_TIME = foo.a:baz.a:bar.a
LIB_CONFIG_SET = $(subst :,_,$(LIBS_THIS_TIME))
include libdeps.d
linkstep:
#echo I am linking $^ now
touch $#
libdeps.d: $(LIB_CONFIG_SET)
-rm libdeps.d
$(foreach lib,$(subst :, ,$(LIBS_THIS_TIME)),echo linkstep: $(lib) >> libdeps.d;)
$(LIB_CONFIG_SET):
touch $#
If make finds that libdeps.d is not up to date to your current library configuration it is remade before make executes any other rule, although it is not the first target in the makefile. This way, if your build process creates a new or different set of libraries, libdeps.d would be remade first and only then make would carry on with the other targets in your top makefile, now with the correct dependecy information.
It sometimes happens that you need to invoke make several times in succession. One possibility to do this is to use conditionals:
ifeq ($(STEP),)
all:
<do-first-step>
$(MAKE) STEP=2 $#
else ifeq ($(STEP),2)
all:
<do-second-step>
$(MAKE) STEP=3 $#
else ifeq ($(STEP),3)
all:
<do-third-step>
endif
In each step you can generate new files and have them existing for the next step.
I`m using a makefile and call some makefiles in subdirectories.
Can anyony explain me, why this works
$(MAKE) -C stub
$(MAKE) -C source
but this not
SUBDIRS = stub source
$(SUBDIRS):
$(MAKE) -C $#
First time I`m working with makefiles.
When you do not explicitly specify the target(s) to make, it will use the first target whose name doesn't start with a dot. Here, that target is stub.
Apparently, you expect all of the targets of the first rule to be used, but there can only be one default target.
The documentation isn't 100% clear on this.
You will get the desired behavior by prepending the rule:
all: $(SUBDIRS)
The question is about parallel making w/ GNU makefile.
Given a folder structure as below, the goal is to deliver a makefile that it supports make release/debug/clean in parallel.
project folder structure:
foo
+-foo1
+-foo2
+-foo3
The makefile may be sth like:
SUBDIR = foo1 foo2 foo3
.PHONY $(SUBDIR) release debug clean
release: $(SUBDIR)
$(SUBDIR):
$(MAKE) -C $# release
debug: $(SUBDIR)
#below is incorrect. $(SUBDIR) is overriden.
$(SUBDIR):
$(MAKE) -C $# debug
..
Sub directory list are set as phony targets for parallel making. but it lost the information of original target (release, debug, clean etc).
One method is to suffix the names for the directories and recover it in commands, but it is weird. another method might be to use variables, but not sure how to work it out.
The questions is:
How to write the rules for directories, that supports parallel making w/ different targets (release/debug/clean)?
Any hints are greatly appreciated.
Setting variables on the command line certainly works. You can also use MAKECMDGOALS (see the GNU make manual):
$(SUBDIR):
$(MAKE) -C $# $(MAKECMDGOALS)
I would like a makefile where I can call:
'make' / 'make <subdir>' / 'make clean' / 'make <subdir> clean'
But I do not want it to attempt to resolve clean when called on a subdir!
Here's an example of my makefile:
SUBDIRS := a b c
all :
#echo building a b and c
clean :
#echo cleaning a b and c
$(SUBDIRS) :
make - C $# $(MAKECMDGOALS)
All the calls work well except make <subdir> clean which calls make -C <subdir> clean, but then attempts to also resolve target clean separately. How can I get make to stop processing later targets?
To both answers:
thank you for your explanation. it is helpful to know what is and what is not meant to be done. i will not attempt this anymore.
Sigh
Yes, you can do this, but it'll be an ugly hack and totally contrary to the way Make is designed to work.
The set of targets you pass to Make is a set of targets, not a structured command with syntax. Make is expected to build them all. If the makefile includes a recipe for the target foo, then Make should build the target foo a certain way, regardless of whether it is invoked as Make foo or Make foo bar or Make bar foo. What you are attempting to do breaks the accepted behavior of Make, so you should try a different approach.
If you still wanted to do it, you could do it like this:
SUBDIRS := a b c
.PHONY: clean
ifneq ($(filter $(SUBDIRS), $(MAKECMDGOALS)),)
$(SUBDIRS) :
#echo make - C $# $(MAKECMDGOALS)
clean:
#: # do nothing
else
all :
#echo default
clean :
#echo cleaning
endif
I fully agree with Beta's statements on misusing make. Another way of achieving what you want, would be to introduce explicit targets for cleaning the subdirectories. This way, for instance, you can call make clean_<subdir>.
SUBDIRS := a b c
all:
#echo building a b and c
clean:
#echo cleaning a b and c
$(SUBDIRS):
make - C $#
$(addprefix clean_, $(SUBDIRS)): clean_%:
make -C $* clean
Let's say you have a Makefile with two pseudo-targets, 'all' and 'debug'. The 'debug' target is meant to build the same project as 'all', except with some different compile switches (like -ggdb, for example). Since the targets use different compile switches, you obviously need to rebuild the entire project if you switch between the two. But GNUmake doesn't naturally recognize this.
So if you type make all you'll get
Building ...
...
Then if you type make debug, you get
make: Nothing to be done for `debug'.
So my question is: how do you implement a clean solution in the Makefile to notice that the last build used a different pseudo-target, or different compile switches, than the one you want currently? If they are different, the Makefile would rebuild everything.
Put the build products into different directory trees (whilst keeping one copy of the source of course). That way you are always just a short compile from an up-to-date build, be it debug or release (or even others). No possibility of confusion either.
EDIT
Sketch of the above.
src := 1.c 2.c 3.c
bare-objs := ${src:%.c=%.o}
release-objs := ${bare-objs:%=Release/%}
debug-objs := ${bare-objs:%=Debug/%}
Release/prog: ${release-objs}
Debug/prog: ${debug-objs}
${release-objs}: Release/%.o: %.c # You gotta lurve static pattern rules
gcc -c $< -o $#
${debug-objs}: Debug/%.o: %.c
gcc -c $< -o $#
Release/prog Debug/prog:
gcc $^ -o $#
.PHONY: all
all: Release/prog ; echo $# Success
.PHONY: debug
debug: Debug/prog ; echo $# Success
(Disclaimer: not tested, nor even run through make.)
There you go. It's even -j safe so you can do make -j5 all debug. There is a lot of obvious boiler plate just crying out for tidying up.
Keeping variant sets of object files (as in bobbogo's solution) is probably the best way, but if for some reason you don't want to do that, you can use empty files as markers, to indicate which way you last built the executable:
%-marker:
#rm -f $(OBJECTS) *-marker
#touch $#
debug: GCCFLAGS += -ggdb
debug: SOMEOTHERFLAG = WHATEVER
all debug: % : %-marker
#echo making $#
#$(MAKE) -S GCCFLAGS='$(GCCFLAGS)' SOMEOTHERFLAG='$(SOMEOTHERFLAG)' main
There are other variants on this idea; you could have a small file containing the flag settings, which the makefile would build and include. That would be clever, but not really any cleaner than this.
The only clean solution is to incorporate the difference into the target names.
E.g. you can define a variable $(DEBUG) and consistently use it in all targets that depend on the compile step.