I know this has been asked before, but please bear with me. run_test is the name of the file I would like to make. I prefer invoking as 'make run_test'. I want the prereq's makefile to be called each time I call make run_test. But I don't want the call to the prereq's make to force run_test to be rebuilt.
run_test: build_prereq prereq_dir/prereq_ts
rebuild_test.sh
.PHONY: build_prereq
build_prereq:
make -C prereq_dir
prereq_dir/prereq_ts is updated if make -C prereq (prereq_dir/makefile) detects that prereq's dependencies are old and prereq needs to be rebuilt.
The problem with this is that rebuilt_test.sh is always called, and I assume that is because one of run_test's dependencies is executed (as it is PHONY).
I have resolved this issue by changing run_test to:
run_test:: build_prereq
run_test:: prereq_dir/prereq_ts
rebuild_test.sh
.PHONY: build_prereq
build_prereq:
make -C prereq_dir
Is this a good solution? This first runs build_prereq, and then it runs rebuild_test.sh if prereq_dir/prereq_ts is updated (which would happen in build_prereq's make call if needed).
Is there a more proper way of doing this. I do not want to create additional targets as that doesn't allow a direct call of make ie 'make run_test'.
Make supports exactly what you want.
.PHONY: build_prereq
build_prereq:
${MAKE} -C prereq_dir
prereq_dir/prereq_ts: build_prereq ;
run_test: prereq_dir/prereq_ts
rebuild_test.sh
So, you ask make to build run_test.
To ensure prereq_dir/prereq_ts is up to date it first runs the
recipe for build_prereq.
Next it runs the recipe for prereq_dir/prereq_ts (which does nothing—see that ;).
Now it checks the time-stamp of prereq_dir/prereq_ts to decide whether run_test is out of date.
Perfect.
If you're willing to be non-portable, GNU make supports order-only prerequisites (see the GNU make manual) which do exactly what you want:
run_test: | build_prereq
build_prereq:
$(MAKE) -C prereq_dir
(note always use $(MAKE) when invoking sub-makes, never make)
Related
I have a Makefile with tons of targets and would like for a certain script to get executed first, irrespective of what target is being called. I like to call it a global prerequisite.
I do not want to create a target for the script and set it as a prerequisite for all existing targets (which, as I said aren't few). Besides, someone else could add a target in future and not add my script as a prerequisite for their target, so the global prerequisite would take care of that.
Does GNU-make provide for a means to achieve this?
Another approach:
-include dummy
.PHONY: dummy
dummy:
run-the-script
Make will always attempt to rebuild any file which the makefile attempts to include (if it is out of date or does not exist). In this case there is no such file, and the rule to build it runs the script and does nothing else.
There is a solution without modifying your existing Makefile (main difference with the answers pointed to by tripleee). Just create a makefile containing:
.PHONY: all
all:
pre-script
#$(MAKE) -f Makefile --no-print-directory $(MAKECMDGOALS) MAKE='$(MAKE) -f Makefile'
post-script
$(MAKECMDGOALS): all ;
The only drawback is that the pre- and post- scripts will always be run, even if there is nothing else to do. But they will not be run if you invoke make with one of the --dry-run options (other difference with the answers pointed to by tripleee).
The GNU make manual says:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file.
What if that's what I want?
For example, what if I have a phony target called lint that lints app/scripts/main.js, and I want it to run every time make goes to update (transpile & minify) dist/scripts/main.js?
I just removed the lint target and put its recipe as the first command of the recipe for the dist/scripts/main.js target.
dist/scripts/main.js : app/scripts/main.js
./node_modules/.bin/eslint $< && \
mkdir -p dist/scripts && \
./node_modules/.bin/babel $< | \
./node_modules/.bin/uglifyjs - --screw-ie8 -o $# -m -c
Updated answer: the current GNU make manual does not specify the cited "rule" from the question but instead explicit says to make use of this in the entry about forced targets, which includes a note:
As you can see, using FORCE this way has the same results as using .PHONY clean.
Using .PHONY is more explicit and more efficient. However, other versions of make do not support .PHONY; thus FORCE appears in many makefiles
Is there a way how to ask gmake to never run two targets from a set in parallel?
I don't want to use .NOTPARALLEL, because it forces the whole Makefile to be run sequentially, not just the required part.
I could also add dependencies so that one depends on another, but then (apart from being ugly) I'd need to build all of them in order to build the last one, which isn't necessary.
The reason why I need this is that (only a) part of my Makefile invokes ghc --make, which takes care of its dependencies itself. And it's not possible to run it in parallel on two different targets, because if the two targets share some dependency, they can rewrite each other's .o file. (But ghc is fine with being called sequentially.)
Update: To give a specific example. Let's say I need to compile two programs in my Makefile:
prog1 depends on prog1.hs and mylib.hs;
prog2 depends on prog2.hs and mylib.hs.
Now if I invoke ghc --make prog1.hs, it checks its dependencies, compiles both prog1.hs and mylib.hs into their respective object and interface files, and links prog1. The same happens when I call ghc --make prog2.hs. So if they the two commands get to run in parallel, one will overwrite mylib.o of the other one, causing it to fail badly.
However, I need that neither prog1 depends on prog2 nor vice versa, because they should be compilable separately. (In reality they're very large with a lot of modules and requiring to compile them all slows development considerably.)
Hmmm, could do with a bit more information, so this is just a stab in the dark.
Make doesn't really support this, but you can sequential-ise two targets in a couple of ways. First off, a real use for recursive make:
targ1: ; recipe1...
targ2: ; recipe2...
both-targets:
${MAKE} targ1
${MAKE} targ2
So here you can just make -j both-targets and all is fine. Fragile though, because make -j targ1 targ2 still runs in parallel. You can use dependencies instead:
targ1: ; recipe1...
targ2: | targ1 ; recipe2...
Now make -j targ1 targ2 does what you want. Disadvantage? make targ2 will always try to build targ1 first (sequentially). This may (or may not) be a show-stopper for you.
EDIT
Another unsatisfactory strategy is to explicitly look at $MAKECMDGOALS, which lists the targets you specified on the command-line. Still a fragile solution as it is broken when someone uses dependencies inside the Makefile to get things built (a not unreasonable action).
Let's say your makefile contains two independent targets targ1 and targ2. Basically they remain independent until someone specifies on the command-line that they must both be built. In this particular case you break this independence. Consider this snippet:
$(and $(filter targ1,${MAKECMDGOALS)),$(filter targ2,${MAKECMDGOALS}),$(eval targ1: | targ2))
Urk! What's going on here?
Make evaluates the $(and)
It first has to expand $(filter targ1,${MAKECMDGOALS})
Iff targ1 was specified, it goes on to expand $(filter targ2,${MAKECMDGOALS})
Iff targ2 was also specified, it goes on to expand the $(eval), forcing the serialization of targ1 and targ2.
Note that the $(eval) expands to nothing (all its work was done as a side-effect), so that the original $(and) always expands to nothing at all, causing no syntax error.
Ugh!
[Now that I've typed that out, the considerably simpler prog2: | $(filter prog1,${MAKECMDGOALS})
occurs to me. Oh well.]
YMMV and all that.
I'm not familiar with ghc, but the correct solution would be to get the two runs of ghc to use different build folders, then they can happily run in parallel.
Since I got stuck at the same problem, here is another pointer in the direction that make does not provide the functionality you describe:
From the GNU Make Manual:
It is important to be careful when using parallel execution (the -j switch; see Parallel Execution) and archives. If multiple ar commands run at the same time on the same archive file, they will not know about each other and can corrupt the file.
Possibly a future version of make will provide a mechanism to circumvent this problem by serializing all recipes that operate on the same archive file. But for the time being, you must either write your makefiles to avoid this problem in some other way, or not use -j.
What you are attempting, and what I was attempting (using make to insert data in a SQLite3 database) suffers from the exact same problem.
I needed to separate the compilation from other steps (cleaning, building dirs and linking), as I wanted to run the compilation with more core processes and the -j flag.
I managed to solve this, with different makefiles including and calling each other. Only the "compile" make file is running in parallel with all the cores, the rest of the process is syncronous.
I divided my makefile in 3 separate scripts:
settings.mk: contains all the variables and flag definitions
makefile: has all the targets except the compilation one (It has .NOTPARALLEL directive). It calls compile.mk with -j flag
compile.mk: contains only the compile operation (without .NOTPARALLEL)
In settings.mk I have:
CC = g++
DB = gdb
RM = rm
MD = mkdir
CP = cp
MAKE = mingw32-make
BUILD = Debug
DEBUG = true
[... all other variables and flags needed, directories etc ...]
In makefile I have Link and compilation target as these:
include .makefiles/settings.mk
[... OTHER TARGETS (clean, directories etc)]
compilation:
#echo Compilation
#$(MAKE) -f .makefiles/compile.mk --silent -j 8 -Oline
#Link
$(TARGET): compilation
#echo -e Linking $(TARGET)
#$(CC) $(LNKFLAGS) -o $(TARGETDIR)/$(TARGET) $(OBJECTS) $(LIBDIRS) $(LIB)
#Non-File Targets
.PHONY: all prebuild release rebuild clean resources directories run debug
.NOTPARALLEL: all
# include dependency files (*.d) if available
-include $(DEPENDS)
And this is my compile.mk:
include .makefiles/settings.mk
#Defauilt
all: $(OBJECTS)
#Compile
$(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(SRCEXT)
#echo -e Compiling: $<
#$(MD) -p $(dir $#)
#$(CC) $(COMFLAGS) $(INCDIRS) -c $< -o $#
#Non-File Targets
.PHONY: all
# include dependency files (*.d) if available
-include $(DEPENDS)
Until now, it's working.
Note that I'm calling compile.mk with -j flag AND -Oline so that parallel processing doesn't mess up with the output.
Any syntax color can be setted in the makefile main script, since the -O flag invalidates escape color codes.
I hope it can help.
I had a similar problem so ended up solving it on the command line, like so:
make target1; make target2
to force it to do the targets sequentially.
(this is similar to GNU make: Execute target but take dependency from file but slightly different).
When I try to force to build a target, make automatically thinks that this target is out of date and forces a run of all targets which depend on it.
In my case, the target is a recursive make call which does a lot of work and might just return with "nothing to be done":
.PHONY: intermediate.dat
intermediate.dat:
$(MAKE) expensive_chain_which_finally_creates_intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
In this case, "make all" runs for 40 seconds although intermediate.dat might not have changed (recursive make returned "nothing to be done"). However, if the recursive make updated intermediate.dat, the target shall be out of date.
Is there really no way to do this?
Make intermediate.dat depend on a phony target instead of being phony itself.
.PHONY : always-remake
intermediate.dat : always-remake
IIRC, the last time I solved the problem, the .PHONY didn't work as intended and I used:
always-remake :
#true
instead. However, I can't recall why, so try the .PHONY first.
The problem with making intermediate.dat itself phony is that make never checks the existence/date of a phony file, which is behaviour that you want. You only need to trigger the rebuild rule, which is done by a prerequisite that is out of date; a phony prerequisite is always out of date, so it does the job.
Another trick is to use an order-only prerequisite, plus a recursive call to make. This can be useful when you want to implement some additional logic around when the intermediate.dat file needs to be created, or when you want to decouple a target from the usual exists && newer than check that make does.
For example:
Let's say that the file contains some date-sensitive material that subsequent tasks depend on, and that it expires after 12 hours. We only want to recreate the file when any of these is true:
The file is older than 12 hours
The file does not exist
To implement the make target & recipes, we'll create a .PHONY check-intermediate.dat target, which runs the time check, conditionally removes the file if it's expired, and then re-runs make for that file.
Note that the check-intermediate.dat target will be defined via the order-only prerequisite syntax (e.g. intermediate.dat: | check-intermediate.dat) which causes it to run always before the target (e.g. intermediate.dat). The time check is inexpensive, and we always want it to run and manage the file for us, but not invalidate the original make check on intermediate.dat if it already exists like a normal prerequisite .PHONY target would.
.PHONY: check-intermediate.dat
check-intermediate.dat:
if [ -e intermediate.dat ]; then find intermediate.dat -mmin +720 -exec bash -c 'rm -f "{}"; $(MAKE) intermediate.dat' \; ; fi
intermediate.dat: | check-intermediate.dat
echo run-expensive-tasks-here-to-create-intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
Now, anything that depends on the intermediate.dat target will first run the check, remove the file if it's expired, and re-run make intermediate.dat before running the rest of the dependent targets. If the intermediate.dat file already exists, it will run the check, and if the check passes, it will continue without running the time expensive recreation task.
The problem with depending on a .PHONY target is that it will always run the tasks which depend on it, because of the way GNU Make handles .PHONY targets. This can cause the make run to always perform the time expensive operations even when it does not need to:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file. As long as a phony target is never a prerequisite of a real target, the phony target recipe will be executed only when the phony target is a specified goal (see Arguments to Specify the Goals).
When I change a Makefile, its rules may have changed, so they should be reevaluated, but make doesn't seem to think so.
Is there any way to say, in a Makefile, that all of its targets, no matter which, depend on the Makefile itself?
(Regardless of its name.)
I'm using GNU make.
This looks like one more simple, useful, logical thing that Make should be able to do, but isn't.
Here is a workaround. If the clean rule is set up correctly, Make can execute it whenever the makefile has been altered, using an empty dummy file as a marker.
-include dummy
dummy: Makefile
#touch $#
#$(MAKE) -s clean
This will work for most targets, that is targets that are actual files and that are removed by clean, and any targets that depend on them. Side-effect targets and some PHONY targets will slip through the net.
Since GNU make version 4.3 it is now possible with the use of those two special variable:
.EXTRA_PREREQS
To add new prerequisite to every target
MAKEFILE_LIST
To get the path of the make file
To have every target depend on the current make file:
Put near the top of the file (before any include since it would affect the MAKEFILE_LIST) the following line:
.EXTRA_PREREQS:= $(abspath $(lastword $(MAKEFILE_LIST)))
To have every target depend on the current make file and also the make files which were included
Put the following line at the end of your file:
.EXTRA_PREREQS+=$(foreach mk, ${MAKEFILE_LIST},$(abspath ${mk}))
The only answer I know to this is to add makefile explicitly to the dependencies. For example,
%.o: %.c makefile
$(CC) $(CFLAGS) -c $<