The GNU make manual says:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file.
What if that's what I want?
For example, what if I have a phony target called lint that lints app/scripts/main.js, and I want it to run every time make goes to update (transpile & minify) dist/scripts/main.js?
I just removed the lint target and put its recipe as the first command of the recipe for the dist/scripts/main.js target.
dist/scripts/main.js : app/scripts/main.js
./node_modules/.bin/eslint $< && \
mkdir -p dist/scripts && \
./node_modules/.bin/babel $< | \
./node_modules/.bin/uglifyjs - --screw-ie8 -o $# -m -c
Updated answer: the current GNU make manual does not specify the cited "rule" from the question but instead explicit says to make use of this in the entry about forced targets, which includes a note:
As you can see, using FORCE this way has the same results as using .PHONY clean.
Using .PHONY is more explicit and more efficient. However, other versions of make do not support .PHONY; thus FORCE appears in many makefiles
Related
I have a Makefile as below, and I am concerned with the dependency order for up-clean:
.PHONY: up
up: down
docker-compose up -d
.PHONY: up-clean
up-clean: down-clean up
.PHONY: down
down:
docker-compose down
.PHONY: down-clean
down-clean:
docker-compose down -v
Obviously it is important that in up-clean: down-clean up, down-clean must be executed before up. For ordinary make targets, the solution would be to add an entry, up: down-clean, but as these are PHONY targets, that would make up functionally equivalent to up-clean, removing volumes every time. Obviously, this is unacceptable.
In practice, GNU make respects the order of dependencies, but it does not guarantee it, and so is not entirely trustworthy, or with -j not trustworthy at all. What methods, if any, exist to ensure the order of execution of dependencies in this situation without changing the result of building other targets?
EDIT: Attempting to use order-only prerequisites does not appear to work, possibly because of an interaction with .PHONY. Adding
up: | down-clean
Causes the execution log to be:
$ make up
docker-compose down
<...>
docker-compose down -v
Removing volume <...>
Which is what is supposed to happen for normal prerequisites, not order-only ones.
The simplest answer is to use recursive invocations of make:
up-clean:
$(MAKE) down-clean
$(MAKE) up
Another alternative would be to model up-clean on up instead of making the latter a prerequisite for the former:
.PHONY: up
up: down
docker-compose up -d
.PHONY: up-clean
up-clean: down-clean
docker-compose up -d
If you want to make that a little DRYer, you could factor out the docker-compose command to a variable:
UP_COMMAND = docker-compose up -d
.PHONY: up
up: down
$(UP_COMMAND)
.PHONY: up-clean
up-clean: down-clean
$(UP_COMMAND)
Starting with GNU make 4.4 you can explicitly serialize your prerequisites with the .WAIT pseudo-target, e.g.,
.PHONY: up-clean
up-clean: down-clean .WAIT up
Before 4.4, the GNU make documentation was avoiding making any commitments about the order or execution to enable parallel execution. However, any POSIX-compliant implementation of make (emphasis mine)
... shall treat all prerequisites as targets themselves and recursively ensure that they are up-to-date, processing them in the order in which they appear in the rule. The make utility shall use the modification times of files to determine whether the corresponding targets are out-of-date.
Of course, in the parallel mode, GNU make can't be fully compliant with this requirement.
But, the 4.4 release also adds the --shuffle option, and from the research made by the implementor of this option, it is evident that the only source of non-determinism is the -j option and the parallel mode of execution, which is also witnessed by the contents of the patch and the tests, which were reviewed by other members of the GNU make project.
Therefore, in version prior to 4.4, we can safely assume that the prerequisites are executed strictly in the order in which they are specified, from left to right, as long as make is executed in non-parallel mode. In versions before 4.4, we can disable parallelism with the .NOTPARALLEL pseudo target, just add it to your file, e.g.,
.NOTPARALLEL: # ensures that all deps are executed strictly in order
.PHONY: up
up: down
docker-compose up -d
.PHONY: up-clean
up-clean: down-clean up
.PHONY: down
down:
docker-compose down
.PHONY: down-clean
down-clean:
docker-compose down -v
Notice also, that --shuffle respects the presence of the .NOTPARALLEL target, which corroborates our hypothesis the order could different from the syntactic order only because of the parallel execution.
I am trying to compile an OpenCobol program using make. I am always getting "make: Nothing to be done for test1.cob". Here is my makefile. I had put a TAB before cobc. But still I am getting that message. Please help.
Thanks.
COBCWARN = -W
%: %.cob
cobc $(COBCWARN) -free -x $^ -o $#
And here is my cobol program.
IDENTIFICATION DIVISION.
PROGRAM-ID. TEST1.
PROCEDURE DIVISION.
DISPLAY 'Hello world!'.
STOP RUN.
Your makefile contains no actual targets. You have defined only a pattern rule which tells make how to build targets that match the pattern. But make doesn't go looking for targets out on the filesystem that could match the pattern, it only checks the pattern against targets that have been specifically requested.
You don't have any specific targets (files) listed in your makefile, so the only way make can know about a target is if you give the target to be built on the command line.
You are running this command from within vim, using the % special token, which expands to the name of the file currently being edited. That means you are running the command:
make test1.cob
because you are editing the file test1.cob. So, you are telling make "please try to create the target (file) test1.cob". But, that file already exists (it's the file you're writing). So make says "nothing to do".
If you run make and ask it to create the target you really want created, it will work:
make test1
Now the file test1 doesn't exist, and make can find a pattern rule that knows how to build it, so make will run that rule.
Alternatively, you can edit your makefile to add the specific target, like this:
COBCWARN = -W
test1: test1.cob
%: %.cob
cobc $(COBCWARN) -free -x $^ -o $#
Then you can run make with no arguments at all. Without any command line arguments, make will look in the makefile for explicit targets and find test1 as the first one. It sees that there is a rule (the pattern rule) that matches that target, so it will build that target.
UPDATE
If you want to allow a simple command make to build multiple programs, write your makefile like this:
COBCWARN = -W
all: test1 test2 test3
.PHONY: all
%: %.cob
cobc $(COBCWARN) -free -x $^ -o $#
Now from vim you can just say :!make and that's it.
If you run make with no arguments then it will find the first explicit target in the makefile and build that. In this example the first target is all, and its prerequisites are the possible programs to build. To build each one make sees that it can apply the pattern rule, and so it will do so (if the .cob file has been modified since the last time the program was built).
Is there a way how to ask gmake to never run two targets from a set in parallel?
I don't want to use .NOTPARALLEL, because it forces the whole Makefile to be run sequentially, not just the required part.
I could also add dependencies so that one depends on another, but then (apart from being ugly) I'd need to build all of them in order to build the last one, which isn't necessary.
The reason why I need this is that (only a) part of my Makefile invokes ghc --make, which takes care of its dependencies itself. And it's not possible to run it in parallel on two different targets, because if the two targets share some dependency, they can rewrite each other's .o file. (But ghc is fine with being called sequentially.)
Update: To give a specific example. Let's say I need to compile two programs in my Makefile:
prog1 depends on prog1.hs and mylib.hs;
prog2 depends on prog2.hs and mylib.hs.
Now if I invoke ghc --make prog1.hs, it checks its dependencies, compiles both prog1.hs and mylib.hs into their respective object and interface files, and links prog1. The same happens when I call ghc --make prog2.hs. So if they the two commands get to run in parallel, one will overwrite mylib.o of the other one, causing it to fail badly.
However, I need that neither prog1 depends on prog2 nor vice versa, because they should be compilable separately. (In reality they're very large with a lot of modules and requiring to compile them all slows development considerably.)
Hmmm, could do with a bit more information, so this is just a stab in the dark.
Make doesn't really support this, but you can sequential-ise two targets in a couple of ways. First off, a real use for recursive make:
targ1: ; recipe1...
targ2: ; recipe2...
both-targets:
${MAKE} targ1
${MAKE} targ2
So here you can just make -j both-targets and all is fine. Fragile though, because make -j targ1 targ2 still runs in parallel. You can use dependencies instead:
targ1: ; recipe1...
targ2: | targ1 ; recipe2...
Now make -j targ1 targ2 does what you want. Disadvantage? make targ2 will always try to build targ1 first (sequentially). This may (or may not) be a show-stopper for you.
EDIT
Another unsatisfactory strategy is to explicitly look at $MAKECMDGOALS, which lists the targets you specified on the command-line. Still a fragile solution as it is broken when someone uses dependencies inside the Makefile to get things built (a not unreasonable action).
Let's say your makefile contains two independent targets targ1 and targ2. Basically they remain independent until someone specifies on the command-line that they must both be built. In this particular case you break this independence. Consider this snippet:
$(and $(filter targ1,${MAKECMDGOALS)),$(filter targ2,${MAKECMDGOALS}),$(eval targ1: | targ2))
Urk! What's going on here?
Make evaluates the $(and)
It first has to expand $(filter targ1,${MAKECMDGOALS})
Iff targ1 was specified, it goes on to expand $(filter targ2,${MAKECMDGOALS})
Iff targ2 was also specified, it goes on to expand the $(eval), forcing the serialization of targ1 and targ2.
Note that the $(eval) expands to nothing (all its work was done as a side-effect), so that the original $(and) always expands to nothing at all, causing no syntax error.
Ugh!
[Now that I've typed that out, the considerably simpler prog2: | $(filter prog1,${MAKECMDGOALS})
occurs to me. Oh well.]
YMMV and all that.
I'm not familiar with ghc, but the correct solution would be to get the two runs of ghc to use different build folders, then they can happily run in parallel.
Since I got stuck at the same problem, here is another pointer in the direction that make does not provide the functionality you describe:
From the GNU Make Manual:
It is important to be careful when using parallel execution (the -j switch; see Parallel Execution) and archives. If multiple ar commands run at the same time on the same archive file, they will not know about each other and can corrupt the file.
Possibly a future version of make will provide a mechanism to circumvent this problem by serializing all recipes that operate on the same archive file. But for the time being, you must either write your makefiles to avoid this problem in some other way, or not use -j.
What you are attempting, and what I was attempting (using make to insert data in a SQLite3 database) suffers from the exact same problem.
I needed to separate the compilation from other steps (cleaning, building dirs and linking), as I wanted to run the compilation with more core processes and the -j flag.
I managed to solve this, with different makefiles including and calling each other. Only the "compile" make file is running in parallel with all the cores, the rest of the process is syncronous.
I divided my makefile in 3 separate scripts:
settings.mk: contains all the variables and flag definitions
makefile: has all the targets except the compilation one (It has .NOTPARALLEL directive). It calls compile.mk with -j flag
compile.mk: contains only the compile operation (without .NOTPARALLEL)
In settings.mk I have:
CC = g++
DB = gdb
RM = rm
MD = mkdir
CP = cp
MAKE = mingw32-make
BUILD = Debug
DEBUG = true
[... all other variables and flags needed, directories etc ...]
In makefile I have Link and compilation target as these:
include .makefiles/settings.mk
[... OTHER TARGETS (clean, directories etc)]
compilation:
#echo Compilation
#$(MAKE) -f .makefiles/compile.mk --silent -j 8 -Oline
#Link
$(TARGET): compilation
#echo -e Linking $(TARGET)
#$(CC) $(LNKFLAGS) -o $(TARGETDIR)/$(TARGET) $(OBJECTS) $(LIBDIRS) $(LIB)
#Non-File Targets
.PHONY: all prebuild release rebuild clean resources directories run debug
.NOTPARALLEL: all
# include dependency files (*.d) if available
-include $(DEPENDS)
And this is my compile.mk:
include .makefiles/settings.mk
#Defauilt
all: $(OBJECTS)
#Compile
$(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(SRCEXT)
#echo -e Compiling: $<
#$(MD) -p $(dir $#)
#$(CC) $(COMFLAGS) $(INCDIRS) -c $< -o $#
#Non-File Targets
.PHONY: all
# include dependency files (*.d) if available
-include $(DEPENDS)
Until now, it's working.
Note that I'm calling compile.mk with -j flag AND -Oline so that parallel processing doesn't mess up with the output.
Any syntax color can be setted in the makefile main script, since the -O flag invalidates escape color codes.
I hope it can help.
I had a similar problem so ended up solving it on the command line, like so:
make target1; make target2
to force it to do the targets sequentially.
I know this has been asked before, but please bear with me. run_test is the name of the file I would like to make. I prefer invoking as 'make run_test'. I want the prereq's makefile to be called each time I call make run_test. But I don't want the call to the prereq's make to force run_test to be rebuilt.
run_test: build_prereq prereq_dir/prereq_ts
rebuild_test.sh
.PHONY: build_prereq
build_prereq:
make -C prereq_dir
prereq_dir/prereq_ts is updated if make -C prereq (prereq_dir/makefile) detects that prereq's dependencies are old and prereq needs to be rebuilt.
The problem with this is that rebuilt_test.sh is always called, and I assume that is because one of run_test's dependencies is executed (as it is PHONY).
I have resolved this issue by changing run_test to:
run_test:: build_prereq
run_test:: prereq_dir/prereq_ts
rebuild_test.sh
.PHONY: build_prereq
build_prereq:
make -C prereq_dir
Is this a good solution? This first runs build_prereq, and then it runs rebuild_test.sh if prereq_dir/prereq_ts is updated (which would happen in build_prereq's make call if needed).
Is there a more proper way of doing this. I do not want to create additional targets as that doesn't allow a direct call of make ie 'make run_test'.
Make supports exactly what you want.
.PHONY: build_prereq
build_prereq:
${MAKE} -C prereq_dir
prereq_dir/prereq_ts: build_prereq ;
run_test: prereq_dir/prereq_ts
rebuild_test.sh
So, you ask make to build run_test.
To ensure prereq_dir/prereq_ts is up to date it first runs the
recipe for build_prereq.
Next it runs the recipe for prereq_dir/prereq_ts (which does nothing—see that ;).
Now it checks the time-stamp of prereq_dir/prereq_ts to decide whether run_test is out of date.
Perfect.
If you're willing to be non-portable, GNU make supports order-only prerequisites (see the GNU make manual) which do exactly what you want:
run_test: | build_prereq
build_prereq:
$(MAKE) -C prereq_dir
(note always use $(MAKE) when invoking sub-makes, never make)
At work we use a common makefile that other makefiles include (via the include statement) and it has a generic "clean" target that kills some common files. I want to add on to that target in my new makefile so I can delete some specific files, but if I add a clean target in my makefile, it just overrides the old one.
I know I can just make a new target with a new name and have it call clean, and then do other stuff, but for sake of consistency I'd like to be able to just call make clean and have it do everything.
Is that possible?
I've seen this done at several shops. The most common approach is to use double-colon rules, assuming you're using something like GNU make. In your common makefile you would have something like this:
clean::
# standard cleanup, like remove all .o's:
rm -f *.o
Note that there are two colons following clean, not just one!
In your other makefile you just declare clean again, as a double-colon rule:
clean::
# custom cleanup, like remove my special generated files:
rm -f *.h.gen
When you invoke make clean, GNU make will automagically run both of these "branches" of the clean rule:
% make clean
rm -f *.o
rm -f *.h.gen
It's simple to set up and it composes quite neatly I think. Note that specifically because it is a double-colon rule, you don't get the "overriding commands" errors you normally get when you define two rules for the same target. That's sort of the point of double-colon rules.
You can write your own clean and make it a preq of the common clean.
clean: myclean
myclean:
rm whatever
Yours will run first. If for some reason you want the common clean to run first then the solution will be more complicated.
EDIT:
Here is the best solution I can see which runs the common rule before the local one:
include Makefile.common
clean:
$(MAKE) -f Makefile.common $#
rm whatever additional things
The include directive is necessary because the local makefile relies on the common one for things other than clean. The local clean rule overrides the common clean rule, but invokes the common clean rule before doing the additional work. (This overriding will cause some warnings, which is a nuisance; I don't know a good way to silence them.)
Use implicit rules:
existing-target: my-extention
my-extention:
echo running command 1
echo running command 2
Very simple make tutorial to ramp up.
When using :: you can run into issues since make complains when you mix single colon : and double colon :: rules:
a:
echo a
a::
echo aa
will result in:
. . .
*** target file `a' has both : and :: entries. Stop.
It seems like the common makefile's rule should be called something like common-clean. Then each main makefile would declare their clean rule as
clean: common-clean
and you're set.
If that isn't an option, you could take a look at double colon rules, but those introduce a whole other set of issues to consider.
Adding another possible solution I've seen for posterity... I know the OP was wary about changing the common makefile, but something like this works and involves minimal changes.
local makefile 1:
CLEAN=MyExe1 MyExe2
....
include /my/common/makefile
local makefile 2:
CLEAN=MyExe3 MyExe4
....
include /my/common/makefile
common makefile:
clean:
rm -f *.dep *.o *.a $(CLEAN)
Basically the idea is to define some variable (in this case CLEAN) in each local makefile with all the specific items you want to delete. Then the common makefile runs rm -f on all the common file types to delete, plus whatever was specifically flagged for deletion in each local makefile via the CLEAN variable. If there's nothing specific to delete, simply omit the variable declaration or leave it empty (CLEAN=)
So now if we run make clean for local makefile 1, it executes
rm -f *.dep *.o *.a MyExe1 MyExe2
And if we run make clean for local makefile 2, it executes
rm -f *.dep *.o *.a MyExe3 MyExe4
I've found a better solution:
.PHONY: my-extra-clean
clean: my-extra-clean
my-extra-clean:
rm <whatever-you-want>
include Makefile.common
The key line is clean: my-extra-clean. Ie, you can add dependencies in separate stanzas in different makefiles to add behaviour. my-extra-clean is run as a dependency of the root clean target.
For ours, we define a variable, EXTRAFILESTOCLEAN, then when the clean rule runs, it has a step to remove anything specified in the EXTRAFILESTOCLEAN variable
clean:
rm -f *.o
ifdef $(EXTRAFILESTOCLEAN)
rm -f $(EXTRAFILESTOCLEAN)
endif
That can cause unexpected problems if you set that variable to weird values, but you could guard against those by adding prefixes or other tests.
It's in the docs: https://www.gnu.org/software/make/manual/html_node/Overriding-Makefiles.html
So instead of include Makefile you use a wildcard target and forward it to the base Makefile:
# -include base.Makefile <--- not this
%:
#$(MAKE) -f base.Makefile $#