(this is similar to GNU make: Execute target but take dependency from file but slightly different).
When I try to force to build a target, make automatically thinks that this target is out of date and forces a run of all targets which depend on it.
In my case, the target is a recursive make call which does a lot of work and might just return with "nothing to be done":
.PHONY: intermediate.dat
intermediate.dat:
$(MAKE) expensive_chain_which_finally_creates_intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
In this case, "make all" runs for 40 seconds although intermediate.dat might not have changed (recursive make returned "nothing to be done"). However, if the recursive make updated intermediate.dat, the target shall be out of date.
Is there really no way to do this?
Make intermediate.dat depend on a phony target instead of being phony itself.
.PHONY : always-remake
intermediate.dat : always-remake
IIRC, the last time I solved the problem, the .PHONY didn't work as intended and I used:
always-remake :
#true
instead. However, I can't recall why, so try the .PHONY first.
The problem with making intermediate.dat itself phony is that make never checks the existence/date of a phony file, which is behaviour that you want. You only need to trigger the rebuild rule, which is done by a prerequisite that is out of date; a phony prerequisite is always out of date, so it does the job.
Another trick is to use an order-only prerequisite, plus a recursive call to make. This can be useful when you want to implement some additional logic around when the intermediate.dat file needs to be created, or when you want to decouple a target from the usual exists && newer than check that make does.
For example:
Let's say that the file contains some date-sensitive material that subsequent tasks depend on, and that it expires after 12 hours. We only want to recreate the file when any of these is true:
The file is older than 12 hours
The file does not exist
To implement the make target & recipes, we'll create a .PHONY check-intermediate.dat target, which runs the time check, conditionally removes the file if it's expired, and then re-runs make for that file.
Note that the check-intermediate.dat target will be defined via the order-only prerequisite syntax (e.g. intermediate.dat: | check-intermediate.dat) which causes it to run always before the target (e.g. intermediate.dat). The time check is inexpensive, and we always want it to run and manage the file for us, but not invalidate the original make check on intermediate.dat if it already exists like a normal prerequisite .PHONY target would.
.PHONY: check-intermediate.dat
check-intermediate.dat:
if [ -e intermediate.dat ]; then find intermediate.dat -mmin +720 -exec bash -c 'rm -f "{}"; $(MAKE) intermediate.dat' \; ; fi
intermediate.dat: | check-intermediate.dat
echo run-expensive-tasks-here-to-create-intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
Now, anything that depends on the intermediate.dat target will first run the check, remove the file if it's expired, and re-run make intermediate.dat before running the rest of the dependent targets. If the intermediate.dat file already exists, it will run the check, and if the check passes, it will continue without running the time expensive recreation task.
The problem with depending on a .PHONY target is that it will always run the tasks which depend on it, because of the way GNU Make handles .PHONY targets. This can cause the make run to always perform the time expensive operations even when it does not need to:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file. As long as a phony target is never a prerequisite of a real target, the phony target recipe will be executed only when the phony target is a specified goal (see Arguments to Specify the Goals).
Related
I have a Makefile with tons of targets and would like for a certain script to get executed first, irrespective of what target is being called. I like to call it a global prerequisite.
I do not want to create a target for the script and set it as a prerequisite for all existing targets (which, as I said aren't few). Besides, someone else could add a target in future and not add my script as a prerequisite for their target, so the global prerequisite would take care of that.
Does GNU-make provide for a means to achieve this?
Another approach:
-include dummy
.PHONY: dummy
dummy:
run-the-script
Make will always attempt to rebuild any file which the makefile attempts to include (if it is out of date or does not exist). In this case there is no such file, and the rule to build it runs the script and does nothing else.
There is a solution without modifying your existing Makefile (main difference with the answers pointed to by tripleee). Just create a makefile containing:
.PHONY: all
all:
pre-script
#$(MAKE) -f Makefile --no-print-directory $(MAKECMDGOALS) MAKE='$(MAKE) -f Makefile'
post-script
$(MAKECMDGOALS): all ;
The only drawback is that the pre- and post- scripts will always be run, even if there is nothing else to do. But they will not be run if you invoke make with one of the --dry-run options (other difference with the answers pointed to by tripleee).
I know what a .PHONY does.
If in the folder where my Makefile is, I add an empty file called clean and after I run make clean all of the clean target will not be executed since there was not any change in the file, so the target will not run and this is correct.
If I add .PHONY: clean, than the clean is seen as a command and this is also correct.
My question is why this behavior does not happen the same to all target, since I added a all file in the folder.So basically the all target still executes like if it was a .PHONY: all
I have the fallowing makefile code.
all: test1 test2
test1: test1.o
test1.o: test1.c
test2: test2.o
test2.o: test2.c
clean:
rm -rf *.o test1 test2
How do you know that the all rule is "still executing"? That rule has no recipe, so there's no way it can be "executed".
If what you mean is that even though the all file exists in the local directory, make is still building the targets test1 and test2, that's how make works (this doesn't have anything to do with phony vs. non-phony targets). When make decides whether or not build a particular target first it tries to build all the prerequisites of that target, and all the prerequisites of those targets, etc. Only after all that is complete, can make know whether or not to build the first target (all in this case).
make clean here doesn't have any dependencies, so putting a file named clean there is enough for the target to be considered built.
make all on the other hand has dependencies. Even if you put a file named all there, Make has to check whether the all file is newer than test1 and test2. This process triggers builds of test1 and test2, and it happens to have the same effect as if all was a phony target.
The basis is that all: test1 test2 is a recipe for building a file named all, that depends on the files test1 and test2.
If you ran make all, Make would do something like this:
Analyse the Makefile.
Find out that all depends on test1 and test2.
Check the timestamp of all and see if it is "up to date".
It is "up to date" if none of the dependencies are newer than itself.
In other words, Make can skip building a file if it's newer than all it's dependencies.
Build outdated or missing files.
Now, if you would like to prevent Make from considering the targets as files, you could specify them as phony targets. That is best practice for non-file targets like all.
(This answer isn't disagreeing with either of the existing answers, but suggesting another way of thinking about this).
When you have a rule like
dst: src
action
you're saying two things (as you know):
if dst doesn't exist, or is older than src, then do action; and
when action completes, the file dst will exist.
With targets such as all or clean, the second statement is of course not true. Make doesn't hold you to the promise in (2), so when you say make all, it'll compute and generate the required dependencies, and not complain that there's no file all in place afterwards. You're lying to Make, but it doesn't mind (it's cool with that...). That is, this is basically a makefile hack.
Where this goes wrong, of course, is if for some reason there happens to be a file called all or clean. Then Make will take the modification date of the file all into account when calculating the dependencies, and possibly come to a conclusion you didn't expect.
So what .PHONY: all does is legitimise the hack, and tells Make ‘even if a file all exists, pretend that it doesn't’; you're basically cancelling promise (2).
Therefore, as the other answers state, mentioning .PHONY isn't necessary. It simply forestalls an error – easy to make but easy to miss – when a file matching a phony target is accidentally created.
I know this has been asked before, but none of the solutions I've found work for me because they're anti-DRY.
I have a number of targets that depend on things that can't readily be timestamped -- such as files copied from another system. What I'd like to be able to do is list dependencies in a variable, like nobuild=this,that, and have those targets be assumed to be up-to-date. Since I have a lot of these, I don't want to ifdef around each one; what would be pseudocodibly preferable would be something like
ignorable-target: dependencies
$(call ifnobuild,$#)
.. rest of normal build steps ..
where the ifnobuild macro expanded to some sort of exit-from-this-recipe-with-success gmake instruction if ignorable-target was mentioned in the nobuild variable.
I also don't want to get into multi-line continued shell commands in order to defer the conditional to the recipe itself; I want to be able to tell make "Assume these targets are up-to-date and don't try to build them," so I can test other aspects with the local copies already obtained from the problematic recipes.
There isn't any sort of exit-recipe-with-success mechanism in gmake, is there?
[Edited to hopefully make the situation more clear.]
Here's an example. Targets remote1 and remote2 each involve using ssh to do something time-consuming on a remote system, and then copying the results locally. Target local1 is built locally, and isn't a time sink. target-under-work depends on all three of the above.
local1: local1.c Makefile
remote1: local1
scp local1 remote-host:/tmp/
ssh remote-host /tmp/local1 some-args # takes a long time
scp remote-host:/tmp/local1.out remote1
remote2: local1
scp local1 other-host:/tmp/
ssh other-host /tmp/local1 other-args # takes a long time
scp other-host:/tmp/local1.out remote2
target-under-work: local1 remote1 remote2
do-something-with remote1,remote2
Now, when I just run make target-under-work, it's going to run the recipes for remote1 and remote2. However, the local copies of those files are 'good enough' for my testing, so I don't want them run every time. Once things go into production, they will be run every time, but while I'm developing target-under-work, I just want to use the copies already built, and I can rebuild them daily (or whatever) for the necessary testing granularity.
The above is over-simplified; there are multiple steps and targets that depend on remote1 and/or remote2. I see how I can get the effect I want by making them order-only prerequisites -- but that would mean changing the dependency list of every target that has them as prerequisites, rather than making a single change to remote1 and remote2 so I can use some variable from the command line to tell their recipes 'pretend this has been built, don't actually build it if there's already a copy.'
I hope this makes my question more clear.
No, this early exit make feature does not exist.
Note that your problem is probably under-specified because you don't explain what behaviour you want when a slow target does not exist yet.
Let's assume that the slow targets listed in nobuild shall be rebuilt if and only if they don't exist. Instead of using make functions to early exit their recipe you could use make functions to "hide" their list of prerequisites. This way, if they already exist, they will not be rebuilt, even if they are outdated. The only subtlety is that you will need the second expansion to use the $# automatic variable in the lists of prerequisites. In the following example slow (your remoteX) depends on fast1 (your local1). fast2 (your target-under-work) depends on fast1 and slow:
host> cat Makefile
# Expands as empty string if $(1) exists and
# listed in $(nobuild). Else expands as $(2).
# $(1): target
# $(2): prerequisites
define HIDE_IF_NOBUILD
$(if $(wildcard $(1)),$(if $(filter $(1),$(nobuild)),,$(2)),$(2))
endef
nobuild :=
fast1:
#echo 'build $#'
#touch $#
fast2: fast1 slow
#echo 'build $#'
#touch $#
.SECONDEXPANSION:
slow: $$(call HIDE_IF_NOBUILD,$$#,fast1)
#echo 'build $#'
#touch $#
# Case 1: slow target not listed in nobuild and not existing
host> rm -f slow; touch fast1; make fast2
build slow
build fast2
# Case 2: slow target not listed in nobuild and existing and outdated
host> touch slow; sleep 2; touch fast1; make fast2
build slow
build fast2
# Case 3: slow target listed in nobuild and not existing
host> rm -f slow; touch fast1; make nobuild="slow" fast2
build slow
build fast2
# Case 4: slow target listed in nobuild and existing and outdated
host> touch slow; sleep 2; touch fast1; make nobuild="slow" fast2
build fast2
Consider the following make workflow, in which I (a) want to be sure to always work on the newest download.zip file, but (b) only run the rest of the workflow when this file changes:
my_output.file: download.zip
some_expensive_operation
download.zip:
wget -N http://some.server/download.zip
In this example, wget has got dependency information that make does not have.
If download.zip is present, the rule will never be executed and wget will never check if there is a newer file available.
I can of course make .PHONY: download.zip to have the rule executed every time, by then some_expensive_operation will be executed as well no matter if the file changed or not.
Is there any way to tell make to run the my_output.file rule above only if download.zip actually changed?
Use the force rule trick:
my_output.file: download.zip
some_expensive_operation
download.zip: FORCE
wget -N http://some.server/download.zip
FORCE: ;
(or you could declare .PHONY: FORCE if you prefer). This ensures that the recipe for download.zip is always run, but it's not marked phony itself so targets that depend on it won't be rebuilt unless it's changed.
Given a makefile:
# Create the following sequence of files, in the following order: 1)'old' then 2)'all', finally 3)'new'.
$(shell touch 'old')
$(shell sleep 1)
$(shell touch 'all')
$(shell sleep 1)
$(shell touch 'new')
all: new
echo '$#'
# Let the modification-time of 'new', to be like 'old' ("older" than 'all').
new : phony
cp -p old new
.PHONY : phony
.INTERMEDIATE: new
Running, we get:
$ make -Wnew
cp -p old new
echo 'all'
all
rm new
Now, although all was older than the file new before parsing the build, things changed after Make finished building the new file, with the recipe cp -p old new, which basically copied the modification-time of old into new, hence: all is now "newer" than the file new.
Then, why does build the up-to-date file all, given, that running through its dependencies - after their respective build - we find that all modification-time is "after" that of its dependency "new".
Explanation of Effect of -Wnew
The -Wnew option tells make to assume that new has been modified and that we must update the targets of which it is a prerequisite. Regardless of what happens to the file new in our makefile make must update all of the targets new is a prerequisite of.
I have provided an explanation and demonstration, using your example, that make does not keep the old modification time but that it is the effect of the -Wnew option which causes make to rebuild all, below.
Process of Execution with -Wnew
So in executing make -Wnew make first reads your makefile and sees
all: new
echo '$#'
so it knows regardless of what happens to new later on it has to update the target all because we've specified the -Wnew option.
make then proceeds to execute your shell commands and creates the files old, all and new. It now traverses through your targets and prerequisites noting that phony does not exist so we must remake it. After making phony we then make new as its target phony is newer than it. This actually modifies new's modification time to be older than all. However, we specified -Wnew so regardless of what happened to new we've told make we want it to rebuild targets that have new as a prerequisite such as all. This is why make then proceeds to build all.
Demonstration That make Does Not Keep Old Modification Times
If we were to execute your example with make, (note no -Wnew option), the output would be
$ make
cp -p old new
Aha! make didn't update all. It realised that by remaking new the modification time of new changed and it is now older than all thus all is up to date and we can finish.