I know this has been asked before, but none of the solutions I've found work for me because they're anti-DRY.
I have a number of targets that depend on things that can't readily be timestamped -- such as files copied from another system. What I'd like to be able to do is list dependencies in a variable, like nobuild=this,that, and have those targets be assumed to be up-to-date. Since I have a lot of these, I don't want to ifdef around each one; what would be pseudocodibly preferable would be something like
ignorable-target: dependencies
$(call ifnobuild,$#)
.. rest of normal build steps ..
where the ifnobuild macro expanded to some sort of exit-from-this-recipe-with-success gmake instruction if ignorable-target was mentioned in the nobuild variable.
I also don't want to get into multi-line continued shell commands in order to defer the conditional to the recipe itself; I want to be able to tell make "Assume these targets are up-to-date and don't try to build them," so I can test other aspects with the local copies already obtained from the problematic recipes.
There isn't any sort of exit-recipe-with-success mechanism in gmake, is there?
[Edited to hopefully make the situation more clear.]
Here's an example. Targets remote1 and remote2 each involve using ssh to do something time-consuming on a remote system, and then copying the results locally. Target local1 is built locally, and isn't a time sink. target-under-work depends on all three of the above.
local1: local1.c Makefile
remote1: local1
scp local1 remote-host:/tmp/
ssh remote-host /tmp/local1 some-args # takes a long time
scp remote-host:/tmp/local1.out remote1
remote2: local1
scp local1 other-host:/tmp/
ssh other-host /tmp/local1 other-args # takes a long time
scp other-host:/tmp/local1.out remote2
target-under-work: local1 remote1 remote2
do-something-with remote1,remote2
Now, when I just run make target-under-work, it's going to run the recipes for remote1 and remote2. However, the local copies of those files are 'good enough' for my testing, so I don't want them run every time. Once things go into production, they will be run every time, but while I'm developing target-under-work, I just want to use the copies already built, and I can rebuild them daily (or whatever) for the necessary testing granularity.
The above is over-simplified; there are multiple steps and targets that depend on remote1 and/or remote2. I see how I can get the effect I want by making them order-only prerequisites -- but that would mean changing the dependency list of every target that has them as prerequisites, rather than making a single change to remote1 and remote2 so I can use some variable from the command line to tell their recipes 'pretend this has been built, don't actually build it if there's already a copy.'
I hope this makes my question more clear.
No, this early exit make feature does not exist.
Note that your problem is probably under-specified because you don't explain what behaviour you want when a slow target does not exist yet.
Let's assume that the slow targets listed in nobuild shall be rebuilt if and only if they don't exist. Instead of using make functions to early exit their recipe you could use make functions to "hide" their list of prerequisites. This way, if they already exist, they will not be rebuilt, even if they are outdated. The only subtlety is that you will need the second expansion to use the $# automatic variable in the lists of prerequisites. In the following example slow (your remoteX) depends on fast1 (your local1). fast2 (your target-under-work) depends on fast1 and slow:
host> cat Makefile
# Expands as empty string if $(1) exists and
# listed in $(nobuild). Else expands as $(2).
# $(1): target
# $(2): prerequisites
define HIDE_IF_NOBUILD
$(if $(wildcard $(1)),$(if $(filter $(1),$(nobuild)),,$(2)),$(2))
endef
nobuild :=
fast1:
#echo 'build $#'
#touch $#
fast2: fast1 slow
#echo 'build $#'
#touch $#
.SECONDEXPANSION:
slow: $$(call HIDE_IF_NOBUILD,$$#,fast1)
#echo 'build $#'
#touch $#
# Case 1: slow target not listed in nobuild and not existing
host> rm -f slow; touch fast1; make fast2
build slow
build fast2
# Case 2: slow target not listed in nobuild and existing and outdated
host> touch slow; sleep 2; touch fast1; make fast2
build slow
build fast2
# Case 3: slow target listed in nobuild and not existing
host> rm -f slow; touch fast1; make nobuild="slow" fast2
build slow
build fast2
# Case 4: slow target listed in nobuild and existing and outdated
host> touch slow; sleep 2; touch fast1; make nobuild="slow" fast2
build fast2
Related
I'm exploring a feature of some software that's used in a very mature Makefile to see if that feature can be incorporated into the flow. There is a target like the following:
my_target: dependency_1 dependency_2
echo foo
touch bar
# etc...
In regular use, dependency_2 would always be called before my_target's commands were executed. But for my exploration purposes, I want to selectively call dependency_2 from its own make command. And then make my_target after, possibly several times, without re-doing dependency_2 each time.
For the time being I just copy/pasted my_target into my_target_2 and removed the dependency from my_target_2 (easy enough), but I'm wondering if there's a command-line option to disable a target's dependency without modifying the file.
EDIT: dependency_2 is a PHONY target
But for my exploration purposes, I want to selectively call dependency_2 from its own make command
This is done as easy as make dependency_2
And then make my_target after, possibly several times, without re-doing dependency_2 each time
Normally, dependency_2 should be a file on the disk, so make would skip rebuilding it until its own prerequisites (source files) are changed.
But if dependency_2 is a .PHONY target ("fake file"), it will be rebuilt on each run. In this case you still can fool it like this:
echo "dependency_2:;#:" | make -f Makefile -f -
However, make will issue a warning about overriding a recipe for dependency_2.
There's no way to do this without editing the makefile, that I can think of.
You can put dependency_2 into a variable then use the variable in the prerequisite list of my_target:
dependency_2 = dependency_2
my_target : dependency_1 $(dependency_2)
then when you don't want to rebuild it, run make dependency_2= to reset the variable to empty.
I have a Makefile with tons of targets and would like for a certain script to get executed first, irrespective of what target is being called. I like to call it a global prerequisite.
I do not want to create a target for the script and set it as a prerequisite for all existing targets (which, as I said aren't few). Besides, someone else could add a target in future and not add my script as a prerequisite for their target, so the global prerequisite would take care of that.
Does GNU-make provide for a means to achieve this?
Another approach:
-include dummy
.PHONY: dummy
dummy:
run-the-script
Make will always attempt to rebuild any file which the makefile attempts to include (if it is out of date or does not exist). In this case there is no such file, and the rule to build it runs the script and does nothing else.
There is a solution without modifying your existing Makefile (main difference with the answers pointed to by tripleee). Just create a makefile containing:
.PHONY: all
all:
pre-script
#$(MAKE) -f Makefile --no-print-directory $(MAKECMDGOALS) MAKE='$(MAKE) -f Makefile'
post-script
$(MAKECMDGOALS): all ;
The only drawback is that the pre- and post- scripts will always be run, even if there is nothing else to do. But they will not be run if you invoke make with one of the --dry-run options (other difference with the answers pointed to by tripleee).
From the docs:
$?
The names of all the prerequisites that are newer than the target,
with spaces between them.
So, given a makefile:
# Force make to search for 'foo' in the VPATH directory
$(shell rm -rf foo)
# If 'D' is a "regular" file, we remove it first.
$(shell rm -rf D)
$(shell mkdir D)
# Suggest a VPATH-file, for Make to "associate" with 'foo'.
$(shell touch D/foo)
$(shell sleep 1)
# Target 'all' is newer than prerequisite 'D/foo'
$(shell touch all)
VPATH = D
all : foo phony
echo '$?'
foo ::
touch '$#'
.PHONY: phony
.PRECIOUS : D/foo
And running, I get:
$ make -r
touch 'D/foo'
echo 'D/foo phony'
D/foo phony
# Try again, but this time, with parallel-execution mode.
$ make -r -j
touch 'D/foo'
echo 'phony'
phony
Here, we have 2 serious issues:
Given the simple and explicit recipe to "touch" the prerequisite foo, which Make clearly executes - hence will guarantee that foo will be "newer" than all - Make still does not expand $? to D/foo, at-least in the 2nd case above (i.e. for the parallel-execution (-j) mode). Why?
If you come up with an explanation for the above, shouldn't it also explain, why in the 1st case (non-parallel execution), $? - does indeed - get expanded to D/foo.
I guess, I had an assumption, that parallel vs. non-parallel aside, Make will always pause before executing a target, and first check if all of its prerequisites had already finished their respective builds.
So, shouldn't the $? variable be identically expanded for both cases?
I think there are two issues going on here.
The first is that double-colon rules appear to act like phony targets in that they force make to consider the target as "newer" regardless of actual modification time. (This is why the non-parallel version behaves the way it does. Change from foo :: to foo : and you don't get foo in the $? output at all.)
The second thing is that, despite that, using parallel mode seems to force make back into considering modification times of its prerequisites (so the previous behavior is avoided).
This is conjecture and not definitive since I haven't dug through the code to see if this is actually happening but it explains the results here (it also explains the results on the other, nearly identical, question here).
The Problem:
Is it possible to give a target a different name or alias, such that it can be invoked using either the original target name or the alias.
For example something like
/very/long/path/my_binary: dep_a dep_b dep_c
# Compile
# Desired command
ALIAS my_binary = /very/long/path/my_binary
# NOTE: Notice the use of 'my_binary' in the dependencies
data1: my_binary datafile
# Build data file using compiled my_binary
Attempt 1: .PHONY
I have tried using a .PHONY target:
.PHONY: my_binary
my_binary: /very/long/path/my_binary
This works great when invoked from the command-line:
# Runs rule 'my_binary' and then *only* runs rule '/very/long/path/my_binary'
# if the rule '/very/long/path/my_binary' needs updating.
make my_binary
However, this does not work well when the alias my_binary is listed as a dependency:
# *Always* thinks that rule 'data1' needs updating, because it always thinks that
# the .PHONY target 'my_binary' "needs updating". As a result, 'data1' is
# rebuilt every time.
make /very/long/path/my_binary
Possible hack?
A possible hack is to use an empty target as suggested in an answer to this question, but that would require introducing fake files with names corresponding to the alias:
my_binary: /very/long/path/my_binary
touch my_binary
This will clutter the main directory with files! Placing the fake files in a sub-directory would defeat the purpose, as the alias would have to be referred to as 'directory/my_binary'
Okay, I needed something similar. The path to my output artifacts were quite long, but I wanted short target names and also benefit easily from bash-completion.
Here is what I'm came up with:
os := [arbitrary long path to an artifact]
platform := [arbitrary long path to a differ artifact]
packer := [common parts of my packer build command]
.PHONY: all
all: $(platform)
.PHONY: platform
platform: $(platform)
$(platform): platform.json $(os)
#$(packer) $<
.PHONY: os
os: $(os)
$(os): os.json
#$(packer) $<
.PHONY: clean
clean:
rm -fr build/
With the Makefile above you can say:
$ make os
$ make platform
Which will be aliases for the long artifact names. I've made the snippet above quite long, because it's important to see the relationships between the .PHONY aliases and the real targets. I hope that works for you.
Note: I did not delete the clean target from the above example, because many people does not make that a .PHONY target. However, semantically it should be.
I don't think there's any way to do it so that you can use the alias from within your makefile as well as the command line, except by creating those temporary files.
Why can't you just set a variable in the makefile, like:
my_binary = /very/long/path/my_binary
then use $(my_binary) everywhere in the makefile? I don't see any point in creating a real alias target for use inside the makefile.
I had a somewhat similar need. I wanted users of my makefile to be able to enter any of the following to accomplish the same result, such that the following were effectively synonyms of each other:
make hit list
make hitlist
make hit_list
What I did in my makefile was the following:
hit_list:
#echo Got here
<the real recipe goes here>
hit: hit_list
hitlist: hit_list
.PHONY: list
list:
#echo > /dev/null
Then, when I tested it using any of the commands "make hit list", "make hitlist", or "make hit_list", I got identical results, as intended.
By extension, if one of your targets was the one with the long name but you used this approach whereby a simple short name identified the target with the long name as a prerequisite, I think that you should be able to say "make short_name" and accomplish what you're asking about.
This differs from your Approach 1 in that none of the synonyms is defined as a phony target (considering that "make hit list" is a command to make two targets, the second being effectively a noop), so the complication that you described would not arise.
(this is similar to GNU make: Execute target but take dependency from file but slightly different).
When I try to force to build a target, make automatically thinks that this target is out of date and forces a run of all targets which depend on it.
In my case, the target is a recursive make call which does a lot of work and might just return with "nothing to be done":
.PHONY: intermediate.dat
intermediate.dat:
$(MAKE) expensive_chain_which_finally_creates_intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
In this case, "make all" runs for 40 seconds although intermediate.dat might not have changed (recursive make returned "nothing to be done"). However, if the recursive make updated intermediate.dat, the target shall be out of date.
Is there really no way to do this?
Make intermediate.dat depend on a phony target instead of being phony itself.
.PHONY : always-remake
intermediate.dat : always-remake
IIRC, the last time I solved the problem, the .PHONY didn't work as intended and I used:
always-remake :
#true
instead. However, I can't recall why, so try the .PHONY first.
The problem with making intermediate.dat itself phony is that make never checks the existence/date of a phony file, which is behaviour that you want. You only need to trigger the rebuild rule, which is done by a prerequisite that is out of date; a phony prerequisite is always out of date, so it does the job.
Another trick is to use an order-only prerequisite, plus a recursive call to make. This can be useful when you want to implement some additional logic around when the intermediate.dat file needs to be created, or when you want to decouple a target from the usual exists && newer than check that make does.
For example:
Let's say that the file contains some date-sensitive material that subsequent tasks depend on, and that it expires after 12 hours. We only want to recreate the file when any of these is true:
The file is older than 12 hours
The file does not exist
To implement the make target & recipes, we'll create a .PHONY check-intermediate.dat target, which runs the time check, conditionally removes the file if it's expired, and then re-runs make for that file.
Note that the check-intermediate.dat target will be defined via the order-only prerequisite syntax (e.g. intermediate.dat: | check-intermediate.dat) which causes it to run always before the target (e.g. intermediate.dat). The time check is inexpensive, and we always want it to run and manage the file for us, but not invalidate the original make check on intermediate.dat if it already exists like a normal prerequisite .PHONY target would.
.PHONY: check-intermediate.dat
check-intermediate.dat:
if [ -e intermediate.dat ]; then find intermediate.dat -mmin +720 -exec bash -c 'rm -f "{}"; $(MAKE) intermediate.dat' \; ; fi
intermediate.dat: | check-intermediate.dat
echo run-expensive-tasks-here-to-create-intermediate.dat
step1.dat: intermediate.dat
sleep 10
step2.dat: step1.dat
sleep 15
step3.dat: step2.dat
sleep 10
all: step3.dat
sleep 5
Now, anything that depends on the intermediate.dat target will first run the check, remove the file if it's expired, and re-run make intermediate.dat before running the rest of the dependent targets. If the intermediate.dat file already exists, it will run the check, and if the check passes, it will continue without running the time expensive recreation task.
The problem with depending on a .PHONY target is that it will always run the tasks which depend on it, because of the way GNU Make handles .PHONY targets. This can cause the make run to always perform the time expensive operations even when it does not need to:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file. As long as a phony target is never a prerequisite of a real target, the phony target recipe will be executed only when the phony target is a specified goal (see Arguments to Specify the Goals).