I'm writing a Makefile that needs to execute the program foo1, which is usually provided by the operating system. Some operating systems provide this binary with a different name, foo2.
I'm trying to figure out how to best test for the correct executable and exist if neither doesn't exist. I've had a few approaches but they all seem ugly and I figured there might be a more appropriate way to do this.
I tried this:
FOO = $(shell which foo foo2)
target:
#which $(FOO) > /dev/null
$(FOO)
I don't find this very elegant because if both foo and foo2 exist on the system, which will actually return two lines. It's also weird that I call which twice and that I rely on an error being emitted when I don't pass anything to the second which.
In short, what's a better way?
How about something like:
FOO := $(firstword $(shell which foo foo1))
ifeq ($(FOO),)
$(error Cannot find foo or foo1)
endif
target:
$(FOO)
Or if you don't like using make's error function you can use:
target:
#[ -n "$(FOO)" ] || { echo "Cannot find foo or foo1"; exit 1; }
Note this will not work if you have any pathnames with spaces in them in your PATH... but that's a much larger problem to solve anyway.
Related
Is there a way to make use of exported variables in shell function without the need for sub-make?
Take the following example.
FOO := BAR
.EXPORT_ALL_VARIABLES:
.PHONY: buzz
buzz:
$(info buzz)
$(error finish)
.PHONY: fizz
fizz: $(if $(shell echo $$FOO),buzz,)
$(info fizz)
$(MAKE) fizz
If I run the fizz target like so I get the following output.
$ make fizz
fizz
make fizz
make[1]: Entering directory '/home/jshbrntt/test'
buzz
Makefile:8: *** finish. Stop.
make[1]: Leaving directory '/home/jshbrntt/test'
make: *** [Makefile:13: fizz] Error 2
As you can see only the second run of make fizz had the shell function expanded and cause the buzz target to also run.
Is there a way to make use of exported variables in shell function without the need for sub-make?
No.
Remember always that GNU make functions such as $(shell) are evaluated while the makefile is being parsed, not when make runs recipes (refer to section 3.7 of the manual), regardless of where in the makefile the $(shell) invocation appears. make determines which variables are exported based on its own environment and the combination of all rules and export / unexport directives in the makefile, potentially including rules and / or directives generated via $(shell) and other functions. In this way it ensures that it is consistent about the environment used to execute recipes.
Although it is conceivable that make would expose expose intermediate forms of its export list to the $(shell) function (and its documentation doesn't clearly specify whether it does so), as a practical matter it would be surprising for it to do so, and your experiment shows that it does not do. And although the manual does not explicitly speak directly to the question, it should be noted that its documentation of export, etc. is in a section entitled Communicating Variables to a Sub-make.
Personally, I recommend avoiding $(shell) (and $(wildcard)) altogether. If you do use $(shell) then I recommend reserving such use to outside recipes. Inside recipes, use shell code directly. This is clearer, certainly in terms of makefile semantics (what is evaluated when), but often in terms of the actual code, too. For your particular example, that might look like so:
FOO := BAR
.EXPORT_ALL_VARIABLES:
.PHONY: fizz
fizz:
#echo fizz
#if test -n "$$FOO"; then echo FOO found; else echo FOO missing; make fizz; fi
In all current versions of GNU make, exported variables are not sent to the shell function. There are some very nasty recursive behaviors that can happen (what if you write export BAR = $(shell echo $$FOO) ???)
In the next release of GNU make, make variables will be exported to the shell function.
However, there's never any good reason to use shell in a recipe. The recipe is running in a shell, so you can just write the commands that you want directly. So if your example is accurate in that you want to use this facility in a recipe, just take out the shell invocation:
.PHONY: fizz
fizz:
#echo fizz
#test -n $$FOO && echo FOO found || echo FOO missing
Make is a beautiful and powerful tool for wide and sundry use. I love it, and this is only a detail.
Suppose I have a recursive make recipe defined as follows:
submake-cmd:
make ${SUBMAKEGOALS} -C foo
# where I have derived ${SUBMAKEGOALS} earlier
And here is the crux: I can either set the make interface up like so...
$: make submake-cmd SUBMAKEGOALS="foo bar"
-- OR --
# preferably
make submake-cmd foo bar biz baz
The only problem is that if I calculate the ${SUBMAKEGOALS}, then I am forced to write the submake-cmd rule like so in order to avoid unpredictable behavior:
submake-cmd:
make ${SUBMAKEGOALS} -C foo
exit 1
Such that make will exit, and I will see make exit with an error.
There must be some way to clear out the make targets or indicate that make has completed successfully when using make recursion...
Does anyone have any ideas?
preferably
make submake-cmd foo bar biz baz
You're probably overthinking something. If you want to pass an unknown number of targets to submake then it could be done like this:
ifneq ($(sub),)
.DEFAULT:
$(MAKE) -C $(sub) $#
endif
And then: make sub=dir1 foo; make sub=dir2 bar biz bazetc. (I assume the root makefile does not contain any of foo/bar/biz/baz of its own; if it's not so, then you have to patch these rules appropriately).
Inside a Makefile I have recently had to look at, it includes this include $(MAKERULES)
Now to me, I would think somewhere at the top of the makefile it has MAKERULES = xyz but it does not. So I print out MAKERULES
test:
#echo "Rules: $(MAKERULES)"
Then I do make test. It runs and it prints out another Makefile Location, Makefile2.
X/Y/Z/Makefile2 path. I go inside this Makefile and attempt to write out some echo statements so I can see what is printing, but nothing prints out.
Is it possible to print out variables from another Makefile (Makefile2) that my local Makefile (Makefile) references to?
Yes, it is possible. You haven't shown us your attempt to display those variables, so we can't tell you why it didn't work. (Well, I can't.)
Here is how I'd do it:
$(info the variable FOO contains $(FOO))
If you want to do this for several variables:
$(foreach X, FOO BAR BAZ, $(info $(X) is $($(X))))
And the list of all currently defined global variables is .VARIABLES, so you can use "$(.VARIABLES)" in place of "FOO BAR BAZ" to print all of them.
From the docs:
$?
The names of all the prerequisites that are newer than the target,
with spaces between them.
So, given a makefile:
# Force make to search for 'foo' in the VPATH directory
$(shell rm -rf foo)
# If 'D' is a "regular" file, we remove it first.
$(shell rm -rf D)
$(shell mkdir D)
# Suggest a VPATH-file, for Make to "associate" with 'foo'.
$(shell touch D/foo)
$(shell sleep 1)
# Target 'all' is newer than prerequisite 'D/foo'
$(shell touch all)
VPATH = D
all : foo phony
echo '$?'
foo ::
touch '$#'
.PHONY: phony
.PRECIOUS : D/foo
And running, I get:
$ make -r
touch 'D/foo'
echo 'D/foo phony'
D/foo phony
# Try again, but this time, with parallel-execution mode.
$ make -r -j
touch 'D/foo'
echo 'phony'
phony
Here, we have 2 serious issues:
Given the simple and explicit recipe to "touch" the prerequisite foo, which Make clearly executes - hence will guarantee that foo will be "newer" than all - Make still does not expand $? to D/foo, at-least in the 2nd case above (i.e. for the parallel-execution (-j) mode). Why?
If you come up with an explanation for the above, shouldn't it also explain, why in the 1st case (non-parallel execution), $? - does indeed - get expanded to D/foo.
I guess, I had an assumption, that parallel vs. non-parallel aside, Make will always pause before executing a target, and first check if all of its prerequisites had already finished their respective builds.
So, shouldn't the $? variable be identically expanded for both cases?
I think there are two issues going on here.
The first is that double-colon rules appear to act like phony targets in that they force make to consider the target as "newer" regardless of actual modification time. (This is why the non-parallel version behaves the way it does. Change from foo :: to foo : and you don't get foo in the $? output at all.)
The second thing is that, despite that, using parallel mode seems to force make back into considering modification times of its prerequisites (so the previous behavior is avoided).
This is conjecture and not definitive since I haven't dug through the code to see if this is actually happening but it explains the results here (it also explains the results on the other, nearly identical, question here).
I've two targets foo and bar. Neither depend on the other, but if bar has to be rebuilt, it has to be done before foo. They are what gnu-make calls phony targets, their rules have always to be executed when they are specified.
Currently, we express a main target which depends on both like this:
# user level targets
all: bar
#$(MAKE) foo
#echo all
alt: foo
#echo alt
# internal targets
foo:
#echo foo
bar: qux
#echo bar
qux:
#echo qux
#touch qux
and we have the required behavior: if qux is not up-to-date: make bar outputs qux bar foo all (in that order) and make alt outputs foo alt; if qux is up-to-date, make bar output bar foo all and make alt outputs foo alt.
This is increasingly uncomfortable as foo has to be handled specifically (all targets which depend on both have to be handled that way, foo can't be put in a variable describing dependencies if bar is also there, the submake is itself an issue and the command line has to be maintained to pass additional variables). We now have another target which has to be handled in the same way and I'm looking for other, more convenient, ways to handle the structure.
Note 1 : In practice, I'm currently using only gnu-make but the only known dependency on a gnu-make extension over POSIX is the possibility to include files (which is quite widely available). I'd prefer something which keep the current state (i.e. widely supported constructs), but if it is not possible, the use of a gnu-make only extension is acceptable.
Note 2: gnu-make has a notion of order-only-prerequisites, but it apparently doesn't provide what we need. With
# user level targets
all: bar foo
#echo all
alt: foo
#echo alt
# internal targets
foo: | bar
#echo foo
bar:
#echo bar
make alt also build bar (if a file bar exist, its date doesn't influence the decision of rebuilding foo, which is the documented behavior).
Note 3: The more I think about it, the less I think it is possible to solve this problem with make without using a recursive call. It seems to me that it need two passes on the dependency graph, one to determine what has to be built, one to determine the ordering and I know nothing in make behavior which can't be done with a one pass algorithm.
Hmmm, how about this hack (for a hack it undoubtedly is :-)).
Basically, you could run make -d -n plus your command arguments. The output will contain several lines like Must remake target 'clean'. This information tells you whether this run of make will attempt to build both foo and bar. If this turns out to be the case, just add a rule to cause the serialisation you want.
A sketch:
this := $(lastword ${MAKEFILE_LIST})
ifndef DONTRECURSE
targets-that-will-get-remade := $(patsubst %',%,$(shell ${MAKE} -f ${this} ${MAKECMDGOALS} --debug=b -n DONTRECURSE=nosiree | grep -Po "Must remake target '\K.*'"))
endif
ifeq (bar foo,$(sort $(filter bar foo,${targets-that-will-get-remade})))
foo: bar
endif
.PHONY: foo bar
foo bar:
sleep 3
: $#
So, you run make. DONTRECURSE is not set so the $(shell …) runs. That runs make a second time with the same makefile and goals, but adds the -d (debug) and -n (don't actually run the recipes) flags. DONTRECURSE is set to prevent a third copy of make running.
The expansion of all that is a list of the targets this run of make will attempt to build on this run. (Extracting the target names is pretty tiresome—there is probably a cleaner way.)
If this list of targets includes both foo and bar, simply add a foo: bar dependency. Job done. The sleep 3 lines show this serialisation working when you use -j4 (say).