In this example, I have a process that takes some time. Let's says 1 second. If I write the following Makefile, FOO will be expanded 3 times for make all and none for make clean.
If I want to save some execution time for all I can assign FOO using := instead of =. However this will cause FOO to be expanded for the target clean even if it doesn't use it.
FOO = $(shell echo -e "+1" >> foo && echo "Hello" && sleep 1)
all:
#echo $(FOO)
#echo $(FOO)
#echo $(FOO)
#cat foo
clean:
rm foo
The output:
$ make
Hello
Hello
Hello
+1
+1
+1
I would like to force Make to expand a variable only once only if required.
Is it possible to do it so?
The "best" solution I can come up with looks like this:
$ cat coin.mk
FOO = FOO:=$(shell echo -e "+1" >> foo && echo "Hello" && sleep 1)
defined = $(and $(filter-out undefined,$(origin $1)),$($1))
all:
echo $(eval $(FOO))$(FOO)
echo $(FOO)
echo $(FOO)
cat foo
clean:
rm foo
$ time make -f coin.mk clean
rm foo
rm: cannot remove `foo': No such file or directory
make: *** [clean] Error 1
real 0m0.003s
user 0m0.003s
sys 0m0.000s
$ time make -f coin.mk
echo Hello
Hello
echo Hello
Hello
echo Hello
Hello
cat foo
+1
real 0m1.009s
user 0m0.002s
sys 0m0.004s
$ time make -f coin.mk clean
rm foo
real 0m0.003s
user 0m0.000s
sys 0m0.003s
Which works but requires special-casing the first use of the variable in the make run ($(eval $(FOO)) run a second time will cause a make error).
I tried briefly to encapsulate the eval logic inside the value of FOO but most attempts were blocked by make complaining that *** Recursive variable 'FOO' references itself (eventually). Stop.
Try this:
FOO = $(eval FOO := $(shell echo "+1" >> foo && echo "Hello" && sleep 1))$(value FOO)
The first time make expands $(FOO) it will first expand the eval, which resets the variable FOO using :=. Then it resolves the value of the FOO variable. In subsequent expansions, due to the eval, FOO expands directly to the value.
I should also point out that if you have at least GNU make 4.0 you can use a new feature added to the POSIX standard for make recently, the != operator:
FOO != echo "+1" >> foo && echo "Hello" && sleep 1
which does exactly what you want here.
Related
In a makefile I'm trying to
run a shell command and capture the output in a make variable
do something if the variable is not empty
I've created this simplified makefile to demonstrate my problem. Neither make a or make b executes the body of the if, I don't understand why not.
.PHONY: a b
a:
$(eval MY_VAR = $(shell echo whatever))
#echo MY_VAR is $(MY_VAR)
$(info $(MY_VAR))
ifneq ($(strip $(MY_VAR)),)
#echo "should be executed"
endif
#echo done
b:
$(eval MY_VAR = $(shell echo ''))
#echo MY_VAR is $(MY_VAR)
$(info $(MY_VAR))
ifneq ($(strip $(MY_VAR)),)
#echo "should not be executed"
endif
#echo done
I'm using
$ make --version
GNU Make 3.81
Edit: as pointed out, the vars don't need to be make vars
If you want to dynamically test the content of MY_VAR, you may have to :
a:
$(eval MY_VAR = $(shell echo ''))
$(if $(strip $(MY_VAR)),echo ok,echo no)
if evaluation will become echo ok if MY_VAR is not empty, otherwise it will become echo no
Note that, due to the time of evaluation, make conditionals (ifeq, ifneq...) cannot be used in recipes the way you tried. Use shell conditionals, instead, as shown below.
As your MY_VAR variable is used only in recipes, is target-dependent and you want it to be computed only when needed, why don't you use shell variables, instead of make variables?
$ cat Makefile
.PHONY: a b
a:
MY_VAR=$$(echo 'whatever') && \
echo '$#: MY_VAR is $$MY_VAR' && \
if [ -n "$$MY_VAR" ]; then \
echo '$#: should be executed'; \
fi && \
echo '$#: done'
b:
MY_VAR=$$(echo '') && \
echo '$#: MY_VAR is $$MY_VAR' && \
if [ -n "$$MY_VAR" ]; then \
echo '$#: should not be executed'; \
fi && \
echo '$#: done'
$ make a
a: MY_VAR is whatever
a: should be executed
a: done
$ make b
b: MY_VAR is
b: done
In case you absolutely need MY_VAR to be a target-specific make variable, but want to execute only once (per target) the shell command that produces its value, MadScientist has a wonderful trick that you should probably look at. Applied to your case, it should look like:
$ make --version
GNU Make 4.1
...
$ cat Makefile
a: MY_VAR = $(eval a: MY_VAR := $$(shell echo 'whatever'))$(MY_VAR)
b: MY_VAR = $(eval b: MY_VAR := $$(shell echo ''))$(MY_VAR)
a:
#echo '$#: MY_VAR is $(MY_VAR)' && \
if [ -n "$(MY_VAR)" ]; then \
echo '$#: should be executed'; \
fi && \
echo '$#: done'
b:
#echo '$#: MY_VAR is $(MY_VAR)' && \
if [ -n "$(MY_VAR)" ]; then \
echo '$#: should not be executed'; \
fi && \
echo '$#: done'
$ make a
a: MY_VAR is whatever
a: should be executed
a: done
$ make b
b: MY_VAR is
b: done
$ make b a
b: MY_VAR is
b: done
a: MY_VAR is whatever
a: should be executed
a: done
It may look extremely strange but it guarantees that MY_VAR is computed if and only if targets a or b are invoked, and only at most once for each. Have a look at MadScientist's post for detailed explanations. Go, it's brilliant.
The ifeq and family of conditionals are evaluated when parsing the Makefile. If you want a conditional for a Make variable when expanding a rule, you'll want to use the $(if ) function:
.PHONY: a b
a b:
#$(if $(strip $(MY_VAR)),echo "MY_VAR isn't empty",)
#echo done
a: MY_VAR =
b: MY_VAR = something
A bit left field, but was useful in my case so I guess it's worth a share: pipe the value to xargs with the --no-run-if-empty option:
echo $(POSSIBLY_EMPTY) | xargs --no-run-if-empty echo
Note that this will not work on OSX. See How to ignore xargs commands if stdin input is empty? for more details on xargs --no-run-if-empty
Shell commands sometimes take a long time to run, so you may not want to do VAR = $(shell slow-cmd) (with =, the slow-cmd will be run every time the variable is referenced). Using VAR := $(shell slow-cmd) can be useful, but if you are building a target that does not ever need the variable expanded, you will get one more invocation of the slow-cmd than is needed. In the following makefile (with gnu-make), you can get the desired behavior: the shell command to define a value for V2 is never invoked more than once, and for the target foo it is not invoked at all. But this is a heinous kludge. Is there a more reasonable way to ensure that a variable is only defined when needed, but never evaluated more than once?
V1 = $(shell echo evaluating V1 > /dev/tty; echo V1 VALUE)
all: foo bar V2
#echo $(V1) $#
#echo $(V2) $#
foo:
#echo $(V1) $#
bar: V2
#echo $(V1) $#
#echo $(V2) $#
V2:
$(eval V2 := $(shell echo evaluating V2 > /dev/tty; echo V2 VALUE))
.PHONY: all foo bar
There's no way to do it without tricks, but there's a cleaner way (maybe) than you're using. You can use:
V1 = $(eval V1 := $$(shell some-comand))$(V1)
For more details and explanation of exactly how this works see this page.
Target-specific deferred variables are an option:
host> cat Makefile
foo: VFOO = $(shell echo "VFOO" >> log.txt; echo "VFOO")
foo:
#echo '$(VFOO)' > $#
bar: VBAR = $(shell echo "VBAR" >> log.txt; echo "VBAR")
bar:
#echo '$(VBAR)' > $#
host> make foo
host> cat log.txt
VFOO
host> make foo
make: 'foo' is up to date.
host> cat log.txt
VFOO
host> make bar
host> cat log.txt
VFOO
VBAR
host> make bar
make: 'bar' is up to date.
host> cat log.txt
VFOO
VBAR
If there a way to affect the '?' variable in bash?
For a regular variable, it is possible to just use FOO=bar, thus 'echo $FOO' will output bar, but for some reason, it is not working with the '?' variable.
I have found two workaround but they are quite unsatisfactory.
First, it is possible to use true and false that will set $? to respectively 0 and 1.
#! /bin/bash
echo $?
true
echo $?
false
echo $?
This will output xxx, 0, 1. This workaround is limited, because it only allow to affect the values 0 and 1.
Then, it is possible to write some code in C (or other) that will just return the value in parameter via exit and then call this function. Example :
#! /bin/bash
rm foo.c
touch foo.c
echo "#include <stdio.h>" >> foo.c
echo "#include <stdlib.h>" >> foo.c
echo "int main(int argc, char **argv)" >> foo.c
echo "{" >> foo.c
echo " return atoi(argv[1]);" >> foo.c
echo "}" >> foo.c
gcc -o foo foo.c
./foo 42
echo $?
That will output 42. Even though it works, it is pretty nasty for doing something so simple, not to mention that this is only a simplified version without all the checkings that would have to be done in order to be sure not to overwrite anything. In addition, this require gcc to be present on the system.
There's no need to involve C at all if you just want to set the most recent return code (although I'm wondering why you're trying to do this). A simple shell function is sufficient:
ret() { return $1; }
Call it like ret 42.
Of course you know that '$?' holds the exit code of the last executed command. If the command is your own written executable, then '$?' holds its exit code.
The default value is 0 (success).
To sum up: use exit in your own scripts.
$ ls
$ echo $?
0
$ echo '#!/bin.bash' > script1.sh && chmod u+x script1.sh
$ ./script1.sh
$ echo $?
0
$ echo -e '#!/bin.bash\nexit 42' > script2.sh && chmod u+x script2.sh
$ ./script2.sh
$ echo $?
42
Also, have a look at Return value in bash script
From the docs:
The 'shell' function performs the same function that backquotes ('`')
perform in most shells: it does "command expansion". This means that it
takes as an argument a shell command and evaluates to the output of the
command. The only processing 'make' does on the result is to convert
each newline (or carriage-return / newline pair) to a single space. If
there is a trailing (carriage-return and) newline it will simply be
removed.
Version 1 of makefile, is:
foo := $(shell echo 'bar'; foo)
all:
#echo 'foo is: $(foo)'
.PHONY: all
Running, we get:
/bin/sh: 1: foo: not found
bar
foo is:
Version 2 of makefile, is:
foo := $(shell echo 'bar'; false)
all:
#echo 'foo is: $(foo)'
.PHONY: all
Running, we get:
foo is: bar
Now, given the above quote from the documentation:
The 'shell' function performs the same function that backquotes ('`')
perform in most shells: it does "command expansion".
We try and compare the following:
# Equivalent to version 1 of makefile above.
$ foo=`echo 'bar'; foo`
sh: 1: foo: not found
$ echo ${foo}
bar
# Equivalent to version 2 of makefile above.
$ foo=`echo 'bar'; false`
$ echo ${foo}
bar
So, not only is the shell function not consistent, but it differs from "command expansion" of shells, referenced to, in the docs, cited above!
Is there a rationale behind all this?
Converting a comment into a substantiated answer.
The exit status from echo 'bar'; foo is 127; the exit status from echo 'bar'; false is 1. Exit status 127 means that the command failed to execute in some way.
In the comment, I said that my suspicion is that make treats the two exit statuses differently.
When I create makefile.v3 that contains:
foo := $(shell echo 'bar'; exit ${exit})
all:
#echo 'foo is: $(foo)'
.PHONY: all
and run it as shown:
$ make -f makefile.v3
foo is: bar
$ make -f makefile.v3 exit=0
foo is: bar
$ make -f makefile.v3 exit=1
foo is: bar
$ make -f makefile.v3 exit=127
bar
foo is:
$ make -f makefile.v3 exit=128
foo is: bar
$ make -f makefile.v3 exit=129
foo is: bar
$ make -f makefile.v3 exit=126
foo is: bar
$ make -f makefile.v3 exit=255
foo is: bar
$
As hypothesized in the comment, the exit status of 127 is treated differently by make.
(For the record: tested with GNU Make 3.81 on Mac OS X 10.10.4.)
To test if a variable is read-only, there are the following ugly hacks:
# True if readonly
readonly -p | egrep "declare -[:lower:]+ ${var}="
# False if readonly
temp="$var"; eval $var=x 2>/dev/null && eval $var=\$temp
Is there a more elegant solution?
Using a subshell seems to work. Both with local and exported variables.
$ foo=123
$ bar=456
$ readonly foo
$ echo $foo $bar
123 456
$ (unset foo 2> /dev/null) || echo "Read only"
Read only
$ (unset bar 2> /dev/null) || echo "Read only"
$
$ echo $foo $bar
123 456 # Still intact :-)
The important thing is that even is that the subshell salvages your RW ($bar in this case) from being unset in your current shell.
Tested with bash and ksh.
You can also add an empty string to the variable, which still leaves its value alone, but is faster than using a subshell, e.g.:
foo+= 2>/dev/null || echo "Read only"
Captured as a function, it'd be:
is-writable() { eval "$1+=" >2/dev/null; }