I have a shell program that takes ages to complete. As written, executing make build takes 4 x 2 seconds to complete because $(value) is computed for each file.
A solution is to declare value a deferred variable by using := instead of =.
Unfortunately this is not a solution either because it slows down the execution of make clean and any other targets by 2 seconds because value is computed for nothing.
value = $(shell sleep 2 && echo 42)
in = a b c d
out = $(addsuffix .out,$(in))
build: $(out)
%.out: %
echo $(value) > $< || [ rm $# -a true ]
init:
touch $(in)
clean:
rm -vf $(out)
How can I set a variable what is assigned only if used, but only computed once ?
Said differently, I would like build to take 2 seconds to complete and clean to be immediate.
I am not interested to a solution that involves conditionals in order to bypass the assignment of value if the target is not build.
An alternative solution would be this. Unfortunately in this case I need to check whether or not the shelve file needs to be regenerated.
value = $(cat shelve)
shelve:
sleep 2 && echo 42 > $# || [ rm $# -a true ]
in = a b c d
out = $(addsuffix .out,$(in))
build: $(out)
%.out: %
echo $(value) > $< || [ rm $# -a true ]
init:
touch $(in)
clean:
rm -vf $(out)
Here's a trick you can play:
value = $(eval value := $(shell cat shelve))$(value)
How this works: value is first assigned using recursive assignment so the value on the RHS is not expanded.
The first time value is expanded the make parser will first run the $(eval ...) which starts up a "new parser" for makefiles. In that parser, the content value := $(cat shelve) is evaluated. Here, value is a simple variable assignment so the RHS is expanded immediately and the $(shell ...) is run and assigned to value.
Remember make doesn't really have a concept of variable scope, so this value is just the same global value variable that we are setting in the outer parser.
Then the eval completes and expands to the empty string, and make continues parsing things. Here it finds the value $(value) and expands that... value now has the result from the eval, not the eval text itself, so that's what will be expanded.
Maybe this will help:
value = $(eval value := $(shell cat shelve))$(value)
Here value contains the string $(eval value := $(shell cat shelve))$(value)
Now you expand it:
%.out: %
echo $(value) > $< ...
Make starts to expand this recipe. It gets to $(value) and sees it needs to expand the variable value: since it's recursive it expands the value:
$(eval value := $(shell cat shelve))$(value)
First it expands the eval, which parses this:
value := $(shell cat shelve)
That sets the value variable as a simply-expanded variable, so the RHS is expanded immediately. Say the results of cat shelve are "foo", so value is now set to foo (and it's marked simply expanded).
That's the end of the eval, so then make starts the next part which is $(value), so it looks up the variable value and discovers it's a simply-expanded variable with the value foo.
One solution would be to turn that value into a regular file target that gets updated only when its prerequisites change. If you insist on rebuilding that target for every build, mark it as phony.
When clean target does not depend on that file, then it won't be rebuilt when you invoke make clean.
In
%.out: %
echo $(value) > $< || [ rm $# -a true ]
echo $(value) > $< updates the prerequisite, whereas make expects it to update the target only. Updating a prerequisite must be done by a separate rule with that prerequisite being the target.
You can make the assignment depend on the target name in $(MAKECMDGOALS):
ifneq ($(MAKECMDGOALS),clean)
value := $(shell sleep 2 && echo 42)
endif
See also the docs for details.
How can I list the current value of all variables (also called macros) in a Makefile when running make?
E.g. if this is in the Makefile:
CUR-DIR := $(shell /bin/pwd)
LOG-DIR := $(CUR-DIR)/make-logs
Then I would like it to tell me:
CUR-DIR = /home/johv/src/test
LOG-DIR = /home/johv/src/test/make-logs
GNU make provides .VARIABLES
which holds all global variables' names.
However, this includes built-in variables(like MAKEFLAGS).
If you have to exclude built-in variables, some filtering like the following
might be needed.
The following makefile prints user-defined variables(CUR-DIR, LOG-DIR)
using info:
# Place this line at the top of your Makefile
VARS_OLD := $(.VARIABLES)
# Define your variables
CUR-DIR := $(shell pwd)
LOG-DIR := $(CUR-DIR)/make-logs
# Put this at the point where you want to see the variable values
$(foreach v, \
$(filter-out $(VARS_OLD) VARS_OLD,$(.VARIABLES)), \
$(info $(v) = $($(v))))
Thanks to #Ise Wisteria, condensed down, this shows all variables, useful for large projects with multiple makefiles (Buildroot).
$(foreach v, $(.VARIABLES), $(info $(v) = $($(v))))
output: BR2_GCC_TARGET_TUNE = "cortex-a8" ...
If you get an error like: insufficient number of arguments (1) to function 'addprefix' this project had some broken variables... I trimmed the list of variables to show, only with a prefix BR2_
$(foreach v, $(filter BR2_%,$(.VARIABLES)), $(info $(v) = $($(v))))
I ended up doing it like this:
gmake -pn | grep -A1 "^# makefile"| grep -v "^#\|^--" | sort | uniq > makevars.txt
which gives:
CUR-DIR := /home/johv/src/test
LOG-DIR := /home/johv/src/test/make-logs
MAKEFILE_LIST := Makefile
MAKEFLAGS = pn
SHELL = /bin/sh
VARS_OLD := [...]
gmake -pn is really verbose and looks kinda like this:
# environment
GNOME2_PATH = /usr/local:/opt/gnome:/usr:/usr/local:/opt/gnome:/usr
# automatic
#F = $(notdir $#)
# makefile
SHELL = /bin/sh
# default
RM = rm -f
It's also doable without saving all the .VARIABLES and filtering them out.
Moreover, if one of the original .VARIABLES was modified in your makefile, the two most voted answers won't catch it.
Check out $(origin) function. This target filters out and prints all the variables that were defined in a makefile:
print_file_vars:
$(foreach v, $(.VARIABLES), $(if $(filter file,$(origin $(v))), $(info $(v)=$($(v)))))
I get only a few excess variables this way: CURDIR SHELL MAKEFILE_LIST .DEFAULT_GOAL MAKEFLAGS.
One can replace file with environment or command line to print the respective kinds of variables.
There are a lot of good answers here, but you're going to have problems using $($(v)) if some of your variables are of the recursive flavor. This is why you should use $(value $(v)).
This variation cleans this up a little bit, sorts variables by name and makes the output a bit more readable.
dump:
$(foreach v, \
$(shell echo "$(filter-out .VARIABLES,$(.VARIABLES))" | tr ' ' '\n' | sort), \
$(info $(shell printf "%-20s" "$(v)")= $(value $(v))) \
)
Thanks to #kevinf for the great idea. I would suggest a minor change to prevent .VARIABLE itself from printing out in the variable list:
$(foreach v, $(filter-out .VARIABLES,$(.VARIABLES)), $(info $(v) = $($(v))))
Thanks to #kevinf for the foreach solution -- if one wants to export this list as a somewhat machine-readable file, one will have a hard time with uneven quotes or newlines when using echo or printf, since Make isn't able to quote the data correctly -- one needs to use the $(file ...) function to write the data to avoid sh/bash complaining about invalid syntax. For example, use this in your rule -- it prints variable name, definition and expanded value:
$(file > $(MAKEFILE_ENV_FILE),)
$(foreach v, $(.VARIABLES), \
$(file >> $(MAKEFILE_ENV_FILE),$(v)) \
$(file >> $(MAKEFILE_ENV_FILE), := $(value $(v))) \
$(file >> $(MAKEFILE_ENV_FILE), == $($(v))) \
$(file >> $(MAKEFILE_ENV_FILE),) \
)
(This will still not allow to always distinguish malicious variables with double newlines from two variables, for this one now add a sufficiently unique separator infront of each Makefile-generated newline just after each comma inside $(file >> NAME,TEXT))
Set MAKEFILE_ENV_FILE to some filename, e.g.:
MAKEFILE_ENV_FILE := $(abspath $(lastword $(MAKEFILE_LIST))).env
In a makefile, I'd like to define a variable specifying whether the current redhat-release is greater than 5.3. (This variable will be passed to gcc as a #define)
So far I've come up with:
# Find out which version of Red-Hat we're running
RH_VER_NUM = $(shell /bin/grep -o [0-9].[0-9] /etc/redhat-release)
RH_GT_5_3 = $RH_VER_NUM > '5.3'
What would be the correct way to define RH_GT_5_3?
GNU Make doesn't contain any string comparisons other than equality, and test can only do less-than/greater-than tests on integers. Split the version number into its integral parts and do the comparison that way. Try this (note also that := is better than = here, as make's lazy evaluation would call your $(shell) commands many more times than required:
RH_VER_MAJOR := $(shell echo $(RH_VER_NUM) | cut -f1 -d.)
RH_VER_MINOR := $(shell echo $(RH_VER_NUM) | cut -f2 -d.)
RH_GT_5_3 := $(shell [ $(RH_VER_MAJOR) -gt 5 -o \( $(RH_VER_MAJOR) -eq 5 -a $(RH_VER_MINOR) -ge 3 \) ] && echo true)
ifeq ($(RH_GT_5_3),true)
CPPFLAGS += -DRH_GT_5_3=1
endif
A little bit shorter solution is:
RH_GT_5_3 := $(shell echo -e "5.4\n$(RH_VER_NUM)"|sort -ct. -k1,1n -k2,2n && echo YES)
This will set RH_GT_5_3 to "YES" if RH_VER_NUM is greater then or equal to 5.4 (so greater then 5.3). Otherwise RH_GT_5_3 will be set to empty.
If multiple version numbers need to be checked we can define a function:
IF_VER_GE = $(shell echo -e "$2\n$1"|sort -ct. -k1,1n -k2,2n && echo YES)
GLIBC := $(word 2,$(shell getconf GNU_LIBC_VERSION))
...
all:
ifeq "$(call IF_VER_GE, $(GLIBC), 2.5)" "YES"
echo "GE"
else
echo "LT"
endif
I used the "$(word 2,..." instead of "$(lastword,..." because the later does not work in make 3.8. And shorter...
... some aeons later
I tried to solve the version comparison with internal makefile functions. I found a project (GNU Make Standard Library (GMSL)), which adds an include to the makefile which implements integer arithmetic. Unfortunately with the common unary numeral system. But comparing version numbers are more complex. As I worked with versions having numbers greater then 100_000 I decided to implement a more general solution. It works with arbitrary number of subversion numbers each with arbitrary digits. Some ideas were borrowed from the GMSL project
It implements the ver.lt function. It returns 'T' if the first version number is less then the second. It returns an empty string otherwise. Of course, the subversion numbers are compared numerically not lexicographically. So 1.20 is greater then 1.3. There is some issues. 1.2 < 1.2.0, 1.0.1 < 1.00.1, 1.9.1 < 1.01.1 (as it is expected that a number starts with a nonzero digit. Except the 0 itself.). I do not want to solve them now.
The solution
It is testAed under gnu make 3.82.90. There are some very long lines as makefile add spaces if the '\' is used. I left some implemented, but not used functions in the code. Maybe I would use better temporary variable names (like GMSL uses _gmsl). Sometimes temporary variables could be left off, but the code would be more cryptic.
.SILENT:
S :=
SP := $S $S
# For non empty strings
#not = $(if $1,$S,T)
#str.ne = $(if $(subst $1,,$2),T,$S)
str.eq = $(if $(subst $1,,$2),$S,T)
str.le = $(call str.eq,$(word 1,$(sort $1 $2)),$1)
#str.ge = $(call str.eq,$(word 1,$(sort $1 $2)),$2)
# Creates a list of digits from a number
mklist = $(eval __tmp := $1)$(foreach i,0 1 2 3 4 5 6 7 8 9,$(eval __tmp := $$(subst $$i,$$i ,$(__tmp))))$(__tmp)
# reverse: $(subst $(SP),,$(list))
#pop = $(wordlist 2, $(words $1), x $1)
#push = $1 $2
shift = $(wordlist 2, $(words $1), $1)
#unshift = $2 $1
num.le = $(eval __tmp1 := $(call mklist,$1))$(eval __tmp2 := $(call mklist,$2))$(if $(call str.eq,$(words $(__tmp1)),$(words $(__tmp2))),$(call str.le,$1,$2),$(call str.le,$(words $(__tmp1)),$(words $(__tmp2))))
#num.ge = $(eval __tmp1 := $(call mklist,$1))$(eval __tmp2 := $(call mklist,$2))$(if $(call str.eq,$(words $(__tmp1)),$(words $(__tmp2))),$(call str.ge,$1,$2),$(call str.ge,$(words $(__tmp1)),$(words $(__tmp2))))
#Strip zeroes from the beginning of a list
list.strip = $(eval __flag := 1)$(foreach d,$1,$(if $(__flag),$(if $(subst 0,,$d),$(eval __flag :=)$d,$S),$d))
#Strip zeroes from the beginning of a number
#num.strip = $(subst $(SP),,$(call list.strip,$(call mklist,$1)))
# temp string: 0 - two number equals, L first LT, G first GT or second is short,
gen.cmpstr = $(eval __Tmp1 := $(subst ., ,$1))$(eval __Tmp2 := $(subst ., ,$2))$(foreach i,$(__Tmp1),$(eval j := $(word 1,$(__Tmp2)))$(if $j,$(if $(call str.eq,$i,$j),0,$(if $(call num.le,$i,$j),L,G)),G)$(eval __Tmp2 := $$(call shift,$(__Tmp2))))$(if $(__Tmp2), L)
ver.lt = $(call str.eq,$(word 1,$(call list.strip,$(call gen.cmpstr,$1,$2))),L)
all:
echo ver.lt,1.20,1.3:$(call ver.lt,1.20,1.3)%
echo ver.lt,1.5.9,1.5:$(call ver.lt,1.5.9,1.5)%
echo ver.lt,1.4.9,1.5:$(call ver.lt,1.4.9,1.5)%
echo ver.lt,1.2,1.2.0:$(call ver.lt,1.2,1.2.0)%
echo ver.lt,1.20.3.4.5,1.10.5:$(call ver.lt,1.20.3.4.5,1.10.5)%
echo ver.lt,1.20.3.4.5,1.0.5:$(call ver.lt,1.20.3.4.5,1.0.5)%
echo ver.lt,1.0,1.0.5:$(call ver.lt,1.0,1.0.5)%
echo ver.lt,1.20,1.10.3:$(call ver.lt,1.20,1.10.3)%
echo ver.lt,1.20,1.30.3::$(call ver.lt,1.20,1.30.3)%
echo ver.lt,1.10.3,1.10.3:$(call ver.lt,1.10.3,1.10.3)%
And the output
ver.lt,1.20,1.3:%
ver.lt,1.5.9,1.5:%
ver.lt,1.4.9,1.5:T%
ver.lt,1.2,1.2.0:T%
ver.lt,1.20.3.4.5,1.10.5:%
ver.lt,1.20.3.4.5,1.0.5:%
ver.lt,1.0,1.0.5:T%
ver.lt,1.20,1.10.3:%
ver.lt,1.20,1.30.3::T%
ver.lt,1.10.3,1.10.3:%
More music
I have found another interesting project called makepp (makepp.sourceforge.net). It enables to implement new functions in perl inside the makefile.
The simplest solution I think of is by using bc:
# Find out which version of Red-Hat we're running
RH_VER_NUM = $(shell /bin/grep -o [0-9].[0-9] /etc/redhat-release)
RH_GT_5_3 = $(shell echo $(RH_VER_NUM)\>=5.3 | bc )
ifeq ($(RH_GT_5_3),1)
CPPFLAGS += -DRH_GT_5_3=1
endif
This will be equal to 1 if bigger than 5.3, and 0 otherwise.
bc is an arbitrary precision calculator language. Here are some documentation about bc:
Official documentation
Man page
Wikipedia
Kudos to TrueY for coming up with a version that doesn't use any expensive shell commands. I have a simpler version that only compares 2 small integers (often you only care about the major version).
The idea is simply
x > y
= y is member of {0,1,2,3,4 ..., x - 1)
The member of operator can be implemented with GNU make's $(filter) and generating the set can be done with $(wordlist)
# returns all integers less than x
LessThanSubset=$(wordlist 1,$(1),0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20)
# if x > y, return 1, empty otherwise
GreaterThan=$(if $(filter $(2),$(call LessThanSubset,$(1))),1)
GreaterOrEqual=$(if $(filter $(2),$(call LessThanSubset,$(1)) $(1)),1)
# example
$(error 5 > 4 = $(call GreaterThan,5,4))
I realize this is an old question, but still. Actually you can resort to lsb_release, which I found to be installed on every single recent RedHat system.
To be precise you'd make use of
lsb_release -sr
inside $(shell ...) and then split at the . like this:
RH_VER:=$(shell lsb_release -sr)
RH_MAJVER:=$(word 1, $(subst ., ,$(RH_VER)))
RH_MINVER:=$(word 2, $(subst ., ,$(RH_VER)))
you now have the major/minor parts of the release version and can check that against whatever you like.
The classic case would be $(filter ...) and the other text functions that GNU make provides.
Nevertheless I agree that a kind of a config.h would make more sense, though Autotools is by no means perfect for all scenarios (there are other build tools that attempt the same, though).
What about using sort function:
ifeq "$(firstword $(sort $(RH_VER_NUM),5.3))" "5.3"
RH_GT_5_3 = 1
endif
(well it would actually test for greater or equal, but you get the idea)
A modern way would be to get the version into make (using one shell invocation), e.g.:
SHELL := /bin/bash
# Define redhat_release as major_version * 100 + minor_version
redhat_release := $(shell [[ "$$(cat /etc/redhat-release)" =~ ([0-9]+)\.([0-9]+) ]] && echo $$(( $${BASH_REMATCH[1]} * 100 + $${BASH_REMATCH[2]} )))
And then have version specific flags:
# Flags for different redhat versions
redhat_flags.604 := abc
redhat_flags.605 := ${redhat_flags.604} def
redhat_flags := ${redhat_flags.${redhat_release}}
And print them out:
$(info redhat_release=${redhat_release})
$(info redhat_flags=${redhat_flags})
Output:
$ make
redhat_release=605
redhat_flags=abc def
Building on TrueY's smart use of sort and making it a re-usable Make function:
version_greater_equal = $(shell if printf '%s\n%s\n' '$(2)' '$(1)' | \
sort -Ct. -k1,1n -k2,2n ; then echo YES; else echo NO; fi )
ifeq (YES,$(call version_greater_equal,${SOME_VERSION},8.2))
...
With the arithmetic capabilities of the GNUmake table toolkit you can write a quite general test - the one thing you need to adapt is the Lisp-like functional style ("function op1,op2,opN"):
include gmtt/gmtt.mk
# Simulate some nasty versioning style:
# X.Y.Z.rN for release versions,
# X.rN-beta.Y.Z for beta versions towards a release
VERSION_NR_REL := $(call glob-match,$(VERSION_NR),*.*.*.r*)
VERSION_NR_BETA := $(call glob-match,$(VERSION_NR),*.r*-beta.*.*)
# glob-match only produces a non-empty result if the match succeeds,
# thus one of the following will be an empty string
$(info Release version: $(VERSION_NR_REL))
$(info Beta version: $(VERSION_NR_BETA))
# we use makes "if" function which sees empty strings as false and everything else as true
BETA_OR_RELEASE := $(if $(VERSION_NR_REL),release,beta)
# react to version numbers on different positions in release and beta
ifeq ($(BETA_OR_RELEASE),beta)
MAJOR_V := $(word 1,$(VERSION_NR_BETA))
MINOR_V := $(word 5,$(VERSION_NR_BETA))
BUGFIX_V := $(word 7,$(VERSION_NR_BETA))
REL_NR := $(word 3,$(VERSION_NR_BETA))
else
MAJOR_V := $(word 1,$(VERSION_NR_REL))
MINOR_V := $(word 3,$(VERSION_NR_REL))
BUGFIX_V := $(word 5,$(VERSION_NR_REL))
REL_NR := $(word 7,$(VERSION_NR_REL))
endif
GT_5_3 := $(if $(or $(call int-gt,$(MAJOR_V),5),\
$(and $(call int-eq,$(MAJOR_V),5),\
$(call int-gt,$(MINOR_V),3))),\
yes,no)
$(info Major: $(MAJOR_V), Minor: $(MINOR_V), Bugfix: $(BUGFIX_V), Release: $(REL_NR), greater than 5.3: $(GT_5_3))
Test with a larger version number:
make VERSION_NR=18.r01-beta.3.5
Output:
Release version:
Beta version: 18 .r 01 -beta. 3 . 5
Major: 18, Minor: 3, Bugfix: 5, Release: 01, greater than 5.3: yes
Test with a version equal to 5.3:
make VERSION_NR=5.3.12.r3
Output:
Release version: 5 . 3 . 12 .r 3
Beta version:
Major: 5, Minor: 3, Bugfix: 12, Release: 3, greater than 5.3: no
I had to compare GLIBC versions, I used:
GLIBC_VERSION := $(shell ldd --version | head -1 | awk '{print $$NF}')
ifeq ($(shell awk -v a="$(GLIBC_VERSION)" -v b="2.35" 'BEGIN{print(a<b)}'), 1)
# do something if GLIBC less than 2.35
endif
this works because awk can handle floating-point comparison and glibc doesn't have a third version number field