So I have a really nice make macro that creates me a pretty message during compilation that is done in such a way that the destination of the message can be implemented externally.
It uses the $(shell ) make function something like this.
send_msg = $(shell $do_send_msg "$1")
Where do_send_msg can be expanded to be echo, wall, email, etc.
I use this lots in the makefiles.
I'd like to be able to use the same macro in recipes as the result of the recipe may change what is needed to be sent, dependent on some test made to the result of an external program.
However variables in recipes are expanded by make entirely before the lines in the recipe are called one at a time.
So if I write (e.g.)
if test_prog; then \
$(call send_msg,PASS);\
else \
$(call send_msg,FAIL);\
fi
Effectively this becomes (not strictly true syntax, but close enough):
(bash -c 'echo "PASS"') &
(bash -c 'echo "FAIL"') &
(bash -c 'if test_prog; then \
\
else \
\
fi)'
This of course will not work, it will run test_prog and, separately, regardless of the exit status both send_msg lines will expand and be executed.
So it will print both PASS and FAIL as the two subshells are run independently and in parallel by make.
I don't want to 'cheat' and use another variable in the $(call ) function or even worse a global that, ok, would allow an if some_var ... in the send_msg implementation but would reduce it's flexability as the implementation would have to understand that variable in all cases.
Another way would be to just have two different send_msg macros, one with and one without the $(shell ) function. Simple, but not elegant.
Right now I am using a 'hack' and calling the same makefile with a variable. If that variable is set then it sends it's contents otherwise does nothing. This works fine but to me it seems clunky and wrong, there must be a better way.....
e.g.
if test_prog; then \
MESSAGE=PASS $(MAKE) message;\
else \
MESSAGE=FAIL $(MAKE) message;\
fi
Where the Makefile says (and this is an abreviated version to convey the idea)
ifneq ($(MESSAGE),)
message:
$(call send_msg,$(MESSAGE))
else:
message:
endif
Question:
How would I make (make) detect if the macro is being expanded inside a Makefile recipe or inside the Makefile body and effectively keep or remove the $(shell ) call that wraps how the work is done?
e.g. (if make_or_shell existed)
ifdef some_test
$(make_or_shell send_msg,"Message from Make")
endif
goal:
$(make_or_shell send_msg,"Message from Recipe")
Related
The example below is directly from the GNU make manual:
program: $(OBJECTS)
$(file >$#.in,$^)
$(CMD) $(CMDFLAGS) #$#.in
#rm $#.in
This works perfectly. However, if I reverse the order of $(file) and the external tool, the behavior is still the same: $(file) is executed first, and only then the external tool.
Is it possible to call an external tool in a recipe, and after the successful completion of the tool, use $(file) to process the log and create further files?
The external tool is a code generator that produces an unknown set of files. The recipe should parse the log file and create make include files. Works with echo/$(shell), but that's limited by the system command line length. Using a separate rule would certainly work, but can it be done in the same recipe?
GNU make will always expand all variables and functions in the entire recipe up front, before it starts to invoke any commands in the recipe. So you cannot have a $(file ...) function (for example) operate on the results of running some command in the same recipe.
So the short answer to your question is no, it can't be done in a single recipe the way you want.
It's possible, if you run the shell command using $(shell ) rather than directly putting it in a recipe. I've been using following wrapper for it, which checks the error code and discards the result:
# Same as `$(shell ...)`, but triggers a error on failure.
ifeq ($(filter --trace,$(MAKEFLAGS)),)
override safe_shell = $(shell $1)$(if $(filter-out 0,$(.SHELLSTATUS)),$(error Unable to execute `$1`, status $(.SHELLSTATUS)))
else
override safe_shell = $(info Shell command: $1)$(shell $1)$(if $(filter-out 0,$(.SHELLSTATUS)),$(error Unable to execute `$1`, status $(.SHELLSTATUS)))
endif
# Same as `safe_shell`, but discards the output and expands to a single space.
override safe_shell_exec = $(call space,$(call safe_shell,$1))
Then $(CMD) $(CMDFLAGS) #$#.in becomes $(call safe_shell_exec,$(CMD) $(CMDFLAGS) #$#.in).
I have a number of makefiles that build and run tests. I would like to create a script that makes each one and notes whether the tests passed or failed. Though I can determine test status within each make file, I am having trouble finding a way to communicate that status to the caller of the make command.
My first thought is to somehow affect the return value of the make command, though this does not seem possible. Can I do this? Is there some other form of communication I can use to express the test status to the bash script that will be calling make? Perhaps by using environment variables?
Thanks
Edit: It seems that I cannot set the return code for make, so for the time being I will have to make the tests, run them in the calling script instead of the makefile, note the results, and then manually run a make clean. I appreciate everyone's assistance.
Make will only return one of the following according to the source
#define MAKE_SUCCESS 0
#define MAKE_TROUBLE 1
#define MAKE_FAILURE 2
MAKE_SUCCESS and MAKE_FAILURE should be self-explanatory; MAKE_TROUBLE is only returned when running make with the -q option.
That's pretty much all you get from make, there doesn't seem to be any way to set the return code.
The default behavior of make is to return failure and abandon any remaining targets if something failed.
for directory in */; do
if ( cd "$directory" && make ); then
echo "$0: Make in $directory succeeded" >&2
else
echo "$0: Make in $directory failed" >&2
fi
done
Simply ensure each test leaves its result in a file unique to that test. Least friction will be to create test.pass if thes test passes, otherwise create test.fail. At the end of the test run gather up all the files and generate a report.
This scheme has two advantages that I can see:
You can run the tests in parallel (You do us the -jn flag, don't you? (hint: it's the whole point of make))
You can use the result files to record whether the test needs to be re-run (standard culling of work (hint: this is nearly the whole point of make))
Assuming the tests are called test-blah where blah is any string, and that you have a list of tests in ${tests} (after all, you have just built them, so it's not an unreasonable assumption).
A sketch:
fail = ${#:%.pass=%.fail}
test-passes := $(addsuffix .pass,${tests})
${test-passes}: test-%.pass: test-%
rm -f ${fail}
touch $#
$* || mv $# ${fail}
.PHONY: all
all: ${test-passes}
all:
# Count the .pass files, and the .fail files
echo '$(words $(wildcard *.pass)) passes'
echo '$(words $(wildcard *.fail)) failures'
In more detail:
test-passes := $(addsuffix .pass,${tests})
If ${tests} contains test-1 test-2 (say), then ${test-passes} will be test-1.pass test-2.pass
${test-passes}: test-%.pass: test-%
You've just gotta love static pattern rules.
This says that the file test-1.pass depends on the file test-1. Similarly for test-2.pass.
If test-1.pass does not exist, or is older than the executable test-1, then make will run the recipe.
rm -f ${fail}
${fail} expands to the target with pass replaced by fail, or test-1.fail in this case. The -f ensures the rm returns no error in the case that the file does not exist.
touch $# — create the .pass file
$< || mv $# ${fail}
Here we run the executable
If it returns success, our work is finished
If it fails, the output file is deleted, and test-1.fail is put in its place
Either way, make sees no error
.PHONY: all — The all target is symbolic and is not a file
all: ${test-passes}
Before we run the recipe for all, we build and run all the tests
echo '$(words $(wildcard *.pass)) passes'
Before passing the text to the shell, make expands $(wildcard) into a list of pass files, and then counts the files with $(words). The shell gets the command echo 4 passes (say)
You run this with
$ make -j9 all
Make will keep 9 jobs running at once — lovely if you have 8 CPUs.
I have a few software projects which are distributed as RPMs. They are versioned using semantic versioning to which we affix a release number. Using the regular conventions, this is MAJOR.MINOR.PATCH-REL_NUM. Though beyond the scope of this article, the release numbers are stored in git. The release target in the makefile looks something like this:
release:
make clean
$(BLD_ROOT)/tools/incr_rel_num
# Although the third step, this was re-ordered to step 1
$(eval RELEASE_NUMBER=$(shell cat $(BLD_ROOT)/path/to/rel_num.txt))
make rpm RPM_RELEASE_NUM=$(RELEASE_NUMBER)
While debugging, I eventually discovered that, although the call to eval was the third step in the recipe, it was actually being evaluated first! This is why the RPM always had a release number one less than the number I was watching get pushed to the remote.
I have done much googling on this and I haven't found any hits that explain the order of evaluation with regard to eval when used in recipes. Perhaps it isn't even with respect to eval but functions in general. Furthermore, I haven't found verbiage on this in the GNU manuals for make either (if it's there, kindly point out what chapter). I've worked around the problem so it's not a bother, I'm just wondering, is this expected and if so, why?
The missing bit, that no one above is getting, is simple: when make is going to run a recipe it expands all lines of the recipe first, before it starts the first line. So:
release:
make clean
$(BLD_ROOT)/tools/incr_rel_num
# Although the third step, this was re-ordered to step 1
$(eval RELEASE_NUMBER=$(shell $(BLD_ROOT)/path/to/rel_num.txt))
make rpm RPM_RELEASE_NUM=$(RELEASE_NUMBER)
when make decides to run the release target it first expands all the lines in the recipe, which means the eval is expanded, then it runs the resulting lines. That's why you're getting the behavior you're seeing.
I don't really see why you need to use eval here at all; why not just use:
release:
$(MAKE) clean
$(BLD_ROOT)/tools/incr_rel_num
$(MAKE) rpm RPM_RELEASE_NUM="$$(cat $(BLD_ROOT)/path/to/rel_num.txt))"
(BTW, you should never use bare make inside your makefiles; you should always use $(MAKE) (or ${MAKE}, same thing).
The $(eval ...) function
generates a fragment of make-sytax which becomes part of the parsed makefile.
The makefile is parsed entirely before any recipes are executed and when recipes
are executed all make-statements, make-expressions and make-variables have been
evaluated away.
So it does not make sense to consider an $(eval ...) call as being one
of the lines of a recipe. It might generate values that are used in the make-expansion
of the recipe, but if so then this happens when the makefile is parsed, before the recipe is run.
Thus in your example, the line:
$(eval RELEASE_NUMBER=$(shell $(BLD_ROOT)/path/to/rel_num.txt))
which I assume should really be:
$(eval RELEASE_NUMBER=$(shell cat $(BLD_ROOT)/path/to/rel_num.txt))
is evaluated when the makefile is parsed, and let's say it results in the
make-variable RELEASE_NUMBER acquiring the value 1.0, because, when the
makefile is parsed, the file $(BLD_ROOT)/path/to/rel_num.txt) contains
1.0. In that case your recipe:
release:
make clean
$(BLD_ROOT)/tools/incr_rel_num
$(eval RELEASE_NUMBER=$(shell cat $(BLD_ROOT)/path/to/rel_num.txt))
make rpm RPM_RELEASE_NUM=$(RELEASE_NUMBER)
will resolve to the like of:
release:
make clean
some_build_dir/tools/incr_rel_num
make rpm RPM_RELEASE_NUM=1.0
You will observe when make runs the recipe that it prints no line that
is "the expansion of" $(eval RELEASE_NUMBER=$(shell cat $(BLD_ROOT)/path/to/rel_num.txt)),
because there is no such thing in the recipe. It doesn't matter that:
some_build_dir/tools/incr_rel_num
is presumably a command that writes, say, 1.1 or 2.0 in the file some_build_dir/path/to/rel_num.txt.
That action simply has no effect on the recipe. Nothing that executed in the recipe
can change the recipe.
$(eval ...) has no business in your recipe. What you want to achieve is simply:
release:
make clean
$(BLD_ROOT)/tools/incr_rel_num
RELEASE_NUMBER=$$(cat $(BLD_ROOT)/path/to/rel_num.txt) && \
make rpm RPM_RELEASE_NUM=$$RELEASE_NUMBER
where $$ is what you do in a makefile to escape $ and, in this case,
leave it for the shell when the recipe is executed.
This recipe expands to 3 shell commands executed in sequence:
$ make clean
$ some_build_dir/tools/incr_rel_num
$ RELEASE_NUMBER=$(cat some_build_dir/path/to/rel_num.txt) && \
make rpm RPM_RELEASE_NUM=$RELEASE_NUMBER
and might as well be simplified further to:
release:
make clean
$(BLD_ROOT)/tools/incr_rel_num
make rpm RPM_RELEASE_NUM=$$(cat $(BLD_ROOT)/path/to/rel_num.txt)
You are correct, there are multiple levels of evaluation. The content on what is inside eval is evaluated a first time before that the function is actually called. If you want the content of eval to be evaluated at the time eval is called, you have to escape the $ sign by putting it twice, like this :
$(eval RELEASE_NUMBER=$$(shell $(BLD_ROOT)/path/to/rel_num.txt))
To view what is really inside eval at the time it's called you can use the same syntax with info instead of eval :
$(info RELEASE_NUMBER=$$(shell $(BLD_ROOT)/path/to/rel_num.txt))
Now I'm not sure about the part which is evaluated too soon so the $ symbols that I doubled may not be the good one(s), but using the info function will help you to find the correct command.
I have a somewhat complex makefile that I want to change and I don't know much about make.
BUILD_TYPE = SERVER
BAS_CSRC = a.c \
b.c \
c.c
What I want to do is conditionally add things to BAS_CSRC like so:
ifeq ($(BUILD_TYPE), SERVER)
USR_CSRC = $(BAS_CSRC) \
d.c \
e.c
endif
all_csrc = $(USR_CSRC) $(foreach var, $(COMMON_OBJECTS), $($(var)_csrc))
But when I compile the d.c and e.c are just ignored so the ifeq fails. Why? What about quoting?
Based on what you've typed here it should work (and I cut and pasted this into a test makefile which worked for me), which likely means there's something different about your real environment, than this example. Can you show us where you use the variable all_csrc? Maybe it's that that's the problem, not the assignment of the variable.
Make sure you don't have any trailing whitespace. Make sure you have matching case in your variables and values (make, like all UNIX tools, is case-sensitive). You can try adding $(info ...) statements to your makefile and it will print out what it's doing. Put one inside the ifeq to see if it fires, and after the endif to see what the value of USR_CSRC is.
Also, in general it's not a good idea to add whitespace into if statements or function calls like foreach. In the above situations it shouldn't matter but in general it's best avoided.
Alright I am stuck on this and I have no idea what I am doing wrong. Everything was going great working on a more complicated makefile but then all of a sudden I got the "Missing separator" error. I was able to isolate it down to a very simple scenario:
test.mk
define push_dir
$(info ${1})
endef
define pop_dir
$(info ${1})
endef
define include_submake
$(call push_dir,${1})
$(call pop_dir,${1})
endef
Simple
include test.mk
INITIAL_SUBMAKE:= includeme.mk
$(call include_submake,${INITIAL_SUBMAKE})
process:
#echo Processed...
And the output:
C:\project>make -f Simple process
includeme.mk
includeme.mk
Simple:4: *** missing separator. Stop.
includeme.mk does not actually exist. I have no idea what is going wrong here I have tried a multitude of things. If I surround the call to include_submake in info like so:
$(info $(call include_submake,${INITIAL_SUBMAKE}))
The missing separator error goes away. Also If in the include_submake define I only call one of the functions it works fine. Additionally if I directly call the functions instead of calling them include_submake it works as well:
include test.mk
INITIAL_SUBMAKE:= includeme.mk
$(call push_dir,${INITIAL_SUBMAKE})
$(call pop_dir,${INITIAL_SUBMAKE})
process:
#echo Processed...
C:\project>make -f Simple process
includeme.mk
includeme.mk
Processed...
I feel like I'm overlooking something fundamental here. Thanks for your help.
The missing separator error happens because of a non-empty return value of include_submake, which is a single line feed character in your case. Make only permits whitespace characters (that is, a space or tab) to occur in an expression which is not assumed to be a part of some rule or another directive.
Rewrite your functions using plain-old Make variable assignment and the error should go away:
push_dir = \
$(info $1)
pop_dir = \
$(info $1)
include_submake = \
$(call push_dir,$1) \
$(call pop_dir,$1)
UPD.: define vs plain old variable assignment
Answering to a question from the first comment. Personally I would prefer using define directive in several cases.
Using with eval function
As the GNU Make manual suggests, define directive is very useful in conjunction with the eval function. Example from the manual (emphasis is mine):
PROGRAMS = server client
server_OBJS = server.o server_priv.o server_access.o
server_LIBS = priv protocol
client_OBJS = client.o client_api.o client_mem.o
client_LIBS = protocol
# Everything after this is generic
.PHONY: all
all: $(PROGRAMS)
define PROGRAM_template
$(1): $$($(1)_OBJS) $$($(1)_LIBS:%=-l%)
ALL_OBJS += $$($(1)_OBJS)
endef
$(foreach prog,$(PROGRAMS),$(eval $(call PROGRAM_template,$(prog))))
$(PROGRAMS):
$(LINK.o) $^ $(LDLIBS) -o $#
clean:
rm -f $(ALL_OBJS) $(PROGRAMS)
Generator templates
Verbatim variables fit perfectly for cases when you want to generate a file from GNU Make. For example, consider generating a header file based on some information from Makefile.
# Args:
# 1. Header identifier.
define header_template
/* This file is generated by GNU Make $(MAKE_VERSION). */
#ifndef $(inclusion_guard)
#define $(inclusion_guard)
$(foreach inc,$($1.includes),
#include <$(inc).h>)
/* Something else... */
#endif /* $(inclusion_guard) */
endef
# 1. Unique header identifier.
inclusion_guard = \
__GEN_$1_H
# Shell escape.
sh_quote = \
'$(subst ','"'"',$1)'
foo.includes := bar baz
HEADERS := foo.h
$(HEADERS) : %.h :
#printf "%s" $(call sh_quote,$(call header_template,$(*F)))> $#
Extended Make syntax
In our project we use our own build system called Mybuild, and it is implemented entirely on top of GNU Make. As one of low-level hacks that we used to improve the poor syntax of the builtin language of Make, we have developed a special script which allows one to use extended syntax for function definitions. The script itself is written in Make too, so it is a sort of meta-programming in Make.
In particular, one can use such features as:
Defining multiline functions without the need to use backslash
Using comments inside functions (in plain-old Make comments can only occur outside variable assignment directives)
Defining custom macros like $(assert ...) or $(lambda ...)
Inlining simple functions like $(eq s1,s2) (string equality check)
This is an example of how a function can be written using the extended syntax. Note that it becomes a valid Make function and can be called as usual after a call to $(def_all).
# Reverses the specified list.
# 1. The list
# Return:
# The list with its elements in reverse order.
define reverse
# Start from the empty list.
$(fold ,$1,
# Prepend each new element ($2) to
# the result of previous computations.
$(lambda $2 $1))
endef
$(def_all)
Using these new features we were able to implement some really cool things (well, at least for Make :-) ) including:
Object-Oriented layer with dynamic object allocation, class inheritance, method invocations and so on
LALR parser runtime engine for parsers generated by GOLD Parser Builder
Modelling library with runtime support for models generated with EMF
Feel free to use any part of the code in your own projects!
I ran into the same problem. I inserted 'tab', deleted 'tab', reinserted to be sure. Same error message.
But, I did all of this inside of XCodem which to my surprise inserted white spaces, not '\t'. Once I used different editor these 'phantom' errors went away.
HTH...