I'm using a Makefile to compile my project. I get to a point which is:
$(MAKE) <some flags>; \
$(UPLOAD_SCRIPT)
The $(MAKE) line actually compiles the code, but I only want the upload script to run if the make was successful (i.e. no errors). Is there a way to do this? I'm imagining something with exit codes, storing the result in a variable, and an if statement, but I'm not super familiar with Makefiles.
Chain the 2 commands using && like this instead:
mytarget:
$(MAKE) <some flags> && $(UPLOAD_SCRIPT)
If you have more lines, and do not want to make your line look very long using &&, you can use set -e, so that the shell stops on the first error.
-e When this option is on, if a simple command fails for any of the
reasons listed in Consequences of Shell Errors or returns an exit
status value >0, and is not part of the compound list following a
while, until, or if keyword, and is not a part of an AND or OR list,
and is not a pipeline preceded by the ! reserved word, then the shell
shall immediately exit.
mytarget:
set -e; \
cmd1; \
cmd2
Related
I am looking for a clean way to change a few target declarations I created in a makefile into a more functional type declaration in which I am able to pass variables and the output will remain consistent.
For example:
default: clean run1 run2
run1:
for entity in $(FLIST_01); do \
echo $entity ; \
done
run2:
for entity in $(FLIST_02); do \
echo $entity ; \
done
Ideally, I would like to remove the repetitive run target declarations and only have 1.
FLIST_01 = my_flist.txt
FLIST_02 = other.txt
default: clean run
run:
$(run_func $(FLIST_01))
$(run_func $(FLIST_02))
How do I create a custom function in make to do what run_func is supposed to be doing (the for loop reading of the file list variable passed to it?
UPDATE:
My attempt so far as been this:
run:
runfunc() { \
echo "test" \
for entity in $1; do \
echo $(entity); \
done \
}
runfunc $(FLIST_01)
runfunc $(FLIST_02)
But I get a syntax error on the do line: syntax error near unexpected token `do'
First off, unless you specify .ONESHELL, the commands in a target will be run in separate subshells. So your runfunc() declaration will run in a shell which declares the function, then exits. The next statement will run in a new shell which will know nothing about the function which was declared and then basically forgotten.
Secondly,
echo "test" \
for entity in $1;
will be expanded by make into
echo "test" for entity in ;
which obviously contains multiple errors. You mean
echo "test"; \
for entity in $$1;
to properly pass the dollar sign through to the shell. But on the whole, I would say your approach is flawed. You can refactor this to a make function like you originally hoped;
define run
for entity in $(1); do \
echo $$entity; \
done
endef
Now you can call this like
run:
$(call run,$(FLIST_01))
$(call run,$(FLIST_02))
But this particular loop can quite easily be replaced with a single shell statement.
run:
printf '%s\n' $(FLIST_01)
printf '%s\n' $(FLIST_02)
Another twopenn'orth.
First off, we want to expand $FLIST_01 for target run1, and FLIST_02 for target run2.
A clean way might be:
FLIST_run1 := a b c
FLIST_run2 := 1 2 3
.PHONY: run1 run2
run1 run2:
for entity in ${FLIST_$#}; do echo $entity; done
For a slightly more obscure solution we notice that what you are trying to do can be described with make's noddy pattern matching:
.PHONY: run1 run2
run%: run1 run2:
for entity in ${FLIST_$*}; do echo $entity; done
Here we use a static pattern rule.
In the recipe, $* expands to whatever matched the % in the pattern.
This solution looks quite clean to my jaundiced eye.
I urge you to seek out idiomatic make rather than idiomatic shell.
Your makefiles will nearly always thank you for it.
In the first instance, keep your two run targets rather than coalesce them into the same recipe.
.PHONY: run1 run2
run%: run1 run2:
for entity in ${FLIST_$*}; do echo $entity; done
.PHONY: run
run: run1
run: run2
run: ; echo $# Success
Here, when you make run for instance, make first carries out the recipe for run1, then the recipe for run2, and finally the recipe for run.
Is there any advantage to this? Sure. If you run make -j2 run then the commands for run1 and run2 will be run in parallel. The whole point of make really.
The final issue is to get rid of the shell loop, and to replace it with separate commands. That way make will check the exit code of the echo for you, and even run them in parallel if that is suitable. Sure, not helpful in this noddy example, but they could be compile commands for instance.
Usually, it is not easier to maintain shell functions inside Makefile. Main reasons:
Each shell '$' need to be escaped to '$$'
The $(...) construct has different meaning in shell vs Make.
Each line in the Makefile is executing by new shell instance, meaning that variables assignment, functions definitions, etc, are not shared between lines.
Consider few options: write shell helper script (recommended), use make user-defined functions or inline the function into the Make. For the specific case of the "echo", it might be possible to use one of the other approaches, see "Alternative Solution"
From personal experience, for any nontrivial functions, better to use shell helper script. They are much easier to develop and test.
Using Shell Helper Script
Write a small helper script "print-list.sh"
#! /bin/bash
echo "test" \
for entity ; do
echo $entity
done
And then invoke it from the target
FLIST_01 = a.txt b.txt c.txt
t1:
print-list.sh ${FLIST_01)
Inlining the shell function
As indicated above, embedding the shell function into the Makefile requires matching make rules about quoting, escapes, etc. It also require lot of effort debugging the script, as error messages are very cryptic in this setup.
Note that RUNFUNC must be included before every action lines that uses the command.
RUNFUNC = runfunc() { \
echo "test" ; \
for entity in $$*; do \
echo $$entity; \
done \
} ;
FLIST_01 = a.txt b.txt c.txt
t2:
$(RUNFUNC) runfunc ${FLIST_01)
Note that the function can be written as one liner, or using the 'define'/'endef' to simplify end-of-line escapes.
Using make user-defined functions
It is possible to create simple functions using make. This option require experience and time.
UFUNC = \
echo "test" ; \
for entity in $1; do \
echo $$entity ; \
done
FLIST_01 = a.txt b.txt c.txt
t3:
$(call UFUNC, ${FLIST_01))
Alternative Solution
Last option is to use existing construct (assuming that the only goal of the function is to convert the space-seperate list of files to new-line separated
t4:
echo ${FLIST_01} | tr " " "\n"
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)
As part of my makefile I need to download and build ZLib. However I want to ensure that when I download ZLib, it is correct by comparing the sha256 of the downloaded .tar.gz against the known correct sha256 value. This need to work on multiple platforms.
I have so far something like the following, however the value of ZLIB_SHA256_ACTUAL always seems to be blank when I compare it with ZLIB_SHA256, so my makefile always exits with an error because the checksums are not the same. I am newish to Makefiles, can someone tell me what I am doing wrong please?
ZLIB_VER = 1.2.11
ZLIB_SHA256 = c3e5e9fdd5004dcb542feda5ee4f0ff0744628baf8ed2dd5d66f8ca1197cb1a1
SHA256_CMD = sha256sum
ifeq ($(PLATFORM), OS_MACOSX)
SHA256_CMD = openssl sha256 -r
endif
ifeq ($(PLATFORM), OS_SOLARIS)
SHA256_CMD = digest -a sha256
endif
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
ZLIB_SHA256_ACTUAL = $(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz
ifneq ($(ZLIB_SHA256), $(ZLIB_SHA256_ACTUAL))
$(error zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected="$(ZLIB_SHA256)" actual="$(ZLIB_SHA256_ACTUAL)")
endif
tar xvzf zlib-$(ZLIB_VER).tar.gz
cd zlib-$(ZLIB_VER) && CFLAGS='-fPIC' ./configure --static && make
cp zlib-$(ZLIB_VER)/libz.a .
A makefile consists of two different programming languages in one file. Most of the file uses makefile syntax, that make understands and parses. But the recipes of the rules use shell syntax, which make doesn't try to interpret: it just passes the contents of the recipe to the shell to interpret.
The recipe is the part of the makefile indented with a TAB character, after a target definition. So in your example above, the target definition is libz.a: and all the lines after that which are indented with a TAB, are recipe lines. They are passed to the shell, not run by make.
The recipe is a single block of lines; you cannot intersperse recipe lines with makefile lines. Once make sees the first non-recipe line, that's the end of the recipe and make starts treating the remaining lines as if they were makefile lines.
Let's look at your rule:
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
OK, this is fine: you've created a target libz.a and provided two command lines, which are valid shell commands, in your recipe.
ZLIB_SHA256_ACTUAL = $(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz
OK, now you have problems; this is a make variable assignment, not a shell command, but since you've indented it with a TAB make will not interpret it: make will just pass it to the shell. That's not a valid shell command (in the shell, variable assignments cannot have spaces around the equal sign); this is trying to run a program named literally ZLIB_SHA256_ACTUAL and pass it the arguments = and the expansion of the SHA256_CMD variable. Even if this was recognized as a make assignment it wouldn't do what you want since it would just set the value of the variable to the string openssl sha256 -r zlib-1.2.11.tar.gz: you want to run that command and set the variable to the output.
Then the next lines:
ifneq ($(ZLIB_SHA256), $(ZLIB_SHA256_ACTUAL))
$(error zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected="$(ZLIB_SHA256)" actual="$(ZLIB_SHA256_ACTUAL)")
endif
Again, this is wrong because these are make commands but you've put them into a recipe which means they'll be passed to the shell, but the shell doesn't know anything about them.
However, they never get the chance to be passed to the shell because the one thing make does with a recipe before it sends it off to the shell is expand all make variables and functions. So, when make expands this it runs the error function and that immediately fails and make never has a chance to try to run the recipe.
This is the tricky part of make. Maybe I've just confused you with all of the above stuff.
The short, simple answer is: you have to use shell commands to perform operations in a recipe. You cannot use make commands (like ifeq etc.), and if you want to set variables in a recipe they have to be shell variables, not make variables.
So, you want something like this, which uses shell syntax not make syntax for the variable assignment and test.
EDIT Note your SHA generation command doesn't print just the SHA it also prints the name of the file, so you can't compare them as strings: they'll never be the same. You need to do something fancier; there are many ways to go about it. Here I decided to use case to do the comparison:
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
ZLIB_SHA256_ACTUAL=`$(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz`; \
case "$$ZLIB_SHA256_ACTUAL " in \
($(ZLIB_SHA256)\ *) : ok ;; \
(*) echo zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected=\"$(ZLIB_SHA256)\" actual=\"$$ZLIB_SHA256_ACTUAL\"; \
exit 1 ;; \
esac
tar xvzf zlib-$(ZLIB_VER).tar.gz
cd zlib-$(ZLIB_VER) && CFLAGS='-fPIC' ./configure --static && $(MAKE)
cp zlib-$(ZLIB_VER)/libz.a .
Note that each logical line in the recipe is passed to a new instance of the shell, so if you want to set a shell variable and test its value you have to combine physical lines into one logical line with the backslash/newline syntax.
Also, when running a sub-make in a recipe you should always use the variable $(MAKE) and never use just make.
I have a makefile that defines several rules where the target is a foreach function.
$(foreach var,$(list), $($(var)_stuff) $($(var)_more_stuff)):
#echo Building $# from $^...
$(CC) $(FLAGS) ...
Is there any way to get make to quit when encountering an error without going through the entire list.
One workaround is to "manually" invoke exit on failure.
For example, assume we have a directory called scripts with a number of shell scripts (with filenames that end with .sh) that we want to execute.
Then a variable declaration like this:
LIST_OF_SCRIPTS ?= $(wildcard scripts/*.sh)
will give us a list of those scripts, and a target like this:
run-all-scripts
#$(foreach scriptfile,$(LIST_OF_SCRIPTS),$(scriptfile);)
will run all of those scripts, but as you note, the foreach loop will keep going whether or not one of the scripts returns an error code. Adding a || exit to the command will force the subcommand to exit on error, which Make will then treat as a failure.
E.g.,
run-all-scripts
#$(foreach scriptfile,$(LIST_OF_SCRIPTS),$(scriptfile) || exit;)
will do what you want (I believe).
Specifically, using your pseudo-code example, I think you want something like this:
$(foreach var,$(list), $($(var)_stuff) $($(var)_more_stuff)):
#echo Building $# from $^...
($(CC) $(FLAGS) ...) || exit
(where all I've changed is wrapping the (CC) $(FLAGS) ... bit in parens and appending || exit to make it fail on error).
The foreach is completely evaluated and substituted before any of the rules are executed. So the behaviour of this should be identical to as if you had hardcoded the rule without using the foreach. In other words, it's not directly relevant to the problem.
There are only a few possible explanations for what you're seeing, mostly described in the manual here:
You are running Make with -k or --keep-going
You are running Make with -i or --ignore-errors
Your targets is defined as prerequisites of the special .IGNORE target
Your recipe starts with a -
Your recipe isn't actually returning a non-zero exit status
Not sure about your example, but maybe problem is in ; - look at Makefile : show and execute:
dirs = $(shell ls)
clean:
$(foreach dir,$(dirs),echo $(dir);)
produce:
$ make clean
echo bin; echo install.sh; echo Makefile; echo README.md; echo utils;
So make check exit code only for last command: echo utils.
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)