Could you explain me, why Makefile rule:
clean:
rm -f foo.{bar1,bar2,bar3}
does not result in removing files: foo.bar1 foo.bar2 and foo.bar3?
I believe I saw pattern like that many times in various Makefiles, but I'm currently writing my own Makefile and can't make that rule work correctly (no files are removed).
I'm using:
gnu make 3.81
gnu bash 4.1.5
Bash evals that pattern as I suspect:
$ echo test.{a,b,c}
test.a test.b test.c
Thanks!
UPDATE
Thank to David's hint I found solution for the problem described above.
The gnu make uses the /bin/sh by default and that is why a.{1,2,3} isn't evaluated to a.1 a.2 a.3.
To make 'make' use bash instead of sh add following line to your Makefile:
SHELL=/bin/bash
from now a.{1,2,3} will be considered as a.1 a.2 a.3
Is there a file named clean in the directory? If so, make will consider that target up to date and won't run the corresponding command. To fix that, add this line to your makefile:
.PHONY: clean
If when you run make clean you get the output
make: `clean' is up to date.
then that's probably your problem.
It's because the shell isn't being invoked here, but rather rm is directly. Since the shell does the {} substitution, what rm 'sees' is the raw foo.{bar1,bar2,bar3} string. As there's no such file, nothing happens.
You should use one of GNUmake's string macros to have it perform the expansion for you.
Related
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)
I have a situation where I need a small program to determine the prerequisites of a file by examining it. Concretely:
%.bar: $(shell python get_preqs.py %.foo) # % in shell is not substituted
python gen_bar.py $^ $#
This doesn't work however, as the % in the shell command doesn't get substituted in by make. Is there any way to achieve this?
You can use the .SECONDEXPANSION feature of GNU make:
.SECONDEXPANSION:
%.bar: $$(shell python get_prereqs.py $$*.foo)
python gen_bar.py $^ $#
Another option is to use a build tool more suited to dynamic targets. For example, I've written a Gnu Make-like tool incorporating some of the concepts from DJB's ReDo, called GoodMake. Your equivalent makefile would just be:
#? *.bar
preqs=$(python get_preqs.py ${1%.bar}.foo)
$0 $preqs
python gen_bar.py $preqs $1
As part of my makefile I need to download and build ZLib. However I want to ensure that when I download ZLib, it is correct by comparing the sha256 of the downloaded .tar.gz against the known correct sha256 value. This need to work on multiple platforms.
I have so far something like the following, however the value of ZLIB_SHA256_ACTUAL always seems to be blank when I compare it with ZLIB_SHA256, so my makefile always exits with an error because the checksums are not the same. I am newish to Makefiles, can someone tell me what I am doing wrong please?
ZLIB_VER = 1.2.11
ZLIB_SHA256 = c3e5e9fdd5004dcb542feda5ee4f0ff0744628baf8ed2dd5d66f8ca1197cb1a1
SHA256_CMD = sha256sum
ifeq ($(PLATFORM), OS_MACOSX)
SHA256_CMD = openssl sha256 -r
endif
ifeq ($(PLATFORM), OS_SOLARIS)
SHA256_CMD = digest -a sha256
endif
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
ZLIB_SHA256_ACTUAL = $(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz
ifneq ($(ZLIB_SHA256), $(ZLIB_SHA256_ACTUAL))
$(error zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected="$(ZLIB_SHA256)" actual="$(ZLIB_SHA256_ACTUAL)")
endif
tar xvzf zlib-$(ZLIB_VER).tar.gz
cd zlib-$(ZLIB_VER) && CFLAGS='-fPIC' ./configure --static && make
cp zlib-$(ZLIB_VER)/libz.a .
A makefile consists of two different programming languages in one file. Most of the file uses makefile syntax, that make understands and parses. But the recipes of the rules use shell syntax, which make doesn't try to interpret: it just passes the contents of the recipe to the shell to interpret.
The recipe is the part of the makefile indented with a TAB character, after a target definition. So in your example above, the target definition is libz.a: and all the lines after that which are indented with a TAB, are recipe lines. They are passed to the shell, not run by make.
The recipe is a single block of lines; you cannot intersperse recipe lines with makefile lines. Once make sees the first non-recipe line, that's the end of the recipe and make starts treating the remaining lines as if they were makefile lines.
Let's look at your rule:
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
OK, this is fine: you've created a target libz.a and provided two command lines, which are valid shell commands, in your recipe.
ZLIB_SHA256_ACTUAL = $(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz
OK, now you have problems; this is a make variable assignment, not a shell command, but since you've indented it with a TAB make will not interpret it: make will just pass it to the shell. That's not a valid shell command (in the shell, variable assignments cannot have spaces around the equal sign); this is trying to run a program named literally ZLIB_SHA256_ACTUAL and pass it the arguments = and the expansion of the SHA256_CMD variable. Even if this was recognized as a make assignment it wouldn't do what you want since it would just set the value of the variable to the string openssl sha256 -r zlib-1.2.11.tar.gz: you want to run that command and set the variable to the output.
Then the next lines:
ifneq ($(ZLIB_SHA256), $(ZLIB_SHA256_ACTUAL))
$(error zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected="$(ZLIB_SHA256)" actual="$(ZLIB_SHA256_ACTUAL)")
endif
Again, this is wrong because these are make commands but you've put them into a recipe which means they'll be passed to the shell, but the shell doesn't know anything about them.
However, they never get the chance to be passed to the shell because the one thing make does with a recipe before it sends it off to the shell is expand all make variables and functions. So, when make expands this it runs the error function and that immediately fails and make never has a chance to try to run the recipe.
This is the tricky part of make. Maybe I've just confused you with all of the above stuff.
The short, simple answer is: you have to use shell commands to perform operations in a recipe. You cannot use make commands (like ifeq etc.), and if you want to set variables in a recipe they have to be shell variables, not make variables.
So, you want something like this, which uses shell syntax not make syntax for the variable assignment and test.
EDIT Note your SHA generation command doesn't print just the SHA it also prints the name of the file, so you can't compare them as strings: they'll never be the same. You need to do something fancier; there are many ways to go about it. Here I decided to use case to do the comparison:
libz.a:
-rm -rf zlib-$(ZLIB_VER)
curl -O -L http://zlib.net/zlib-$(ZLIB_VER).tar.gz
ZLIB_SHA256_ACTUAL=`$(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz`; \
case "$$ZLIB_SHA256_ACTUAL " in \
($(ZLIB_SHA256)\ *) : ok ;; \
(*) echo zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected=\"$(ZLIB_SHA256)\" actual=\"$$ZLIB_SHA256_ACTUAL\"; \
exit 1 ;; \
esac
tar xvzf zlib-$(ZLIB_VER).tar.gz
cd zlib-$(ZLIB_VER) && CFLAGS='-fPIC' ./configure --static && $(MAKE)
cp zlib-$(ZLIB_VER)/libz.a .
Note that each logical line in the recipe is passed to a new instance of the shell, so if you want to set a shell variable and test its value you have to combine physical lines into one logical line with the backslash/newline syntax.
Also, when running a sub-make in a recipe you should always use the variable $(MAKE) and never use just make.
I have a variable containing list of files separated with string _NEWLINE_. I need to output that variable into a file so that each file is in a separate line. The trick is that it needs to works on FreeBSD and Solaris.
This is what I am trying now:
echo "lib/alarms-1.2/priv/snmp_conf/agent.conf: lib/alarms/priv/snmp_conf/agent.conf_NEWLINE_lib/alarms-1.2/priv/snmp_conf/agent.conf.src: lib/alarms/priv/snmp_conf/agent.conf.src_NEWLINE_lib/alarms-1.2/priv/snmp_conf/community.conf: lib/alarms/priv/snmp_conf/community.conf" | sed 's|_NEWLINE_|\'$'\n|g'
This works on FreeBSD and in shell on Solaris. But when run in GNUmakefile on Solaris I am getting this (notice $ at the end of each line):
lib/alarms-1.2/priv/snmp_conf/agent.conf: lib/alarms/priv/snmp_conf/agent.conf$
lib/alarms-1.2/priv/snmp_conf/agent.conf.src: lib/alarms/priv/snmp_conf/agent.conf.src$
lib/alarms-1.2/priv/snmp_conf/community.conf: lib/alarms/priv/snmp_conf/community.conf$
If I remove \'$' from sed then it works on Solaris but doesn't on FreeBSD. Maybe there is a way of telling which version to use depending on which system the makefile is executed?
EDIT:
Thanks to the solution proposed by bobbogo I created an exemplary makefile that provides the desired outcome and seems to be working on both FreeBSD and Solaris:
one-line := lib/alarms-1.2/priv/snmp_conf/agent.conf: lib/alarms/priv/snmp_conf/agent.conf_NEWLINE_lib/alarms-1.2/\
priv/snmp_conf/agent.conf.src: lib/alarms/priv/snmp_conf/agent.conf.src_NEWLINE_lib/alarms-1.2/priv/snmp_conf/comm\
unity.conf: lib/alarms/priv/snmp_conf/community.conf
many-lines := { echo '$(subst _NEWLINE_,' && echo ',${one-line})'; }
.PHONY: all
all:
$(shell $(many-lines) > test.txt)
If this is GNU make, then do it all in make.
one-line := lib/alarms-1.2/priv/snmp_conf/agent.conf: lib/alarms/priv/snmp_conf/agent.conf_NEWLINE_lib/alarms-1.2/priv/snmp_conf/agent.conf.src: lib/alarms/priv/snmp_conf/agent.conf.src_NEWLINE_lib/alarms-1.2/priv/snmp_conf/community.conf: lib/alarms/priv/snmp_conf/community.conf
define \n
endef
many-lines := $(subst _NEWLINE_,${\n},${one-line})
Now ${many-lines} has just what you want. Annoyingly, it's quite hard to use in shell lines. If you do this:
tgt:
echo '${many-lines}'
make will invoke a separate shell for each line. The first shell invocation will get an un-paired ' and exit with an error.
.ONESHELL:
tgt:
echo '${many-lines}'
will work in an invasive sort of way. The proper fix is to ensure each line of ${many-lines} has valid sh syntax. Some mouthfull like:
echolines = $(subst ${\n},'${\n}echo ',echo '${many-lines}')
.PHONY: aa
aa:
$(call echolines,${many-lines})
Sheesh.
Tried many different solutions, including defining \n as mentioned in this answer: Add a newline in Makefile 'foreach' loop
The real problem is inconsistent implementation of the echo command across platforms, and the fact that by default make invokes shell commands using sh, which itself is quite inflexible.
I found a better way thanks to this answer: "echo -e" when called directly and when called via a shell-script
The better way is to use printf instead of echo
Construct the string with \n instead of _NEWLINE_ to separate parts that go into separate lines in the output file:
some_string = lib/alarms-1.2/priv/snmp_conf/target_params.conf: lib/alarms/priv/snmp_conf/target_params.conf\nlib/alarms-1.2/priv/snmp_conf/community.conf: lib/alarms/priv/snmp_conf/community.conf\n
and then in the makefile print it simply as this:
#printf "$(some_string)" >> $(some_file)
Works on both, FreeBSD and Solaris.
Disclaimer: I have no experience with Solaris or FreeBSD... here goes anyway.
In make, you can use $(patsubst pattern,replacement,text) to substitute a pattern.
try this...
FILENAMES := "lib/alarms-1.2/priv/snmp_conf/agent.conf: lib/alarms/priv/snmp_conf/agent.conf_NEWLINE_lib/alarms-1.2/priv/snmp_conf/agent.conf.src: lib/alarms/priv/snmp_conf/agent.conf.src_NEWLINE_lib/alarms-1.2/priv/snmp_conf/community.conf: lib/alarms/priv/snmp_conf/community.conf"
.PHONY: all
all:
#echo $(patsubst _NEWLINE_,${\n},$(FILENAMES))
As an alternative, I think your first approach will work, if you just double the $ to "escape" it:
sed 's|_NEWLINE_|\'$$'\n|g'
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)