Here's the code snippet from my Makefile:
%/foo: %/bar.yaml
$(BINARY) generate -g go \
--package-name {COOL_VALUE}
# COOL_VALUE should be the value for a unique `x-cool-value` key from `bar.yaml`.
E.g., here's bar.yaml:
abc:
xyz: Hi StackOverflow!
x-cool-value: v1
and I'd like to extract v1 value.
E.g.,
make foo1/foo2/foo
> ./binary generate -g go \
--package-name v1
# since foo1/foo2/foo/bar.yaml had a `v1` value for `abc.x-cool-value` key.
You can't do things like that using makefile syntax (at least it's not worth it). Figure out how to do it using a shell command, then put that shell command in your makefile recipe.
For example:
$ sed -n 's/^ *x-cool-value: *//p' foo1/foo2/foo/bar.yaml
v1
So, in your makefile:
%/foo: %/bar.yaml
$(BINARY) generate -g go \
--package-name $$(sed -n 's/^ *x-cool-value: *//p' $<)
Or you can try to find some command-line tool that parses YAML and lets you query things; there probably is one.
Related
I am currently using a makefile for cocotb, similar to this.
That consists of a bunch of variable definitions followed by and
include $(shell cocotb-config --makefiles)/Makefile.sim
Where the final commands will be executed.
I run it in Visual studio code, the simulator output message format is (%file,%line|%column). I want them to be shown as (%file:%line:%col). I can easily do it piping the output to sed.
make MODULE=x TESTCASE=y | sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g'
I was expecting to be able to change the make file so that the output will be edited before shown
Something like
...
.DEFAULT:
make -C $PWD -f $(shell cocotb-config --makefiles)/Makefile.sim \
(all variables/environment) (rule) \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g
Where "magic rule" would apply the specified rule and pipe its output to sed.
How can I achieve this?
Thank you.
Given your example, it seems like the .DEFAULT target would do what you want:
.DEFAULT:
$(MAKE) -f $$(cocotb-config --makefiles)/Makefile.sim $# \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g
Edit (from the OP)
The default works when the rule is given, e.g. if I invoke make sim instead of make. For this I can create a default-delegate rule.
More importantly, I have to pass all the variables to the sub make, that can be achieved by using export by itself.
Then, replacing the include command but the following snippets, works for the more common cases.
export
default-delegate: sim
.DEFAULT:
$(MAKE) -f $$(cocotb-config --makefiles)/Makefile.sim $# \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g'
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)
Here's a snippet from a Makefile which in my environment is recursive and it appears this piece of code has a n issue and I don't understand why foreach doesn't evaluate. The variables TOOLVERSION and TOOLSDIR are assigned values based on running a Ruby script. These provide the include paths for the C_FLAGS. The target test2 produces the right result where every element of C_FLAGS seperated by space appears on a new line which is the desired result. However test1 does not evaluate TOOLVERSION and TOOLSDIR so produces garbage. If I use $(shell getenvval.rb version) for assigning TOOLVERSION and TOOLSDIR the foreach work but the value of running the script is incorrect. I thought this was because something defined in the Makefile environment doesn't get to the shell so I used export but it didn't make a difference.
So the question comes down to why does the following foreach loop not work:
#$(foreach flag, $(C_FLAGS), `echo $(flag) >> $(FILE_C1_LIST)`)
while this works:
#echo $(C_FLAGS) >> $(FILE_C2_LIST)
Appreciate any help in understanding the evaluation.
Snippet from Makefile:
export
TOOLVERSION:= `getenvval.rb version`
TOOLSDIR:= `getenvval.rb directory`
FILE_C1_LIST := test_c1.f
FILE_C2_LIST := test_c2.f
C_FILES =\
./a.c \
./b.c
C_FLAGS := \
-I$(TOOLSDIR)/$(TOOLVERSION)/tools/include \
-I./aa/include \
-I./bb/editline \
-g -DDEBUG -DPLISIM -DINCLUDE_EDITLINE -DSYS_UNIX
$(FILE_C1_LIST): $(C_FILES)
rm -f $(FILE_C1_LIST)
touch $(FILE_C1_LIST)
#(echo $(C_FLAGS) )
#$(foreach flag, $(C_FLAGS), `echo $(flag) >> $(FILE_C1_LIST)`)
#$(foreach file, $(C_FILES), `echo $(file) >> $(FILE_C1_LIST)` )
$(FILE_C2_LIST): $(C_FILES)
#rm -f $(FILE_C2_LIST)
#touch $(FILE_C2_LIST)
#echo $(C_FLAGS) >> $(FILE_C2_LIST)
#$(foreach file, $(C_FILES), `echo $(file) >> $(FILE_C2_LIST)` )
test1: $(FILE_C1_LIST)
test2: $(FILE_C2_LIST)
Your expectations are odd. You appear to try to use shell syntax in constructs which are not going to be evaluated by a shell; and even if they were, the expressions would not be syntactically valid.
Without delving too much into the details, try this instead.
printf '%s\n' $(C_FLAGS) >> $(FILE_C1_LIST)
In some more detail, your loop would create the text
`echo one >>file` `echo two >>file` `echo three >>file`
where the backticks imply that the output from echo should be the text of a command which you want the shell to execute; but of course, because of the redirection, there is no output to standard output from any of these commands. (The superficial problem of having all the commands on a single line could be worked around with a semicolon after each.)
As a further aside, there is no need to rm or touch a file you are going to overwrite. Just write it. Your recipes can be substantially simplified.
$(FILE_C1_LIST): $(C_FILES)
printf '%s\n' $(C_FLAGS) $(C_FILES) >$#
$(FILE_C2_LIST): $(C_FILES)
echo $(C_FLAGS) >$#
printf '%s\n' $(C_FILES) >>$#
The parentheses you had around the first echo would needlessly run that in a separate subshell. I'm guessing you had that purely to see what you were doing; removing the # on your recipes does that much better. You can run make silently witth make -s once you have everything debugged and sorted.
I am trying to write a makefile that does something like the following:
%-foo-(k).out : %-foo-(k-1).out
# do something, e.g.
cat $< $#
i.e. there are files with arbitrary stems, then -foo-, then an integer, followed by .out. Each file depends on the one with the same name, with integer one smaller.
For instance, if the file blah/bleh-foo-1.out exists, then
make blah/bleh-foo-2.out
would work.
I could do this with multiple stems if there were such a thing... what's another way to do this sort of thing in (gnu) make?
There is no easy way to do something like this. You basically have two options: you can use auto-generated makefiles, or you can use $(eval ...). To me auto-generated makefiles are easier, so here's a solution:
SOURCELIST = blah/bleh-foo-1.out
all:
-include generated.mk
generated.mk: Makefile
for f in $(SOURCELIST); do \
n=`echo "$$f" | sed -n 's/.*-\([0-9]*\)\.out$/\1/p'`; \
echo "$${f%-foo-[0-9]*.out}-foo-`expr $$n + 1`.out: $$f ; cat $$< > $$#"; \
done > $#
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)