I am looking for a clean way to change a few target declarations I created in a makefile into a more functional type declaration in which I am able to pass variables and the output will remain consistent.
For example:
default: clean run1 run2
run1:
for entity in $(FLIST_01); do \
echo $entity ; \
done
run2:
for entity in $(FLIST_02); do \
echo $entity ; \
done
Ideally, I would like to remove the repetitive run target declarations and only have 1.
FLIST_01 = my_flist.txt
FLIST_02 = other.txt
default: clean run
run:
$(run_func $(FLIST_01))
$(run_func $(FLIST_02))
How do I create a custom function in make to do what run_func is supposed to be doing (the for loop reading of the file list variable passed to it?
UPDATE:
My attempt so far as been this:
run:
runfunc() { \
echo "test" \
for entity in $1; do \
echo $(entity); \
done \
}
runfunc $(FLIST_01)
runfunc $(FLIST_02)
But I get a syntax error on the do line: syntax error near unexpected token `do'
First off, unless you specify .ONESHELL, the commands in a target will be run in separate subshells. So your runfunc() declaration will run in a shell which declares the function, then exits. The next statement will run in a new shell which will know nothing about the function which was declared and then basically forgotten.
Secondly,
echo "test" \
for entity in $1;
will be expanded by make into
echo "test" for entity in ;
which obviously contains multiple errors. You mean
echo "test"; \
for entity in $$1;
to properly pass the dollar sign through to the shell. But on the whole, I would say your approach is flawed. You can refactor this to a make function like you originally hoped;
define run
for entity in $(1); do \
echo $$entity; \
done
endef
Now you can call this like
run:
$(call run,$(FLIST_01))
$(call run,$(FLIST_02))
But this particular loop can quite easily be replaced with a single shell statement.
run:
printf '%s\n' $(FLIST_01)
printf '%s\n' $(FLIST_02)
Another twopenn'orth.
First off, we want to expand $FLIST_01 for target run1, and FLIST_02 for target run2.
A clean way might be:
FLIST_run1 := a b c
FLIST_run2 := 1 2 3
.PHONY: run1 run2
run1 run2:
for entity in ${FLIST_$#}; do echo $entity; done
For a slightly more obscure solution we notice that what you are trying to do can be described with make's noddy pattern matching:
.PHONY: run1 run2
run%: run1 run2:
for entity in ${FLIST_$*}; do echo $entity; done
Here we use a static pattern rule.
In the recipe, $* expands to whatever matched the % in the pattern.
This solution looks quite clean to my jaundiced eye.
I urge you to seek out idiomatic make rather than idiomatic shell.
Your makefiles will nearly always thank you for it.
In the first instance, keep your two run targets rather than coalesce them into the same recipe.
.PHONY: run1 run2
run%: run1 run2:
for entity in ${FLIST_$*}; do echo $entity; done
.PHONY: run
run: run1
run: run2
run: ; echo $# Success
Here, when you make run for instance, make first carries out the recipe for run1, then the recipe for run2, and finally the recipe for run.
Is there any advantage to this? Sure. If you run make -j2 run then the commands for run1 and run2 will be run in parallel. The whole point of make really.
The final issue is to get rid of the shell loop, and to replace it with separate commands. That way make will check the exit code of the echo for you, and even run them in parallel if that is suitable. Sure, not helpful in this noddy example, but they could be compile commands for instance.
Usually, it is not easier to maintain shell functions inside Makefile. Main reasons:
Each shell '$' need to be escaped to '$$'
The $(...) construct has different meaning in shell vs Make.
Each line in the Makefile is executing by new shell instance, meaning that variables assignment, functions definitions, etc, are not shared between lines.
Consider few options: write shell helper script (recommended), use make user-defined functions or inline the function into the Make. For the specific case of the "echo", it might be possible to use one of the other approaches, see "Alternative Solution"
From personal experience, for any nontrivial functions, better to use shell helper script. They are much easier to develop and test.
Using Shell Helper Script
Write a small helper script "print-list.sh"
#! /bin/bash
echo "test" \
for entity ; do
echo $entity
done
And then invoke it from the target
FLIST_01 = a.txt b.txt c.txt
t1:
print-list.sh ${FLIST_01)
Inlining the shell function
As indicated above, embedding the shell function into the Makefile requires matching make rules about quoting, escapes, etc. It also require lot of effort debugging the script, as error messages are very cryptic in this setup.
Note that RUNFUNC must be included before every action lines that uses the command.
RUNFUNC = runfunc() { \
echo "test" ; \
for entity in $$*; do \
echo $$entity; \
done \
} ;
FLIST_01 = a.txt b.txt c.txt
t2:
$(RUNFUNC) runfunc ${FLIST_01)
Note that the function can be written as one liner, or using the 'define'/'endef' to simplify end-of-line escapes.
Using make user-defined functions
It is possible to create simple functions using make. This option require experience and time.
UFUNC = \
echo "test" ; \
for entity in $1; do \
echo $$entity ; \
done
FLIST_01 = a.txt b.txt c.txt
t3:
$(call UFUNC, ${FLIST_01))
Alternative Solution
Last option is to use existing construct (assuming that the only goal of the function is to convert the space-seperate list of files to new-line separated
t4:
echo ${FLIST_01} | tr " " "\n"
Related
Is is possible to use shell arguments inside a makefile function?
e.g.
mywords:=hello there neighbour
myrecipe:
for ((i = 0 ; i < 3 ; i++)); do \
this_word=$(word $${i}, $(mywords)); \
done
Except myrecipe gives the error non-numeric first argument to 'word' function: '${i}'. Stop
Remember that make first expands the entire recipe (yes, that's all of the recipe lines) before ever doing anything with the shell.
Only after this full expansion does make then look for individual recipe lines, running each one in turn.
Thus make behaves sensibly when you have a macro that expands to more than one line.
It also hints that you should use make functionality wherever possible.
mywords := hello there neighbour
define mkcommand
echo $1
something $1
endef
myrecipe:
$(foreach _,${mywords},$(call mkcommand,$_))
Now, when you ask make to build myrecipe,
make sees something like:
myrecipe:
echo hello
something hello
echo there
something there
⋮
IMHO it's nearly always a mistake if you find yourself writing shell loops in a makefile.
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)
Usually, I use foreach call in a makefile. Notwithstanding, I wanted to try different approaches.
For that, I have created the code bellow.
I am using GNU Make 3.81
I have replaced the eval by info to understand what is happening.
I cannot avoid both the call and the eval expansion to keep the $file
I mean, when I open the file I am getting:
"for file in *.v; do echo ; echo ; done; > trial.sh;"
define create_area
$(1): $(2)
mkdir -p $$(#D); \
cp -nLr -v $(2)/* $$(#D); \
cd $$(#D); \
for file in *.v; do echo $$(file); \
done; \
" > trial.sh;
$(foreach a,$(A1) $(A2),\
$(foreach b,$(B1),\
$(eval $(call create_area,$(a),$(b)))))
Thank you in advance,
You're almost there.
If you run the loop in a shell, you must precede the variable name with '$' so that it will be expanded.
If you put that command in a makefile recipe, to prevent Make from expanding the variable name too soon (when the variable has not yet been given a value) you must precede ("escape") that '$' with another '$'.
If you put that recipe in a definition which you will expand with call, you must double each of them:
for file in *.v; do echo $$$$file; done
My original use case rather complicated but could be distilled to the following:
a:
export AAA=true
b:
echo ${AAA}
ifdef AAA
echo "AAA mode is on."
else
echo "AAA mode is off."
endif
What I want to achieve is to have conditional b target depends on whether a was invoked or not.
However behaviour is not intuitive and whilst AAA=true make b does work make a b does not work as intended (I prefer the latter).
Any helpful advises will be appreciated!
If you're willing to restrict yourself to GNU make you can use the eval function. I don't generally think this is a good idea and I agree with Beta that this may be an XY problem, but:
AAA = false
a:
$(eval AAA = true)
b:
echo ${AAA}
Note you definitely cannot use make preprocessor statements like ifeq or ifdef: those are evaluated as the makefile is parsed, even ones that are contained within a recipe, which obviously happens well before any recipe is invoked which might set AAA etc.
ETA:
Here's an example of the above makefile in action to show it works:
$ make b
false
$ make b a
false
$ make a b
true
As mentioned in comments elsewhere, the value of AAA is reset in the recipe for b, IFF the recipe for a has already been invoked. That's how I interpreted your request What I want to achieve is to have conditional b target depends on whether a was invoked or not, but maybe you had a different behavior in mind.
Ok, I was trying to play with evals (didn't work) and file markers (not ideal because I have hierarchy of makefiles and some of them invoked twice hence $(shell rm -f .target-marker) invoked multiple times) and I think for my use case the best is to explore current goals invoked:
a:
#market target
b:
#if [[ "${MAKECMDGOALS}" = a* ]]; then \
echo "AAA mode is on"; \
else \
echo "AAA mode is off"; \
fi;
MAKECMDGOALS contains all goals and I simply can check if certain goal was in list or not.
You can't expose one shell's environment to another, but you can use a flag file.
# Kludge: Force removal when make starts
$(shell rm -f .done-a)
a:
: things
touch .done-a
b:
if [ -e .done-a ]; then : stuff ...; \
else : other stuff ...; \
fi
Two variations on your theme with a single definitions. As #tripleee said you can't pass externs, but you can have a common definition in the makefile. I also try to avoid empty shell variables where possible. In this case a straight up true/false value can be used.
#last definition "wins"
AAA=false
AAA=true
a:
export AAA=$(AAA)
b:
export AAA=$(AAA)
echo $${AAA}; #note double dollar within shell commands
$$AAA && echo "AAA mode is on." || echo "AAA mode is off."
nested_c:
$(MAKE) AAA=$(AAA) c
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)