In my Makefile I am trying to copy a list of files from location1 to location2, then to location2 to location3. I got the following strange behavior:
FILES_LIST=dir1/file1 dir2/file2 dir3/file3 ........
mytarget:
for file in $(FILES_LIST) ; do \
#this works
cp -vf location1/$$file location2/$(shell $$file##*/) ; \
#this does not work
cp -vf location2/$(shell $$(file)##*/) location3/ ; \
done
I am using "$(shell $$(file)##/)" to strip out "dir1/" part of each item in FILES_LIST.
The first cp works (from location1 to 2), however, the send does not, build log shows "$(shell $$(file)##/)" is evaluated to empty.
I am using GNU Make 3.81
The problem is $$(file). That's not a variable evaluation. That's a Command Substitution. You meant $${file} (well not quite but we'll get to that).
There is also absolutely no reason to be using $(shell) here at all as you are already in a shell context when those lines run.
Not to mention that those $(shell) calls aren't doing anything even remotely like what you want (they aren't operating at the right time to do that).
You want this:
FILES_LIST=dir1/file1 dir2/file2 dir3/file3 ........
mytarget:
for file in $(FILES_LIST) ; do \
#this works
cp -vf location1/$$file location2/$${file##*/} ; \
#this does not work
cp -vf location2/$${file)##*/} location3/ ; \
done
Your $file variable is a shell variable not a make one. The call to $(shell) does not see it. You are effectively running $(shell $file##*/) which runs the $file##*/ command through the shell. That shell has no $file variable so that becomes ##*/ which is a comment and the whole thing returns nothing. (Actually I think the comment may be stripped first but that doesn't change anything.)
Use $(notdir $file) command.
notdir will strip out the directories and return the filename. For example, $(notdir dir1/file.h) will return file.h
Reference link for more detailed info:
https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html
Related
I am currently using a makefile for cocotb, similar to this.
That consists of a bunch of variable definitions followed by and
include $(shell cocotb-config --makefiles)/Makefile.sim
Where the final commands will be executed.
I run it in Visual studio code, the simulator output message format is (%file,%line|%column). I want them to be shown as (%file:%line:%col). I can easily do it piping the output to sed.
make MODULE=x TESTCASE=y | sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g'
I was expecting to be able to change the make file so that the output will be edited before shown
Something like
...
.DEFAULT:
make -C $PWD -f $(shell cocotb-config --makefiles)/Makefile.sim \
(all variables/environment) (rule) \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g
Where "magic rule" would apply the specified rule and pipe its output to sed.
How can I achieve this?
Thank you.
Given your example, it seems like the .DEFAULT target would do what you want:
.DEFAULT:
$(MAKE) -f $$(cocotb-config --makefiles)/Makefile.sim $# \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g
Edit (from the OP)
The default works when the rule is given, e.g. if I invoke make sim instead of make. For this I can create a default-delegate rule.
More importantly, I have to pass all the variables to the sub make, that can be achieved by using export by itself.
Then, replacing the include command but the following snippets, works for the more common cases.
export
default-delegate: sim
.DEFAULT:
$(MAKE) -f $$(cocotb-config --makefiles)/Makefile.sim $# \
| sed -e 's/,\([0-9]\+\)|\([0-9]\+\)):/:\1:\2):/g'
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)
Here's my code:
for num in {0001..1000}; do
cd ${num}_1000_solar
sed -i '827s/.*/ if(BigUni==1) fp910=fopen("biguni_${num}.dat","w");/' binary.c
gcc singl.c binary.c -lm
cd ..
done
sed command writes '{num}' into a file instead of putting there an appropriate value from the loop. How can I replace a line with a string + some variable?
Cheers!
If ${num} is to be interpreted by the shell, it must be in double quotes (or outside quotes), not single quotes.
for num in {0001..1000}
do
(
cd ${num}_1000_solar
sed -i '827s/.*/ if (BigUni==1) fp910=fopen("biguni_'"${num}"'.dat","w");/' binary.c
gcc singl.c binary.c -lm
)
done
Having 1000 programs where you change the file name in the source code and recompile (to a.out each time) is an abuse of C. You should pass the file name in as an argument to the program. In other words, the exercise is only necessary because the setup is deeply flawed.
Also, as a general rule, I avoid cd subdir followed later by cd .. in scripts when it's feasible. By running a sub-shell (the ( and ) in the revised script), the calling shell process is unaffected by any changes of directory. Ultimately, it's more reliable.
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)
This question already has answers here:
How to use shell commands in Makefile
(2 answers)
Closed 9 months ago.
I would like to use a loop to find some files and rename them:
for i in `find $# -name *_cu.*`;do mv $i "$(echo $i|sed s/_cu//)"
done
This works in the shell. But how can I do this in a makefile recipe?
There are two main things you need to know when putting non-trivial shell fragments into make recipes:
Commands in the recipe are (of course!) executed one at a time, where command means "tab-prefixed line in the recipe", possibly spread over several makefile lines with backslashes.
So your shell fragment has to be written all on one (possibly backslashed) line. Moreover it's effectively presented to the shell as a single line (the backslashed-newlines are not plain newlines so are not used as command terminators by the shell), so must be syntactically correct as such.
Both shell variables and make variables are introduced by dollar signs ($#, $i), so you need to hide your shell variables from make by writing them as $$i. (More precisely, any dollar sign you want to be seen by the shell must be escaped from make by writing it as $$.)
Normally in a shell script you would write separate commands on separate lines, but here you effectively only get a single line so must separate the individual shell commands with semicolons instead. Putting all this together for your example produces:
foo: bar
for i in `find $# -name *_cu.*`; do mv $$i "$$(echo $$i|sed s/_cu//)"; done
or equivalently:
foo: bar
for i in `find $# -name *_cu.*`; do \
mv $$i "$$(echo $$i|sed s/_cu//)"; \
done
Notice that the latter, even though it's laid out readably on several lines, requires the same careful use of semicolons to keep the shell happy.
I found this useful, trying to use for loops to build multiple files:
PROGRAMS = foo bar other
.PHONY all
all: $(PROGRAMS)
$(PROGRAMS):
gcc -o $# $#.c
It will compile foo.c, bar.c, other.c into foor bar other executables
I spend good time on this and finally had it working. I had an easy solution using the global variable in makefile available for all targets, however I don`t want that so this is how I did it.
target:
$(eval test_cont=$(shell sh -c "docker ps | grep test" | awk '{print $$1}'))
for container in $(test_cont);do \
docker cp ssh/id_rsa.pub $${container}:/root/.ssh/authorized_keys; \
docker exec -it $${container} chown root.root /root/.ssh/authorized_keys; \
done