I want to use the bash timing variables in my makefile
for example in my terminal I can do this and it works
MY_TIME=$SECONDS
echo $MY_TIME
but when I write this on my makefile it does not work
how can I use these two lines in my make file?
this is what I'm doing
.PHONY: myProg
myProg:
MY_TIME=$SECONDS
echo $MY_TIME
After Etan Reisner' answer
This is what I have now
.PHONY: myProg
myProg:
MY_TIME= date; echo $MY_TIME
but the result of my echo is an empty line, it does not look like it is storing the date
the dollar sign ($MY_TIME) refers to make variables, which are not the same as bash variables.
To access a bash variable you must escape the dollar using the double dollar notation ($$MY_TIME).
.PHONY: myProg
myProg:
MY_TIME=$$SECONDS ; echo $$MY_TIME
As already mentioned in Etan answer you can't split the code into multiple lines (unless you are using the backslash) since each command executes in a different subshell, making variables inaccessible to other lines.
In the following example the value of SECONDS will be always 0, since it get reset by the spawn of the shell for the second line.
.PHONY: myProg
myProg: # WRONG
MY_TIME=$$SECONDS
echo $$MY_TIME
By default make uses /bin/sh as the shell which executes recipe lines.
Presumably /bin/sh doesn't support the SECONDS variable.
You can tell make to use a different shell by assigning a value to the SHELL variable (i.e. SHELL := /bin/bash).
Doing that will make SECONDS available but will still not allow you to carry a variable value between recipe lines as each recipe line is run in its own shell.
So to do what you want you would need to write both of those lines on one line or continue the line over the newline.
.PHONY: myProg
myProg:
MY_TIME=$SECONDS; echo $MY_TIME
or
.PHONY: myProg
myProg:
MY_TIME=$SECONDS; \
echo $MY_TIME
That being said you would almost certainly be better off not doing this and instead using something like date invoked at the start/end of the recipe or time invoked on the command to be timed directly instead.
.PHONY: myProg
myProg:
date
# Do something
date
or
.PHONY: myProg
myProg:
time some_command
PROGRAM_NAME = myLittleProgram
...
$(PROGRAM_NAME) : myLittleProgram.o
I know the above works, as it is in my own makefile (program names and object names changed to protect the innocent).
"Variable references can be used in any context: targets, dependencies, commands, most directives, and new variable values. Here is an example of a common case, where a variable holds the names of all the object files in a program:"
objects = program.o foo.o utils.o
program : $(objects)
cc -o program $(objects)
$(objects) : defs.h
http://web.mit.edu/gnu/doc/html/make_6.html
Related
Say I have the following function defined in my .bashrc:
foo (){
touch $1
}
I want to call this function in a makefile like so:
%.png : %.tex
bash -i -c foo $<
But this complains about a missing operand. (The error is "touch: missing file operand", so it is calling touch...). I've seen some solutions to similar issues, but they aren't applicable since I need the automatic variable $< (or rather, its value) to be passed as an argument to the shell script.
I know I could work around this problem (copy the function wholesale into the makefile, for example) but I'm curious to see how I could solve my problem within the makefile.
Your problem is probably because you don't have enough quoting. The shell -c option takes its script as a single argument.
Consider:
$ bash -c echo hi
<prints nothing>
Versus:
$ bash -c 'echo hi'
hi
I have a makefile that executes some shell command and I want to store the output to a global variable:
GLOBVAR = a
all:
GLOBVAR=$(shell echo 'X')
$(info $(GLOBVAR))
GLOBVAR is empty. What am I doing wrong?
You are mixing up make and shell variables. In GLOBVAR=$(shell echo 'X') it is a shell variable that you assign, while in $(info $(GLOBVAR)) it is a make variable that you expand.
Try this, instead:
GLOBVAR = $(shell echo 'X')
all:
$(info $(GLOBVAR))
But there are several other issues with your Makefile that you should probably consider.
Using $(shell...) in recipes is not recommended because recipes are already shell scripts. So, if you want to assign a shell variable in a recipe, just:
all:
GLOBVAR="$$(echo 'X')"
Note the $$ to escape the expansion that make performs before passing the recipes to the shell.
The different lines of the recipe are executed in different shells. So, if you want to use in a line a shell variable that was assigned in a previous line you must join them:
all:
GLOBVAR="$$(echo 'X')"; echo $$GLOBVAR
(same remark as before about $$). You can use line continuation if you prefer:
all:
GLOBVAR="$$(echo 'X')"; \
echo $$GLOBVAR
And finally, if you want to assign make variables in recipes you can, with the eval make function, but I strongly discourage you to do so until you perfectly understand when make does what:
$ cat Makefile
.PHONY: all lla
all:
$(eval GLOBVAR = $(shell echo 'X'))
#echo all: $(GLOBVAR)
lla:
#echo lla: $(GLOBVAR)
$ make all
all: X
$ make lla
lla:
$ make all lla
all: X
lla: X
$ make lla all
lla:
all: X
And I let you imagine what the results could be with parallel make... In summary, if you start using make functions in recipes you are probably wandering into dangerous areas.
I want to declare a variable using a for from bash in a makefile. That's what I tried:
SRCS="path/to/foo.c path/to/boo.c path/to/bar.c"
OBJS=$(for file in $SRCS; do TEMP="$TEMP $(basename $file .c).o"; done; echo $TEMP)
This command:
for file in $SRCS; do TEMP="$TEMP $(basename $file .c).o"; done
works on bash when echoing TEMP. But OBJS in makefile is empty. What am I missing here?
Desired output:
foo.o boo.o bar.o
First of all there's better ways to do this without using bash's for. You could use the make constructs to generate these lists. See the documentation on $(notdir), $(addprefix...) and $(addsuffix ...)
However, to answer your question on this particular example:
First, you are putting quotes around sources. Make does not interpret quotes as special characters, and thus SRCS will expand to "path/to/foo.c path/to/boo.c path/to/bar.c" (including the quotes). This will mess up your for loop later on.
The next thing is the reference to $SRCS -- make will interpret this as $S followed by the literal RCS (which is not what you want). You have to use braces around multi0letter variables in bash as so -- $(SRCS)
Next, TEMP=$TEMP.... When make sees $TEMP, it will immediately attempt to expand it. Because TEMP is not set to anything in the make context, it will expand to a null string -- this is before it invokes the bash shell... What you wanted to do in this case is use $$TEMP, which make will expand to $TEMP.
The following does what you want: note the $(info) lines are just for debugging.
SRCS:=path/to/foo.c path/to/boo.c path/to/bar.c
OBJS:=$(shell for file in $(SRCS); do TEMP="$$TEMP $$(basename $$file .c).o"; done; echo $$TEMP)
$(info for file in $(SRCS); do TEMP="$$TEMP $$(basename $$file .c).o"; done; echo $$TEMP)
$(info OBJS=$(OBJS))
If you wanted to do this in makefile without bash or sh, you might try:
OBJS:=$(notdir $(SRCS:.c=.o))
Note: this is gnu-make specific syntax and may not work on other makes.
I have a bash shell script which I usually source into my shell, with lots of environment variables defined, which are not exported. I do not want to:
Export the variables, because this would make the exportable environment too big, and eventually make the whole system slower (it must be exported when running every command from the shell)
Redefine those variables in the makefile (DRY)
I would like to source the same shell script into the environment of the makefile, so that I can access those variables. Is this possible? How can I do that? Ideally I would do in the makefile:
source setup-env.sh
There is not source command for makefiles, but maybe something equivalent? Any special hack I can use to simulate the same effect?
As per the additional question in the comment, here is one way to effectively mark the whole environment as exported:
for var in $(compgen -v); do export $var; done
compgen -v simply outputs all variable names, as per the bash manual, section 8.7 Programmable Completion Builtins. Then we simply loop over this list and export each one.
Credit to https://stackoverflow.com/a/16337687/2113226 - compgen is new to me.
There are two ways I can think of to integrate this into your make workflow:
- Shell script wrapper
Simply write a shell script which sources your setup-env.sh, exports all variables as above, then calls make itself. Something like:
#!/bin/bash
./source setup-env.sh
for var in $(compgen -v); do export $var; done
make $#
- Recursive make
It may be that you don't want a shell script wrapper, and want to directly invoke make for whatever reason. You can do this all in one Makefile which calls itself recursively:
$(info MAKELEVEL=$(MAKELEVEL) myvar=$(myvar))
ifeq ($(MAKELEVEL), 0)
all:
bash -c "source ./setup-env.sh; \
for var in \$$(compgen -v); do export \$$var; done; \
$(MAKE) $#"
else
all: myprog
myprog:
echo "Recipe for myprog. myvar=$(myvar)"
endif
Output for this Makefile is:
$ make
MAKELEVEL=0 myvar=
bash -c "source ./setup-env.sh; \
for var in \$(compgen -v); do export \$var; done; \
make all"
MAKELEVEL=1 myvar=Hello World
make[1]: Entering directory `/home/ubuntu/makesource'
echo "Recipe for myprog. myvar=Hello World"
Recipe for myprog. myvar=Hello World
make[1]: Leaving directory `/home/ubuntu/makesource'
$
We check the GNU Make builtin variable MAKELEVEL to see what level of recursion we are at. if the level is 0, then we recursively call make for all targets, but first source ./setup-env.sh and export all variables. If the recursion level is anything else, we just do the normal makefile stuff, but you see that the variables you need are now available. This is highlighted by the $(info ) line at the top of the Makefile, which shows the recursion level, and the value (or not) of myvar.
Notes:
We have to use bash -c because compgen is strictly a bash builtin, and not available in Posix mode - i.e. when make invokes the shell as sh -c by default.
The $ in the first all: recipe need to be escaped very carefully. The $$ escapes the $ from being expanded by make, and the \$$ escapes the $ from being expanded by the implicit sh
There is plenty of literature arguing that "Recursive make is considered harmful". E.g. http://aegis.sourceforge.net/auug97.pdf
Considering that every command is run in its own shell, what is the best way to run a multi-line bash command in a makefile? For example, like this:
for i in `find`
do
all="$all $i"
done
gcc $all
You can use backslash for line continuation. However note that the shell receives the whole command concatenated into a single line, so you also need to terminate some of the lines with a semicolon:
foo:
for i in `find`; \
do \
all="$$all $$i"; \
done; \
gcc $$all
But if you just want to take the whole list returned by the find invocation and pass it to gcc, you actually don't necessarily need a multiline command:
foo:
gcc `find`
Or, using a more shell-conventional $(command) approach (notice the $ escaping though):
foo:
gcc $$(find)
As indicated in the question, every sub-command is run in its own shell. This makes writing non-trivial shell scripts a little bit messy -- but it is possible! The solution is to consolidate your script into what make will consider a single sub-command (a single line).
Tips for writing shell scripts within makefiles:
Escape the script's use of $ by replacing with $$
Convert the script to work as a single line by inserting ; between commands
If you want to write the script on multiple lines, escape end-of-line with \
Optionally start with set -e to match make's provision to abort on sub-command failure
This is totally optional, but you could bracket the script with () or {} to emphasize the cohesiveness of a multiple line sequence -- that this is not a typical makefile command sequence
Here's an example inspired by the OP:
mytarget:
{ \
set -e ;\
msg="header:" ;\
for i in $$(seq 1 3) ; do msg="$$msg pre_$${i}_post" ; done ;\
msg="$$msg :footer" ;\
echo msg=$$msg ;\
}
The ONESHELL directive allows to write multiple line recipes to be executed in the same shell invocation.
all: foo
SOURCE_FILES = $(shell find . -name '*.c')
.ONESHELL:
foo: ${SOURCE_FILES}
FILES=()
for F in $^; do
FILES+=($${F})
done
gcc "$${FILES[#]}" -o $#
There is a drawback though : special prefix characters (‘#’, ‘-’, and ‘+’) are interpreted differently.
https://www.gnu.org/software/make/manual/html_node/One-Shell.html
Of course, the proper way to write a Makefile is to actually document which targets depend on which sources. In the trivial case, the proposed solution will make foo depend on itself, but of course, make is smart enough to drop a circular dependency. But if you add a temporary file to your directory, it will "magically" become part of the dependency chain. Better to create an explicit list of dependencies once and for all, perhaps via a script.
GNU make knows how to run gcc to produce an executable out of a set of .c and .h files, so maybe all you really need amounts to
foo: $(wildcard *.h) $(wildcard *.c)
What's wrong with just invoking the commands?
foo:
echo line1
echo line2
....
And for your second question, you need to escape the $ by using $$ instead, i.e. bash -c '... echo $$a ...'.
EDIT: Your example could be rewritten to a single line script like this:
gcc $(for i in `find`; do echo $i; done)