Is there make debug option that prints all executions of shell function?
For example if I have makefile
contents := $(shell echo foo)
all:
I want it to print "echo foo".
You can supply your own program as a shell, that will echo the commands and do whatever else you need.
https://www.gnu.org/software/make/manual/html_node/Choosing-the-Shell.html:
5.3.2 Choosing the Shell
The program used as the shell is taken from the variable SHELL. If this variable is not set in your makefile, the program /bin/sh is used as the shell. The argument(s) passed to the shell are taken from the variable .SHELLFLAGS. The default value of .SHELLFLAGS is -c normally, or -ec in POSIX-conforming mode.
So, run:
SHELL=mysneakyprogram make all
Each command will be passed to mysneakyprogram with a -c argument.
The catch is: this will also apply to normal commands that build targets, not only $(shell ...); I don't think you can distinguish the two cases.
Related
Suppose I would like to set a shell in make:
SHELL:=/usr/bin/env bash
Next, suppose I have some runcom/bash file I would like to source as well. This file optionally activates a virtual environment:
if [ -d venv ]; then source venv/bin/activate fi;
However, if I write:
SHELL:=/usr/bin/env bash && source runcom/bash
This fails. However, if I deposited the venv logic into the local ~/.bashrc and write:
SHELL:=/usr/bin/env bash -l
I can get the exact functionality I need.
But, I have to deposit something that should remain downstream from the local user into the user's upstream environment -- I'd rather not.
Is there a way to get the make shell to source a file at the declaration step in the make start-up process?
This can't work:
SHELL:=/usr/bin/env bash && source runcom/bash
Because SHELL tells make how to invoke the shell; if you make the contents of SHELL be a shell script then make has to invoke the shell to interpret how to invoke the shell, which means it has to invoke the shell to invoke the shell to interpret how to invoke the shell, etc.
So, SHELL must be either a simple command or, at most, a simple set of arguments that can converted into an argv list and passed to exec(2).
So, this is really a shell question not a make question: how can you get the shell to source arbitrary stuff when it starts without changing ~/.profile or whatever?
Luckily, this is possible; see the bash man page:
BASH_ENV
If this parameter is set when bash is executing a shell script,
its value is interpreted as a filename containing commands to
initialize the shell, as in ~/.bashrc. The value of BASH_ENV is
subjected to parameter expansion, command substitution, and
arithmetic expansion before being interpreted as a filename.
PATH is not used to search for the resultant filename.
ENV Similar to BASH_ENV; used when the shell is invoked in posix
mode.
So, in your makefile you can use something like:
SHELL := /bin/bash
export BASH_ENV := runcom/bash
and that should be sufficient.
I used below sample code in makefile in MacOS to construct path, but it always not works.
all:
ROOT=$(shell pwd)
#echo $(ROOT)
AGENT:=$(ROOT)/agent
#echo $(AGENT)
#mkdir -p $(AGENT)
It will output as AGENT:=/agent, obviously, the AGENT variable is not expanded as ROOT
Several common mistakes here:
Recipe lines run in separate shells, you cannot expect a shell variable assigned in one to be still defined in another.
No need to use the $(shell...) make function in recipes: recipes are already shell scripts.
You apparently mix-up shell variables and make variables.
Shell variables are assigned with var=xxx (no spaces around the =), only in recipes (well, almost only). They are expanded with $var or ${var} but never with $(var), still almost only in recipes.
Make variables are assigned with var := xxx or var = xxx (with or without spaces around := or =), only outside from recipes (well, almost only). They are expanded with $(var) or ${var}, never with $var, anywhere. Note that = and := are different but this is out of scope.
If you are a make beginner please ignore the "almost" in the above explanation, you will not need this for now.
A Makefile is written with two different languages: the shell language in recipes and the make language everywhere else. A bit like a web page is written in HTML and in CSS. The shell language and the make language look sometimes similar but they are different. Something important to consider is that make will pass the recipes to the shell... after having expanded them first to substitute make variables or make function calls. It is thus frequently necessary to protect parts of the shell syntax from this make expansion (see below for an example).
Anyway, try something like this, maybe:
With make variables
ROOT := $(shell pwd)
AGENT := $(ROOT)/agent
all:
#echo $(ROOT)
#echo $(AGENT)
#mkdir -p $(AGENT)
ROOT and AGENT are make variables, assigned outside any recipe with := and available anywhere, including in recipes. They are expanded with $(var). Before passing the recipe to the shell make will expand it as:
echo /some/where
echo /some/where/agent
mkdir -p /some/where/agent
Each line will be executed in a separate shell but it does not matter because there is no shell variables passing between them.
Note: if you are using GNU make, the CURDIR make variable is defined and expands as the current working directory. No need to use pwd, just use $(CURDIR) instead of $(ROOT):
AGENT := $(CURDIR)/agent
all:
#echo $(CURDIR)
#echo $(AGENT)
#mkdir -p $(AGENT)
Note: if ROOT refers to where the Makefile is located and if you invoke make from elsewhere using the -f make option, $(CURDIR) does not work any more because it points to where you invoked make from, not where the Makefile is located. But you can also get the correct ROOT with:
ROOT := $(dir $(lastword $(MAKEFILE_LIST)))
at the very beginning of your Makefile (before any include statement).
With shell variables
Use only shell syntax and a one-line recipe:
all:
#ROOT="`pwd`"; AGENT="$$ROOT"/agent; echo "$$ROOT"; echo "$$AGENT"; mkdir -p "$$AGENT"
ROOT and AGENT are shell variables, assigned with = inside the recipe and available only in the very same line of the recipe. Note the double $ used to expand shell variables. They are needed to escape the first expansion that make performs before passing the result to the shell. After expansion by make the recipe is passed to the shell as:
ROOT="`pwd`"; AGENT="$ROOT"/agent; echo "$ROOT"; echo "$AGENT"; mkdir -p "$AGENT"
(make expands $$ as $ and stops there) which is what you want. It is executed in one single shell, reason why the shell variables are passed from one command of the list to the next.
If you wish, for better readability, you can use the \ line continuation to split the recipe on several lines, but still have only one single recipe executed in one single shell:
all:
#ROOT="`pwd`"; \
AGENT="$$ROOT/agent"; \
echo "$$ROOT"; \
echo "$$AGENT"; \
mkdir -p "$$AGENT"
This is strictly equivalent to the other form, just a bit easier to read. But it is completely different from:
all:
#ROOT="`pwd`"
#AGENT="$$ROOT/agent"
#echo "$$ROOT"
#echo "$$AGENT"
#mkdir -p "$$AGENT"
because with this last form the 5 lines are executed by 5 different shells. When the second one runs, for example, the ROOT shell variable assigned by the first shell is not defined any more and the result is the same as:
AGENT="/agent"
Then, the two echo lines echo nothing at all, just a newline. And finally you get an error because:
mkdir -p
is not valid.
What is the difference between these two commands in makefiles:
#echo "Hello World"
$(info Hello World)
As it seems, echo and info print the same output, so where is the difference?
And when to use which one?
Well, echo is a shell command. So if you put it in a recipe, a shell will be invoked to run it and the shell command will generate the output:
foo: ; #echo "Hello World"
runs /bin/sh -c 'echo "Hello World"'. It can only be used in a recipe. It will work in any version of make, and with any POSIX shell. Because it invokes a shell, you may need to be concerned with quoting issues, etc. (not in this simple example of course).
info is a GNU make function. It is handled directly by make: no shell is invoked. It can appear anywhere in a makefile, not just in a recipe. It is not portable to other versions of make. Because no shell is invoked, there are no quoting issues.
However, because info is a make function it is parsed by make before the shell is invoked: that means it can't show shell variables that are set within a recipe; for example:
foo: ; #for i in a b c d; do $(info $$i); done
cannot work; you must use echo here.
This question already has an answer here:
command substitution doesn't work with echo in makefile [duplicate]
(1 answer)
Closed 6 years ago.
Inside of a makefile, I'm trying to check if fileA was modified more recently than fileB. Using a few prior posts (this and this) as references, I've come up with this as an attempt to store the time since last file modification as a variable:
(I'd rather this be in a function outside of a make recipe, but one step at a time.)
.PHONY: all clean
all : (stuff happens here)
radio :
BASE_MOD_TIME="$( expr $(date +%s) - $(date +%s -r src/radio_interface/profile_init.c) )"
#echo "$(BASE_MOD_TIME)"
I thought that I would be assigning the output of the expr command to a variable, BASE_MOD_TIME, but the output is:
bash-4.1$
BASE_MOD_TIME=""
echo ""
What am I doing wrong here? Simple attempts to save the output of ls -l also didn't work like this.
Make variables are normally global, and you don't normally set make variables in a recipe. A recipe is simply a list of commands to be executed by a shell, and so what looks like a variable assignment in a recipe is a shell variable assignment.
However, each line in a make recipe is run in its own shell subprocess. So a variable set in one line won't be visible in another line; they are not persistent. That makes setting shell variables in recipes less useful. [Note 1]
But you can combine multiple lines of a recipe into a single shell command using the backslash escape at the end of the line, and remembering to terminate the individual commands with semicolons (or, better, link them with &&), because the backslash-escaped newline will not be passed to the shell. Also, don't forget to escape the $ characters so they will be passed to the shell, rather than being interpreted by make.
So you could do the following:
radio:
#BASE_MOD_TIME="$$( expr $$(date +%s) - $$(date +%s -r src/radio_interface/profile_init.c) )"; \
echo "$$BASE_MOD_TIME"; \
# More commands in the same subprocess
But that gets quite awkward if there are more than a couple of commands, and a better solution is usually to write a shell script and invoke it from the recipe (although that means that the Makefile is no longer self-contained.)
Gnu make provides two ways to set make variables in a recipe:
1. Target-specific variables.
You can create a target-specific variable (which is not exactly local to the target) by adding a line like:
target: var := value
To set the variable from a shell command, use the shell function:
target: var := $(shell ....)
This variable will be available in the target recipe and all dependencies triggered by the target. Note that a dependency is only evaluated once, so if it could be triggered by a different target, the target-specific variable might or might not be available in the dependency, depending on the order in which make resolves dependencies.
2. Using the eval function
Since the expansion of recipes is always deferred, you can use the eval function inside a recipe to defer the assignment of a make variable. The eval function can be placed pretty well anywhere in a recipe because its value is the empty string. However, evaling a variable assignment makes the variable assignment global; it will be visible throughout the makefile, but its value in other recipes will depend, again, on the order in which make evaluates recipes, which is not necessarily predictable.
For example:
radio:
$(eval var = $(shell ...))
Notes:
You can change this behaviour using the .ONESHELL: pseudo-target, but that will apply to the entire Makefile; there is no way to mark a single recipe as being executed in a single subprocess. Since changing the behaviour can break recipes in unexpected ways, I don't usually recommend this feature.
What's wrong with this?
fileB: fileA
#echo $< was modified more recently than $#
Instead of forcing the makefile to do all of the heavy lifting via some bash commands, I just called a separate bash script. This provided a lot more clarity for a newbie to bash scripting like myself since I didn't have to worry about escaping the current shell being used by make.
Final solution:
.PHONY: all clean
all : (stuff happens here)
radio :
./radio_init_check.sh
$(MKDIR_P) $(OBJDIR)
make $(radio_10)
with radio_init_check.sh being my sew script.
I'm trying to access a variable declared by previous command (inside a Makefile).
Here's the Makefile:
all:
./script1.sh
./script2.sh
Here's the script declaring the variable I want to access,script1.sh:
#!/usr/bin/env bash
myVar=1234
Here's the script trying to access the variable previously defined, script2.sh:
#!/usr/bin/env bash
echo $myVar
Unfortunately when I run make, myVar isn't accessible. Is there an other way around? Thanks.
Make will run each shell command in its own shell. And when the shell exits, its environment is lost.
If you want variables from one script to be available in the next, there are constructs which will do this. For example:
all:
( . ./script1.sh; ./script2.sh )
This causes Make to launch a single shell to handle both scripts.
Note also that you will need to export the variable in order for it to be visible in the second script; unexported variables are available only to the local script, and not to subshells that it launches.
UPDATE (per Kusalananda's comment):
If you want your shell commands to populate MAKE variables instead of merely environment variables, you may have options that depend on the version of Make that you are running. For example, in BSD make and GNU make, you can use "variable assignment modifiers" including (from the BSD make man page):
!= Expand the value and pass it to the shell for execution and
assign the result to the variable. Any newlines in the result
are replaced with spaces.
Thus, with BSD make and GNU make, you could do this:
$ cat Makefile
foo!= . ./script1.sh; ./script2.sh
all:
#echo "foo=${foo}"
$
$ cat script1.sh
export test=bar
$
$ cat script2.sh
#!/usr/bin/env bash
echo "$test"
$
$ make
foo=bar
$
Note that script1.sh does not include any shebang because it's being sourced, and is therefore running in the calling shell, whatever that is. That makes the shebang line merely a comment. If you're on a system where the default shell is POSIX but not bash (like Ubuntu, Solaris, FreeBSD, etc), this should still work because POSIX shells should all understand the concept of exporting variables.
The two separate invocations of the scripts create two separate environments. The first script sets a variable in its environment and exits (the environment is lost). The second script does not have that variable in its environment, so it outputs an empty string.
You can not have environment variables pass between environments other than between the environments of a parent shell to its child shell (not the other way around). The variables passed over into the child shell are only those that the parent shell has export-ed. So, if the first script invoked the second script, the value would be outputted (if it was export-ed in the first script).
In a shell, you would source the first file to set the variables therein in the current environment (and then export them!). However, in Makefiles it's a bit trickier since there's no convenient source command.
Instead you may want to read this StackOverflow question.
EDIT in light of #ghoti's answer: #ghoti has a good solution, but I'll leave my answer in here as it explains a bit more verbosely about environment variables and what we can do and not do with them with regards to passing them between environments.