I'm trying to get the command line parameters used to invoke make utility inside a Makefile. Example:
make -C /some/folder -f someMakeFile.mk SOME_VAR=someValue
Inside someMakeFile.mk, I would like to get my hands on the following string:
-C /some/folder -f someMakeFile.mk
or
-C /some/folder -f someMakeFile.mk SOME_VAR=someValue
I'm not interested in SOME_VAR value, that is already well known. I'm interested in the rest of the command line switches for make utility itself (-C and -f).
The reason I'm asking is because I would like to slightly alter the behavior of someMakeFile.mk when certain cmd line switches are used. For example, I would like to return dummy values from complex macro definitions when make is invoked in dry-run mode, rather than do the actual work. This would speed up the TAB auto-completion in bash when you type make than TAB+TAB to show the list of targets.
Or, if this is not possible, is there a way (a variable or function) available for someMakeFile.mk to detect that is only a dry-run?
Best regards
You can check the MAKEFLAGS variable to see if the first word contains an n (for dry run):
ifeq (n,$(findstring n,$(firstword $(MAKEFLAGS))))
$(info in dry-run mode)
else
$(info not in dry-run mode)
endif
Related
I have a shell command where it outputs multiple lines. I want to store it in a variable in makefile for later processing in the target.
A simplified example:
I have this file called zfile1
#zfile1
some text
$echo 123
more text
$$$#&^$
more text
The makefile:
a:
#$(eval v1 = $(shell cat zfile1))
# need to process the variable here, example:
#echo "$(v1)"
# I want to prevent expansion of values in the file and print in multi-line
If you have GNU make 4.2 or above you can use the $(file <zfile1) function. See https://www.gnu.org/software/make/manual/html_node/File-Function.html
If you don't have a new-enough version of GNU make, you can't do it. Of course in your example there's no real need to put the contents of the file into a make variable at all: you can just have your recipe use the file itself. But maybe your real use-case isn't so simple.
ETA
You should never use either the make function eval or the make function shell in a recipe [1].
You can just write:
v1 := $(file <zfile1)
.ONESHELL:
a:
#echo "$(v1)"
You must have the .ONESHELL because otherwise each line of the recipe (after it expands into multiple lines) is considered a separate recipe line. Note that .ONESHELL is in effect for the entire make process so could cause other recipes to break if they rely on each line being invoked in a different shell.
Another option is to export the result into the environment, and use a shell variable like this:
export v1 := $(file <zfile1)
a:
#echo "$$v1"
There are probably better ways to do it but since you've only given us this example and not what you really want to do, that's about all we can say.
[1] There are times where it can be useful but if you have a sufficiently sophisticated requirement to need this you'll probably know how to use it.
I think you're making things too complicated.
Start by writing your recipes as proper self-contained shell scripts.
You can then either store the whole script in a file and run it from make, or you can include it directly in your makefile as a single logical line, as in the following:
a:
#v1=$$(< zfile1); \
echo $$v1
Note the need to "escape" the dollar sign by repeating it.
You could also use global make variables, depending on the actual logic of your real-world use.
I have a Makefile that defines docker-compose project.
It essentially assembles me a command:
COMMAND := docker-compose --project-name=$(PREFIX) --file=$(FILE_PATH)
up:
$(COMMAND) up -d
I would like to add a target named dc to which I would be able to pass any arguments I want.
I know there is one solution:
target:
$(COMMAND) $(ARGS)
And then call it with make target ARGS="--help" for example.
But isn't there an easier way like in bash $# ? I would like to skip the ARGS=... part and send everything to the command after target name.
Not really. The make program interprets all arguments (that don't contain =) as target names to be built and there's no way you can override that. So even though you can obtain the list of arguments given on the command line (via the GNU make-specific $(MAKECMDGOALS) variable) you can't prevent those arguments from being considered targets.
You could do something like this, which is incredibly hacky:
KNOWN_TARGETS = target
ARGS := $(filter-out $(KNOWN_TARGETS),$(MAKECMDGOALS))
.DEFAULT: ;: do nothing
.SUFFIXES:
target:
$(COMMAND) $(ARGS)
(untested). The problem here is you have to keep KNOWN_TARGETS up to date with all the "real" targets so you can remove them from the list of targets given on the command line. Then add the .DEFAULT target which will be run for any target make doesn't know how to build, which does nothing. Reset the .SUFFIXES meta-target to remove built-in rules.
I suspect this still will have weird edge-cases where it doesn't work.
Also note you can't just add options like --help to the make command line, because make will interpret them itself. You'll have to prefix them with -- to force make to ignore them:
make target -- --help
Another option would be to add a target like this:
target%:
$(COMMAND) $*
Then you can run this:
make "target --help"
But you have to include the quotes.
In general I just recommend you reconsider what you want to do.
You could write a bash wrapper script to do what you'd like:
#/bin/bash
make target ARGS=\"$#\"
The reason you don't want to do it in make, is that make parses the command line parameters before it parse the makefile itself, so by the time you read the makefile, the targets, variables, etc have already been set. This means that make will have already interpreted the extra parameters as new targets, variables etc.
A target that re-run make containerized
.PHONY: all containerized
ifeq ($(filter containerized,$(MAKECMDGOALS)),containerized)
.NOTPARALLEL: containerized
MAKEOVERRIDES ?=
containerized: ## Build inside a container
#docker run image_with_make make $(MAKEOVERRIDES) $(filter-out containerized,$(MAKECMDGOALS))
else
# other targets here
all: xxxx
endif
Executing
make containerized all runs make all in container
The first answer is correct, no passthru of args. However, here is a plausible path for experimentation, use of branch by include selection:
# Makefile:
COMMAND := $(PYTHON) this_shit_got_real.py
LOCAL_MK ?= local.mk
# '-' important, absence of LOCAL_MK is not cause for error, just run with no overrides
- include $(LOCAL_MK)
target:
$(COMMAND) $(ARGS)
Now see how you add branching with env:
echo "ARGS=--help">>local.mk
# make target
And the other cli controlled branch
echo "ARGS=--doit">>runner.mk
# LOCAL_MK=runner.mk make target
I am unable to prevent make from communicating any variables to a submake. I've read the manual and I've followed their advice (resetting MAKEOVERRIDES and MAKEFLAGS) but it's still not working has I think it should.
Consider the following prototype Makefile:
${warning $(MAKEOVERRIDES)}
${warning $(MAKEFLAGS)}
${warning $(VAR)}
none:
$(MAKE) -f Makefile MAKEOVERRIDES= MAKEFLAGS= all
all:
echo done!
If I make VAR=10 none, I get the following:
Makefile:2: VAR=10
Makefile:3:
Makefile:4: 10
make -f Makefile MAKEOVERRIDES= MAKEFLAGS= all
make[1]: Entering directory `/home/adriano/sandbox/makes'
Makefile:2:
Makefile:3:
Makefile:4: 10
echo done!
done!
make[1]: Leaving directory `/home/adriano/sandbox/makes'
Meaning that make is communication VAR to the submake. Is this the correct behaviour?
I've tried unexport VAR and bash -c make ... without any luck.
EDIT: I've modified none's recipe to: bash -c "echo $$MAKEOVERRIDES $$MAKEFLAGS $$VAR" ; make ...
This way I found out that VAR is actually being passed through the environment that make creates for the commands to be executed and not through the other variables (the other variables are also passed this way to make).
I think my question now is: how can I create a fresh shell/environment to run my sub make?
EDIT: Someone asked why am I trying to this; I'll try to answer to that here.
I have a "module" which uses a variable named CONFIG. In order to build this module I need to build another partially unrelated "module" which also uses CONFIG, but with a different value. The problem is that when I try to build the "sub-module" CONFIG contains the value of the "super-module." I could specify CONFIG when making the "sub-module" however both modules use many variables with the same name and trying to specify them all would make the modules tightly coupled which is something I cannot afford.
How can this be so difficult...
This is wrong:
none:
$(MAKE) -f Makefile MAKEOVERRIDES= MAKEFLAGS= all
These variables (MAKEOVERRIDES and MAKEFLAGS) are set in the environment by the parent make to be passed down to the sub-makes. Setting overrides on these values inside the recipe won't help, because make has to set the environment for the recipe before it actually starts the commands in the recipe (of course).
You have to override/remove these values in the parent makefile, so that those changes are seen by the parent make before it constructs the sub-make's environment:
MAKEOVERRIDES =
none:
$(MAKE) -f Makefile all
There's no perfect way to do this. However, you can play a trick that will work most of the time:
unexport $(shell echo '$(MAKEOVERRIDES)' | sed 's/=[^ ]*//g')
MAKEOVERRIDES =
The first line tries to unexport all the variables in MAKEOVERRIDES and the second line resets MAKEOVERRIDES. There are a few issues with this. One is that if MAKEOVERRIDES is empty, it will use "unexport" by itself which unexports everything. That can be easily worked around by sticking some bogus variable before the shell function. The other is that if any variable's value contains whitespace, the expansion will consider it a variable to be unexported. That's probably OK, but it's odd.
I can't think of any better way to do it.
You don't really say why you want to do this. Have you considered doing something different, such as running the commands where you want to have a "vanilla" environment using env; for example if you want to run a command with a limited and specific set of env vars, you can run:
test:
env -i PATH='$(PATH)' LANG='$(LANG)' runMyCommand --with --my arguments
Unfortunately some versions of env use - instead of -i; check your man page.
Alternatively, you can try to start a login shell which will re-read the user's shell setup environment from scratch:
test:
/bin/sh -lc 'runMyCommand --with --my arguments'
EDIT: It's difficult because what you're asking to do (restrict the environment of the sub-make) is tricky.
Luckily based on your description, it doesn't seem necessary. Make has a hierarchy of importance for finding variable values. The command line is the highest level (well, there's override but we'll ignore that). After that comes variables set in the makefile itself. And last and lowest comes variables imported from the environment (well, default variables are even lower but we'll ignore that too).
So if your goal is to allow the variables in the sub-makes to not be affected by command line variables given to the upper-level makes, then all this rigmarole of getting the variables out of the environment is not necessary. Variables set in the sub-makefiles will take precedence over the values in the environment. So all you have to do is get rid of the variables set on the command line, which I've already shown how to do above, by setting MAKEOVERRIDES.
How can I write the contents of a makefile variable to file, without invoking a shell command?
The problem is that the contents of the variable is possible longer than the shell allows for a command (i.e. longer than MAX_ARG_STRLEN (131072) characters).
In particular, in a makefile I have a variable containing a long list of filenames to process (including their absolute pathes for out-of-source builds). Now I need to write those filenames to a (temporary) file, which I can then pass to another command.
So far, we had a rule like ($COLLATED_FILES is the variable containing the paths):
$(outdir)/collated-files.tely: $(COLLATED_FILES)
$(LYS_TO_TELY) --name=$(outdir)/collated-files.tely --title="$(TITLE)" \
--author="$(AUTHOR)" $^
This breaks if COLLATED_FILES is longer than about 130000 characters, we get the error message:
make[2]: execvp: /bin/sh: Argument list too long
As a solution, we are now trying to write the contents of the variable to a file and use that file in the $(LYS_TO_TELY) command. Unfortunately, I have not yet found a way to do this without invoking the shell.
My attempts include:
$(outdir)/collated-files.list: $(COLLATED_FILES)
echo "" > $#
$(foreach f,$^,echo $f >> $#;)
But this also invokes all echo commands at once in a shell, so the shell command is just as long.
Is there any way to write the contents of $(COLLATED_FILES) to a file on disk without passing them on the command line to a shell command?
I also searched whether I could pipe the contents of the variable to the shell, but I couldn't find anything in that direction, either...
Assuming you are using GNU Make, there is the file function!
https://www.gnu.org/software/make/manual/html_node/File-Function.html
$(file op filename,text)
where op is either > or >>.
This requires GNU Make 4.0+
You could move whatever makefile code you use to build up the value of COLLATED_FILES to a trivial helper makefile, then invoke make recursively from your original makefile and use trivial shell redirection to capture the stdout of the recursive make invocation -- basically using make as a rudimentary text-processing tool in that context. For example, create a makefile called get_collated_files.mk with these contents:
COLLATED_FILES=abc
COLLATED_FILES+=def
COLLATED_FILES+=ghi
# ... etc ...
# Use $(info) to print the list to stdout. If you want each filename on a
# separate line, use this instead:
#
# $(foreach name,$(COLLATED_FILES),$(info $(name)))
$(info $(COLLATED_FILES))
all: ;##shell no-op to quell make complaints
Then, in your original makefile:
collated-files.list:
$(MAKE) -f get_collated_files.mk > $#
$(outdir)/collated-files.tely: collated-files.list
$(LYS_TO_TELY) --name=$(outdir)/collated-files.tely --title="$(TITLE)" \
--author="$(AUTHOR)" --filelist=collated-files.list
This will be quite a lot more efficient than using hundreds or thousands of individual echo invocations to append to the file one path at a time.
EDIT: One final option, if you really want to have each filename on a separate line, and you have a lot of control over how COLLATED_FILES is defined:
define NL
endef
COLLATED_FILES=abc
COLLATED_FILES+=$(NL)def
COLLATED_FILES+=$(NL)ghi
$(info $(COLLATED_FILES))
all: ;##no-op
This approach allows you to again use just one call to $(info), if that's important to you for some reason.
Here's a patch to gnu make that lets you directly write a variable into a file. It creates a new 'writefile' function, similar to the existing 'info' function, except it takes a filename argument and writes to the file:
https://savannah.gnu.org/bugs/?35384
It looks to me as if you should rethink your build design-- surely there's a better way than letting a variable get this big. But here's a way to do it:
# Make sure this doesn't collide with any of your other targets.
NAMES_TO_WRITE = $(addprefix write_,$(COLLATED_FILES))
collated-files.list: $(NAMES_TO_WRITE)
write_blank:
echo "" > collated-files.list
.PHONY: $(NAMES_TO_WRITE)
$(NAMES_TO_WRITE) : write_% : write_blank
echo $* >> collated-files.list
I need to execute some make rules conditionally, only if the Python installed is greater than a certain version (say 2.5).
I thought I could do something like executing:
python -c 'import sys; print int(sys.version_info >= (2,5))'
and then using the output ('1' if ok, '0' otherwise) in a ifeq make statement.
In a simple bash shell script it's just:
MY_VAR=`python -c 'import sys; print int(sys.version_info >= (2,5))'`
but that doesn't work in a Makefile.
Any suggestions? I could use any other sensible workaround to achieve this.
Use the Make shell builtin like in MY_VAR=$(shell echo whatever)
me#Zack:~$make
MY_VAR IS whatever
me#Zack:~$ cat Makefile
MY_VAR := $(shell echo whatever)
all:
#echo MY_VAR IS $(MY_VAR)
Beware of recipes like this
target:
MY_ID=$(GENERATE_ID);
echo $MY_ID;
It does two things wrong. The first line in the recipe is executed in a separate shell instance from the second line. The variable is lost in the meantime. Second thing wrong is that the $ is not escaped.
target:
MY_ID=$(GENERATE_ID); \
echo $$MY_ID;
Both problems have been fixed and the variable is useable. The backslash combines both lines to run in one single shell, hence the setting of the variable and the reading of the variable afterwords, works.
I realize the original post said how to get the results of a shell command into a MAKE variable, and this answer shows how to get it into a shell variable. But other readers may benefit.
One final improvement, if the consumer expects an "environment variable" to be set, then you have to export it.
my_shell_script
echo $MY_ID
would need this in the makefile
target:
export MY_ID=$(GENERATE_ID); \
./my_shell_script;
Hope that helps someone. In general, one should avoid doing any real work outside of recipes, because if someone use the makefile with '--dry-run' option, to only SEE what it will do, it won't have any undesirable side effects. Every $(shell) call is evaluated at compile time and some real work could accidentally be done. Better to leave the real work, like generating ids, to the inside of the recipes when possible.
Wrapping the assignment in an eval is working for me.
# dependency on .PHONY prevents Make from
# thinking there's `nothing to be done`
set_opts: .PHONY
$(eval DOCKER_OPTS = -v $(shell mktemp -d -p /scratch):/output)
With GNU Make, you can use shell and eval to store, run, and assign output from arbitrary command line invocations. The difference between the example below and those which use := is the := assignment happens once (when it is encountered) and for all. Recursively expanded variables set with = are a bit more "lazy"; references to other variables remain until the variable itself is referenced, and the subsequent recursive expansion takes place each time the variable is referenced, which is desirable for making "consistent, callable, snippets". See the manual on setting variables for more info.
# Generate a random number.
# This is not run initially.
GENERATE_ID = $(shell od -vAn -N2 -tu2 < /dev/urandom)
# Generate a random number, and assign it to MY_ID
# This is not run initially.
SET_ID = $(eval MY_ID=$(GENERATE_ID))
# You can use .PHONY to tell make that we aren't building a target output file
.PHONY: mytarget
mytarget:
# This is empty when we begin
#echo $(MY_ID)
# This recursively expands SET_ID, which calls the shell command and sets MY_ID
$(SET_ID)
# This will now be a random number
#echo $(MY_ID)
# Recursively expand SET_ID again, which calls the shell command (again) and sets MY_ID (again)
$(SET_ID)
# This will now be a different random number
#echo $(MY_ID)
Here's a bit more complicated example with piping and variable assignment inside recipe:
getpodname:
# Getting pod name
#eval $$(minikube docker-env) ;\
$(eval PODNAME=$(shell sh -c "kubectl get pods | grep profile-posts-api | grep Running" | awk '{print $$1}'))
echo $(PODNAME)
I'm writing an answer to increase visibility to the actual syntax that solves the problem. Unfortunately, what someone might see as trivial can become a very significant headache to someone looking for a simple answer to a reasonable question.
Put the following into the file "Makefile".
MY_VAR := $(shell python -c 'import sys; print int(sys.version_info >= (2,5))')
all:
#echo MY_VAR IS $(MY_VAR)
The behavior you would like to see is the following (assuming you have recent python installed).
make
MY_VAR IS 1
If you copy and paste the above text into the Makefile, will you get this? Probably not. You will probably get an error like what is reported here:
makefile:4: *** missing separator. Stop
Why: Because although I personally used a genuine tab, Stack Overflow (attempting to be helpful) converts my tab into a number of spaces. You, frustrated internet citizen, now copy this, thinking that you now have the same text that I used. The make command, now reads the spaces and finds that the "all" command is incorrectly formatted. So copy the above text, paste it, and then convert the whitespace before "#echo" to a tab, and this example should, at last, hopefully, work for you.
In the below example, I have stored the Makefile folder path to LOCAL_PKG_DIR and then use LOCAL_PKG_DIR variable in targets.
Makefile:
LOCAL_PKG_DIR := $(shell eval pwd)
.PHONY: print
print:
#echo $(LOCAL_PKG_DIR)
Terminal output:
$ make print
/home/amrit/folder
From the make manual
The shell assignment operator ‘!=’ can be used to execute a shell script and set a >variable to its output. This operator first evaluates the right-hand side, then passes >that result to the shell for execution. If the result of the execution ends in a >newline, that one newline is removed; all other newlines are replaced by spaces. The >resulting string is then placed into the named recursively-expanded variable. For >example:
hash != printf '\043'
file_list != find . -name '*.c'
source