I have an environment variable GITHUB_REFS that I want to perform some bashism on and capture the result in another variable GITHUB_BRANCH from a GNU makefile. My naive approach looks like this:
SHELL:=/bin/bash
GITHUB_BRANCH:=$(shell echo "${GITHUB_REF#refs/heads/}")
If I run the bashism by itself, it works fine however when running the makefile above it fails with:
Makefile:2: *** unterminated call to function 'shell': missing ')'. Stop.
I tried escaping the # as \#, since it is a plausible culprit, and indeed then the Makefile works however the bashism does not. Double escaping it gives the same error again.
So how can I pull this off?
You also need to double the dollar sign to pass it through to the shell.
GITHUB_BRANCH:=$(shell echo "$${GITHUB_REF\#refs/heads/}")
For what it's worth, this simple parameter expansion is portable to any reasonably modern sh, so not at all an exclusive Bash feature.
Of course, make is perfectly capable of performing the same substitution, without invoking an external process.
GITHUB_BRANCH := $(patsubst refs/heads/%,%,${GITHUB_REF})
The # can be escaped using \, but you also forgot to escape the $.
This Makefile works:
SHELL:=/bin/bash
foo := $(shell echo "$${SHELL\#/bin/}")
all:
echo $(foo)
Just to note that this (the need to escape the #) is a bug, and will be fixed in an upcoming version of GNU make. If you want to allow your makefile to be portable before/after the bug is fixed, you should hide it in a variable like this:
HASH := \#
foo := $(shell echo "${GITHUB_REF$(HASH)refs/heads/}")
This will work in all versions of GNU make.
Related
I'm trying to print out the perforce file version of the make file when it's executed. I'm using the $Id$ tag, which expands to $Id: //repository/path/check.make#6 $ or the like and I want to print //repository/path/check.make#6 to a file (currently using echo). I can't figure out how to get make to take the # as part of the string and not the beginning of a comment. I tried:
str1 = $(subst \#,\\\#,'$Id: //repository/path/check.make#6 $')
and other variations but I keep getting:
unterminated call to function `subst': missing `)'. Stop.
It would help if you provided a full example of what you want. I don't really understand why you're trying to subst a hash with a backslash hash. If you showed us a full example, including how you get the string and also what you want to do with the variable ar1, we could actually give you advice.
But, the way to use hashes in GNU make is to put them into a variable:
HASH := \#
$(info HASH = $(HASH))
That's all I can say without more info.
ETA
Yes, I'm very familiar with keyword expansion... it originated with SCCS/RCS back in the day :).
I see, you mean, you want to put the $Id$ into your makefile, then when your makefile is checked out the value will be replaced. That wasn't clear to me.
I'm sorry to say that what you want to do is close to impossible. The problem is that you can't escape the value in the makefile because you're not writing the value into the makefile, Perforce is. And Perforce is not escaping it.
You have only two options that I can see:
First, don't try to put this into a make variable. There are many ways to do this, depending on what you really want. One way is to create a header file that contains const char* foo = "$Id$"; and let that be replaced. If you really want the ID of the makefile, but you only need it within a certain recipe, you can put it directly into that recipe:
myrecipe: ; echo '$$Id$$'
(I'm not actually sure the $$ trick here will work, it depends on how Perforce replaces things... if it doesn't you can use echo '$Id$x' you'll lose the dollar signs but keep the rest).
The only other option is to upgrade your version of GNU make to the latest (4.3). In that release, some broken handling of hash characters in the $(shell ...) function was fixed, which means you can use:
var1 := $(shell echo '$$Id$$')
and it will work (same caveats, and solutions, for $$ here as above).
Maybe I didn't get you correctly but the following works for an outside actor replacing $Id$ without escaping:
define PERFORCE_ID
$Id$
endef
PERFORCE_ID := $(word 2,$(value PERFORCE_ID))
$(info $(PERFORCE_ID))
As a test, I simply put in the text substitution from Perforce myself:
define PERFORCE_ID
$Id: //repository/path/check.make#6 $
endef
PERFORCE_ID := $(word 2,$(value PERFORCE_ID))
$(info Perforce id is: $(PERFORCE_ID))
Output:
Perforce id is: //repository/path/check.make#6
You can't have an unescaped literal # in a make assignment and not have it be interpreted as a comment character. But as a hack, you can have the shell extract this token from the current Makefile.
# $Id: //repository/path/check.make#6 $
str1 := $(shell sed '/[$$]Id[$$:]/!d;s/^\# [$$]Id: \(.*\) [$$].*/\1/' Makefile)
The sed script looks for the $Id$ or $Id: token in the Makefile itself by way of a regex which doesn't match itself; the doubled dollar sign is how you put a literal dollar sign in a Makefile. It extracts the contents of the field, and make assigns the output to str1. Because there is no literal # in the code which assigns the variable, no further escaping is necessary.
Demo: https://ideone.com/hWjnCp
This requires GNU Make, but that's apparently what you are using already. (Please tag such questions explicitly as gnu-make.)
I am trying to remove the path prefix. Here is a small example showing just the issue.
Makefile
dist_directory = ./dist
default: build
build: $(patsubst %.md, $(dist_directory)/%.html, $(wildcard *.md))
$(dist_directory)/%.html: %.md
#echo start
#echo $#
#echo ${$#//$(dist_directory)/}
#echo end
Create a file: touch stuff.md
Then build: make
The output is:
start
dist/stuff.html
end
The expected output is:
start
dist/stuff.html
/stuff.html
end
There are similar posts on Stack Exchange. However, they have not worked for me in a Makefile for some reason. I'm probably doing something wrong.
https://unix.stackexchange.com/questions/311758/remove-specific-word-in-variable
Remove a fixed prefix/suffix from a string in Bash
Remove substring matching pattern both in the beginning and the end of the variable
You have many issues here. The most fundamental one is that if you want to use shell variables you have to escape the dollar sign so that make doesn't interpret it. And, you can only use shell variable substitutions on shell variables, while $# is a make variable, so you need:
#foo='$#' ; echo $${foo//$(dist_directory)/}
The more subtle one is that make always uses /bin/sh (POSIX standard shell) when it invokes recipes, and the above syntax is specific to bash. One way around that would be to explicitly set SHELL := /bin/bash in your makefile to force make to use bash. Luckily that is not necessary because POSIX sh can also do this, as mentioned by Reda in another answer:
#foo='$#' ; echo $${###*/}
But even more, you don't need any of this because make sets the automatic variable $* to the part of the target that matches the stem (the %):
#echo $*.html
It also sets $(#F) to the filename part of the $# variable:
#echo $(#F)
ETA
If you want to do something very similar to your shell variable expansion using GNU make you can use:
#echo $(patsubst $(dist_directory)/%,%,$#)
I'm trying to make sense out of the multi-line define directive of GNU make and I cannot. Example:
define A
1
2
endef
all:
#echo W=$(word 1,$(A))
Running make produces a result I have expected the least:
W=1
make: 2: Command not found
make: *** [all] Error 127
It appears that part of $(A) has spilled outside the $(word) function.
Is it a bug or intended behavior? If the "spill" is intentional, how does it really works?
P.S. GNU make v3.81 on Linux/x64
The thing to remember here is that make stores each recipe as a single recursive variable. At the point that make decides that it must run your recipe, it expands that variable. Make then passes each line in the resulting expansion to a separate shell, stopping if any of those shell executions return an error.
In your example, before running anything make expands #echo W=$(word 1,$(A)).
$(A) becomes 1¶2 (dunno what this looks like on your browser, but I'm using ¶ to represent a newline character)
Now, 1¶2 is a single word as far as make is concerned, so $(word 1,1¶2) naturally expands to 1¶2 (can you see where this is going yet?)
This leaves make with the string #echo W=1¶2. Make dutifully passes the first line of this to the shell (without the # as that is special to make). The shell executes echo W=1.
make executes 2 in a new shell.
The second shell complains that it can't find the command 2.
So, yes, expected behaviour.
[Warning: slight simplification in the above where I gloss over the bit where make is able to elide the shell and invoke the command itself if the string has no shell metacharacters in it]
The $(word) function is splitting on spaces. Not whitespace, spaces.
There are no spaces in your A macro so nothing gets split.
Add a trailing space on the 1 line or a leading space on the 2 line and you get your expected behaviour.
This is consistent across GNU make 3.81, 3.82, 4.0, and 4.1 in some quick testing here.
The reason you see the "spill" as you called it is because of how the define is expanded. It is expanded literally, newline and all. (Think template expansion.)
So make expands the define into the call to $(word 1,...) then expands that result (the whole define including the newline) into the recipe template and ends up with two lines that it executes as the recipe.
Consider a macro like this:
define somecommands
echo foo
echo bar
echo baz
endef
all:
$(somecommands)
What would you expect to happen here? How many lines is the body of all? How many shells are run here? What commands are executed? The answer is three lines, three shells and three echo commands.
If the newlines weren't counted then you would effectively run echo foo echo bar echo baz in one command and get foo echo bar echo baz as output instead of the expected (and far more useful) foo, bar, and baz on three different lines.
I'm trying to create a generic build template for my Makefiles, kind of like they discuss in the eval documentation.
I can't seem to get the wildcard function to work within an eval. The basic code I'm having issues with looks like this.
SRC_DIR = ./src/
PROG_NAME = test
define PROGRAM_template
$(1)_SRC_DIR = $(join $(SRC_DIR), $(1)/)
$(1)_SRC_FILES = $(wildcard $$($(1)_SRC_DIR)*.c)
endef
$(eval $(call PROGRAM_template, $(PROG_NAME)))
all:
#echo $(test_SRC_DIR)
#echo $(test_SRC_FILES)
#echo $(wildcard $(wildcard $(test_SRC_DIR)*.c)
When I run make with this, the output is
./src/test
[correct list of all .c files in ./src/test/]
Basically, the wildcard call within PROGRAM_template is not being eval'd as I expect it. The call results in an empty list.
The join call is being eval'd correctly though.
So, what am I doing wrong? My guess is that
$$($(1)_SRC_DIR)
is not correct, but I can't figure out the right way to do it.
EDIT
Once this was solved, it didn't take long for me to hit another issue with eval.
I posted it as a new question at
Workaround for GNU Make 3.80 eval bug
You need to double escape virtually all of the functions and variables when you use eval. In most cases, the only things that don't need to be double-escaped are function arguments (because the call function will fully expand them). In this case, you technically don't need to double-escape join or SRC_DIR either, but it will simplify your life if you just always double-escape all variables and functions when using eval.
The reason you need the double escapes is that expansion happens twice when using eval. The eval function itself performs expansion, and then expansion is done again when the block is finally parsed as makefile syntax (i.e. when it is actually evaluated).
The way you've got it written, wildcard is invoked on the string literal $( test_SRC_DIR)*.c. If you want, you can see this for yourself by replacing wildcard with info in your version and see what happens.
You need to hold off on actually invoking wildcard until the second expansion, so that it's argument is the result of the expansion of $(test_SRC_DIR).
Try this:
SRC_DIR = ./src/
PROG_NAME = test
define PROGRAM_template
$(1)_SRC_DIR = $$(join $$(SRC_DIR),$(1)/)
$(1)_SRC_FILES = $$(wildcard $$($(1)_SRC_DIR)*.c)
endef
$(eval $(call PROGRAM_template,$(PROG_NAME)))
all:
#echo $(test_SRC_DIR)
#echo $(test_SRC_FILES)
#echo $(wildcard $(test_SRC_DIR)*.c)
EDIT: After posting this, I thought I'd better test it out to make sure it actually works. In doing so, I discovered another problem. You should avoid putting spaces between the comma and argument when calling functions. It causes a literal space character to be prepended to the argument that is passed to the function and leads to unintended results. I've removed the spaces after the commas in the function calls in my version (while this isn't a problem for the call to join, I removed the space there as well just because it's a good habit to get into).
I need to execute some make rules conditionally, only if the Python installed is greater than a certain version (say 2.5).
I thought I could do something like executing:
python -c 'import sys; print int(sys.version_info >= (2,5))'
and then using the output ('1' if ok, '0' otherwise) in a ifeq make statement.
In a simple bash shell script it's just:
MY_VAR=`python -c 'import sys; print int(sys.version_info >= (2,5))'`
but that doesn't work in a Makefile.
Any suggestions? I could use any other sensible workaround to achieve this.
Use the Make shell builtin like in MY_VAR=$(shell echo whatever)
me#Zack:~$make
MY_VAR IS whatever
me#Zack:~$ cat Makefile
MY_VAR := $(shell echo whatever)
all:
#echo MY_VAR IS $(MY_VAR)
Beware of recipes like this
target:
MY_ID=$(GENERATE_ID);
echo $MY_ID;
It does two things wrong. The first line in the recipe is executed in a separate shell instance from the second line. The variable is lost in the meantime. Second thing wrong is that the $ is not escaped.
target:
MY_ID=$(GENERATE_ID); \
echo $$MY_ID;
Both problems have been fixed and the variable is useable. The backslash combines both lines to run in one single shell, hence the setting of the variable and the reading of the variable afterwords, works.
I realize the original post said how to get the results of a shell command into a MAKE variable, and this answer shows how to get it into a shell variable. But other readers may benefit.
One final improvement, if the consumer expects an "environment variable" to be set, then you have to export it.
my_shell_script
echo $MY_ID
would need this in the makefile
target:
export MY_ID=$(GENERATE_ID); \
./my_shell_script;
Hope that helps someone. In general, one should avoid doing any real work outside of recipes, because if someone use the makefile with '--dry-run' option, to only SEE what it will do, it won't have any undesirable side effects. Every $(shell) call is evaluated at compile time and some real work could accidentally be done. Better to leave the real work, like generating ids, to the inside of the recipes when possible.
Wrapping the assignment in an eval is working for me.
# dependency on .PHONY prevents Make from
# thinking there's `nothing to be done`
set_opts: .PHONY
$(eval DOCKER_OPTS = -v $(shell mktemp -d -p /scratch):/output)
With GNU Make, you can use shell and eval to store, run, and assign output from arbitrary command line invocations. The difference between the example below and those which use := is the := assignment happens once (when it is encountered) and for all. Recursively expanded variables set with = are a bit more "lazy"; references to other variables remain until the variable itself is referenced, and the subsequent recursive expansion takes place each time the variable is referenced, which is desirable for making "consistent, callable, snippets". See the manual on setting variables for more info.
# Generate a random number.
# This is not run initially.
GENERATE_ID = $(shell od -vAn -N2 -tu2 < /dev/urandom)
# Generate a random number, and assign it to MY_ID
# This is not run initially.
SET_ID = $(eval MY_ID=$(GENERATE_ID))
# You can use .PHONY to tell make that we aren't building a target output file
.PHONY: mytarget
mytarget:
# This is empty when we begin
#echo $(MY_ID)
# This recursively expands SET_ID, which calls the shell command and sets MY_ID
$(SET_ID)
# This will now be a random number
#echo $(MY_ID)
# Recursively expand SET_ID again, which calls the shell command (again) and sets MY_ID (again)
$(SET_ID)
# This will now be a different random number
#echo $(MY_ID)
Here's a bit more complicated example with piping and variable assignment inside recipe:
getpodname:
# Getting pod name
#eval $$(minikube docker-env) ;\
$(eval PODNAME=$(shell sh -c "kubectl get pods | grep profile-posts-api | grep Running" | awk '{print $$1}'))
echo $(PODNAME)
I'm writing an answer to increase visibility to the actual syntax that solves the problem. Unfortunately, what someone might see as trivial can become a very significant headache to someone looking for a simple answer to a reasonable question.
Put the following into the file "Makefile".
MY_VAR := $(shell python -c 'import sys; print int(sys.version_info >= (2,5))')
all:
#echo MY_VAR IS $(MY_VAR)
The behavior you would like to see is the following (assuming you have recent python installed).
make
MY_VAR IS 1
If you copy and paste the above text into the Makefile, will you get this? Probably not. You will probably get an error like what is reported here:
makefile:4: *** missing separator. Stop
Why: Because although I personally used a genuine tab, Stack Overflow (attempting to be helpful) converts my tab into a number of spaces. You, frustrated internet citizen, now copy this, thinking that you now have the same text that I used. The make command, now reads the spaces and finds that the "all" command is incorrectly formatted. So copy the above text, paste it, and then convert the whitespace before "#echo" to a tab, and this example should, at last, hopefully, work for you.
In the below example, I have stored the Makefile folder path to LOCAL_PKG_DIR and then use LOCAL_PKG_DIR variable in targets.
Makefile:
LOCAL_PKG_DIR := $(shell eval pwd)
.PHONY: print
print:
#echo $(LOCAL_PKG_DIR)
Terminal output:
$ make print
/home/amrit/folder
From the make manual
The shell assignment operator ‘!=’ can be used to execute a shell script and set a >variable to its output. This operator first evaluates the right-hand side, then passes >that result to the shell for execution. If the result of the execution ends in a >newline, that one newline is removed; all other newlines are replaced by spaces. The >resulting string is then placed into the named recursively-expanded variable. For >example:
hash != printf '\043'
file_list != find . -name '*.c'
source