I am reading the document of GNU Make. Here is an example
%.d: %.c
#set -e; rm -f $#; \
$(CC) -M $(CPPFLAGS) $< > $#.$$$$; \
sed ’s,\($*\)\.o[ :]*,\1.o $# : ,g’ < $#.$$$$ > $#; \
rm -f $#.$$$$
I tried this on a C++ program, and got the list of files
init3d.d init3d.d.18449 input.d input.d.18444 main.d main.d.18439
Here is what I found but don't understand in the same document
If you have enabled secondary expansion and you want a literal dollar sign in the prerequisites list, you must actually write four dollar signs (‘$$$$’).
I wonder what the four dollar signs "$$$$" mean actually? How do they 18449, 18444 or 18439?
Thanks and regards!
If make "secondary expansion" is enabled, $$$$ is required in order to generate a single $ in the actual output. $ is normally used to expand variables, call make functions, etc. $$ with secondary expansion enabled does something else, but otherwise it generates an actual $ in the output.
The shell that make uses to execute command-lines on Unix-like systems normally interprets $$ as expand to shell process ID. So, without secondary expansion enabled, $$$$ will turn into $$ in the output, which the shell will expand to the process ID.
(Using the shell process ID as a suffix is a simple way of trying to guarantee uniqueness of file name for a temporary file.)
$$ will be converted to $, but in Makefile rules (which are shell expressions) you'll have to also escape the resulting $ using a \ or by using single quotes ' around your expression.
Here is an example that demonstrates it:
DOLLAR:=$$
dollar:
echo '$$' > $#
echo "\$$" >> $#
echo '$(DOLLAR)' >> $#
echo "\$(DOLLAR)" >> $#
cat dollar
18449, 18444 or 18439 look like process ids so maybe a process ID?
Related
What does the # sign inside of the substitution reference in this Makefile recipe mean? I know it's not echo suppression and I'm fairly confident it's not an automatic variable (at least one that I know of).
I've tried replacing the # sign with $#, as basically what the substitution reference is supposed to do is to simply replace the suffix of the target from .d to .o, but this seems to output an empty string. In it's current state with just # it works perfectly (but I want to know why).
$(BUILDDIR)/%.d: %.cpp | $(BUILDDIR)
$(CXX) $(CXXFLAGS) -MM -MT $(#:.d=.o) $< > $#
$# is an automatic variable, see https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
The file name of the target of the rule.
As far as the behavior caused by prepending $ to #, that's how variables work in make, see https://ftp.gnu.org/old-gnu/Manuals/make-3.79.1/html_chapter/make_6.html#SEC58
To substitute a variable's value, write a dollar sign followed by the name of the variable in parentheses or braces: either $(foo) or ${foo} is a valid reference to the variable foo. This special significance of $ is why you must write $$ to have the effect of a single dollar sign in a file name or command.
You can test this with an example:
SOMEVAR:=hello
test:
echo ${SOMEVAR}
echo $SOMEVAR
echo $(SOMEVAR:hello=world)
echo $($SOMEVAR:hello=world)
The behavior (echo output omitted) is:
echo hello
echo OMEVAR
echo world
echo
As you can see the last result is blank because make tries to resolve the OMEVAR variable, which is unset.
Makefile:
.PHONY: all
SHELL:=/usr/bin/env bash
all:
$(eval x=$(shell cat file))
#echo "$x"
File:
foo
bar
Output:
foo bar
How do I get the contents of the file into the make variable without losing the newlines?
You can't do this with shell, as described in its documentation.
If you have a sufficiently new version of GNU make, you can use the file function however.
Make converts newlines from shell outputs to spaces (see here):
The shell function performs the same function that backquotes (‘`’)
perform in most shells: it does command expansion. This means that it
takes as an argument a shell command and evaluates to the output of
the command. The only processing make does on the result is to convert
each newline (or carriage-return / newline pair) to a single space. If
there is a trailing (carriage-return and) newline it will simply be
removed.
So, you cannot preserve spaces from the $(shell) command directly. That being said, make does allow multiline variables using define -- but beware, attempting to use such variables is problematic. Consider:
define x
foo
bar
endef
all:
#echo "$x"
Make expands the $x in place, and you end up with:
all:
#echo " foo
bar"
(where the newline is considered the end of the recipe line..).
Depending on what you want this for, you may be able to get around this is using a bash variable:
all:
#x=$$(cat file); \
echo $$x
Or potentially storing your output in a file, and referencing that when necessary.
all:
eval (cat file >> output.txt)
cat output.txt
(and yes, the last one is convoluted as written, but I'm not sure what you're trying to do, and this allows the output of your command to be persistent across recipe lines).
If the file contents are ensured not to contain any binary data, and if you're willing to to extra processing each time you access the variable, then you could:
foo:=$(shell cat file | tr '\n' '\1')
all:
#echo "$(shell echo "$(foo)" | tr '\1' '\n')"
Note that you cannot use nulls \0, and I suspect that probably means there's a buffer overflow bug in my copy of Make.
Does anyone know how to use a here-document redirection on a recipe?
test:
sh <<EOF
echo I Need This
echo To Work
ls
EOF
I can't find any solution trying the usual backslash method (which basically ends with a command in a single line).
Rationale:
I have a set of multi-line recipes that I want to proxy through another command (e.g., sh, docker).
onelinerecipe := echo l1
define twolinerecipe :=
echo l1
echo l2
endef
define threelinerecipe :=
echo l1
echo l2
echo l3
endef
# sh as proxy command and proof of concept
proxy := sh
test1:
$(proxy) <<EOF
$(onelinerecipe)
EOF
test2:
$(proxy) <<EOF
$(twolinerecipe)
EOF
test3:
$(proxy) <<EOF
$(threelinerecipe)
EOF
The solution I would love to avoid: transform multiline macros into single lines.
define threelinerecipe :=
echo l1;
echo l2;
echo l3
endef
test3:
$(proxy) <<< "$(strip $(threelinerecipe))"
This works (I use gmake 4.0 and bash as make's shell) but it requires changing my recipes and I have a lot.
Strip removes the newlines, from the macro, then everything is written in a single line.
My end goal is: proxy := docker run ...
Using the line .ONESHELL: somewhere in your Makefile will send all recipe lines to a single shell invocation, you should find your original Makefile works as expected.
When make sees a multi-line block in a recipe
(i.e., a block of lines all ending in \, apart from the last),
it passes that block un-modifed to the shell.
This generally works in bash,
apart from here docs.
One way around this is to strip any trailing \s,
then pass the resulting string to bash's eval.
You do this in make by playing with ${.SHELLFLAGS} and ${SHELL}.
You can use both of these in target-specific form if you only want it to kick in for a few targets.
.PHONY: heredoc
heredoc: .SHELLFLAGS = -c eval
heredoc: SHELL = bash -c 'eval "$${#//\\\\/}"'
heredoc:
#echo First
#cat <<-there \
here line1 \
here anotherline \
there
#echo Last
giving
$ make
First
here line1
here anotherline
Last
Careful with that quoting, Eugene.
Note the cheat here:
I am removing all backslashes,
not just the ones at the ends of the line.
YMMV.
With GNU make, you can combine multi-line variables with the export directive to use a multi-line command without having to turn on .ONESHELL globally:
define script
cat <<'EOF'
here document in multi-line shell snippet
called from the "$#" target
EOF
endef
export script
run:; # eval "$$script"
will give
here document in multi-line shell snippet
called from the "run" target
You can also combine it with the value function to prevent its value from being expanded by make:
define _script
cat <<EOF
SHELL var expanded by the shell to $SHELL, pid is $$
EOF
endef
export script = $(value _script)
run:; # eval "$$script"
will give
SHELL var expanded by the shell to /bin/sh, pid is 12712
Not a here doc but this might be a useful workaround.
And it doesn’t require any GNU Make’isms.
Put the lines in a subshell with parens, prepend each line with echo.
You’ll need trailing sloshes and semi-colon and slosh where appropriate.
test:
( \
echo echo I Need This ;\
echo echo To Work ;\
echo ls \
) \
| sh
The following command prints the absolute path of a particular C++ header, according to where g++ believes it to be.
echo \#include\<ham/hamsterdb.h\> | g++ -M -x c++-header - | grep hamsterdb.hpp | sed -e 's/-: //' -e 's/ \\//'
On my system, this outputs: /usr/local/include/ham/hamsterdb.hpp
I run into a problem when trying to run this inside a Makefile in order to set a variable:
FILE=$(shell echo \#include\<ham/hamsterdb.h\> | g++ -M -x c++-header - | grep hamsterdb.hpp | sed -e 's/-: //' -e 's/ \\//')
.PHONY spec
spec:
#echo $(FILE)
This outputs a new line. I think it's the hash ('#') character which is messing with make; if I rewrite the FILE=... line like this:
FILE=$(shell echo \#include\<ham/hamsterdb.h\>)
the output is still nothing.
You have to escape the hash twice in order to use it inside functions: once for Make and once again for the shell.
That is,
FILE = $(shell echo \\\#include\ \<ham/hamsterdb.h\>)
Notice three backslashes instead of two as one could expect. The third backslash is needed because otherwise the first one would escape the second and the hash would be still unescaped.
UPD.
Another possible solution is to only escape the hash for Make and use Bash single quotes to prevent interpreting the hash as a shell comment. This also eliminates the need of escaping spaces, < and >:
FILE = $(shell echo '\#include <ham/hamsterdb.h>')
You need just a bit more quoting:
FILE=$(shell echo \\\#include\<ham/hamsterdb.h\> ...
^^^
Quote once for make itself, a second time for the shell (the shell needs to 'see' \#).
package_version := $(version)x0d$(date)
what is the x0d part between version and date vars? is it just string?
What $(dotin_files:.in=) does below
code
dotin_files := $(shell find . -type f -name \*.in)
dotin_files := $(dotin_files:.in=)
what this means $(dotin_files:=.in)
code
$(dotin_files): $(dotin_files:=.in)
$(substitute) $#.in > $#
can target contain multiple files?
what is the meaning of declaring target variable as PHONY?
code
.PHONY: $(dotin_files)
In the regex replacement code below
code
substitute := perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
what are $$ENV{$$1} and $$&? I guess it's Perl scope...
thanks for your time
Variable Expansion
$() is variable expansion in make, this should just be string substitution - if your makefile is
version=1
date=1.1.10
package_version:=$(version)x0d$(date)
then the variable package_version will expand to 1x0d1.1.10.
Substitution
The syntax $(var:a=b) is a substitution reference and will expand to var with a suffix a substituted with b.
For example, in
foobar:= foo bar
faabar:=$(foobar:oo=aa)
$(faabar) will expand to the string faa bar.
Multiple Targets
Multiple targets in a make rule is equivalent to having n rules with a single target, eg
foo bar:foo.c bar.c
$(CC) -o $# $^
is equivalent to
foo:foo.c bar.c
$(CC) -o $# $^
bar:foo.c bar.c
$(CC) -o $# $^
remember that any variables here are expanded.
Phony Targets
The .PHONY target declares that a rule doesn't produce an actual file, so it will always be built. As always, variables are expanded first. In your case this will expand to something like
.PHONY: foo bar
Escaping
A dollar sign is an escape character in makefiles, the $$ in your perl example is a literal $, eg substitute will be the string
perl -p -e 's/#([^#]+)#/defined $ENV{$1} ? $ENV{$1} : $&/ge'
The dollar signs here are processed by perl, and probably give environment variables (I don't know perl).
x0d part between version and date vars, is it just string?
Yes.
What $(dotin_files:.in=) does below
Removes the .in from the filenames found with the shell find.
what this means $(dotin_files:=.in)
I think you meant $(dotin_files:.in=). As already answered, within the variable dotin_files it replaces any occurrence of ".in" with an empty string(the part between the "=" and ")".
can target contain multiple files?
Yes
what is the meaning of declaring target variable as PHONY?
make will ignore targets time-stamp and consider them as new
thus rebuilding them each time.
In the regex replacement code below what are $$ENV{$$1} and $$&?
To avoid expansion of $ENV, the $ is doubled, think of '%' in C format strings, thus the string
perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
when called as a shell command will become:
perl -p -e 's/#([^#]+)#/defined $ENV{$1} ? $ENV{$1} : $&/ge'
$ENV is the perl Environment hash, $1 I think it's a backreference in the s/// regexp group.
Michael, you've been asking a lot of basic Makefile questions, and the ones you're asking now are ones you should be able to answer for yourself by experiment.
can target contain multiple files?
Try it:
dotin_files := foo.in bar.in
$(dotin_files):
#echo $#
Now try make foo.in and make bar.in. What happens?
What $(dotin_files:.in=) does
It's a substitution reference. Try it yourself and see what happens, like this:
dotin_files := foo.in bar.in
dotin_files := $(dotin_files:.in=)
all:
#echo $(dotin_files)
What did it do?
substitute := perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
what are $$ENV{$$1} and $$&? I guess it's Perl scope...
Let's take a look:
all:
#echo $(substitute)
If you want to know more about Perl, or regexs, or find, or make, or whatever, feel free to ask here, but please take a little time to try to figure it out first.