Makefile: couple syntax questions - syntax

package_version := $(version)x0d$(date)
what is the x0d part between version and date vars? is it just string?
What $(dotin_files:.in=) does below
code
dotin_files := $(shell find . -type f -name \*.in)
dotin_files := $(dotin_files:.in=)
what this means $(dotin_files:=.in)
code
$(dotin_files): $(dotin_files:=.in)
$(substitute) $#.in > $#
can target contain multiple files?
what is the meaning of declaring target variable as PHONY?
code
.PHONY: $(dotin_files)
In the regex replacement code below
code
substitute := perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
what are $$ENV{$$1} and $$&? I guess it's Perl scope...
thanks for your time

Variable Expansion
$() is variable expansion in make, this should just be string substitution - if your makefile is
version=1
date=1.1.10
package_version:=$(version)x0d$(date)
then the variable package_version will expand to 1x0d1.1.10.
Substitution
The syntax $(var:a=b) is a substitution reference and will expand to var with a suffix a substituted with b.
For example, in
foobar:= foo bar
faabar:=$(foobar:oo=aa)
$(faabar) will expand to the string faa bar.
Multiple Targets
Multiple targets in a make rule is equivalent to having n rules with a single target, eg
foo bar:foo.c bar.c
$(CC) -o $# $^
is equivalent to
foo:foo.c bar.c
$(CC) -o $# $^
bar:foo.c bar.c
$(CC) -o $# $^
remember that any variables here are expanded.
Phony Targets
The .PHONY target declares that a rule doesn't produce an actual file, so it will always be built. As always, variables are expanded first. In your case this will expand to something like
.PHONY: foo bar
Escaping
A dollar sign is an escape character in makefiles, the $$ in your perl example is a literal $, eg substitute will be the string
perl -p -e 's/#([^#]+)#/defined $ENV{$1} ? $ENV{$1} : $&/ge'
The dollar signs here are processed by perl, and probably give environment variables (I don't know perl).

x0d part between version and date vars, is it just string?
Yes.
What $(dotin_files:.in=) does below
Removes the .in from the filenames found with the shell find.
what this means $(dotin_files:=.in)
I think you meant $(dotin_files:.in=). As already answered, within the variable dotin_files it replaces any occurrence of ".in" with an empty string(the part between the "=" and ")".
can target contain multiple files?
Yes
what is the meaning of declaring target variable as PHONY?
make will ignore targets time-stamp and consider them as new
thus rebuilding them each time.
In the regex replacement code below what are $$ENV{$$1} and $$&?
To avoid expansion of $ENV, the $ is doubled, think of '%' in C format strings, thus the string
perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
when called as a shell command will become:
perl -p -e 's/#([^#]+)#/defined $ENV{$1} ? $ENV{$1} : $&/ge'
$ENV is the perl Environment hash, $1 I think it's a backreference in the s/// regexp group.

Michael, you've been asking a lot of basic Makefile questions, and the ones you're asking now are ones you should be able to answer for yourself by experiment.
can target contain multiple files?
Try it:
dotin_files := foo.in bar.in
$(dotin_files):
#echo $#
Now try make foo.in and make bar.in. What happens?
What $(dotin_files:.in=) does
It's a substitution reference. Try it yourself and see what happens, like this:
dotin_files := foo.in bar.in
dotin_files := $(dotin_files:.in=)
all:
#echo $(dotin_files)
What did it do?
substitute := perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
what are $$ENV{$$1} and $$&? I guess it's Perl scope...
Let's take a look:
all:
#echo $(substitute)
If you want to know more about Perl, or regexs, or find, or make, or whatever, feel free to ask here, but please take a little time to try to figure it out first.

Related

how to record exact recipe line in Makefile

I'm looking to record the exact commands used to build artifacts within a makefile. I'd like this to be stored in a file for later consumption. I am running into issues due to quotes. Basically, what I want is:
define record_and_run_recipe
#echo '$(2)' > $1
$2
endef
all:
$(call record_and_run_recipe,out.cmd,\
#echo 'hello world "$$1"' )
cat out.cmd
I would like this to output (exactly)
#echo 'hello world "$1"'
Of course, the quotes end up matching with the quotes in the expansion of the variable, and this messes everything up. (I get #echo hello world instead). Bash doesn't like '\'' either, so I can't simply do $(2:'=\'). I also seem to have issues with , characters...
I'm not looking to debug the entire makefile, just dump a couple of recipes. I'm wondering if anyone has a robust way of accomplishing this.
As I said in my comment above, you can use GNU make's $(info ...) function. It's not exactly clear from your example above what you want to do; why are you trying to put the output into a file, then cat it? Is that important?
If you can't use info, the canonical way to handle quoting in shell is to surround the string with single quotes, then replace every single quote with '\''. You say "bash doesn't like" that, but I don't know what that means. Normally you'd do something like:
define record_and_run_recipe
#echo '$(subst ','\'',$2)' > $1
$2
endef
As far as commas you will absolutely have a problem with commas if you want to use the $(call ...) function. The only way to avoid that is to put the string into a variable, like:
output = foo, bar
... $(call blah,$(output))
to "hide" the comma from call.

"%" in $(wildcard) not expanded?

I want to encode the the rule "to make <name>.done, you need all files of the pattern <name>.needed.*. I've attempted to write this with this Makefile:
%.done: $(wildcard %.needed.*)
cat $^ > $#
Yet when I run touch foo.needed.bar && make foo.done, all I get is
cat > foo.done
It appears the % inside $(wildcard) is being interpreted as a literal "%". How can I get it expanded to the right value ("foo" in this case)?
The % is just a placeholder for "any string" in pattern matching. It has no special meaning in the wildcard function and is interpreted literally.
You might attempt using $* instead (which would expand to the stem of the filename), but unfortunately it won't work either:
%.done: $(wildcard $*.needed.*)
The reason it doesn't work is that the automatic variables ($* is one of them) are not available for use in the dependency list.
The workaround is to request a secondary expansion for the target:
.SECONDEXPANSION:
%.done: $$(wildcard $$*.needed.*)
This will prompt GNU Make to go over the rule a second time after processing the Makefile as usual, expanding any escaped variables that weren't expanded the first time around. The second time around, the automatic variable have their appropriate values.

Using ifdef and ifndef directives

I'm trying to check whether a variable is defined using ifndef/ifdef, but I keep getting a not found error from the execution. I'm using GNU Make 3.81, and here is a snippet of what I have:
all: _images
$(call clean, .)
$(call compile, .)
#$(OPENER) *.pdf &
_images:
$(call clean, "images")
$(call compile, "images")
define clean
#rm -f ${1}/*.log ${1}/*.aux ${1}/*.pdf
endef
define compile
ifdef ${1}
dir = ${1}
else
dir = .
endif
ifdef ${2}
outdir = ${2}
else
outdir = ${1}
endif
#$(COMPILER) -output-directory ${outdir} ${dir}/*.tex
endef
And the exact error:
$ make
ifdef "images"
/bin/sh: 1: ifdef: not found
make: *** [_images] Error 127
Edit:
Considering Barmar comments, here goes the conclusions:
The contents of a define are shell command lines, not make directives;
to break lines inside commands within a define block, the linebreak must be escaped -- with \;
also, each block corresponding to one-liner commands is executed separately, each in a different shell execution, which means that, defining local variables won't work if the intention is to access the variable value in the next one-liner block.
Thanks tripleee for the nice work around.
You can combine the shell's facilities with Make's to get a fairly succinct definition.
define compile
#dir="${1}"; outdir="${2}"; outdir=$${outdir:-"$dir"}; \
$(COMPILER) -output-directory "$${outdir}" "$${dir:-.}/*.tex
The double-dollar is an escape which passes a single dollar sign to the shell. The construct ${variable:-value} returns the value of $variable unless it is unset or empty, in which case it returns value. Because ${1} and ${2} are replaced by static strings before the shell evaluates this expression, we have to take the roundabout route of assigning them to variables before examining them.
This also demonstrates how to combine two "one-liners" into a single shell invocation. The semicolon is a statement terminator (basically equivalent to a newline) and the sequence of a backslash and a newline causes the next line to be merged with the current line into a single "logical line".
This is complex enough that I would recommend you omit the leading # but I left it in just to show where it belongs. If you want silent operation, once you have it properly debugged, run with make -s.

Can GNU make handle filenames with spaces?

I have a directory containing several files, some of which have spaces in their names:
Test workspace/
Another directory/
file1.ext
file2.ext
demo 2012-03-23.odp
I use GNU's $(wildcard) command on this directory, and then iterate over the result using $(foreach), printing everything out. Here's the code:
FOO := $(wildcard *)
$(info FOO = $(FOO))
$(foreach PLACE,$(FOO),$(info PLACE = $(PLACE)))
Here's what I would expect to see printed out:
Test workspace
Another directory
file1.ext
file2.ext
demo 2012-03-23.odp
Here's what I would actually get:
Test
workspace
Another
directory
file1.ext
file2.ext
demo
2012-03-23.odp
The latter is obviously of no use to me. The documentation for $(wildcard) flat-out states that it returns a "space-separated list of names" but completely fails to acknowledge the huge problems this raises. Nor does the documentation for $(foreach).
Is it possible to work around this? If so, how? Renaming every file and directory to remove the spaces is not an option.
The bug #712 suggests that make does not handle names with spaces. Nowhere, never.
I found a blog post saying it's partially implemented by escaping the spaces with \ (\\ seems to be typo or formatting artefact), but:
It does not work in any functions except $(wildcard).
It does not work when expanding lists of names from variables, which includes the special variables $?, $^ and $+ as well as any user-defined variable. Which in turn means that while $(wildcard) will match correct files, you won't be able to interpret the result anyway.
So with explicit or very simple pattern rules you can get it to work, but beyond that you are out of luck. You'll have to look for some other build system that does support spaces. I am not sure whether jam/bjam does, scons, waf, ant, nant and msbuild all should work.
GNU Make does very poorly with space-separated filenames.
Spaces are used as delimiters in word list all over the place.
This blog post summarizes the situation well, but WARNING: it incorrectly uses \\ rather than \
target: some\ file some\ other\ file
some\ file some\ other\ file:
echo done
You can also use variables, so this would also work
VAR := some\ file some\ other\ file
target: $(VAR)
$(VAR):
echo done
Only the wildcard function recognizes the escaping, so you can't do anything fancy without lots of pain.
But don't forget that your shell uses spaces as delimiters too.
If I wanted to change the echo done to touch $#, I'd have to add slash to escape it for my shell.
VAR := some\ file
target: $(VAR)
$(VAR):
touch $(subst \,\\,$#)
or, more likely, use quotes
VAR := some\ file some\ other\ file
target: $(VAR)
$(VAR):
touch '$#'
In the end, if you want to avoid a lot of pain, both in GNU make, and in your shell, don't put spaces in your filenames. If you do, hopefully the limited capabilities of Make will be sufficient.
This method will also allow use of listed file names such as $? and user variables that are lists of files.
The best way to deal with spaces in Make is to substitute spaces for other characters.
s+ = $(subst \ ,+,$1)
+s = $(subst +,\ ,$1)
$(call s+,foo bar): $(call s+,bar baz) $(call s+,bar\ baz2)
# Will also shows list of dependencies with spaces.
#echo Making $(call +s,$#) from $(call +s,$?)
$(call s+,bar\ baz):
#echo Making $(call +s,$#)
$(call s+,bar\ baz2):
#echo Making $(call +s,$#)
Outputs
Making bar baz
Making bar baz2
Making foo bar from bar baz bar baz2
You can then safely manipulate lists of file names using all the GNU Make
functions. Just be sure to remove the +'s before using these names in a rule.
SRCS := a\ b.c c\ d.c e\ f.c
SRCS := $(call s+,$(SRCS))
# Can manipulate list with substituted spaces
OBJS := $(SRCS:.c=.o)
# Rule that has object files as dependencies.
exampleRule:$(call +s,$(OBJS))
# You can now use the list of OBJS (spaces are converted back).
#echo Object files: $(call +s,$(OBJS))
a\ b.o:
# a b.o rule commands go here...
#echo in rule: a b.o
c\ d.o:
e\ f.o:
Outputs
in rule: a b.o
Object files: a b.o c d.o e f.o
This info is all from the blog that everyone else was posting.
Most people seem to be recommending using no spaces in paths or using Windows 8.3 paths, but if you must use spaces, escaping spaces and substitution works.
If you are willing to rely on your shell a bit more, this gives a list which can hold names with spaces just fine:
$(shell find | sed 's: :\\ :g')
The original question said that "renaming is not an option", yet many commenters have pointed out that renaming is pretty much the only way Make can handle spaces. I suggest a middle way: Use Make to temporarily rename the files and then rename them back. This gives you all the power of Make with implicit rules and other goodness, but doesn't mess up your file naming scheme.
# Make cannot handle spaces in filenames, so temporarily rename them
nospaces:
rename -v 's/ /%20/g' *\ *
# After Make is done, rename files back to having spaces
yesspaces:
rename -v 's/%20/ /g' *%20*
You could call these targets by hand with make nospaces and make yesspaces, or you can have other targets depends on them. For example, you might want to have a "push" target which makes sure to put the spaces back in filenames before syncing files back with a server:
# Put spaces back in filenames before uploading
push: yesspaces
git push
[Sidenote: I tried the answer which suggested using +s and s+ but it made my Makefile harder to read and debug. I gave up on it when it gave me guff over implicit rules likes: %.wav : %.ogg ; oggdec "$<".]

Four Dollar signs in Makefile

I am reading the document of GNU Make. Here is an example
%.d: %.c
#set -e; rm -f $#; \
$(CC) -M $(CPPFLAGS) $< > $#.$$$$; \
sed ā€™s,\($*\)\.o[ :]*,\1.o $# : ,gā€™ < $#.$$$$ > $#; \
rm -f $#.$$$$
I tried this on a C++ program, and got the list of files
init3d.d init3d.d.18449 input.d input.d.18444 main.d main.d.18439
Here is what I found but don't understand in the same document
If you have enabled secondary expansion and you want a literal dollar sign in the prerequisites list, you must actually write four dollar signs (ā€˜$$$$ā€™).
I wonder what the four dollar signs "$$$$" mean actually? How do they 18449, 18444 or 18439?
Thanks and regards!
If make "secondary expansion" is enabled, $$$$ is required in order to generate a single $ in the actual output. $ is normally used to expand variables, call make functions, etc. $$ with secondary expansion enabled does something else, but otherwise it generates an actual $ in the output.
The shell that make uses to execute command-lines on Unix-like systems normally interprets $$ as expand to shell process ID. So, without secondary expansion enabled, $$$$ will turn into $$ in the output, which the shell will expand to the process ID.
(Using the shell process ID as a suffix is a simple way of trying to guarantee uniqueness of file name for a temporary file.)
$$ will be converted to $, but in Makefile rules (which are shell expressions) you'll have to also escape the resulting $ using a \ or by using single quotes ' around your expression.
Here is an example that demonstrates it:
DOLLAR:=$$
dollar:
echo '$$' > $#
echo "\$$" >> $#
echo '$(DOLLAR)' >> $#
echo "\$(DOLLAR)" >> $#
cat dollar
18449, 18444 or 18439 look like process ids so maybe a process ID?

Resources