Given a colon-delimited list of paths, getting a space-delimited list with GNU Make is straightforward:
CPATHS := /usr/bin/foo:/usr/bin/baz:/usr/bin/baz
SPATHS := $(subst :, ,$(CPATHS))
However, I couldn't find a nice way to go the opposite direction. The following hack does work (at least if sed is installed) but I'm pretty sure there will be a nicer way to solve this just using Make's internal functions.
SPATHS := /usr/bin/foo /usr/bin/baz /usr/bin/baz
CPATHS := $(shell echo $(SPATHS) > tmp; sed 's/ \+/:/g' tmp; rm tmp)
The only tricky part here is to define a literal space:
space := $(subst ,, )
SPATHS := /usr/bin/foo /usr/bin/baz /usr/bin/baz
CPATHS := $(subst $(space),:,$(SPATHS))
The shortest way to get a literal space would be via $() $(). Thus:
$(subst $() $(),:,$(CPATHS))
Or, for brevity:
_=$() $()
$(subst $(_),:,$(CPATHS))
It is perhaps an interesting curiosity that the same trick works with cmake's macros, i.e. that ${} is a separator but introduces no whitespace by itself.
Related
Here's what I am trying to do:
EXTRA_INCLUDE_PATHS = ../dir1/path with spaces/ \
../dir2/other path with spaces/
CPPFLAGS += $(addprefix -I,$(EXTRA_INCLUDE_PATHS))
I want to get “-I../dir1/path with spaces/ […]”, but I get instead: “-I../dir/path -Iwith -Ispaces/ […]”.
How do I espace spaces in addprefix? I've tried doing this trick, but it produces same result:
space =
space +=
#produces “-Isome -Ipath”
CPPFLAGS += $(addprefix -I,some$(space)path)
Don't do it! As #MadScientist points out, you need to eschew all space-containing filenames in a makefile unless you want to be very sad. A space is a list separator (including in lists of targets), and there is just no way around that. Use symbolic links in your file system to avoid them! (mklink on windows, or use cygwin make which understands cygwin symbolic links).
That said, in the current case (where you are not defining a list of targets, but merely a list of include dirs), you could use a single character to represent a space, only translating it right at the end.
EXTRA_INCLUDE_PATHS = ../dir1/path|with|spaces/ ../dir2/other|path|with|spaces/
CPPFLAGS += $(subst |, ,$(patsubst %,-I'%',${EXTRA_INCLUDE_PATHS}))
Check the result:
$(error [${CPPFLAGS}])
gives Makefile:3: *** [-I'../dir1/path with spaces/' -I'../dir2/other path with spaces/']. Stop.. Here, $CPPFLAGS is in sh format—the spaces are quoted with ' so that the compiler sees each -I as a single parameter. make simply does not have this level of quoting.
If all your include dirs start with the same character sequence, you can exploit this for use with the substitute command:
CPPFLAGS += $(subst ..,-I ..,$(EXTRA_INCLUDE_PATHS))
Check the result with:
$(info ${CPPFLAGS})
(this is a slightly different (maybe more elegant) version of #bobbogo's answer)
If you escape spaces with a backslash (\ ), you can replace the escaped space by a pipe symbol (|), add the prefix and then undo the replace operation:
EXTRA_INCLUDE_PATHS = ../dir1/path\ with\ spaces/ ../dir2/other\ path\ with spaces/
CPPFLAGS += $(subst |,\ ,$(addprefix -I,$(subst \ ,|,${EXTRA_INCLUDE_PATHS})))
I am trying to use makefile "subst" function to split the text. Here is my code.
$(subst :,\nvalue:,peter:value:2)
what I want to do is that is to split from the first of occurrence of the ':'. but it also splits on the second occurrence. Can someone help me how to solve that issue. the result she be similar to below.
peter\nvalue:value:2
As the documentation states, the subst function replaces every occurrence of the original text with the new text.
If you want only the first one you'll have to get a lot more fancy, if you want to do it completely within make and you don't know anything about the text before or after the first colon. Something like this should work:
VAL := peter:value:2
NEW := $(patsubst $(firstword $(subst :, ,$(VAL)))%,$(firstword $(subst :, ,$(VAL)))\nvalue%,$(VAL))
There might be a simpler way to do it; I'll have to think about it.
If you don't mind having make invoke a shell to do this, you could do:
VAL := peter:value:2
NEW := $(shell echo "$(VAL)" | sed -e 's/:/\\nvalue:/')
I'm not sure if your \n is a literal "\n" or a newline. I'm assuming the former. This means the \n in the sed expression needs to be escaped as \\n.
Another way of doing it completely within make (without any shell invocations):
empty:=
space:= $(empty) $(empty)
VAL := peter:value:2
VL := $(subst :, ,$(VAL))
NEW := $(firstword $(VL))\nvalue:$(subst $(space),:,$(wordlist 2,$(words $(VL)),$(VL)))
Caveat: This method will give incorrect results if the original value contains whitespace.
I have a directory containing several files, some of which have spaces in their names:
Test workspace/
Another directory/
file1.ext
file2.ext
demo 2012-03-23.odp
I use GNU's $(wildcard) command on this directory, and then iterate over the result using $(foreach), printing everything out. Here's the code:
FOO := $(wildcard *)
$(info FOO = $(FOO))
$(foreach PLACE,$(FOO),$(info PLACE = $(PLACE)))
Here's what I would expect to see printed out:
Test workspace
Another directory
file1.ext
file2.ext
demo 2012-03-23.odp
Here's what I would actually get:
Test
workspace
Another
directory
file1.ext
file2.ext
demo
2012-03-23.odp
The latter is obviously of no use to me. The documentation for $(wildcard) flat-out states that it returns a "space-separated list of names" but completely fails to acknowledge the huge problems this raises. Nor does the documentation for $(foreach).
Is it possible to work around this? If so, how? Renaming every file and directory to remove the spaces is not an option.
The bug #712 suggests that make does not handle names with spaces. Nowhere, never.
I found a blog post saying it's partially implemented by escaping the spaces with \ (\\ seems to be typo or formatting artefact), but:
It does not work in any functions except $(wildcard).
It does not work when expanding lists of names from variables, which includes the special variables $?, $^ and $+ as well as any user-defined variable. Which in turn means that while $(wildcard) will match correct files, you won't be able to interpret the result anyway.
So with explicit or very simple pattern rules you can get it to work, but beyond that you are out of luck. You'll have to look for some other build system that does support spaces. I am not sure whether jam/bjam does, scons, waf, ant, nant and msbuild all should work.
GNU Make does very poorly with space-separated filenames.
Spaces are used as delimiters in word list all over the place.
This blog post summarizes the situation well, but WARNING: it incorrectly uses \\ rather than \
target: some\ file some\ other\ file
some\ file some\ other\ file:
echo done
You can also use variables, so this would also work
VAR := some\ file some\ other\ file
target: $(VAR)
$(VAR):
echo done
Only the wildcard function recognizes the escaping, so you can't do anything fancy without lots of pain.
But don't forget that your shell uses spaces as delimiters too.
If I wanted to change the echo done to touch $#, I'd have to add slash to escape it for my shell.
VAR := some\ file
target: $(VAR)
$(VAR):
touch $(subst \,\\,$#)
or, more likely, use quotes
VAR := some\ file some\ other\ file
target: $(VAR)
$(VAR):
touch '$#'
In the end, if you want to avoid a lot of pain, both in GNU make, and in your shell, don't put spaces in your filenames. If you do, hopefully the limited capabilities of Make will be sufficient.
This method will also allow use of listed file names such as $? and user variables that are lists of files.
The best way to deal with spaces in Make is to substitute spaces for other characters.
s+ = $(subst \ ,+,$1)
+s = $(subst +,\ ,$1)
$(call s+,foo bar): $(call s+,bar baz) $(call s+,bar\ baz2)
# Will also shows list of dependencies with spaces.
#echo Making $(call +s,$#) from $(call +s,$?)
$(call s+,bar\ baz):
#echo Making $(call +s,$#)
$(call s+,bar\ baz2):
#echo Making $(call +s,$#)
Outputs
Making bar baz
Making bar baz2
Making foo bar from bar baz bar baz2
You can then safely manipulate lists of file names using all the GNU Make
functions. Just be sure to remove the +'s before using these names in a rule.
SRCS := a\ b.c c\ d.c e\ f.c
SRCS := $(call s+,$(SRCS))
# Can manipulate list with substituted spaces
OBJS := $(SRCS:.c=.o)
# Rule that has object files as dependencies.
exampleRule:$(call +s,$(OBJS))
# You can now use the list of OBJS (spaces are converted back).
#echo Object files: $(call +s,$(OBJS))
a\ b.o:
# a b.o rule commands go here...
#echo in rule: a b.o
c\ d.o:
e\ f.o:
Outputs
in rule: a b.o
Object files: a b.o c d.o e f.o
This info is all from the blog that everyone else was posting.
Most people seem to be recommending using no spaces in paths or using Windows 8.3 paths, but if you must use spaces, escaping spaces and substitution works.
If you are willing to rely on your shell a bit more, this gives a list which can hold names with spaces just fine:
$(shell find | sed 's: :\\ :g')
The original question said that "renaming is not an option", yet many commenters have pointed out that renaming is pretty much the only way Make can handle spaces. I suggest a middle way: Use Make to temporarily rename the files and then rename them back. This gives you all the power of Make with implicit rules and other goodness, but doesn't mess up your file naming scheme.
# Make cannot handle spaces in filenames, so temporarily rename them
nospaces:
rename -v 's/ /%20/g' *\ *
# After Make is done, rename files back to having spaces
yesspaces:
rename -v 's/%20/ /g' *%20*
You could call these targets by hand with make nospaces and make yesspaces, or you can have other targets depends on them. For example, you might want to have a "push" target which makes sure to put the spaces back in filenames before syncing files back with a server:
# Put spaces back in filenames before uploading
push: yesspaces
git push
[Sidenote: I tried the answer which suggested using +s and s+ but it made my Makefile harder to read and debug. I gave up on it when it gave me guff over implicit rules likes: %.wav : %.ogg ; oggdec "$<".]
I'm dynamically generating config.mk with a bash script which will be used by a Makefile. The file is constructed with:
cat > config.mk <<CFG
SOMEVAR := $value_from_bash1
ANOTHER := $value_from_bash2
CFG
How do I ensure that the generated file really contains the contents of $value_from_bash*, and not something expanded / interpreted? I probably need to escape $ to $$ and \ to \\, but are there other characters that needs to be escaped? Perhaps there is a special literal assignment I've not heard of?
Spaces seems to be troublesome too:
$ ls -1
a b
a
$ cat Makefile
f := a b
default_target:
echo "$(firstword $(wildcard ${f}))"
$ make
a
If I use f := a\ b it works (using quotes like f := 'a b' did not work either, makefile just treats it as a regular character)
Okay, it turned out that Makefiles need little escaping for itself, but the commands which are executed by the shell interpreter need to be escaped.
Characters which have a special meaning in Makefile and that need to be escaped are:
sharp (#, comment) becomes \#
dollar ($, begin of variable) becomes $$
Newlines cannot be inserted in a variable, but to avoid breaking the rest of the Makefile, prepend it with a backslash so the line break will be ignored.
Too bad a backslash itself cannot be escaped (\\ will still be \\ and not \ as you might expect). This makes it not possible to put a literal slash on the end of a string as it will either eat the newline or the hash of a following comment. A space can be put on the end of the line, but that'll also be put in the variable itself.
The recipe itself is interpreted as a shell command, without any fancy escaping, so you've to escape data yourself, just imagine that you're writing a shellscript and inserting the variables from other files. The strategy here would be putting the variables between single quotes and escape only ' with '\'' (close the string, insert a literal ' and start a new string). Example: mornin' all becomes 'morning'\'' all' which is equivalent to "morning' all".
The firstword+wildcard issue is caused by the fact that filenames with spaces in them are treated as separate filenames by firstword. Furthermore, wildcard expands escapes using \ so x\ y is matches as one word, x y and not two words.
It seems that the full answer to this question is found nowhere on the internet, so I finally sat down and figured it out for the Windows case.
Specifically, the "Windows case" refers to file names that are valid in Windows, meaning that they do not contain the characters \, /, *, ?, ", ^, <, >, |, or line breaks. It also means \ and / are both considered valid directory separators for the purposes of Make.
An example will clear it up better than I can explain. Basically, if you are trying to match this file path:
Child\a$b {'}(a.o#$#,&+=~`),[].c
Then you have to write these rules:
all: Child\\a$$b\\\ \\\ {'}(a.o\#$$#,&+=~`),[].o
%.o: %.c
$(CC) '$(subst ','"'"',$(subst \,,$(subst \\,/,$+)))'
Stare at it for a long time and it'll sort of start making some remote sense.
This works in my MSYS2 environment, so I presume it is correct.
I don't see how that makefile can work as you say. A pattern rule cannot be the default.
You're missing a `$` in `$(wildcard ...)`, so I think you haven't posted what you're really testing.
You should escape newlines too.
package_version := $(version)x0d$(date)
what is the x0d part between version and date vars? is it just string?
What $(dotin_files:.in=) does below
code
dotin_files := $(shell find . -type f -name \*.in)
dotin_files := $(dotin_files:.in=)
what this means $(dotin_files:=.in)
code
$(dotin_files): $(dotin_files:=.in)
$(substitute) $#.in > $#
can target contain multiple files?
what is the meaning of declaring target variable as PHONY?
code
.PHONY: $(dotin_files)
In the regex replacement code below
code
substitute := perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
what are $$ENV{$$1} and $$&? I guess it's Perl scope...
thanks for your time
Variable Expansion
$() is variable expansion in make, this should just be string substitution - if your makefile is
version=1
date=1.1.10
package_version:=$(version)x0d$(date)
then the variable package_version will expand to 1x0d1.1.10.
Substitution
The syntax $(var:a=b) is a substitution reference and will expand to var with a suffix a substituted with b.
For example, in
foobar:= foo bar
faabar:=$(foobar:oo=aa)
$(faabar) will expand to the string faa bar.
Multiple Targets
Multiple targets in a make rule is equivalent to having n rules with a single target, eg
foo bar:foo.c bar.c
$(CC) -o $# $^
is equivalent to
foo:foo.c bar.c
$(CC) -o $# $^
bar:foo.c bar.c
$(CC) -o $# $^
remember that any variables here are expanded.
Phony Targets
The .PHONY target declares that a rule doesn't produce an actual file, so it will always be built. As always, variables are expanded first. In your case this will expand to something like
.PHONY: foo bar
Escaping
A dollar sign is an escape character in makefiles, the $$ in your perl example is a literal $, eg substitute will be the string
perl -p -e 's/#([^#]+)#/defined $ENV{$1} ? $ENV{$1} : $&/ge'
The dollar signs here are processed by perl, and probably give environment variables (I don't know perl).
x0d part between version and date vars, is it just string?
Yes.
What $(dotin_files:.in=) does below
Removes the .in from the filenames found with the shell find.
what this means $(dotin_files:=.in)
I think you meant $(dotin_files:.in=). As already answered, within the variable dotin_files it replaces any occurrence of ".in" with an empty string(the part between the "=" and ")".
can target contain multiple files?
Yes
what is the meaning of declaring target variable as PHONY?
make will ignore targets time-stamp and consider them as new
thus rebuilding them each time.
In the regex replacement code below what are $$ENV{$$1} and $$&?
To avoid expansion of $ENV, the $ is doubled, think of '%' in C format strings, thus the string
perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
when called as a shell command will become:
perl -p -e 's/#([^#]+)#/defined $ENV{$1} ? $ENV{$1} : $&/ge'
$ENV is the perl Environment hash, $1 I think it's a backreference in the s/// regexp group.
Michael, you've been asking a lot of basic Makefile questions, and the ones you're asking now are ones you should be able to answer for yourself by experiment.
can target contain multiple files?
Try it:
dotin_files := foo.in bar.in
$(dotin_files):
#echo $#
Now try make foo.in and make bar.in. What happens?
What $(dotin_files:.in=) does
It's a substitution reference. Try it yourself and see what happens, like this:
dotin_files := foo.in bar.in
dotin_files := $(dotin_files:.in=)
all:
#echo $(dotin_files)
What did it do?
substitute := perl -p -e 's/#([^#]+)#/defined $$ENV{$$1} ? $$ENV{$$1} : $$&/ge'
what are $$ENV{$$1} and $$&? I guess it's Perl scope...
Let's take a look:
all:
#echo $(substitute)
If you want to know more about Perl, or regexs, or find, or make, or whatever, feel free to ask here, but please take a little time to try to figure it out first.