My makefile contains these snippets (among others):
SRC = src
OBJ = obj
DEPS = $(wildcard $(SRC)/*.cpp)
# ...
all : $(BINARIES)
#echo $(pathsubst $(SRC)/%.cpp,$(OBJ)/%.d,$(DEPS))
#echo $(DEPS:$(SRC)/%.cpp=$(OBJ)/%.d)
When I make all, only the second #echo outputs something:
$ make all
obj/sample1.d obj/sample1_U.d
The (gnu make) manual states:
Another type of substitution reference lets you use the full power of the patsubst function. It has the same form ‘$(var:a=b)’ described above, except that now a must contain a single ‘%’ character. This case is equivalent to ‘$(patsubst a,b,$(var))’
From this explanation, I would expect that both #echo statements produce the same output, which they clearly don't. What is wrong with the first form using the explicit pathsubst?
(I am using gnu make 3.81 on OS X.)
Presumably you want patsubst, not pathsubst.
Related
I'm trying to write a Makefile where multiple source files (in my case they are markdown) create multiple target files (pdfs). However, the target files generated have extra characters in the file name that can't be predicted (it happens to be a version number encoded in the source), but ideally the Makefile would not have to read the source itself.
So, for example:
file1.md => file1-v1.pdf
file2.md => file2-v2.pdf
...
I can calculate source name given a target name (by excluding anything after the hyphen and adding .md), but cannot calculate target name given the source.
Is it possible to write a Makefile that builds only the targets where the source have been updated?
This will be ugly, but it will work.
As it often is with Make, our problem divides into these two problems:
1. construct a list of targets
2. build them
Suppose we have five md files which map to pdf files (whose names we don't know beforehand):
file1.md => file1-v1.pdf
file2.md => file2-v1.pdf
file3.md => file3-v1.pdf
file4.md => file4-v1.pdf
file5.md => file5-v1.pdf
We can't use the real output file names as targets, because we don't know them beforehand, but we see five input files and know that we must build one output file for each. For now, a fake target name will do:
file1-dummy.pdf: file1.md
zap file1.md
When Make executes this rule, it produces the file file1-v1.pdf. The fact that it doesn't produce a file named file1-dummy.pdf is disquieting, but not a serious problem. We can turn this into a pattern rule:
%-dummy.pdf: %.md
zap $<
Then all we have to do is turn the list of existing input files (file1.md, file2.md, ...) into a list of dummy targets (file1-dummy.pdf, file2-dummy.pdf, ...), and build them. So far, so good.
But suppose some of the output files already exist. If file2-v2.pdf already exists -- and is newer than file2.md -- then we would prefer that Make not rebuild it (by attempting to build file2-dummy.pdf). In that case we would prefer that file2-v2.pdf be in the target list, with a rule that worked like this:
file2-v2.pdf: file2.md
zap $<
This is not easy to turn into a pattern rule, because Make does not handle wildcards very well, and cannot cope with multiple wildcards in a single phrase, not without a lot of clumsiness. But there is a way to write one rule that will cover both cases. First note that we can obtain the part of a variable before the hyphen with this kludge:
$(basename $(subst -,.,$(VAR)))
Armed with this, and with secondary expansion, we can write a pattern rule that will work with both cases, and construct a target list that will exploit it:
# There are other ways to construct these two lists, but this will do.
MD := $(wildcard *.md)
PDF := $(wildcard *.pdf)
PDFROOTS := $(basename $(subst -,.,$(basename $(PDF))))
MDROOTS := $(filter-out $(PDFROOTS), $(basename $(MD)))
TARGETS:= $(addsuffix -foo.pdf, $(MDROOTS)) $(PDF)
.SECONDEXPANSION:
%.pdf: $$(basename $$(subst -,., $$*)).md
# perform actions on $<
Make's algorithm always starts with the final output product and works its way backwards to the source files, to see what needs to be updated.
Therefore, you HAVE to be able to enumerate the final output product as a target name and correlate that back to the inputs that generate that output, for make to work.
This is also why make is not a great tool for building Java, for example, since the output filenames don't map easily to the input file names.
So, you must have at least one target/prerequisite pair which is derivable (for implicit rules), or state-able (for explicit rules)--that is, known at the time you write the makefile. If you don't then a marker file is your only alternative. Note you CAN add extra generated, non-derivative prerequisites (for example, in compilers you can add header files as prerequisites that are not related to the source file name), in addition to the known prerequisite.
#Beta's answer is informative and helpful, but I needed a solution (using GNU Make 4.1) that worked when the destination filename bears no resemblance to the input filename, for example, if it is generated from its content. I came up with the following, which takes every file matching *.in, and creates a file by reading the contents of the source file, appending a .txt, and using it as a filename to create. (For example, if test.in exists and contains foo, the makefile will create a foo.txt file.)
SRCS := $(wildcard *.in)
.PHONY: all
all: all_s
define TXT_template =
$(2).txt: $(1)
touch $$#
ALL += $(2).txt
endef
$(foreach src,$(SRCS),$(eval $(call TXT_template, $(src), $(shell cat $(src)))))
.SECONDARY_EXPANSION:
all_s: $(ALL)
The explanation:
The define block defines the recipe needed to make the text file from the .in file. It's a function that takes two parameters; $(1) is the .in. file and $(2) is the contents of it, or the base of the output filename. Replace touch with whatever makes the output. We have to use $$# because eval will expand everything once, but we want $# to left after this expansion. Since we have to collect all the generated targets so we known what all the make, the ALL line accumulates the targets into one variable. The foreach line goes through each source file, calls the function with the source filename and the contents of the file (i.e. what we want to be the name of the target, here you'd normally use whatever script generates the desired filename), and then evaluates the resulting block, dynamically adding the recipe to make. Thanks to Beta for explaining .SECONDARY_EXPANSION; I needed it for reasons not entirely clear to me, but it works (putting all: $(ALL) at the top doesn't work). The all: at the top depends on the secondary expansion of all_s: at the bottom and somehow this magic makes it work. Comments welcome.
maybe try this ? or something along those lines
# makefile
SRCS=$(wildcard *.md)
PDFS=$(shell printf *.pdf)
$(PDFS): $(SRCS)
command ...
the printf *.pdf is meant to either expand to the first of the pdf files if they exist, else fail if they don't and that will signal to make that it should build. if this doesn't work i suggest maybe experimenting with find, ls or other listing tools (e.g. compgen, complete), maybe even in combination with xargs to get everything on one line.
In my makefile, I am stripping the file paths from a list of objects, and replacing that path with a build directory.
Using patsubst, this seems to work fine, however using substitution replacements it doesnt seem to work, e.g:
OBJS=/path/to/obj1.o /another/path/obj2.o
BUILD_DIR=build
$(info patsubst = $(patsubst %, $(BUILD_DIR)/%, $(notdir $(OBJS))))
$(info substref = $( $(notdir $(OBJS)):%=$(BUILD_DIR)/%) )
The output of this is :
patsubst = build/obj1.o build/obj2.o
substref =
Is this a shortcoming of substitution references or am I doing something wrong?
Im using GNU Make 4.1, but would like my makefile to be valid with other/older versions.
That syntax is not right. A substitution reference requires a variable name in the first part, and make will expand that variable and the substitution will be performed on the expansion. You are giving $(notdir $(OBJS)) as the thing to substitute on, and that expands to a set of strings, not a variable name.
patsubst works fine here so why do you need to use a substitution reference?
I am currently working on a project using templates quite extensively, and running into memory constraints during instantiation. I have split up the instantiation into a number of very simple files, which are all three-liners consisting of includes only.
I let these be generated by a rule in Makefile.am. Now I have a quite long list of files that should be generated in my Makefile, and would like to refactor this list with a foreach-like expression. In more specific terms: I have a line like
libfoo_la_SOURCES = a_0.cpp a_1.cpp ... b_0.cpp b_1.cpp ... c_0.cpp c_1.cpp ...
which could be more concisely expressed as
libfoo_la_SOURCES = $(foreach i,a b ...,$(foreach j,0 1 ...,$i_$j.cpp))
However, the second construct is not only warned against by automake, but also does not work: The files given in this manner are neither compiled nor cleaned.
My current workaround is generating this file list by a shell script.
Any ideas how to implement this iteration?
I would forget about making loops: the GNU extension is not standard, and not understood by Automake. One standard (and portable) make construction you can use here is the macro expansion with substitution: $(var:subst1=subst2) will expand to the value of $(var) after replacing any suffix subst1 of a word by subst2. Automake understands this.
If subst1 is empty, as in $(var:=subst2), you are appending subst2 to all files in $(var). You can use this to construct your list of files as follows:
f = a b c d e f
g = $(f:=_0) $(f:=_1) $(f:=_2) $(f:=_3)
all_files = $(g:=.cpp)
echo:
#echo $(all_files)
Running make echo with the above Makefile will display all files from a_0.cpp to f_3.cpp.
Like you, I discovered that the GNU make foreach function will not work like this because
the sources need to be there at the time the Makefile is generated. So, I use GNU Autogen (also here) to generate a makefile fragment which is subsequently included in Makefile.am. So it's probably not that different than your shell script approach.
I'm trying to create a generic build template for my Makefiles, kind of like they discuss in the eval documentation.
I can't seem to get the wildcard function to work within an eval. The basic code I'm having issues with looks like this.
SRC_DIR = ./src/
PROG_NAME = test
define PROGRAM_template
$(1)_SRC_DIR = $(join $(SRC_DIR), $(1)/)
$(1)_SRC_FILES = $(wildcard $$($(1)_SRC_DIR)*.c)
endef
$(eval $(call PROGRAM_template, $(PROG_NAME)))
all:
#echo $(test_SRC_DIR)
#echo $(test_SRC_FILES)
#echo $(wildcard $(wildcard $(test_SRC_DIR)*.c)
When I run make with this, the output is
./src/test
[correct list of all .c files in ./src/test/]
Basically, the wildcard call within PROGRAM_template is not being eval'd as I expect it. The call results in an empty list.
The join call is being eval'd correctly though.
So, what am I doing wrong? My guess is that
$$($(1)_SRC_DIR)
is not correct, but I can't figure out the right way to do it.
EDIT
Once this was solved, it didn't take long for me to hit another issue with eval.
I posted it as a new question at
Workaround for GNU Make 3.80 eval bug
You need to double escape virtually all of the functions and variables when you use eval. In most cases, the only things that don't need to be double-escaped are function arguments (because the call function will fully expand them). In this case, you technically don't need to double-escape join or SRC_DIR either, but it will simplify your life if you just always double-escape all variables and functions when using eval.
The reason you need the double escapes is that expansion happens twice when using eval. The eval function itself performs expansion, and then expansion is done again when the block is finally parsed as makefile syntax (i.e. when it is actually evaluated).
The way you've got it written, wildcard is invoked on the string literal $( test_SRC_DIR)*.c. If you want, you can see this for yourself by replacing wildcard with info in your version and see what happens.
You need to hold off on actually invoking wildcard until the second expansion, so that it's argument is the result of the expansion of $(test_SRC_DIR).
Try this:
SRC_DIR = ./src/
PROG_NAME = test
define PROGRAM_template
$(1)_SRC_DIR = $$(join $$(SRC_DIR),$(1)/)
$(1)_SRC_FILES = $$(wildcard $$($(1)_SRC_DIR)*.c)
endef
$(eval $(call PROGRAM_template,$(PROG_NAME)))
all:
#echo $(test_SRC_DIR)
#echo $(test_SRC_FILES)
#echo $(wildcard $(test_SRC_DIR)*.c)
EDIT: After posting this, I thought I'd better test it out to make sure it actually works. In doing so, I discovered another problem. You should avoid putting spaces between the comma and argument when calling functions. It causes a literal space character to be prepended to the argument that is passed to the function and leads to unintended results. I've removed the spaces after the commas in the function calls in my version (while this isn't a problem for the call to join, I removed the space there as well just because it's a good habit to get into).
I am distributing my cpp files along with a makefile. Now the makefile is located in the same directory as the cpp file.
What is the variable (if any) in makefile that allows me to retrieve the current directory where the makefile is located? In this way I can use that variable to specify my cpp path for compilation.
My makefile is as follows:
all:
g++ ($makeFileDir)/main.cpp ($makeFileDir)/hello.cpp ($makeFileDir)/factorial.cpp -o ($makeFileDir)/hello.exe
Edit: I am running my makefiles on Windows
I remember I had the exact same problem. It's not possible, as far as I remember.
The best bet you can have is to pass it as a variable. That is both cross platform and guaranteed to work, as you know the makefile dir at invoke time (otherwise you can't invoke it).
In alternative, you can do a very dirty trick, meaning you try to combine your current path (you can obtain with $(CURDIR) in gnu make) with the path of the invocation of the makefile (which can be tricky, and depends on your make)
Here is a cross-platform way to get the directory of the Makefile, which should be fully shell-agnostic.
makeFileDir := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
Note that this will give you the directory of the Makefile being currently interpreted. You might have bad (or good!) surprises if you include a Malefile using this statement from another.
That should be enough if you use a recent implementation of make for windows, i.e. Chocolatey's.
Issues with older make for Windows
Depending on the version of make you're using on Windows, there can be inconsistencies in the handling of backslashes. You might need one of the following variant. That's the case for GnuWin's make 3.81 binary for example.
Make the path separator consistent. The statement below uses forward slashes only, just swap \ and / to get the opposite behavior. From my experience (with GnuWin's make), you might have to use forward slashes to use such a variable for make include statements or to use it in VPATH.
But you would of course need backslashes in the DOS shell, and therefore in recipes... You might need two versions of the variable, but at least the substitution makes sure that the path separator is consistent!
makeFileDir := $(subst \,/,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
The abspath function of GnuWin make 3.81 is broken and doesn't handle paths with drive letters in it. Here is a workaround to handle Windows absolute paths (with drive letter) as well. You can then use it to get the directory of the Makefile (here with the path separator substitution as well).
I won't explain the details, but the workaround simply returns the argument if that's already a Windows absolute path, i.e. if there is : in the root of the path, and uses the builtin abspath otherwise.
define fixabspath
$(if $(findstring :,$(firstword $(subst /, ,$(subst \,/,$(1))))),$\$
$(1),$\
$(abspath $(1)))
makeFileDir := $(subst \,/,$(dir $(call fixabspath,$(lastword $(MAKEFILE_LIST)))))
Remarks
There might be sources I'm omitting here and I'm sorry for that. It's been a long time ago.
In the fixabspath definition, $\ are just here to split the line for readability.
The MAKEFILE_LIST variable contains a list of the Makefiles being interpreted, the last one being the current one. See the corresponding manual page.
If I remember correctly, this also works with macOS' native make.
For 'cygwin' and 'linux' use I've solves this by calling pwd directly from the rule in the makefile:
do.%: %.cpp
echo "Running command in " `pwd`
somecommand $^
you can use $(srcdir)
then ./configure --srcdir="/your/path/to/the/source/directory"