I have a folder of markdown files that are being converted to HTML files using a Makefile. To figure out the source and dest, I use the following Makefile construct
TARGETS_TO_BUILD := $(patsubst src/%.md, out/%.html, $(wildcard src/*.md src/**/*.md))
If you echo out $(TARGETS_TO_BUILD) you get a list of paths ./out/index.html ./out/folder1/somepage.html and so on. Works fine.
However, if I start putting my source files into deeper and deeper folders, such that I end up with a deep tree, things stop working. That wildcard (src/**/*.md) doesn't work. I have to start doing things like this:
TARGETS_TO_BUILD := $(patsubst src/%.md, out/%.html, $(wildcard src/*.md src/**/*.md src/**/**/*.md src/**/**/**/*.md))
I have to keep adding more and more of those.
That glob string isn't working as expected. I thought they would work with infinite depth.
You are mistaken. The docs for wildcard support are quite clear on the syntax, and ** is not listed. ** is an enhanced wildcard that is not supported by POSIX glob(3) and fnmatch(3) functions (which is what GNU make uses to implement globbing), and is not supported by POSIX sh. In fact that syntax is not even available (by default) in bash, the standard shell in most GNU/Linux and MacOS systems.
In standard glob syntax, there is no difference between * and **. In fact ***, ****, etc. all mean the same thing as *: there is no difference between matching zero or more characters once, twice, or 100 times.
If you want to recurse infinitely, the simplest thing to use is the find command via a $(shell ...) function:
TARGETS_TO_BUILD := $(patsubst src/%.md, out/%.html, $(shell find src -name '*.md' -print))
Related
Suppose I have the following directory structure, with a root-node:
/root/
Makefile
branch1/
branch2/
.../
And I write the following minimal makefile:
branches=$(shell find * -maxdepth 1 -type d -printf "%f "}
%:
-${MAKE} ${MAKECMDGOALS} -C branch1
-${MAKE} ${MAKECMDGOALS} -C branch2
...
Having prepared to dynamically perform this relay with the branches variable and having tried several wild-card and descendant rule variations without success, my question amounts to: how do I capture any command goal from outer scope (like I am doing now), and perform the make relay for each of the files my find expression detects?
In pseudo code (did not work with my version of make, which is the latest greatest available on Cygwin):
branches=$(shell find * -maxdepth 1 -type d -printf "%f "}
branch-%:
-${MAKE} ${MAKECMDGOALS} -C $*
%: ${foreach branch,${branches}, branch-${branch}}
Unlike the original makefile, this does not work. However, it seems like it should. Is there a way to do this?
And there is a second issue
Make's parallelism will be broken by my pseudo-code method (if it worked) with an exponential fan out using the -j option, whereas the first method I used will not break parallelism.
However, ideally, this makefile should be able to dynamically execute one make relay for each file in the branches list. However, I don't currently see a way to implement this dynamically.
First, I'm not sure why you're using a complex shell function; why not just:
branches := $(wildcard */.)
Or, if you don't want the /. at the end:
branches := $(patsubst %/.,%,$(wildcard */.))
Second, the reason your second attempt doesn't work is that it's not valid to create a pattern rule with no recipe. See Canceling Pattern Rules.
Instead, you can use the .DEFAULT special target. It would look something like this:
branch-%:
-${MAKE} -C $* $(CMD)
.DEFAULT:
#$(MAKE) CMD=$# ${addprefix branch-,${branches}}
This does use recursive make and it behaves slightly differently than your original.
I'm not sure I understand your second point about -j. GNU make (unless you're using a truly old version) can communicate among all the submakes to ensure they are starting as many, but not more, jobs than you requested.
Oh, I forgot, there's another obvious way to do it if you don't want to use .DEFAULT and recursion:
$(MAKECMDGOALS): $(addprefix branch-,$branches))
I am trying to generate list of object files from source files in my makefile using patsubst
OUT_DIR=Out/
SRC=../../../Client2.4/Client/src/BrokerModule/BrokerApp.cpp
../../../Client2.4/Client/src/CommandMsgManager/CConfigModuleInfo.cpp
OBJ:= $(patsubst %src/%.cpp,${OUT_DIR}$%.o,$(SRC))
I want my OBJ variable to be
OBJ=Out/BrokerModule/BrokerApp.o Out/CommandMsgManager/CConfigModuleInfo.o
after patsubst but above patsubst is not producing the desired result. Please help.
There are some problems with the usage of patsubst, see my suggestion as followed,
OUT_DIR=Out/
SRC=../../../Client2.4/Client/src/BrokerModule/BrokerApp.cpp \
../../../Client2.4/Client/src/CommandMsgManager/CConfigModuleInfo.cpp
# add the definition of src
src=../../../Client2.4/Client/src/
# Modify the definition of OBJ
OBJ:= $(patsubst ${src}%.cpp,${OUT_DIR}%.o,$(SRC))
Filtered out the prepended ${src} and appended .cpp, and keep only
BrokerModule/BrokerApp.cpp & CommandMsgManager/CConfigModuleInfo.cpp.
And % is replaced by the text that matched the % in the previous step.
Patsubst can only handle patterns with one wildcard in it, unluckily. Moreover you are trying to take apart path names not the usual way at the file level. That means, as long as you neither know the prefix nor the postfix parts of the /src/ in your strings, you are out of luck as you can never say 'replace unknown prefix and conserve unknown postfix' (or the other way round).
The usual solution is to 'know' the prefix:
OUT_DIR=Out/
SRC_PATH := ../../../Client2.4/Client/src
SRC=../../../Client2.4/Client/src/BrokerModule/BrokerApp.cpp \
../../../Client2.4/Client/src/CommandMsgManager/CConfigModuleInfo.cpp
OBJ:= $(patsubst $(SRC_PATH)/%,${OUT_DIR}%,$(SRC))
$(info $(OBJ))
Another solution is to use e.g. the GNUmake table toolkit library of make functions (still beta but your problem can be solved):
include gmtt.mk
OUT_DIR=Out
SRC=../../../Client2.4/Client/src/BrokerModule/BrokerApp.cpp \
../../../Client5.6/Client/src/CommandMsgManager/CConfigModuleInfo.cpp
strip-till-last-src = src/$(call implode,$(call down-to,src/,$(call explode,/,$1)))
OBJ:= $(foreach a-path,$(SRC),$(OUT_DIR)/$(call strip-till-last-src,$(a-path)))
$(info $(OBJ))
I've git a project where, at some point in its Makefile, I'm filtering out stuff from a certain directory:
relevant = $(filter-out irrelevant/%,$^)
Now I want to use this in a VPATH-enabled environment. So the paths of my dependencies in $^ might not start with irrelevant any more, but instead something like ../src/irrelevant or similar.
Is there a way to filter-out anything that contains irrelevant, in any position? I.e. something like the following?
relevant = $(filter-out %irrelevant/%,$^)
This doesn't work, since apparently patterns for filter-out can contain only a single % wildcard. I know I could possibly achieve this via a shell invocation, grep or whatever, but I was hoping for some combinations of functions inside the Makefile.
Try
relevant = $(foreach a,$^,$(if $(findstring irrelevant,$a),,$a))
I have a list of filenames:
FILES := a.b c.d e.f
and I want to remove the extensions (suffixes) of all words to obtain:
a c e
what is the best way to do that?
The best I could come up with was "cheating" with shell:
$(shell for f in $(INS_NODIR); do echo -n "$${f%.*} "; done )
but I am surprised there was not a more "built-in" way of doing this only with make built-in functions.
thing I tried:
patsubst. It seems that it can only have one single wildcard, others being treated literally, and I'd like to do something like %.%, %
looking for a notsufix function.
I was surprised that this does not exist, since the dir function has notdir counterpart, but the suffix function that exactly extracts extensions does not have a notsuffix counterpart
Simple, just:
NAMES = $(basename $(FILES))
See the GNU make manual section on Functions for File Names
I am distributing my cpp files along with a makefile. Now the makefile is located in the same directory as the cpp file.
What is the variable (if any) in makefile that allows me to retrieve the current directory where the makefile is located? In this way I can use that variable to specify my cpp path for compilation.
My makefile is as follows:
all:
g++ ($makeFileDir)/main.cpp ($makeFileDir)/hello.cpp ($makeFileDir)/factorial.cpp -o ($makeFileDir)/hello.exe
Edit: I am running my makefiles on Windows
I remember I had the exact same problem. It's not possible, as far as I remember.
The best bet you can have is to pass it as a variable. That is both cross platform and guaranteed to work, as you know the makefile dir at invoke time (otherwise you can't invoke it).
In alternative, you can do a very dirty trick, meaning you try to combine your current path (you can obtain with $(CURDIR) in gnu make) with the path of the invocation of the makefile (which can be tricky, and depends on your make)
Here is a cross-platform way to get the directory of the Makefile, which should be fully shell-agnostic.
makeFileDir := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
Note that this will give you the directory of the Makefile being currently interpreted. You might have bad (or good!) surprises if you include a Malefile using this statement from another.
That should be enough if you use a recent implementation of make for windows, i.e. Chocolatey's.
Issues with older make for Windows
Depending on the version of make you're using on Windows, there can be inconsistencies in the handling of backslashes. You might need one of the following variant. That's the case for GnuWin's make 3.81 binary for example.
Make the path separator consistent. The statement below uses forward slashes only, just swap \ and / to get the opposite behavior. From my experience (with GnuWin's make), you might have to use forward slashes to use such a variable for make include statements or to use it in VPATH.
But you would of course need backslashes in the DOS shell, and therefore in recipes... You might need two versions of the variable, but at least the substitution makes sure that the path separator is consistent!
makeFileDir := $(subst \,/,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
The abspath function of GnuWin make 3.81 is broken and doesn't handle paths with drive letters in it. Here is a workaround to handle Windows absolute paths (with drive letter) as well. You can then use it to get the directory of the Makefile (here with the path separator substitution as well).
I won't explain the details, but the workaround simply returns the argument if that's already a Windows absolute path, i.e. if there is : in the root of the path, and uses the builtin abspath otherwise.
define fixabspath
$(if $(findstring :,$(firstword $(subst /, ,$(subst \,/,$(1))))),$\$
$(1),$\
$(abspath $(1)))
makeFileDir := $(subst \,/,$(dir $(call fixabspath,$(lastword $(MAKEFILE_LIST)))))
Remarks
There might be sources I'm omitting here and I'm sorry for that. It's been a long time ago.
In the fixabspath definition, $\ are just here to split the line for readability.
The MAKEFILE_LIST variable contains a list of the Makefiles being interpreted, the last one being the current one. See the corresponding manual page.
If I remember correctly, this also works with macOS' native make.
For 'cygwin' and 'linux' use I've solves this by calling pwd directly from the rule in the makefile:
do.%: %.cpp
echo "Running command in " `pwd`
somecommand $^
you can use $(srcdir)
then ./configure --srcdir="/your/path/to/the/source/directory"