I need to recursively get list of all subdirectories of a given top level directory. On an UNIX system, I can do it like this:
SUBDIRS := $(shell find $(TOP_DIR) -type d)
However, I need a solution which would be platform independent or at least a solution which works on Windows.
It should be possible to do using recursive wildcards (see this question) but I need to adapt this solution to work for directories instead of files.
If you do not have spaces in your directory names:
define dfind
$(foreach d,$1,$(wildcard $d**/) $(call dfind,$(wildcard $d**/)))
endef
SUBDIRS := $(call dfind,$(TOP_DIR)/)
You may try detect which shell is in use and define variables or functions depending on which. From How to detect shell used in GNU make?
# detect what shell is used
ifeq ($(findstring cmd.exe,$(SHELL)),cmd.exe)
$(info "shell Windows cmd.exe")
DEVNUL := NUL
WHICH := where
else
$(info "shell Bash")
DEVNUL := /dev/null
WHICH := which
endif
Instead (or in addition to) DEVNUL and WHICH you may define FINDDIR. Because of how the arguments work I guess functions is the way to go, see https://www.gnu.org/software/make/manual/html_node/Call-Function.html
Related
I have a folder of markdown files that are being converted to HTML files using a Makefile. To figure out the source and dest, I use the following Makefile construct
TARGETS_TO_BUILD := $(patsubst src/%.md, out/%.html, $(wildcard src/*.md src/**/*.md))
If you echo out $(TARGETS_TO_BUILD) you get a list of paths ./out/index.html ./out/folder1/somepage.html and so on. Works fine.
However, if I start putting my source files into deeper and deeper folders, such that I end up with a deep tree, things stop working. That wildcard (src/**/*.md) doesn't work. I have to start doing things like this:
TARGETS_TO_BUILD := $(patsubst src/%.md, out/%.html, $(wildcard src/*.md src/**/*.md src/**/**/*.md src/**/**/**/*.md))
I have to keep adding more and more of those.
That glob string isn't working as expected. I thought they would work with infinite depth.
You are mistaken. The docs for wildcard support are quite clear on the syntax, and ** is not listed. ** is an enhanced wildcard that is not supported by POSIX glob(3) and fnmatch(3) functions (which is what GNU make uses to implement globbing), and is not supported by POSIX sh. In fact that syntax is not even available (by default) in bash, the standard shell in most GNU/Linux and MacOS systems.
In standard glob syntax, there is no difference between * and **. In fact ***, ****, etc. all mean the same thing as *: there is no difference between matching zero or more characters once, twice, or 100 times.
If you want to recurse infinitely, the simplest thing to use is the find command via a $(shell ...) function:
TARGETS_TO_BUILD := $(patsubst src/%.md, out/%.html, $(shell find src -name '*.md' -print))
I am working on a new Makefile insfrastructure, which I've managed so far to make very portable in the sense that it works (with GNU-make):
on Linux
on Windows (only with CMD shell + GnuWin32 CoreUtils + GnuWin32 Make)
on Windows (with MSYS2 shell)
I am using canned recipes, and the recipes use arguments provided from the top. These arguments are typically header include dirs and library include dirs.
So far I've assumed that all the paths provided to the canned recipes are relative to where the Makefile resides - Within the recipes, these paths always prefixed with:
ROOT_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
Very recently, I've come across an example where the include path that I need to specify is not simple to specify in relative terms.
If coreutils "realpath --relative-to" was working on Windows I would use it, but it isn't. Also on Windows, relative paths are not always possible anyway e.g. if the include dirs and the makefile are on different drives.
So my preferred approach at the moment would be to detect in the canned recipe if the path provided as argument is absolute or relative. Only if it's relative, it gets prefixed with ROOT_DIR otherwise it's used as is.
Any suggestions how to do this, in a robust and portable way ?
We could check various cases:
leading /,
leading ~,
leading X: where X is an upper case letter (Windows drive),
leading \\ (Windows network drive)
and for each set a variable to yes or the empty string:
IS_ROOT := $(if $(patsubst /%,,$(THE_PATH_TO_CHECK)),,yes)
IS_HOME := $(if $(patsubst ~%,,$(THE_PATH_TO_CHECK)),,yes)
IS_NETWORK := $(if $(patsubst \\\\%,,$(THE_PATH_TO_CHECK)),,yes)
IS_DRIVE := $(foreach d,A B C D E...Z,$(if $(patsubst $(d):/%,,$(THE_PATH_TO_CHECK)),,yes))
Then, we can test if the concatenation of these variables equals yes or not:
ifeq ($(strip $(IS_ROOT)$(IS_HOME)$(IS_NETWORK)$(IS_DRIVE)),yes)
<absolute>
else
<relative>
endif
Of course, if you have other cases in mind you can add them using similar combinations of make functions.
I have a C++ library built using a Makefile. Until recently, all the sources were in a single directory, and the Makefile did something like this
SOURCES = $(wildcard *.cpp)
which worked fine.
Now I've added some sources that are in a subdirectory, say subdir. I know I can do this
SOURCES = $(wildcard *.cpp) $(wildcard subdir/*.cpp)
but I'm looking for a way to avoid specifying subdir manually, that is, make wildcard look into subdirectories, or generating a list of subdirectories somehow and expanding it with several wildcard functions. At this point, having a non-recursive solution (that is, expanding only the first level) would be fine.
I haven't found anything - my best guess is using find -type d to list the subdirectories, but it feels like a hack. Is there any built-in way to do this?
This should do it:
SOURCES = $(wildcard *.cpp) $(wildcard */*.cpp)
If you change you mind and want a recursive solution (i.e. to any depth), it can be done but it involves some of the more powerful Make functions. You know, the ones that allow you to do things you really shouldn't.
EDIT:
Jack Kelly points out that $(wildcard **/*.cpp) works to any depth, at least on some platforms, using GNUMake 3.81. (How he figured that out, I have no idea.)
Recursive wildcards can be done purely in Make, without calling the shell or the find command. Doing the search using only Make means that this solution works on Windows as well, not just *nix.
# Make does not offer a recursive wildcard function, so here's one:
rwildcard=$(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))
# How to recursively find all files with the same name in a given folder
ALL_INDEX_HTMLS := $(call rwildcard,foo/,index.html)
# How to recursively find all files that match a pattern
ALL_HTMLS := $(call rwildcard,foo/,*.html)
The trailing slash in the folder name is required. This rwildcard function does not support multiple wildcards the way that Make's built-in wildcard function does, but adding that support would be straightforward with a couple more uses of foreach.
If you don't want to use recursive makefiles, this might give you some ideas:
subdirs := $(wildcard */)
sources := $(wildcard $(addsuffix *.cpp,$(subdirs)))
objects := $(patsubst %.cpp,%.o,$(sources))
$(objects) : %.o : %.cpp
You can use several rules in wildcard:
SOURCES := $(wildcard *.cpp */*.cpp)
if you need more depth:
SOURCES := $(wildcard *.cpp */*.cpp */*/*.cpp */*/*/*.cpp)
Unfortunately, and unlike what we sometimes read, glob (**) is not supported by makefile and will be interpreted as normal wildcard (*).
For example **/*.cpp match dir/file.cpp but neither file.cpp nor dir/sub/file.cpp.
If you need infinite depth use shell and find:
SOURCES := $(shell find . -name "*.cpp")
Common practice is to put a Makefile in each subdir with sources, then
all: recursive
$(MAKE) -C componentX
# stuff for current dir
or
all: recursive
cd componentX && $(MAKE)
# stuff for current dir
recursive: true
It may be wise to put settings for each Makefile in a Makefile.inc in the root source directory. The recursive target forces make to go into the subdirectories. Make sure that it doesn't recompile anything in a target requiring recursive.
If you can use find shell command, you may define a function to use it.
recurfind = $(shell find $(1) -name '$(2)')
SRCS := $(call recurfind,subdir1,*.c) $(call recurfind,subdir2,*.cc) $(call recurfind,subdir2,*.cu) \
...
I am distributing my cpp files along with a makefile. Now the makefile is located in the same directory as the cpp file.
What is the variable (if any) in makefile that allows me to retrieve the current directory where the makefile is located? In this way I can use that variable to specify my cpp path for compilation.
My makefile is as follows:
all:
g++ ($makeFileDir)/main.cpp ($makeFileDir)/hello.cpp ($makeFileDir)/factorial.cpp -o ($makeFileDir)/hello.exe
Edit: I am running my makefiles on Windows
I remember I had the exact same problem. It's not possible, as far as I remember.
The best bet you can have is to pass it as a variable. That is both cross platform and guaranteed to work, as you know the makefile dir at invoke time (otherwise you can't invoke it).
In alternative, you can do a very dirty trick, meaning you try to combine your current path (you can obtain with $(CURDIR) in gnu make) with the path of the invocation of the makefile (which can be tricky, and depends on your make)
Here is a cross-platform way to get the directory of the Makefile, which should be fully shell-agnostic.
makeFileDir := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
Note that this will give you the directory of the Makefile being currently interpreted. You might have bad (or good!) surprises if you include a Malefile using this statement from another.
That should be enough if you use a recent implementation of make for windows, i.e. Chocolatey's.
Issues with older make for Windows
Depending on the version of make you're using on Windows, there can be inconsistencies in the handling of backslashes. You might need one of the following variant. That's the case for GnuWin's make 3.81 binary for example.
Make the path separator consistent. The statement below uses forward slashes only, just swap \ and / to get the opposite behavior. From my experience (with GnuWin's make), you might have to use forward slashes to use such a variable for make include statements or to use it in VPATH.
But you would of course need backslashes in the DOS shell, and therefore in recipes... You might need two versions of the variable, but at least the substitution makes sure that the path separator is consistent!
makeFileDir := $(subst \,/,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
The abspath function of GnuWin make 3.81 is broken and doesn't handle paths with drive letters in it. Here is a workaround to handle Windows absolute paths (with drive letter) as well. You can then use it to get the directory of the Makefile (here with the path separator substitution as well).
I won't explain the details, but the workaround simply returns the argument if that's already a Windows absolute path, i.e. if there is : in the root of the path, and uses the builtin abspath otherwise.
define fixabspath
$(if $(findstring :,$(firstword $(subst /, ,$(subst \,/,$(1))))),$\$
$(1),$\
$(abspath $(1)))
makeFileDir := $(subst \,/,$(dir $(call fixabspath,$(lastword $(MAKEFILE_LIST)))))
Remarks
There might be sources I'm omitting here and I'm sorry for that. It's been a long time ago.
In the fixabspath definition, $\ are just here to split the line for readability.
The MAKEFILE_LIST variable contains a list of the Makefiles being interpreted, the last one being the current one. See the corresponding manual page.
If I remember correctly, this also works with macOS' native make.
For 'cygwin' and 'linux' use I've solves this by calling pwd directly from the rule in the makefile:
do.%: %.cpp
echo "Running command in " `pwd`
somecommand $^
you can use $(srcdir)
then ./configure --srcdir="/your/path/to/the/source/directory"
I'm trying to consolidate some build information by using a common makefile. My problem is that I want to use that makefile from different subdirectory levels, which makes the working directory value (pwd) unpredictable. For example:
# Makefile.common
TOP := $(shell pwd)
COMPONENT_DIR := $(TOP)/component
COMPONENT_INC := $(COMPONENT_DIR)/include
COMPONENT_LIB := $(COMPONENT_DIR)/libcomponent.a
If I include Makefile.common from a subdirectory, like so, the $(TOP) directory is incorrect and everything else follows suit:
# other_component/Makefile
include ../Makefile.common
# $(COMPONENT_LIB) is incorrectly other_component/component
What's the best way to get Makefile.common to use its own directory path instead of the more fickle pwd?
You should be able to use the MAKEFILE_LIST variable, like this:
# This must be the first line in Makefile.common
TOP := $(dir $(firstword $(MAKEFILE_LIST)))
From the documentation:
As make reads various makefiles, including any obtained from the MAKEFILES variable, the command line, the default files, or from include directives, their names will be automatically appended to the MAKEFILE_LIST variable. They are added right before make begins to parse them. This means that if the first thing a makefile does is examine the last word in this variable, it will be the name of the current makefile. Once the current makefile has used include, however, the last word will be the just-included makefile.
Try this:
ROOT_DIR := $(dir $(realpath $(lastword $(MAKEFILE_LIST))))
Edit: Be sure to use := instead of = because the latter causes make to use late-binding and MAKEFILE_LIST may have changed due to later includes.
Have you tried doing:
# Makefile.common
TOP ?= $(shell pwd)
COMPONENT_DIR := $(TOP)/component
COMPONENT_INC := $(COMPONENT_DIR)/include
COMPONENT_LIB := $(COMPONENT_DIR)/libcomponent.a
# other_component/Makefile
TOP ?= ..
include ../Makefile.common
Using the ?= construct will keep TOP from being redefined if it is already set. You can set it to the appropriate value based on where you are in the tree when you invoke make. I confess it's been awhile since I've used GNU make so this may not work or may need some tweaks.
My solution:
cwd := $(shell readlink -en $(dir $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))))
This also works for calls like make -f /opt/some/dir/Makefile whenn your in /opt/other/path/subdir.
write the common stuff in common.mk. Then put the common.mk in the default directories that Make looks for when it encounters an include statement. See the manual for common directories Make looks for.
You could also put the common.mk in custom directory, and then type make -I customdir.
Inside the Makefile in each subfolder, you do
include common.mk
That is all. No need to worry about path and moving things around.