I'm using this rwildcard make function (taken from https://stackoverflow.com/a/18258352)
ifeq ($(OS),Windows_NT)
SHELL := cmd
endif
rwildcard=$(foreach d,$(wildcard $(1:=/*)),$(call rwildcard,$d,$2) $(filter $(subst *,%,$2),$d))
INPUT_JavaFileStorageTest-AS = $(call rwildcard,src/java/android-filechooser-AS/app/src,*) $(call rwildcard,src/java/JavaFileStorage/app/src,*) $(call rwildcard,src/java/JavaFileStorageTest-AS/app/src,*.java)
However, I noticed that:
while on linux this is rather fast (cannot perceive the duration)
on Windows this is actually very slow. (10 seconds)
[SHELL is voluntarily set to cmd on windows because the user might not have a POSIX shell in its path].
Any idea why this? How can I improve this?
This should work with both GNU make 3.x & 4.x (because on macOS it is version 3.x that is shipped in the devel command line tools, there is no 4.x there)
EDIT
After investigation the problem wasn't the recursive wildcard. I could speed up by running make --no-builtin-rules or adding MAKEFLAGS += --no-builtin-rules in the Makefile, or adding .SUFFIXES: (with empty value). This removed all the lag observed on Windows.
There is a wildcard-rec function in gmtt which maybe does what you want. The big upside of it is that it is platform-agnostic.
include gmtt.mk
INPUT_JavaFileStorageTest-AS := $(call wildcard-rec,src/java/android-filechooser-AS/app/src/**) \
$(call wildcard-rec,src/java/JavaFileStorage/app/src/**) \
$(call wildcard-rec,src/java/JavaFileStorageTest-AS/app/src/**.java)
** is the recursive-descent glob code which means that the routine will step down into all subdirectories.
Related
I need to recursively get list of all subdirectories of a given top level directory. On an UNIX system, I can do it like this:
SUBDIRS := $(shell find $(TOP_DIR) -type d)
However, I need a solution which would be platform independent or at least a solution which works on Windows.
It should be possible to do using recursive wildcards (see this question) but I need to adapt this solution to work for directories instead of files.
If you do not have spaces in your directory names:
define dfind
$(foreach d,$1,$(wildcard $d**/) $(call dfind,$(wildcard $d**/)))
endef
SUBDIRS := $(call dfind,$(TOP_DIR)/)
You may try detect which shell is in use and define variables or functions depending on which. From How to detect shell used in GNU make?
# detect what shell is used
ifeq ($(findstring cmd.exe,$(SHELL)),cmd.exe)
$(info "shell Windows cmd.exe")
DEVNUL := NUL
WHICH := where
else
$(info "shell Bash")
DEVNUL := /dev/null
WHICH := which
endif
Instead (or in addition to) DEVNUL and WHICH you may define FINDDIR. Because of how the arguments work I guess functions is the way to go, see https://www.gnu.org/software/make/manual/html_node/Call-Function.html
I'm trying to use find_j=$(findstring j,$(filter-out --%,$(MAKEFLAGS))) to find if there is -j option, so when I echo $(find_j) the value is j
but when I compare it ifeq (j, $(find_j)) this returnes false
I cant understand where is the problem
my version of make is make-3.99.90
find_j=$(findstring -j,$(filter-out --%,$(MAKEFLAGS)))
ifneq ( , $(find_j))
PARALLEL_ENABLED=true
endif
.PHONY: PRINT
PRINT:
$(info $(PARALLEL_ENABLED))
$(info $(MAKEFLAGS))
$(info $(find_j))
---empty line---
--warn-undefined-variables -ws --jobserver-fds=5,6 -j
-j
One thing to note is that the release of GNU make you're using is a beta release of GNU make 4.0, which itself was released in 2013... so you're using a beta of a release that itself is 8.5 years old.
However, that's not related to this problem.
The issue is that the MAKEFLAGS variable's final value is not set until after all makefiles are parsed. If you try to examine it before all makefiles are parsed, it will contain only a subset of the total set of options.
When you expand that variable as part of an ifeq or ifneq statement, that happens as the makefile is being parsed and so (as per the above) only the simple options (ones that don't take an argument: -j accepts an argument so is not "simple") are available.
When you expand the variable as part of a recipe, that happens after all makefiles are parsed: at that time the final value of MAKEFLAGS is set. So your $(info ...) functions inside the recipe do the right thing.
This is easy to see:
$(info no recipe MAKEFLAGS is '$(MAKEFLAGS)')
all: ; $(info in recipe MAKEFLAGS is '$(MAKEFLAGS)')
If you run with -j10 you'll get:
no recipe MAKEFLAGS is ''
in recipe MAKEFLAGS is '-j10 --jobserver-auth=3,4'
(your "in recipe" flags might look different because you're using such an old version of GNU make).
In the next release of GNU make, the value of MAKEFLAGS is kept up-to-date constantly so you can check it at any time and it will be accurate. But that release is not available yet.
This seems to be imprecisely documented. While MAKEFLAGS has the flags like e.g. -s and -k as ks in it, the -j flag gets processed in another way: it is not stripped of the leading dash - AND it is not visible in the first pass of processing the makefile. Only when rules are executed, MAKEFLAGS receives a value, albeit a processed form of the one you gave. -j3 elicits a -j3 --jobserver-auth=3,4 response from the command line transcriber of make, while -j stays -j. So what does this mean for us? Obviously the feature to detect the requested parallelism at runtime is not stable or there are some good reasons not to access them (which is the case most of the time when you encounter exceptional behaviour in GNU tools), so maybe you can give us more information on what you are trying to achieve - maybe there is a way to circumvent accessing the command line.
I have a makefile which calls itself, in order to obtain a license for the compiler before compiling anything, and release the license even when compilation fails, like this:
.PHONY main_target
main_target:
#license_grab &
#sleep 2
-#$(MAKE) real_target
#license_release
This works great if the makefile is named "makefile". But if I make a copy of the makefile to experiment with something, and invoke it with make -f makefile_copy, then the wrong makefile gets used in the recursive call. How do I prevent this without hard-coding the makefile name in the makefile itself?
Edit: Unfortunately I'm stuck using GNU Make version 3.79.1, so I cannot use MAKEFILE_LIST, which was apparently introduced in version 3.80. Therefore none of the answers in this question will work for me.
You can use the MAKEFILE_LIST variable:
THIS_MAKEFILE := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
.PHONY main_target
main_target:
#license_grab &
#sleep 2
-#$(MAKE) -f $(THIS_MAKEFILE) real_target
#license_release
You can set the MAKE variable outside the makefile, to include the makefile name (unless of course, it gets overridden). Something like this (for bash):
MAKE="make -f makefile_copy" make -e -f makefile_copy
or this (in pretty much any shell):
make MAKE="make -f makefile_copy" -f makefile_copy
I'm trying to debug makefiles for a large project and I'm struggling define TEMPLATE/endef and foreach/eval/call constructs. In particular I think I'm having a hard time figuring out which variables I need to reference with $ and which I need to reference with $$.
I think would be easier for me to debug if I could see the actual results of the the eval/call expansion, before variable expansion.
For instance if we use the example in the eval documentation for gnu-make, we have the following makefile-fragment:
PROGRAMS = server client
server_OBJS = server.o server_priv.o server_access.o
server_LIBS = priv protocol
client_OBJS = client.o client_api.o client_mem.o
client_LIBS = protocol
...
define PROGRAM_template =
$(1): $$($(1)_OBJS) $$($(1)_LIBS:%=-l%)
ALL_OBJS += $$($(1)_OBJS)
endef
$(foreach prog,$(PROGRAMS),$(eval $(call PROGRAM_template,$(prog))))
I think this foreach should effectively expand to the following, before variable expansion:
server: $(server_OBJS) $(server_LIBS:%=-l%)
ALL_OBJS += $(server_OBJS)
client: $(client_OBJS) $(client_LIBS:%=-l%)
ALL_OBJS += $(client_OBJS)
I figured out the above expansion by hand (corrections welcome), but I'm looking for a general method to display this expansion for more complex examples. Does such a method exist?
I have looked into make -d and make -pn options, and as far as I can tell, I don't think either of these will provide this particular output.
I'm using make-3.81.
Replace all your $(eval ...) calls with $(info ...). Actually if you're stuck with 3.81 you may have to use $(warning ...) and ignore the extra output, because I think $(info ...) might not have existed until version 3.82.
Anyway, writing:
$(foreach prog,$(PROGRAMS),$(info $(call PROGRAM_template,$(prog))))
(or warning) you'll see what make is parsing.
I've written a macro to help with seeing the results of $(eval ...):
# This version prints out rules for debugging
#c = $(eval $(call $(strip $1),$(strip $2),$(strip $3),$(strip $4),$(strip $5),$(strip $6),$(strip $7),$(strip $8),$(strip $9))) $(info $(call $(strip $1),$(strip $2),$(strip $3),$(strip $4),$(strip $5),$(strip $6),$(strip $7),$(strip $8),$(strip $9)))
# This version is the production version
c = $(eval $(call $(strip $1),$(strip $2),$(strip $3),$(strip $4),$(strip $5),$(strip $6),$(strip $7),$(strip $8),$(strip $9)))
Use it like this:
$(call c,function,arg1,arg2,...)
To see what is being generated by eval, simply use the debugging version. I needed this instead of simply replacing $(eval ...) by $(info ...) because functionality later in the makefile depended on the results of the $(eval ...)
I have an X11 app I developed on a RedHat box. When I moved to CentOS, the X11 libraries moved as well. From /usr/X11R6/lib64 to /usr/X11R6/lib (even though both were running 64-bit), and I have seen them elsewhere on other Linux/Unix systems.
I had used the -I and -L compile flags to point to the X11 lib directories directly in the makefile, but have since realized, this requires changes to the makefile when switching machines.
My question is, for the sake of portability... should I keep using -I and -L in the makefile? And if so, is there a way to detect the location automatically? Or, should I just rely on the user having these in their LD_LIBRARY_PATH?
If you're relying on GNU make, you could do something like:
X11LIBS ?= $(word 1, $(dir $(wildcard /usr/X11R6/lib64/libX11.so.*)))
X11LIBS ?= $(word 1, $(dir $(wildcard /usr/X11R6/lib/libX11.so.*)))
X11LIBS ?= $(word 1, $(dir $(wildcard /usr/lib/libX11.so.*)))
# and so on for whatever paths you come across
LDFLAGS += -L$(X11LIBS) -R$(X11LIBS)
This allows the user to override X11LIBS on the command line (make X11LIBS=/usr/local/X11R6/lib), and tries a couple of locations if no user option is given.
You could even abort the build if no path is found, like
ifeq ($(X11LIBS),)
$(error Could not find X11 library path, please specify in X11LIBS
endif
All this is only an option if you're content with restricting the Makefile's portability: non-GNU make very likely won't be happy with these macros.