My Makefile is bundling recipes to process large numbers of files in parallel (make <recipe> -j8, etc). But before files can be processed, they need to be found with find. This is a time consuming operation, so I only want to run it for the exact recipe that the user invoked. I can do it like this:
test: FILES=$(shell find "$(SEARCHDIR)/" -mindepth 3 -maxdepth 3 -type f ! -regex $(SOMEREGEX))
test: $(FILES)
$(FILES):
echo "do thing with file $# here"
The problem is that since the files inherently exist already, they need to be declared as .PHONY in order for the recipe to run, like this:
.PHONY: $(FILES)
But in order for this to work, the FILES variable needs to exist and be populated, which requires me to run the find command. This goes against my goal of not performing the search to find FILES until test is invoked. What I need is something like this:
test: FILES=$(shell find "$(SEARCHDIR)/" -mindepth 3 -maxdepth 3 -type f ! -regex $(SOMEREGEX))
test: .PHONY: $(FILES)
test: $(FILES)
$(FILES):
echo "do thing with file $# here"
But test: .PHONY: $(FILES) is invalid syntax and does not work.
Your makefile already doesn't work, even without the .PHONY thing; just:
test: FILES=$(shell find "$(SEARCHDIR)/" -mindepth 3 -maxdepth 3 -type f ! -regex $(SOMEREGEX))
$(FILES):
echo "do thing with file $# here"
fails because $(FILES) in the target of the rule expands to the empty string when make parses the makefile, before it's started to run the test target.
I recommend that you use recursive make here; write your makefile like this:
FIND_FILES :=
FILES :=
ifneq ($(FIND_FILES),)
FILES := $(shell find "$(SEARCHDIR)/" -mindepth 3 -maxdepth 3 -type f ! -regex $(SOMEREGEX))
endif
test:
$(MAKE) test-recurse FIND_FILES=1
test-recurse: $(FILES)
$(FILES):
echo "do thing with file $# here"
.PHONY: test test-recurse $(FILES)
Related
Hi I am trying to write a generic (basic) Makefile to compile basic, practice c programs.
I have a src directory that I am practicing c programming in. My directory structure looks something like this:
practice/src/ex1.c
ex2.c
ex3.c
practice/build
my Makefile looks like this:
blddir = ./build
srcdir = ./src
# RM="find $(blddir) -maxdepth 1 -executable -type f -name 'ex*' -delete"
CFLAGS=-Wall -g
$(csrc): %.c
$(CC) -o $(blddir)/$# $(srcdir)/$^ # all prereqs
clean:
find $(blddir) -maxdepth 1 -executable -type f -name 'ex*' -delete
When I run: make ex6, I am greeted with : make: *** No rule to make target 'ex6'. Stop.
When I run: make -p ex6 | less
...and search for ex6 I see:
.DEFAULT_GOAL := src/ex6.c
src/ex6.c: %.c
# Implicit rule search has not been done.
# Modification time never checked.
# File has not been updated.
# recipe to execute (from 'Makefile', line 15):
$(CC) -o $(blddir)/$# $(srcdir)/$^ # all prereqs
What am I missing?
Thanks!
...more reading more testing, new(er) Makefile:
blddir = ./build
srcdir = ./src
CFLAGS=-Wall -g
% : $(srcdir)/%.c
$(CC) -o $(blddir)/$# $^ # all prereqs
make ex6
...and it works
I have a makefile that loops over all child directories that have files called Makefile in them, and runs a new make in that directory with my current target. It works fine when it works.
However, if the child make process finds an error it stops... but my calling make doesn't. The calling make continues to run, giving a false impression it is working. How can I make the calling make halt in this case?
# Make the same target in all subdirectories with a Makefile.
SubMakefiles = $(shell find -L . -maxdepth 2 -type f -name Makefile)
SubMakeDirs = $(filter-out ./,$(dir $(SubMakefiles)))
clean depend all::
#for dir in $(SubMakeDirs); do \
make -C $$dir $#
done
Without changing your targeting, you can capture the return value of one or more commands in sh with parenthesis, and exit with their error code should there be an error code, as follows:
# Make the same target in all subdirectories with a Makefile.
SubMakefiles = $(shell find -L . -maxdepth 2 -type f -name Makefile)
SubMakeDirs = $(filter-out ./,$(dir $(SubMakefiles)))
clean depend all::
#for dir in $(SubMakeDirs); do \
( make -C $$dir $# ) || exit $$?; \
done
Ultimately I settled on the following approach. Since the recursive make is run as a command directly by make, any error code produced will cause the master make to abort as well.
Walkthrough: we recurse to any sub-directory with a Makefile in it.
MAKECMDGOALS is a gmake-managed variable holding the command goal. A little counterintuitively, this top-level makefile says that whatever command you're trying to run depends on all the subdirectories with makefiles, and the rule to "create" those subdirectories (of course they already exist, but gmake isn't specifically aware of that) it recursively runs the same command in the subdirectory (specified with -C).
SUBDIRS = ${sort ${dir ${wildcard */Makefile}}}
all $(MAKECMDGOALS):: $(SUBDIRS)
.PHONY: $(SUBDIRS)
$(SUBDIRS)::
$(MAKE) --no-print-directory -C $# $(MAKECMDGOALS)
I have this command from my bash script
find . -type f -name "*.c" -execdir bash -c "f={}; kos-cc $KOS_CFLAGS -c {} -o $PWD/\${f%.c}.o" \;
Its job it to recursively search the current directory ($PWD) for .c files, compile them with my "kos-cc" compiler then output all the .o files into the current working directory.
I want to move parts of my bash script into a makefile and this line is the last one that stumps me. I know how to make rules that compiles c files from directory A and outputs the .o files into directory B, but here directory A isn't always the same (Since I need to handle sub directories too). How do I do the equivalent task in a Makefile either with a rule or a command?
It is not trivial because you have your sources in different directories and simple make pattern rules cannot easily handle this. But using slightly more advanced make features can make it:
SRC := $(shell find . -type f -name "*.c")
OBJ := $(patsubst %.c,%.o,$(notdir $(SRC)))
.PHONY: all
all: $(OBJ)
define COMPILE_rule
$$(patsubst %.c,%.o,$$(notdir $(1))): $(1)
kos-cc $$(KOS_CFLAGS) -c -o $$# $$<
endef
$(foreach s,$(SRC),$(eval $(call COMPILE_rule,$(s))))
.PHONY: clean
clean:
rm -f $(OBJ)
Beware: you could have several source files with the same base name in different directories. And if this happens, you will end up with a fatal conflict of object file names...
EDIT (add conflicts detection):
The following detects conflicts situations and issues an error (replace error by warning or info depending on the severity level you assign to these conflicts):
SRC := $(shell find . -type f -name "*.c")
OBJ := $(patsubst %.c,%.o,$(notdir $(SRC)))
ifneq ($(words $(OBJ)),$(words $(sort $(OBJ))))
$(error object file name conflicts detected)
endif
(sort sorts and also removes duplicates and words returns the number of space-separated words in its string parameter).
Although the below script, is intended find a makefile in the sub directory level of 1 and make them recursively, I'm not able to find any success with it.
for file in `find $(pwd)/ -maxdepth 2 -mindepth 2 -type f -name 'Makefile'`; do\
make --directory=$(dirname $file);\
done
What might be the problem here?
In this recipe you do not intend $(pwd), $(dirname ...) and $file to be expanded by make,
but they are are, because you have not escaped $. (Strictly, you do not intend the $f in $file to be expanded by make). Re-write as:
for file in `find $$(pwd)/ -maxdepth 2 -mindepth 2 -type f -name 'Makefile'`; do\
make --directory=$$(dirname $$file);\
done
Later
Is there a better and simple way of making sub modules of a project?
Yes. The canonical way would be like:
# Identify the sub-directories some way, not necessarily like this:
SUBDIRS := $(dir $(shell find $$(pwd)/ -maxdepth 2 -mindepth 2 -type f -name 'Makefile'))
.PHONY: $(SUBDIRS)
all: $(SUBDIRS)
# Maybe some recipe...
$(SUBDIRS):
$(MAKE) -C $#
I have several widgets denoted by a config.xml in their root in a directory layout.
The GNUmakefile I have here is able to build them. Though if I update the folders, the dependencies aren't tracked. I don't want to depend on a clean target obviously, so how do I track the contents of each folder?
WGTS := $(shell find -name 'config.xml' | while read wgtdir; do echo `dirname $$wgtdir`.wgt; done )
all: $(WGTS)
%.wgt:
#cd $* && zip -q -r ../$(shell basename $*).wgt .
#echo Created $#
clean:
rm -f $(WGTS)
I hoped something like:
%.wgt: $(shell find $* -type f)
Would work, but it doesn't. Help.
Combining Beta's idea with mine:
WGTS := $(shell find -name config.xml)
WGTS := $(WGTS:/config.xml=.wgt)
WGTS_d := $(WGTS:.wgt=.wgt.d)
all: $(WGTS)
clean:
rm -f $(WGTS) $(WGTS_d)
-include $(WGTS_d)
define WGT_RULE
$(1): $(shell find $(1:.wgt=))
$(1:.wgt=)/%:
#
endef
$(foreach targ,$(WGTS),$(eval $(call WGT_RULE,$(targ))))
%.wgt:
#echo Creating $#
#(echo -n "$#: "; find $* -type f | tr '\n' ' ') > $#.d
#cd $* && zip -q -r ../$(shell basename $*).wgt .
Example:
$ mkdir -p foo bar/nested
$ touch {foo,bar/nested}/config.xml
$ make
Creating bar/nested.wgt
Creating foo.wgt
$ make
make: Nothing to be done for `all'.
$ touch foo/a
$ make
Creating foo.wgt
$ rm foo/a
$ make
Creating foo.wgt
$ make
make: Nothing to be done for `all'.
The only potential problem here is the dummy rule that lets make ignore targets it doesn't know how to build which are nested inside the directories. (foo/a in my example.) If those are real targets that make needs to know how to build, the duplicate recipe definition may be a problem.
Probably the best way to do this is to create the prerequisite lists explicitly, beforehand:
define WGT_RULE
$(1).wgt: $(wildcard $(1)/*)
endef
$(foreach targ,$(WGTS),$(eval $(call WGT_RULE,$(targ))))
There is another way that's very clever (a phrase that makes a good programmer wary). Years ago I came up with a left-handed kludge for treating a directory as a prerequisite. I'll see if I can dig up my old notebooks if the above isn't good enough.
EDIT:
Sorry, I didn't consider subdirectories. Here's a complete makefile (I left out the clean rule) that should do the trick.
WGTS := $(shell find -name 'config.xml' | while read wgtdir; do echo `dirname $\
$wgtdir`.wgt; done )
all: $(WGTS)
# This constructs a rule without commands ("foo.wgt: foo/bar.txt foo/baz.dat...").
define WGT_RULE
$(1).wgt: $(shell find $(1))
endef
# This invokes the above to create a rule for each widget.
$(foreach targ,$(WGTS),$(eval $(call WGT_RULE,$(targ))))
%.wgt:
#cd $* && zip -q -r ../$(shell basename $*).wgt .
#echo Created $#