GNU Make pattern rule with non-matching patterns - makefile

I'm trying to use a codegen tool for Go to automatically generate some code based on the contents of other go files. The codegen tool will get standard arguments which can be deduced from the name of the file its generating and the name of the file that it's parsing. If I were doing it all manually, it would look like:
foo-tool -name FooInterface -file foo/api.go
foo-tool -name BarInterface -file foo/api.go
foo-tool -name BingInterface -file foo/bing.go
foo-tool -name BazInterface -file foo/baz.go
But I don't want to do it manually, I want to use Make! So I tried to accomplish the same thing with a Makefile and a pattern rule.
foo_FooInterface.go : foo/api.go
foo_BarInterface.go : foo/api.go
foo_BingInterface.go : foo/bing.go
foo_BazInterface.go : foo/baz.go
foo_%.go : %.go
$(eval foo_name=$(subst mock_,,$(subst .go,,$(#F))))
build-foo -name $(foo_name) -file $<
In my mind, the first 3 rules would set up the dependency graph, and the pattern rule would tell Make what to do with the dependencies. But when I try running make foo_BarInterface.go, I get make: Nothing to be done for foo_BarInterface.go. I understand why this happens, because Make is expecting to match foo_FooInterface.go with FooInterface.go, but I don't want to restructure my project's files like that.
Is this possible, or do I need to do something like:
foo_FooInterface.go : foo/api.go
build-foo -name FooInterface -file foo/api.go
foo_BarInterface.go : foo/api.go
build-foo -name BarInterface -file foo/api.go
foo_BingInterface.go : foo/bing.go
build-foo -name BingInterface -file foo/bing.go
foo_BazInterface.go : foo/baz.go
build-foo -name BingInterface -file foo/baz.go
Which I really don't want to do, because new Interfaces will be added as the codebase grows, and I don't want to require people to type all of that every time.
Edit: I wouldn't mind specifying the rule manually every time, but I need a rule that collects all the generated files together, and I don't want to specify every foo_*.go in that one. Is there a way to say "This rule depends on all rules (not files) matching a pattern?" I was able to do
foo_files := $(shell grep 'foo_\w\+.go' Makefile | cut -d : -f1)
But this seems bad to me.

If each of your target files has different prerequisites, you cannot avoid to specify them individually. You can do that with empty rules, as you did:
foo_FooInterface.go : foo/api.go
foo_BarInterface.go : foo/api.go
foo_BingInterface.go : foo/bing.go
foo_BazInterface.go : foo/baz.go
However, with this style of makefile, the rules to build your targets cannot have prerequisites: if they do, the prerequisites specified in the empty rules are ignored. So what you need is the simple rule
foo_%.go:
foo-tool -name $* -file $<
(note that $^ is expanded as the list of all prerequisites, $< only the first one. In your example there is no difference, but there might be.)
I am not aware of any way to specify “all rules in a makefile matching a pattern”. But it costs little to “register” all desired names in a variable, as #Beta has suggested. To put it all together:
# Register all your components here
INTERFACES := Foo Bar Bing Baz
# Make ’em all
.PHONY : all
all: $(patsubst %, foo_%Interface.go, $(INTERFACES))
# Specify individual dependencies with empty rules
foo_FooInterface.go : foo/api.go
foo_BarInterface.go : foo/api.go
foo_BingInterface.go : foo/bing.go
foo_BazInterface.go : foo/baz.go
# This is your catch-(almost-)all rule
foo_%.go:
foo-tool -name $* -file $<
So, every new target requires only registering its name in INTERFACES and adding a line with its dependency. In my opinion, trying to add the dependency line only and extrapolate the existence of a new target from it (as you did by grepping the makefile for foo_\w\+.go), decreases the overall readability with no real benefit.

Nothing easier:
foo_%.go : foo/api.go
foo-tool -name $* -file $<
And if you want a rule that gathers them all together:
INTERFACES := Foo Bar Bing
FILES := $(patsubst %, foo_%Interface.go, $(INTERFACES))
all: $(FILES)
#echo do something with $^

You can write a single common recipe for the three rules:
foo_FooInterface.go foo_BarInterface.go foo_BingInterface.go: foo/api.go
foo-tool -name $(#:foo_%.go=%) -file $<
The common recipe is executed separately for each of the individual targets individually, i.e.: once for foo_FooInterface.go, once for foo_BarInterface.go and once for foo_BingInterface.go, not only once for all of them together.

Related

GNU make: how to ensure pattern to match only filename without directory

I just want to copy files from other directories (before doing something). Because it's tedious to write the copy command for each file, I tried
%: ../src1/%
#echo cp $^ .
%: ../src2/%
#echo cp $^ .
all: file1 file2 file3 file4
# do something
but this doesn't work because make tries to look into ../src2/../src2/../src2/../ . . . . (I included echo for testing to prevent actual copying from happening. I keep forgetting what the "dry run" command line options is . . .)
I naïvely thought that there must be a way to force matches only to filenames that don't include directories.
Is there a way?
You can mark the "make anything" rules terminal with a double colon:
%:: ../src1/%
#echo cp $^ .
%:: ../src2/%
#echo cp $^ .
This does not answer your specific question of how to get Make to match only filenames without directories, but it does get Make to do what you want.
There is another approach that works and is closer to what you asked for: add pattern rules to satisfy Make:
../src1/%:
#: # do nothing
%: ../src1/%
#echo cp $^ .
EDIT: Or better still, us one dummy pattern rule to cover all source directories:
../%:
#: # do nothing

Linux bash-script to run make in all subdirectories

I'm trying to write a bash-script in Linux which traverses the current directory and, in every subdirectory, it launches the existing makefile. It should work for each subdirectory, regardless of depth.
Some restrictions:
I cannot use Python;
I don't know in advance how many subdirectories and their names;
I don't know in advance the name of current directory;
the make command for each directory should only be launched if there is makefile in such folder.
Any ideas on how to do it?
Using -exec and GNU make
find -type f \( -name 'GNUmakefile' -o -name 'makefile' -o -name 'Makefile' \) \
-exec bash -c 'cd "$(dirname "{}")" && make' \;
Given that this is make-related. I'd try to use a makefile at the top-level instead of a script. Something like this:
MAKEFILES:=$(shell find . -mindepth 2 -name Makefile -type f)
DIRS:=$(foreach m,$(MAKEFILES),$(realpath $(dir $(m))))
.PHONY: all
all: $(DIRS)
.PHONY: $(DIRS)
$(DIRS):
$(MAKE) -C $#
I'd accept what #MLSC says about using for with find, and that kind of applies here too .. the problem with that is when you have a space in the directory name. However, in many cases that's not going to happen, and IMHO there are benefits in using a makefile instead of a script. (There might be a solution using make that can cope with spaces in the directory name, but I can't think of it off the top of my head.)
You can use this script https://gist.github.com/legeyda/8b2cf2c213476c6fe6e25619fe22efd0.
Example usage is:
foreach */ 'test -f Makefile && make'
This should work if dont care about the execution order or if parent directory also has a Makefile.
#!/bin/bash
for f in $(find . -name Makefile); do
pushd $(dirname $f)
make
popd
done

make: How to deal with an ever-changing list of dependencies

I have a set of files of type *.x, processing which yields a corresponding set of files *.x.y (where each *.x file produces a corresponding *.x.y file). Simple enough so far.
The two issues I have are:
The set of *.x files keeps changing, so I don't want to use their names as a static list in the dependency portion of a rule.
I only want to process those *.x files that have a size greater than 0.
Currently I use:
for a in $(find . -iname '*.x' -size +0); do make $a.y; done
and it works, but I'd prefer a cleaner, more make-y solution.
Using GNU make 3.81 on OS X 10.7.5.
You can compute the list of source files with a shell function:
SRCFILES := $(shell find . -iname '*.x' -size +0)
OUTFILES := $(SRCFILES:%=%.y)
all: $(OUTFILES)
%.x.y : %.x
$(CONVERT) -to $# $<

How do I get the files in a directory and all of its subdirectories in bash?

I'm working on a C kernel and I want to make it easier to compile all of the sources by using a bash script file. I need to know how to do a foreach loop and only get the files with a .c extension, and then get the filename of each file I find so I can make gcc compile each one.
Use find to walk through your tree
and then read the list it generates using while read:
find . -name \*.c | while read file
do
echo process $file
done
If the action that you want to do with file is not so complex
and can be expressed using ore or two commands, you can avoid while
and make all things with the find itself. For that you will use -exec:
find . -name \*.c -exec command {} \;
Here you write your command instead of command.
You can also use -execdir:
find . -name \*.c -execdir command {} \;
In this case command will be executed in the directory of found file (for each file that was found).
If you're using GNU make, you can do this using only make's built-in functions, which has the advantage of making it independent of the shell (but the disadvantage of being slower than find for large trees):
# Usage: $(call find-recursive,DIRECTORY,PATTERN)
find-recursive = \
$(foreach f,$(wildcard $(1)/*),\
$(if $(wildcard $(f)/.),\
$(call find-recursive,$(f),$(2)),\
$(filter $(2),$(f))))
all:
#echo $(call find-recursive,.,%.c)

Makefile: depend on every file of a directory

I'd like to do a Makefile that runs either with gnumake or makepp that packs all the files under given directiories:
DIRS:=$(shell find . -mindepth 2 -maxdepth 2 -not -name mp3 -not -name ".*" -type d)
PACKAGES = $(DIRS:%=%.npk)
all: packages
packages: $(PACKAGES)
%.npk: %/*
npack c $# #^
.PHONY: all packages
the problem is that there's no such thing as %/* in the dependencies.
I need the targets (X.npk) to depend on every file in directory X, but I don't know what the files are when I write the Makefile, 'cause they're generated later.
An example:
./dirA/x
./dirA/y
./dirB/e
./dirB/f
I'd like to create ./dirA.npk (depending on x,y), ./dirB.npk (e,f)
There's nothing I know about the dirs or the files in advance except that the find used in the 1st line finds all the dirs.
Try using the wildcard directive:
DEPS := $(foreach dir, $(DIRS), $(wildcard $(dir)/*))
%.npk: $(DEPS)
npack c $# $^
EDIT:
The above is just an example of using wildcard and makes each .npk file dependent on the files in all of the other folders. Your usage would be slightly different.
I think there may be an easier way to go about this. Why are you wanting to have a dependency on all of the files in the folder? Is it just to use the $^ operator? Or do you need to rebuild the .npk if any of the files changed?
One alternate (and possibly cleaner) solution would be to use the find utility in your recipe instead of $^ and use the .FORCE directive to always force the .npk file to be rebuilt. The downside is that .npk files may be rebuilt unnecessarily.
EDIT 2:
If there's not a way to do this cleanly with make commands, you can work around it by using .FORCE to ensure that the recipe is always run and move the "should I rebuild this file" check into the body of the recipe:
%.npk: .FORCE
check_for_rebuild.sh $# && npack c $# $^
where check_for_rebuild.sh is a shell script that does something like this:
#!/bin/bash
# Returns non-zero if the archive needs to be rebuilt
if [ -e $1 ]; then
folder_name=$(basename $1 .npk)
[ -z "$(find $folder_name -newer $1 -not -type d)" ] && return 0
fi
return 1
I don't really like that solution because it works around the problem instead of solving it directly, but it may be able to get you going in the meantime. If you are going to go that route, it's probably cleaner and easier to do everything in the shell script and either have the makefile simply invoke the script or get rid of the makefile entirely.
This is the solution I found:
it is based on the makedepend idea, with some "meta" scripting. Not very nice, but works.
PACKAGES :=
all: packages
-include Makefile.depend
packages: Makefile.depend $(PACKAGES)
depend: clean Makefile.depend
Makefile.depend:
#(PACKAGES= ; \
for DIR in `find . -mindepth 2 -maxdepth 2 -not -name mp3 -not -name ".*" -type d` ; \
do \
PACKAGE=`basename $${DIR}.npk` ; \
PACKAGES="$${PACKAGES} $${PACKAGE}" ; \
DEPS=`find $${DIR} -not -type d | sed -e 's#\([: ]\)#\\\\\1#' -e 's#^\./\(.*\)# \1#' | tr -d "\n"` ; \
SUBDIR=`echo $${DIR} | sed -e 's#^\./\([^/]\+\)/.*#\1#'` ; \
FILES=`echo \ $${DEPS} | sed -e "s# $${SUBDIR}/# #g"` ; \
echo "$${PACKAGE}:$${DEPS}" ; \
echo " #cd $${SUBDIR} ; \\" ; \
echo " npack c ../\$$# $${FILES} ; \\" ; \
echo ; \
done ; \
echo "PACKAGES = $${PACKAGES}" \
)>> Makefile.depend ; \
cleanall: clean
rm -f *.npk
clean:
#rm -f Makefile.depend
.PHONY: all packages depend clean
With makepp you can do this in 2 steps, via the :foreach rule modifier:
$(foreach).txt: $(foreach)/*: foreach */
&echo $(inputs) -o $(output)
This provides a rule for every subdirectory, which reexecutes whenever there is a change in the list of files therein.

Resources