I want a pattern rule for a Makefile for turning several sources into a single object.
So instead of a filename like foo.c I have a folder name foo/.
Additionally, there's path of the directory stem src/bar/foo/ where not only the shared src/ needs to be ignored but also the variable bar/.
Directory Structure
I have several objects to create - let's call them this:
bin/lib1.so
bin/lib2.so
bin/lib3.so
The source directory has a structure like this with folders grouping the libs:
src/
foo/
lib1/
a.c
b.c
bar/
lib2/
d.c
e.c
lib3/
f.c
g.c
If the grouping of libs didn't exist it would be straightforward, but it does exist.
Makefile
The targets get built by make all which is given for this example explicitly (I actually use shell find as in this answer, because there's several of them, but for this example I keep it simple):
all: bin/lib1.so bin/lib2.so bin/lib3.so
Now I would like to clean up the Makefile by creating a pattern rule for making all of these targets, kind of like this:
bin/%.so: $$(wildcard src/*/%/*.c)
# testing the rule:
echo $< > $#
But this doesn't work.
I'm skipping the step of c->o for simplicity because the point is linking together several files.
There's this question which has a similar problem of subdirectories, but it's only about 1:1 mapping of files instead of n:1, and I couldn't get it to work for my problem.
Desired Result
To test the rule it should just show in each target file what sources went into it:
bin/lib1.so
src/foo/lib1/a.c
src/foo/lib1/b.c
bin/lib2.so
src/bar/lib2/d.c
src/bar/lib2/e.c
bin/lib3.so
src/bar/lib3/f.c
src/bar/lib3/g.c
Here is a working version that supports one or multiple C files as input:
bin/%.so: src/*/%/*.c
#echo "START"
#echo $^
#echo $#
#echo "END"
all: bin/lib1.so bin/lib2.so bin/lib3.so
Result:
$ make
START
src/foo/lib1/main.c
bin/lib1.so
END
START
src/bar/lib2/plip.c src/bar/lib2/plop.c
bin/lib2.so
END
START
src/bar/lib3/kilo.c
bin/lib3.so
END
So I am trying to write a Makefile to remove files nested in directories with the same names. As an example, I have a directory with two files whose paths are ./1234/1234.txt and ./567/567.txt. I want to move those files up to the top directory so that they are just ./1234.txt and ./567.txt. I tried something like this but it didn't work:
variable = ./1234/1234.txt ./567/567.txt
run: $(variable)
$(variable): $$(notdir %): %
mv $< $#
It seems like something that would be a common problem but I haven't been able to find an answer.
Thanks so much for the help! :)
You have your targets the wrong way round, and trying to use functions in a rule won't work unless you use secondary expansion. You'll also need to conditionally add the dependencies because otherwise your rule will give an error if you've already moved the files.
variable = ./1234.txt ./567.txt
run: $(variable)
.SECONDEXPANSION:
$(variable): ./%.txt: $$(wildcard ./$$*/$$*.txt)
mv $< $#
I have another make-like tool that produces an XML as an artifact after parsing my makefile which I'll then further process with Python.
It'd simplify things for me - a lot - if I could have make consider every single prerequisite to be an actual target because then this other tool
will classify each and every file as a "job".
This is a fragment of my makefile:
.obj/eventlookupmodel.o: C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h \
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h \
...
I'd want for make to think I have a dummy rule for each prerequisite such as below:
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qvariant.h:
#echo target pre= $#
C:/Users/User1/Desktop/A/PROJ/qt5binaries/include/QtCore/qabstractitemmodel.h:
#echo target pre=$#
C:/Users/User1/Desktop/A/PROJ/src/AL2HMIBridge/LookupModels/eventlookupmodel.cpp :
#echo target pre=$#
C:\Users\User1\Desktop\A\PROJ\src\AL2HMIBridge\LookupModels\eventlookupmodel.h:
#echo target pre=$#
I don't care about the exact form of the rule just that each file is considered an actual target.
My method of passing in this rule would be by setting the MAKEFILES variable like so
make all MAKEFILES=Dummy.mk
with Dummy.mk containing this rule so that I do not modify the makefiles.
I've tried the following so far.
Dummy.mk:
%.h:
#echo header xyz = $#
%:
#echo other xyz= $#
This partially works.
I run make all --trace --print-data-base MAKEFILES=Dummy.mk and I can see that
make does "bind" the %.h: rule to the header files. In the --print-data-base section, I see that rule being assigned to the header files.
C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef.h:
# Implicit rule search has been done.
# Implicit/static pattern stem: 'C:/Users/User1/Desktop/A/QNX_SDK/target/qnx6/usr/include/stddef'
# Last modified 2016-05-27 12:39:16
# File has been updated.
# Successfully updated.
# recipe to execute (from '#$(QMAKE) top_builddir=C:/Users/User1/Desktop/A/HMI_FORGF/src/../lib/armle-v7/release/ top_srcdir=C:/Users/User1/Desktop/A/HMI_FORGF/ -Wall CONFIG+=release CONFIG+=qnx_build_release_with_symbols CONFIG+=rtc_build -o Makefile C:/Users/User1/Desktop/A/HMI_FORGF/src/HmiLogging/HmiLogging.pro
', line 2):
#echo header xyz = $#
However, I do NOT see the "echo header xyz $#"-rule being executed.
Regarding the %: rule, it is neither executed for the .cpp files nor "bound" to them in the --print-data-base section.
However, it is bound and executed for existing targets which have no suffix i.e.
all: library binary
binary: | library
ifs: | library
For the %: rule, the reason for this behavior is because of 10.5.5 Match-Anything Pattern Rules: If you do not mark the match-anything rule as terminal, then it is non-terminal. A non-terminal match-anything rule cannot apply to a file name that indicates a specific type of data. A file name indicates a specific type of data if some non-match-anything implicit rule target matches it.
If I make it non-terminal - no double colon - then the rule doesn't apply to built-in types like .cppunless I un-define the built-in rules that negate my intended %: rule.
If I make it terminal, "it does not apply unless its prerequisites actually exist". But a .h or .cpp doesn't technically have prerequisites; can I just create a dummy file and have that as its prerequisite?
NOTE: This has NOTHING to do with gcc -M generation. Yes the -M option would help in the specific case of header and source files but this question is for more generic targets and prerequisites that already exist in the makefile when make is launched.
This may take a few iterations. Try:
%.h: null
#echo header xyz = $#
%: null
#echo other xyz= $#
null:
#:
Try generating static pattern rules for the header files. See one of the answers to Make ignoring Prerequisite that doesn't exist.
Static pattern rules only apply to an explicit list of target files like this:
$(OBJECTS): %.o: %.c
*recipe here*
where the variable OBJECTS is defined earlier in the makefile to be a list of target files (separated by spaces), for example:
OBJECTS := src/fileA.c src/fileB.c src/fileC.c
Note that you can use the various make utility functions to build that list of target files. For example, $(wildcard pattern), $(addsuffix), etc.
You should also ensure that the recipe "touches" the header file to change the timestamp.
I've found that using static pattern rules instead of pattern rules fixes problems where make doesn’t build prerequisites that don’t exist, or deletes files that you want.
Here is an example of using wildcard to copy files from one directory to another.
# Copy images to build/images
img_files := $(wildcard src/images/*.png src/images/*.gif src/images/*.jpg \
src/images/*.mp3)
build_images := $(subst src/,$(BUILD_DIR)/,$(img_files))
$(build_images): $(BUILD_DIR)/images/% : src/images/%
mkdir -p $(dir $#)
cp -v -a $< $#
There are other make functions like addprefix that could be used to generate a more complex file specification.
Here's a snippet of a GNU Makefile I'm working with. Basically, I have a directory of images that i want copied into a dist directory when running make, but I'd prefer not to list each image individually.
DISTDIR := dist
IMG := $(shell find app/img -type f)
$(subst app/img, $(DISTDIR), $(filter-out %.svg, $(IMG)))): $(filter-out %.svg, $(IMG))
#cp $? $(DISTDIR)
This sort of works, except any time a source file is changed (in app/img), make always remakes the target file. What's the correct way to handle this situation?
It would be easier to answer if you provided some sample results for the find. Let's say that the find operation returns the files: app/img/foo.svg, app/img/bar.jpg, and app/img/baz.png.
In that case the filter-out functions will return the files app/img/bar.jpg and app/img/baz.png. This means that the rule generated will look like this:
$(DISTDIR)/bar.jpg $(DISTDIR)/baz.png: app/img/bar.jpg app/img/baz.png
#cp $? $(DISTDIR)
Here's the thing: when you define N targets in an explicit rule, it's like defining N rules, one for each target, with all the same prerequisites. So the above is identical to writing this:
$(DISTDIR)/bar.jpg: app/img/bar.jpg app/img/baz.png
#cp $? $(DISTDIR)
$(DISTDIR)/baz.png: app/img/bar.jpg app/img/baz.png
#cp $? $(DISTDIR)
Now you can see why you get the behavior that you do: each target lists ALL the files as prerequisites, so whenever ANY file is changed the target is rebuilt.
The answer is that in make, you should generally try to write rules that create exactly one target from just that target's prerequisites. One simple way to do that is with pattern rules; a pattern rule for the above might look like:
DISTDIR := dist
$(DISTDIR)/% : app/img/%
#cp $< $#
After that, all you have to do is declare a target which depends on the files you want created:
IMG := $(shell find app/img -type f)
all: $(IMG:app/img/%=$(DISTDIR)/%)
and make will figure out the rest!
I am trying to compile for different software directories with different optimization levels etc. I created the following makefile to do so:
OWNER = betsy molly fred
DOG = poodle mutt doberman
COLOUR = brown red yellow
ATTR = big small
LEGS = 0 3
#we want every possible combination to be excercised
OUTPUT_STUFF = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))
.PHONY: all
all: $(OUTPUT_STUFF)
define PROGRAM_template
own = $(1)
dog = $(2)
col = $(3)
attr = $(4)
legs = $(5)
BUILD_DIR = new/$(own)/$(dog)/$(col)/$(attr)/$(legs)
#for each build directory, we are going to put a file in it containing the build dir. string
$$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $$(BUILD_DIR)
#echo "$$(BUILD_DIR)" > $$(BUILD_DIR)/dogInfo.txt
endef
#call the function many times
$(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),$(eval $(call PROGRAM_template,$(own),$(dog),$(col),$(attr),$(legs))))))))
As you can see, this simple test program loops through different combinations of owner, dog etc. The end goal is to have a directory, new, that has all owners as dirs, and in those, all dogs, etc. At the bottom is just a file with the path in it.
When I run this, the output is:
new/betsy/poodle/brown/big/0/dogInfo.txt
mkdir new/fred/doberman/yellow/small/3
mkdir: cannot create directory `new/fred/doberman/yellow/small/3': No such file or directory
make: *** [new/betsy/poodle/brown/big/0/dogInfo.txt] Error 1
So, for some reason, the target is ok, but the seemingly exact same variable is the last in my loops. Fundamentally, I don't understand what is happening that well.
Weird foreach + user-defined function behavior in Makefiles seems to answer, but I don't fully get it. In my mind, when the function is called, it fills in all instances with one $, and the escaped ones become $(BUILD_DIR). It then 'pastes' the code to the temporary file, and after it's done all the calls it evaluates the file, substituting the variables as normal.
One (ugly) solution I thought of is to make the BUILD_DIR variable different every time like so:
B_D_$(1)_$(2)_$(3)_$(4)_$(5) = ~~~
Alex is correct (although I think he means recipe, not receipt :-)). The best way to debug complex eval issues is to replace the eval function with a call to info instead. So if you have something like:
$(foreach A,$(STUFF),$(eval $(call func,$A)))
then you can rewrite this as:
$(foreach A,$(STUFF),$(info $(call func,$A)))
Now make will print out to you exactly what the eval is going to parse. It's usually pretty clear, looking at the makefile output, what the problem is. In your case you'll see something like this in the output (leaving out all the extra variable settings):
BUILD_DIR = new/betsy/poodle/brown/big/0
$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $(BUILD_DIR)
#echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt
BUILD_DIR = new/betsy/poodle/brown/big/3
$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $(BUILD_DIR)
#echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt
etc. Notice how you're setting the global variable BUILD_DIR every time. In make, variables have only one value (at a time). While make is reading the makefile it expands the target and prerequisite lists immediately, so whatever value BUILD_DIR has at that time will be used for targets/prerequisites, so this works for you.
But when make finishes reading the makefile, the value of BUILD_DIR will always be the last thing you set it to; in this case new/fred/doberman/yellow/small/3. Now make starts to invoke the recipes for each target, and when it does that it will expand BUILD_DIR in the recipes then, and so ALL the recipes will get that same value.
As Alex points out, you should ensure that your recipe uses only automatic variables like $#, which are set correctly for each rule. If you do that you'll notice that you don't really need to redefine the rule at all because it's actually the same recipe for all the targets. And if you notice THAT, you'll notice you don't need the whole eval or call complexity in the first place.
All you have to do is compute the names of all the targets, then write a single rule:
ALLDOGINFO = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))
$(ALLDOGINFO):
#echo "$#"
mkdir $(dir $#)
#echo "$(dir $#)" > $#
If you don't want the trailing slash you have to use $(patsubst %/,%,$(dir $#)) instead.
The problem is that when $$(BUILD_DIR) is evaluated in receipt, the loop is already complete. The solution is to rewrite the receipt:
$$(BUILD_DIR)/dogInfo.txt:
#echo "$$#"
mkdir $$(#D)
#echo "$$(#D)" > $$#
I don't think your problem is necessarily with something to do with make. This command:
mkdir new/fred/doberman/yellow/small/3
will fail if one of the parent directories (for example, yellow) doesn't already exist. The error it spits out in this case is the one you're getting, so it seems likely this is the case. If you want a command that makes all parent directories of a given directory as needed, you should run mkdir -p, like this:
mkdir -p $$(BUILD_DIR)
See the mkdir man page for a full description of what -p does.