Generate a directory if a target name has a directory part - makefile

I have this makefile code:
$(DIRS):
#echo " MKDIR build/tmp/base/socket/$#"
$(Q)mkdir -p $#/
%.a.s:
#echo " CC build/tmp/base/socket/$#"
$(Q)$(CC) $(CFLAGS_A) -S $< -o $#
%.so.s:
#echo " CC build/tmp/base/socket/$#"
$(Q)$(CC) $(CFLAGS_SO) -S $< -o $#
%.o: %.s
#echo " AS build/tmp/base/socket/$#"
$(Q)$(AS) $< -o $#
tcp/client.a.s: $(TCP_CLIENT_SRC) $(TCP_CLIENT_INC)
tcp/client.so.s: $(TCP_CLIENT_SRC) $(TCP_CLIENT_INC)
tcp/server.a.s: $(TCP_SERVER_SRC) $(TCP_SERVER_INC)
tcp/server.so.s: $(TCP_SERVER_SRC) $(TCP_SERVER_INC)
I'd like to add a | tcp in the prerequisites of all targets that have a leading tcp/ in their name, but I'd like to be able to do it in one line. I wouldn't want to append that manually to every line that needs it.
I thought of adding this new rule:
tcp/%.s: | tcp
but it isn't doing anything.
I also thought of the more generic, which would be nicer, but same results:
%.s: | $(#D)
How should I write that?
A workaround would be to call mkdir every time (include it in both %.a.s and %.so.s rules), but that would add unnecessary calls to mkdir, wouldn't it?

I am not a fan of pattern rules. They are too arbitrary for my tastes.
(What actually happens depends on what files you may have lying around on your hard disk.)
You cannot just add a prerequisite to a given set of targets using a pattern rule
Well you can if you use static pattern rules. This is a much nicer idiom. Here we prefix a pattern rule with the actual list of sources you want the pattern rule to apply to. This is good where you can describe dependencies using make's noddy pattern matching.
A sketch:
%.a: ; date >$# # Pattern rule
tcp: ; mkdir -p $# # Explicit rule
tcp/a.a: tcp/%.a: | tcp ; # Static pattern rule!
.PHONY: all
all: tcp/a.a
all: c.a
all: dir/b.a
all: ; : $# Success
And we have:
$ make all
mkdir -p tcp
date >tcp/a.a
date >c.a
date >dir/b.a
/bin/sh: dir/b.a: No such file or directory
make: *** [Makefile:3: dir/b.a] Error 1
Here we have told make that before it builds (i.e., "runs the recipe for") tcp/a.a, it must first build tcp. That works. We didn't tell make about the directory for dir/b.a, so that failed. Note that the .a file's recipe is still in an ordinary pattern rule. This is just by way of exposition. I would definitely change this.
Yeah, in this case the pattern rule for tcp/ is over-the-top. Consider though that before you create tcp/a.a, you might first need to create a auto/a.src (say).
tcp/a.a: tcp/%.a: auto/%.src | tcp ; # Static pattern rule!
Easily extensible.
targets := tcp/a.a tcp/b.a tcp/c.a
${targets}: tcp/%.a: auto/%.src | tcp ; # Static pattern rule!
[By the way, in your original makefile, your archive and shared object should depend on the .o files, not the source (???)]

You cannot just add a prerequisite to a given set of targets using a pattern rule. That's not how pattern rules work: they are rules: they must have a recipe associated with them. A pattern rule without a recipe actually deletes that pattern rule (see Canceling Implicit Rules).
You can either create two sets of pattern rules, one for all targets and a second one for just targets that start with tcp/ that have an extra prerequisite, but you have to write the entire pattern rule twice, including a recipe, not just the pattern line.
Or just put the mkdir into the recipe. A mkdir on a directory that already exists won't even be noticeable.

This answer is only to show a working Makefile implementing #bobbogo 's answer.
This Makefile is a leaf of a Makefile tree, so all variables not defined here are exported by upper Makefiles.
Makefile:
#! /usr/bin/make -f
################################################################################
# *AUTHOR*
# FULL NAME "Alejandro Colomar Andrés"
################################################################################
################################################################################
DIRS = \
$(CURDIR)/tcp
OBJ = \
$(CURDIR)/tcp/client.o \
$(CURDIR)/tcp/server.o \
$(CURDIR)/foo.o
SRC = \
$(SRC_DIR)/base/socket/tcp/client.c \
$(SRC_DIR)/base/socket/tcp/server.c \
$(SRC_DIR)/base/socket/foo.c
DEP = $(OBJ:.o=.d)
BOTH_OBJ = $(subst .a.o,.a.o ,$(join $(OBJ:.o=.a.o),$(OBJ:.o=.so.o)))
BOTH_ASM = $(subst .a.s,.a.s ,$(join $(OBJ:.o=.a.s),$(OBJ:.o=.so.s)))
NEEDDIR = $(DEP) $(BOTH_ASM)
################################################################################
PHONY := all
all: $(BOTH_OBJ)
#:
$(DIRS): $(CURDIR)/%:
#echo " MKDIR build/tmp/base/socket/$*"
$(Q)mkdir -p $#
$(NEEDDIR): | $(DIRS)
$(CURDIR)/%.d: $(SRC_DIR)/base/socket/%.c
#echo " CC -M build/tmp/base/socket/$*.d"
$(Q)$(CC) $(CFLAGS_A) -MG -MT"$#" \
-MT"$(CURDIR)/$*.a.s" -MT"$(CURDIR)/$*.so.s" \
-M $< -MF $#
$(CURDIR)/%.a.s: $(SRC_DIR)/base/socket/%.c $(CURDIR)/%.d
#echo " CC build/tmp/base/socket/$*.a.s"
$(Q)$(CC) $(CFLAGS_A) -S $< -o $#
$(CURDIR)/%.so.s: $(SRC_DIR)/base/socket/%.c $(CURDIR)/%.d
#echo " CC build/tmp/base/socket/$*.so.s"
$(Q)$(CC) $(CFLAGS_SO) -S $< -o $#
$(CURDIR)/%.o: $(CURDIR)/%.s
#echo " AS build/tmp/base/socket/$*.o"
$(Q)$(AS) $< -o $#
include $(DEP)
PHONY += clean
clean:
$(Q)rm -rf *.o *.s *.d
################################################################################
# Declare the contents of the PHONY variable as phony.
.PHONY: $(PHONY)
################################################################################
######## End of file ###########################################################
################################################################################
output:
MKDIR build/tmp/base/socket/tcp
CC -M build/tmp/base/socket/foo.d
CC -M build/tmp/base/socket/tcp/server.d
CC -M build/tmp/base/socket/tcp/client.d
CC build/tmp/base/socket/tcp/client.a.s
AS build/tmp/base/socket/tcp/client.a.o
CC build/tmp/base/socket/tcp/client.so.s
AS build/tmp/base/socket/tcp/client.so.o
CC build/tmp/base/socket/tcp/server.a.s
AS build/tmp/base/socket/tcp/server.a.o
CC build/tmp/base/socket/tcp/server.so.s
AS build/tmp/base/socket/tcp/server.so.o
CC build/tmp/base/socket/foo.a.s
AS build/tmp/base/socket/foo.a.o
CC build/tmp/base/socket/foo.so.s
AS build/tmp/base/socket/foo.so.o

Related

Wildcard Pattern Matching in Make

Here is my project structure.
| - src
| - boot
| - table.s
| - boot.s
| - machine
| - revb
| - memmap
| - vars.s
| - os
| - utils
| - ...lots here
I am grouping features by folder, and have a special folder for machine specific code, link scripts, anything.
The problem I am having with make is that I can't seem to get the Pattern match to work.
The below runs and builds the .o files.
#This gets repetative as every file needs to have a recipe by itself.
$(TARGET_DIR)/hash.o : $(SRC_DIR)/utils/hash.s machine
#echo "compiling $<"
$(CC) $(CFLAGS) $< -o $#
#echo "assembly dump $#"
$(DUMP) -D $# > $#.list
The below does NOT work. It doesn't run any of the commands.
#if this works that would be perfect every folder/file will be a recipe!
$(TARGET_DIR)/%.o : $(SRC_DIR)/%.s machine
#echo "compiling $<"
$(CC) $(CFLAGS) $< -o $#
#echo "assembly dump $#"
$(DUMP) -D $# > $#.list
Nothing runs at all, For some reason no files seem to match that pattern.
I have also tried this.
# if this works, it would be a bit annoying as this is per feature recipe/target.
$(TARGET_DIR)/%.o : $(SRC_DIR)/boot/%.s machine
#echo "compiling $<"
$(CC) $(CFLAGS) $< -o $#
#echo "assembly dump $#"
$(DUMP) -D $# > $#.list
Edit
full makefile for reference
CFLAGS = -march=rv32i -mabi=ilp32
CC = riscv32-unknown-linux-gnu-as
LINKER = riscv32-unknown-linux-gnu-ld
DUMP = riscv32-unknown-linux-gnu-objdump
COPY = riscv32-unknown-linux-gnu-objcopy
SRC_DIR = src
TARGET_DIR = target
MACHINE_FILES_DIR = machine
TARGET_MACHINE = revb
# vizoros : $(TARGET_DIR)/%.o
# $(LINKER) $(TARGET_DIR)/boot.o $(TARGET_DIR)/table.o $(TARGET_DIR)/os.o $(TARGET_DIR)/hash.o -T $(TARGET_DIR)/memmap -o $(TARGET_DIR)/$#.elf
# $(DUMP) -D $(TARGET_DIR)/$#.elf > $(TARGET_DIR)/$#.list
$(TARGET_DIR)/%.o : $(SRC_DIR)/boot/%.s machine
#echo "compiling $<"
$(CC) $(CFLAGS) $< -o $#
#echo "assembly dump $#"
$(DUMP) -D $# > $#.list
machine: folders
cp -r $(SRC_DIR)/$(MACHINE_FILES_DIR)/$(TARGET_MACHINE)/. $(TARGET_DIR)
folders:
mkdir -p $(TARGET_DIR)
.phony: clean
clean:
rm -rf $(TARGET_DIR)
As with your other question, you are expecting too much fancy capability from make. Make is a very simple tool. It will not go looking around your directories for files that could be built. It will build only exactly what you ask it to build. It will not infer matching files based on heuristics: it will match only exact strings.
Make always works backwards: it starts with the final target you ask it to build and finds a rule that can build that. Then it looks at each of the prerequisites of that final target and finds a rule that can build each one of those. Then it looks at any prerequisites of each of the prerequisites of those, etc. Once it has built (or found source files for) all the prerequisites of a target, it builds that target, then walks back up until finally it builds the final target you asked for.
In your makefile above you've commented out the "final target" you want built (vizoros), so of course make will not build it.
I'm just going to give you a makefile that solves your problem... if you want to understand what it does please consult the GNU make manual. Note I haven't actually tried this, I just wrote it here. Also note I omitted the whole machine thing because I don't understand what it's supposed to do and you didn't really define it.
CFLAGS = -march=rv32i -mabi=ilp32
CC = riscv32-unknown-linux-gnu-as
LINKER = riscv32-unknown-linux-gnu-ld
DUMP = riscv32-unknown-linux-gnu-objdump
COPY = riscv32-unknown-linux-gnu-objcopy
SRC_DIR = src
TARGET_DIR = target
MACHINE_FILES_DIR = machine
TARGET_MACHINE = revb
# Find all .s files under SRC_DIR
SRCS := $(shell find $(SRC_DIR) -name \*.s)
# Use VPATH with a list of directories to be searched for
VPATH := $(sort $(dir $(SRCS)))
# Convert all .s files into .o files directly under TARGET_DIR
# First strip off the directory, then convert
OBJS := $(patsubst %.s,$(TARGET_DIR)/%.o,$(notdir $(SRCS)))
# Define the final target and provide all the object files as prerequisites
$(TARGET_DIR)/vizoros.elf : $(OBJS)
mkdir -p $(#D)
$(LINKER) $^ -T $(TARGET_DIR)/memmap -o $#
$(DUMP) -D $# > $(#:.elf=.list)
# Define a pattern rule to build an object file in TARGET_DIR
# The source file will be searched for via VPATH
$(TARGET_DIR)/%.o : %.s
mkdir -p $(#D)
#echo "compiling $<"
$(CC) $(CFLAGS) -c $< -o $#
#echo "assembly dump $#"
$(DUMP) -D $# > $#.list
# It must be .PHONY, not .phony: make, like all POSIX tools, is
# case-sensitive
.PHONY: clean
clean:
rm -rf $(TARGET_DIR)
References:
GNU make manual
= vs := variables
shell function
dir and notdir functions
sort and patsubst functions
VPATH
Substitution references
Automatic variables
Phony targets
Also note you had an error in your compile command; you were missing the -c option which tells the compiler to generate an object file rather than a final executable file.

Add rule to generate specific headers if missing

I have a Makefile that mixes Java and C++ with JNI and I would like to generate the JNI header automatically when needed.
Currently, I have the following:
$(OBJDIR)/org_some_package_%.cpp: $(INCDIR)/jni/org_some_package_%.h
$(OBJDIR)/%.o: $(SRCDIR)/%.cpp
#mkdir -p $(OBJDIR)
$(CC) -c -o $# $(CFLAGS) \
-I$(INCDIR) -I../build/include -I"$(JAVA_HOME)/include" -I"$(JAVA_HOME)/include/linux" \
-MD -MF $(patsubst %.o, %.d, $#) $<
.SECONDEXPANSION:
$(INCDIR)/jni/%.h: $(JAVA_TEST_BIN)/$$(subst _,/,%).class
#mkdir -p $(INCDIR)/jni
$(JAVAH) -d inc/jni -classpath TestJNI/bin/tests:$(JUNIT_JAR):$(HAMCREST_JAR) $(basename $(subst _,.,$(subst $(INCDIR)/jni/,,$#)))
The problem is that when a .o file is needed, the first rule is never applied:
$(OBJDIR)/org_some_package_%.cpp: $(INCDIR)/jni/org_some_package_%.h
If I replace this rule with a specific one (by replacing % by the name of a class), it works.
How can I make this generic rule work?
I don't want to do this:
$(OBJDIR)/%.o: $(SRCDIR)/%.cpp $(INCDIR)/jni/%.h
...because I may have .cpp files that do not correspond to JNI header.
You cannot create simple prerequisites using patterns. A pattern rule without any recipe doesn't define a prerequisite, it cancels a previously-existing pattern rule. The GNU make manual says:
You can cancel a built-in implicit rule by defining a pattern rule with the same target and prerequisites, but no recipe.
The only way to create a prerequisite is to make it explicit.
So, your $(INCDIR)/jni/a_b_c.h depend on $(JAVA_TEST_BIN)/a/b/c.class? You will have to use advanced make features. I assume that all *.class file under $(JAVA_TEST_BIN) correspond to a built header file. This (not tested) should do what you show in your question:
CLASSES := $(patsubst $(JAVA_TEST_BIN)/%.class,%,$(shell find $(JAVA_TEST_BIN) -type f -name '*.class'))
BUILT_HEADERS :=
# $(1): class
define BUILD_HEADER_rule
header := $$(INCDIR)/jni/$$(subst /,_,$(1)).h
$$(header): $$(JAVA_TEST_BIN)/$(1).class
BUILT_HEADERS += $$(header)
endef
$(foreach c,$(CLASSES),$(eval $(call BUILD_HEADER_rule,$(c))))
$(BUILT_HEADERS):
#mkdir -p $(dir $#)
$(JAVAH) -d inc/jni -classpath TestJNI/bin/tests:$(JUNIT_JAR):$(HAMCREST_JAR) $(basename $(subst _,.,$(subst $(INCDIR)/jni/,,$#)))

Create dependency files in Makefile

I am using a Makefile including a rule to create a dependency file. The compiler is GCC.
%.d: %.c
mkdir -p $(#D)
$(CC) $(CFLAGS) $(CPPFLAGS) -M $< | \
sed 's,\($(notdir $*)\.o\) *:,$(dir $#)\1 $#: ,' > $#.tmp
mv $#.tmp $#
I am quite new in Makefile technique and I find it a little bit difficult to understand this rule composed of a mixture of several options.
Can somebody give a simple explanation how this rule works?
Thanks in advance.
%.d: %.c
All files ending in .d can be made using this rule, as long as there is a .c file with the same directory and stem relative to the current directory or one of the vpaths.
mkdir -p $(#D)
Create the topmost and all intermediate directories for $(#D), which is an automatic make variable that expands to the directory part of the current target.
$(CC) $(CFLAGS) $(CPPFLAGS) -M $< | \
Invoke the C compiler on the first prerequisite (whatever.c) and tell it to output a list of make dependencies to the standard output. Pipe this output to
sed 's,\($(notdir $*)\.o\) *:,$(dir $#)\1 $#: ,' > $#.tmp
sed. The rules in the output will have the same path as the source files, but whoever wrote this rule wants the objects files in a different directory so we need sed to substitute the paths.
Sed captures any rules with targets ending with .o that match $(notdir $*). $* is another automatic variable that expands to the pattern that matches % in the current rule, and notdir will strip any directory parts so you're left with the filename stem with no extension.
Sed then prepends $(dir $#) to the object file target, which is the same thing as $(#D) we saw above, and adds the dependency file itself ($#) as a target of the same prerequisites. This output is redirected to a file with the name of the current target + .tmp.
mv $#.tmp $#
Moves the previous file to the real target of the rule.
Side note: if you don't mind the dependency files being generated in the same directory as the object files then these recipes are outdated, you can achieve the same thing with something like:
sources := src/somefile1.c src/somefile2.c
objects := $(sources:src/%.c=obj/%.o)
deps := $(objects:.o=.d)
CFLAGS := -MMD -MP
.PHONY: all
all $(objects)
$(objects): obj/%.o: src/%.c
$(COMPILE.c) $(OUTPUT_OPTION) $<
-include $(deps)

Makefile automatic variable changed by prerequisite

first time here. I am relatively new to makefiles. Here is my current makefile:
# Closure compiler php script path
closure = ../../cli/scripts/Compilers/closure.php
# Destination directory
dest = ../../static/js/
# Build directory
build = build/
# Tell "make" to search build and destination dirs
vpath %.o $(build)
vpath %.js $(dest)
all: main.js
#echo "Done.";
main.js: \
date.o \
jquery.autocomplete.o \
jquery.bullseye.o \
jquery.clickopen.o \
jquery.fbmodal.o \
jquery.helpers.o \
jquery.pulljson.o \
jquery.thumbrotate.o \
jquery.timefmt.o \
jquery.tools.o \
layout.main.o
cat $^ > $(dest)$#
%.o: %.js
php $(closure) $*.js $(build)$#
clean:
rm -rf $(build)*.o
rm -rf $(dest)*.js
The problem is with the following line:
cat $^ > $(dest)$#.
It is supposed to cat all the prerequisite objects (minified javascript) into one final js library. According to makefile docs, $^ is an automatic variable which contains a list of prerequisites with directories they are in. From my experience, it behaves differently depending on wether prerequisite needs to be compiled or not.
If prerequisite is up-to-date, this code works perfectly and $^ contains a list like:
build/date.o build/jquery.autocomplete.o build/jquery.bullseye.o....
However if prerequisite needs a fresh compile, then $^ gets directory part stripped and looks like:
date.o jquery.autocomplete.o jquery.bullseye.o
Only the file which needs a fresh compile gets directory part stripped.
I have managed to work around this issue by replacing
cat $^ > $(dest)$#
with
cat $(addprefix $(build), $(^F) ) > $(dest)$#.
I don't like it because:
It's a hack
$(^F) is semi-deprecated
I want to understand why make behaves like this.
thanks
Look here:
# Tell "make" to search build and destination dirs
vpath %.o $(build)
If Make is looking for foo.o, it will look in the local directory first. If it finds no foo.o there, it will look in $(build) (i.e. build/, and you might reconsider your variable names).
And how would Make build foo.o, if it couldn't find it anywhere? With this rule:
%.o: %.js
php $(closure) $*.js $(build)$#
This rule violates an important guideline of makefiles, in that the target (foo.o) is not the name of the thing actually built (build/foo.o).
Now consider what happens when Make tries to execute this rule:
main.js: date.o ...
cat $^ > $(dest)$#
So if date.o is up to date, it's in build/. Make finds it there, and the automatic variable $^ expands to build/date.o ...
But if date.o must be rebuilt, then Make looks to the %.o rule, which promises to build date.o (not build/date.o), so Make takes that rule at its word and $^ expands to date.o ...
There are several ways to solve this problem. I'd do something like this:
OBJS := date.o jquery.autocomplete.o jquery.bullseye.o ...
OBJS := $(addprefix $(build),$(OBJS))
$(dest)main.js: $(OBJS)
cat $^ > $#
# you might have to tinker with this rule a little
$(build)%.o: %.js
php $(closure) $< $#

Iterate to next prerequisite file when matching when given variables for statement

I'm trying to do something like this with make:
SRC := $(src/*.md)
DIST := $(subst -,/,$(patsubst src/%.md, dist/%/index.html, $(SRC)))
all: $(DIST)
$(DIST): $(SRC)
mkdir -p $(#D) && pandoc $< -o $#
E.g., the prerequisite src/2014-04-myfile.md is put into target dist/2014/04/myfile/index.html with the transform pandoc
But when I use $< it only refers to the first argument in the $(SRC) variable.
I know normally we would do something like:
dist/%.html: src/%.md
but since I changed the file name in the output to just index.html for all files and used the original file name to create a new path I'm not sure how to go about iterating over the prerequisites.
Here's one way it could be done. The way this works is that it iterates over $(SRC) to create one rule per source file. The $$ in MAKE_DEP are necessary to prevent make from interpreting the functions when it first reads the contents of MAKE_DEP. The documentation on call and eval are also useful.
SRC := $(wildcard src/*.md)
# Set the default goal if no goal has been specified...
.DEFAULT_GOAL:=all
#
# This is a macro that we use to create the rules.
#
define MAKE_DEP
# _target is a temporary "internal" variable used to avoid recomputing
# the current target multiple times.
_target:=$$(subst -,/,$$(patsubst src/%.md, dist/%/index.html, $1))
# Add the current target to the list of targets.
TARGETS:=$$(TARGETS) $$(_target)
# Create the rule proper.
$$(_target):$1
mkdir -p $$(#D) && pandoc $$< -o $$#
endef # MAKE_DEP
# Iterate over $(SRC) to create each rule.
$(foreach x,$(SRC),$(eval $(call MAKE_DEP,$x)))
.PHONY: all
all: $(TARGETS)
If I create:
src/2000-01-bar.md
src/2014-04-foo.md
and run $ make -n, I get:
mkdir -p dist/2000/01/bar && pandoc src/2000-01-bar.md -o dist/2000/01/bar/index.html
mkdir -p dist/2014/04/foo && pandoc src/2014-04-foo.md -o dist/2014/04/foo/index.html
This could also be done using secondary expansion but it did not appear to me to be simpler or nicer.

Resources