I am trying to automate building man pages in my project with make.
I have the following directory structure:
my_project
man/
man1/
man3/
man5/
man7/
Inside each of those directories, there is one or more .md files.
I would like to generate *.1, *.3, ..., pages when the corresponding man/man{1,3,5,7}/*.md is updated.
The relevant parts of my Makefile are:
MY_DOC := $(wildcard $(MY_PROJECT)/man/*/*.md)
all: $(MY_DOC:%.md=%.1)
%.1 %.3 %.5 %.7 : %.md
#ronn --roff $?
The above is building .1 pages as intended, but .3, .5 and .7 always.
I can see $(MY_DOC:%.md=%.1) is replacing all .md with .1 which is one of the problems, but I don't know how to change that to mean 1, 3, 5 or 7 depending on the parent directory.
If you can be sure the directory name is always part of the filename you're using, which seems to be the case here, then it is just a matter of using GNU Make's patsubst function; the syntax you're using now is just a convenient shorthand for simple cases, and your case is more complex.
I was able to build man pages as intended and still use the substitution syntax.
MY_DOC := $(MY_PROJECT)/man
MAN1 := $(wildcard $(MY_DOC)/man1/*.md)
MAN3 := $(wildcard $(MY_DOC)/man3/*.md)
MAN5 := $(wildcard $(MY_DOC)/man5/*.md)
MAN7 := $(wildcard $(MY_DOC)/man7/*.md)
all: $(MAN1:%.md=%.1) $(MAN3:%.md=%.3) $(MAN5:%.md=%.5) $(MAN7:%.md=%.7)
%.1 %.3 %.5 %.7 : %.md
#ronn --roff $?
I think there must be a simpler way to achieve this, but this is the best I was able to come up with.
Related
i have a list of input*.in files, and for each of them i want to generate an output*.out file using make.
target1.out : input1.in
{external program} < input1.in > target1.out
I have 0 experience with gnu make, so would much appreciate help
How should i do it best using makefile?
Are you looking for this?
target%.out: input%.in
{external program} < $< >$#
Now make knows how to generate targetXX.out from inputXX.in. You still have to add an actual target, either on the command line (like make targetShirley.out assuming you have inputShirley.in) or in a separate recipe. A common arrangement is to have
sources := $(wildcard input*.in)
targets := $(patsubst input%.in,target%.out,$(sources))
.PHONY: all
all: $(targets)
(typically at the top of the Makefile) and then make all will produce all the files it can.
A common antipattern is to write a single recipe with a loop, like
everything:
for file in input*.in; do \
t=$${file%.in}; \
t=target$${t#input}.out; \
{external program} >"$$t"; \
done
(notice also how the shell's dollar signs have to be doubled to prevent make from attempting to interpret them). Besides being clumsy, this disables the single crucial reason to use make in the first place, namely to avoid creating files which are already up to date.
I am aware of tools like CMake and GNU Autotools but I'm trying to write a universal build system myself, to use for my C and C++ projects. I'll briefly explain how it works and hopefully, someone can suggest either improvements or a better design altogether.
The build system proper lives in one of the project's subdirectories (I import it as a Git submodule). The project's root directory has a wrapper makefile that defines a couple of macros and includes the main makefile from said subdirectory. That does most of the work: it follows the directory organization scheme (i.e., it outputs libraries in lib, binaries in bin, etc.), it handles automatic dependencies for the source code and the DocBook documentation, and provides the de facto standard targets: all, test, clean, install, as well as others.
Here's what a wrapper makefile that builds two binaries, foo and bar, might look like:
# foo-specific macros
FOO_SRC_FILES = foo1.c foo2.c foo3.c
FOO_OBJ_FILES = $(FOO_SRC_FILES:.c=.o)
FOO_BIN_FILE = foo
# bar-specific macros
BAR_SRC_FILES = bar1.c bar2.c
BAR_OBJ_FILES = $(BAR_SRC_FILES:.c=.o)
BAR_BIN_FILE = bar
# Inform the build system about them
SRC_FILES = $(FOO_SRC_FILES) $(BAR_SRC_FILES)
OBJ_FILES = R(BAR_OBJ_FILES) $(BAR_OBJ_FILES)
BIN_FILES = $(FOO_BIN_FILE) $(BAR_BIN_FILE)
# Only install the binaries. If I were building a library, I would instead
# select the "lib" and perhaps "include" directories.
INSTALL = bin
INSTALL_DIR = /usr/share
# Use the build system
include build/build.mk
Now here's the problem. While build.mk can use pattern rules to create dependency and object files, there's only one OBJ_FILES and only one BIN_FILES. So if I put a pattern rule like the following in the build system that looks like this:
$(BIN_DIR)/$(BIN_FILES): $(OBJ_FILES:%=$(OBJ_DIR)/%) $(LIB_FILES:%=$(LIB_DIR)/%) | $(BIN_DIR)
$(CC) $(LDFLAGS) -o $# $(OBJ_FILES:%=$(OBJ_DIR)/%) -L $(LIB_DIR) $(LIB_FILES:lib%.a=-l %)
then foo would depend on and link with everything that bar does and vice versa. So what I end up doing is asking the user to put these rules in the wrapper makefile, even though they feel like they belong in build.mk:
$(BIN_DIR)/$(FOO_BIN_FILE): $(FOO_OBJ_FILES:%=$(OBJ_DIR)/%) $(FOO_LIB_FILES:%=$(LIB_DIR)/%) | $(BIN_DIR)
$(CC) $(LDFLAGS) -o $# $(FOO_OBJ_FILES:%=$(OBJ_DIR)/%) -L $(LIB_DIR) $(FOO_LIB_FILES:lib%.a=-l %)
$(BIN_DIR)/$(BAR_BIN_FILE): $(BAR_OBJ_FILES:%=$(OBJ_DIR)/%) $(BAR_LIB_FILES:%=$(LIB_DIR)/%) | $(BIN_DIR)
$(CC) $(LDFLAGS) -o $# $(BAR_OBJ_FILES:%=$(OBJ_DIR)/%) -L $(LIB_DIR) $(BAR_LIB_FILES:lib%.a=-l %)
The same issue applies to libraries as well, of course. The upside is that these rules can be copied and pasted almost verbatim; only the prefixes need to be changed (e.g., FOO or BAR).
Ideas to fix this include:
Asking the user to have separate wrapper makefiles for separate things (e.g., one for foo and another for bar) but that is just terrible.
Changing things up a little bit and then using m4 to do some preprocessing but I don't want to go through that unless a more elegant solution doesn't exist.
I would really appreciate some ideas.
PS: I know that the pattern matching expressions in the last two code samples can be replaced with text functions but those are GNU Make-specific. The style I used is more portable and is in fact on the list of additions for the next version of the POSIX standard.
I have begin to develop a similar system for my own C projects, but the logic I use does rely on some features which I believe are specific to GNU Make.
The main idea is to use a combinaison of $(eval) and $(call), by defining the logic of the build system, and then applying to the project tree.
To do so, I have in each of my directories and subdirectories a piece of Makefile of the following form, which I name Srcs.mk:
SRC := foo.c foo_bar.c bar.c
TARGET := foo_bar
SRC_DIR := src
OBJ_DIR := obj
I define a variable, which is in fact a macro, which is expanded with $(call) and then passed to $(eval). It's defined this way:
define get_local_variables
include Srcs.mk
$1SRC := $(SRC)
$1SRC_DIR := $(SRC_DIR)
$1OBJ_DIR := $(OBJ_DIR)
$1TARGET := $(TARGET)
TARGET :=
SRC :=
SRC_DIR :=
OBJ_DIR :=
$(call get_local_variables, $(DIR)) will expand to the above, with $1 replaced by the content of $(DIR). Then it will be treated as a Makefile fragment by $(eval)
This way, I fill per-directory variables for each of my directory.
I have then a handful or other rules which use this variables, using the same principles.
### Macros ###
obj = $(patsubst %.c,$($1OBJ_DIR)/%.o,$($1SRC))
define standard_rules
$($1TARGET): $(obj)
$$(LINK)
$(obj): $($1OBJ_DIR)/%.o:$($1SRC_DIR)/%.c | $($1OBJ_DIR)
$$(COMPILE)
endef
The variable are computed $(call), then expanded and read as makefile fragments by $(eval).
(I use static pattern rules but that it not intrinsic to the idea).
The whole idea is basically to define directories as a kind of namespace, with data attached to them, and then run function over them.
My actual system is a bit more complicated, but that the whole idea.
If you have a way to emulate $(eval) and $(call) (I think these are specific to GNU make, but not sure), you could try that approach.
You can also implement non recursive make this way, by adding a SUBDIRS variables in each directory and running recursively the same macro which is run on the current one. But it should been done carefully, not to mess it up with the order of expansion and evaluation in make.
So get_local_variables need to be evaluated before the rest of the macros are expanded.
(My project is visible on my Github account if you want to take a look, under make-build-system. But it is far from be complete enough^).
Be aware, though, that this is quite painful to debug when things go wrong. Make (at least, GNU) basically catch the error (when there is one) on the higher $(call) or $(eval) expansion.
I have developed my own non-recursive build system for GNU make, called prorab, where I solved the problem you described as follows.
The approach to solve your problem is somewhat similar to what #VannTen described in his answer, except that I use a macro to clean all state variables before defining build rules for the next binary.
For example, a makefile which builds two binaries could look like this:
include prorab.mk
this_name := AppName
this_ldlibs += -lsomelib1
this_cxxflags += -I../src -DDEBUG
this_srcs := main1.cpp MyClass1.cpp
$(eval $(prorab-build-app))
$(eval $(prorab-clear-this-vars))
this_name := AnotherppName
this_ldlibs += -lsomelib1
this_cxxflags += -I../src -DDEBUG
this_srcs := main2.cpp MyClass2.cpp
$(eval $(prorab-build-app))
So, in this example it will build two binaries: AppName and AnotherppName.
As you can see the build is configured by setting a number of this_-prefixed variables and the calling the $(eval $(prorab-build-app)) which expands to defining all the build, install, clean etc. rules.
Then a call to $(eval $(prorab-clear-this-vars)) clears all this_-prefixed variables, so that those can be defined again from scratch for the next binary, and so on.
Also, the very first line which includes the prorab.mk also cleans all this_-prefixed variables of course, so that the makefiles can be safely included into each other.
You can read more about that build system concepts here https://github.com/cppfw/prorab/blob/master/wiki/HomePage.adoc
I have a simple makefile that I use to build some latex files. The syntax looks like this:
pdf: thesis.tex chapters/a.tex chapters/b.tex chapters/c.tex
latexmk -pdf -pdflatex="pdflatex thesis.tex
open:
open thesis.pdf
The files inside chapters folder can increase further with d.tex, e.tex and may even contain subfolders f\section1.tex, f\section2.tex etc.
I manually add all the requried tex files inside my thesis.tex like this which is not a problem.
\input{chapters/a.tex}
\input{chapters/b.tex}
\input{chapters/c.tex}
\input{chapters/d.tex}
\input{chapters/e.tex}
How can I get make target pdf to depend upon any file changes inside chapters and its subdirectories?
How do I write inter task dependency in makefile. If target open depends upon target pdf, how do I write it?
open: pdf will sort-of do what you want for your second question.
Though it would be better to not use the phony pdf target for this.
Instead have a thesis.pdf: target which depends on the right prerequisites and have both pdf: thesis.pdf and open: thesis.pdf targets.
For the first question you can either use something like:
SRCS := $(shell find chapters -name '*.tex')
or use from here:
rwildcard=$(strip $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2) $(filter $(subst *,%,$2),$d)))
SRCS := $(call rwildcard,chapters,*.tex)
and then:
thesis.pdf: thesis.tex $(SRCS)
to use that variable as the prereq.
If you wanted to get even fancier you could write a script to pull out the actual filenames from the \input{} directives in thesis.tex and use that as your SRCS variable (but that's probably not worth the effort unless you know you will have other, unrelated, .tex files).
I have a C++ library built using a Makefile. Until recently, all the sources were in a single directory, and the Makefile did something like this
SOURCES = $(wildcard *.cpp)
which worked fine.
Now I've added some sources that are in a subdirectory, say subdir. I know I can do this
SOURCES = $(wildcard *.cpp) $(wildcard subdir/*.cpp)
but I'm looking for a way to avoid specifying subdir manually, that is, make wildcard look into subdirectories, or generating a list of subdirectories somehow and expanding it with several wildcard functions. At this point, having a non-recursive solution (that is, expanding only the first level) would be fine.
I haven't found anything - my best guess is using find -type d to list the subdirectories, but it feels like a hack. Is there any built-in way to do this?
This should do it:
SOURCES = $(wildcard *.cpp) $(wildcard */*.cpp)
If you change you mind and want a recursive solution (i.e. to any depth), it can be done but it involves some of the more powerful Make functions. You know, the ones that allow you to do things you really shouldn't.
EDIT:
Jack Kelly points out that $(wildcard **/*.cpp) works to any depth, at least on some platforms, using GNUMake 3.81. (How he figured that out, I have no idea.)
Recursive wildcards can be done purely in Make, without calling the shell or the find command. Doing the search using only Make means that this solution works on Windows as well, not just *nix.
# Make does not offer a recursive wildcard function, so here's one:
rwildcard=$(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))
# How to recursively find all files with the same name in a given folder
ALL_INDEX_HTMLS := $(call rwildcard,foo/,index.html)
# How to recursively find all files that match a pattern
ALL_HTMLS := $(call rwildcard,foo/,*.html)
The trailing slash in the folder name is required. This rwildcard function does not support multiple wildcards the way that Make's built-in wildcard function does, but adding that support would be straightforward with a couple more uses of foreach.
If you don't want to use recursive makefiles, this might give you some ideas:
subdirs := $(wildcard */)
sources := $(wildcard $(addsuffix *.cpp,$(subdirs)))
objects := $(patsubst %.cpp,%.o,$(sources))
$(objects) : %.o : %.cpp
You can use several rules in wildcard:
SOURCES := $(wildcard *.cpp */*.cpp)
if you need more depth:
SOURCES := $(wildcard *.cpp */*.cpp */*/*.cpp */*/*/*.cpp)
Unfortunately, and unlike what we sometimes read, glob (**) is not supported by makefile and will be interpreted as normal wildcard (*).
For example **/*.cpp match dir/file.cpp but neither file.cpp nor dir/sub/file.cpp.
If you need infinite depth use shell and find:
SOURCES := $(shell find . -name "*.cpp")
Common practice is to put a Makefile in each subdir with sources, then
all: recursive
$(MAKE) -C componentX
# stuff for current dir
or
all: recursive
cd componentX && $(MAKE)
# stuff for current dir
recursive: true
It may be wise to put settings for each Makefile in a Makefile.inc in the root source directory. The recursive target forces make to go into the subdirectories. Make sure that it doesn't recompile anything in a target requiring recursive.
If you can use find shell command, you may define a function to use it.
recurfind = $(shell find $(1) -name '$(2)')
SRCS := $(call recurfind,subdir1,*.c) $(call recurfind,subdir2,*.cc) $(call recurfind,subdir2,*.cu) \
...
I'm trying to consolidate some build information by using a common makefile. My problem is that I want to use that makefile from different subdirectory levels, which makes the working directory value (pwd) unpredictable. For example:
# Makefile.common
TOP := $(shell pwd)
COMPONENT_DIR := $(TOP)/component
COMPONENT_INC := $(COMPONENT_DIR)/include
COMPONENT_LIB := $(COMPONENT_DIR)/libcomponent.a
If I include Makefile.common from a subdirectory, like so, the $(TOP) directory is incorrect and everything else follows suit:
# other_component/Makefile
include ../Makefile.common
# $(COMPONENT_LIB) is incorrectly other_component/component
What's the best way to get Makefile.common to use its own directory path instead of the more fickle pwd?
You should be able to use the MAKEFILE_LIST variable, like this:
# This must be the first line in Makefile.common
TOP := $(dir $(firstword $(MAKEFILE_LIST)))
From the documentation:
As make reads various makefiles, including any obtained from the MAKEFILES variable, the command line, the default files, or from include directives, their names will be automatically appended to the MAKEFILE_LIST variable. They are added right before make begins to parse them. This means that if the first thing a makefile does is examine the last word in this variable, it will be the name of the current makefile. Once the current makefile has used include, however, the last word will be the just-included makefile.
Try this:
ROOT_DIR := $(dir $(realpath $(lastword $(MAKEFILE_LIST))))
Edit: Be sure to use := instead of = because the latter causes make to use late-binding and MAKEFILE_LIST may have changed due to later includes.
Have you tried doing:
# Makefile.common
TOP ?= $(shell pwd)
COMPONENT_DIR := $(TOP)/component
COMPONENT_INC := $(COMPONENT_DIR)/include
COMPONENT_LIB := $(COMPONENT_DIR)/libcomponent.a
# other_component/Makefile
TOP ?= ..
include ../Makefile.common
Using the ?= construct will keep TOP from being redefined if it is already set. You can set it to the appropriate value based on where you are in the tree when you invoke make. I confess it's been awhile since I've used GNU make so this may not work or may need some tweaks.
My solution:
cwd := $(shell readlink -en $(dir $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))))
This also works for calls like make -f /opt/some/dir/Makefile whenn your in /opt/other/path/subdir.
write the common stuff in common.mk. Then put the common.mk in the default directories that Make looks for when it encounters an include statement. See the manual for common directories Make looks for.
You could also put the common.mk in custom directory, and then type make -I customdir.
Inside the Makefile in each subfolder, you do
include common.mk
That is all. No need to worry about path and moving things around.