Background
I am writing several books in Markdown. My files are structured as follows:
Description
writing/
Makefile 1. main Makefile (shown below)
book.template 2. pandoc template that uses TITLE
books/
current.txt 3. contains the current book name
book1/
meta.mk 4. sub-Makefile that defines TITLE
chapters/
01.md 5. actual text of book 1, chapter 1
02.md
...
book2/
meta.mk
chapters/
01.md
02.md
...
...
Here is the Makefile:
CURR_BOOK_NAME:=$(shell cat books/current.txt)
CURR_BOOK_DIR:=books/$(CURR_BOOK_NAME)/
CURR_CHAPTERS_DIR:=$(CURR_BOOK_DIR)chapters/
CURR_CHAPTERS:=$(wildcard $(CURR_CHAPTERS_DIR)*.pdf)
# suppose that each meta.mk defines the TITLE variable
include $(CURR_BOOK_DIR)/meta.mk
all: pdfs
...
pdfs: $(CURR_CHAPTERS)
%.pdf: %.md book.template
pandoc -o $# $< ... \
--template=book.template \
--variable=title:$(TITLE)
I usually work on only one book at a time. Thus, it was convenient to create a file current.txt with the name of the current book. Now I just type make to compile the current book to PDF by having the Makefile read in current.txt. Note that the PDF depends on a variable defined in the book-specific meta.mk.
Question
Occasionally, I want to make a small change to another book. How should I modify the Makefile so that I don’t have to update current.txt and then change it back each time? To be more precise, I would like to detect whether the arguments passed to make on the command line are phony targets or paths. For example, I would like the process to look like:
$ cat books/current.txt
book1
$ ls books/*/chapters/*
books/book1/chapters/01.md books/book1/chapters/02.md
books/book2/chapters/01.md books/book2/chapters/02.md
$ make
pandoc -o books/book1/chapters/01.pdf ... --variable=title:One
pandoc -o books/book1/chapters/02.pdf ... --variable=title:One
$ ls books/*/chapters/*
books/book1/chapters/01.md books/book1/chapters/02.md
books/book1/chapters/01.pdf books/book1/chapters/02.pdf
books/book2/chapters/01.md books/book2/chapters/02.md
$ make books/book2/chapters/01.pdf
pandoc -o books/book2/chapters/01.pdf ... --variable=title:Two
$ ls books/*/chapters/*
books/book1/chapters/01.md books/book1/chapters/02.md
books/book1/chapters/01.pdf books/book1/chapters/02.pdf
books/book2/chapters/01.md books/book2/chapters/02.md
books/book2/chapters/01.pdf
Possible solutions
It was suggested to override the variable on the command line:
make CURR_BOOK_NAME=book2 books/book2/chapters/01.pdf
However, I think this is too verbose and redundant, since it requires repeating the name of the book twice, and typing the name of the internal variable CURR_BOOK_NAME once.
Note
This is a simplified example. Please ask if you want to see the actual Makefile. Also, feel free to clarify the question title.
The following valuable answer was posted earlier, but after a short discussion of pros/cons in the comments, the answerer deleted it and left a downvote without opting to comment. I reproduce it here in case it helps other users. I am still looking for a “more complicated” solution that avoids the redundancy and allows building individual chapters.
A simple solution would be copy your Makefile to a new file, say book.mak and delete the first line CURR_BOOK_NAME:=$(shell cat books/current.txt). Then create new Makefile like this:
CURR_BOOK_NAME:=$(shell cat books/current.txt)
current:
$(MAKE) -f book.mak CURR_BOOK_NAME="$(CURR_BOOK_NAME)"
book1:
$(MAKE) -f book.mak CURR_BOOK_NAME="book1"
book2:
$(MAKE) -f book.mak CURR_BOOK_NAME="book2"
Then when you change something in book1 while book2 is current just type make book1. The makefile will figure out what's changed and update it.
If you really want to be able to type make books/book2/chapters/01.pdf then it's a fair bit more complicated.
I would restructure this so that your top-level Makefile is referenced from the top-level directory of each individual project. Think of it as a support library for each book project and manage it accordingly.
The individual Makefile can then be as simple as
include /usr/local/share/lib/bookmaker/main.mk
... assuming you call the library bookmaker and install it at this path. (It could live in a tree somewhere below your home directory just as well.)
I would think of this as a normalization of your de facto project structure, more than a new arrangement. Your individual books already depend on the bookmaker Makefile, but forcing them to live in physical subdirectories makes it harder to work on an experimental clone (assuming you manage each individual book as a separate Git project -- if not, switching to this model probably makes even more sense!). You can also get rid of the minor but pesky inconvenience of the "current" state file you will now obviously no longer need.
Related
I am working on a writing project and would like to use make for running pandoc on files. So far I've tried to pass arguments to make like I do with a bash script.
For example:
$ make chapter 2
In the make file chapter is the target and 2 would be the argument.
I don't know if makefiles have the facility to take cli arguments. I haven't been able to find what I'm looking for in the documentation.
So far I have tried to run make with this recipe.
chapter:
#pandoc -s -o "$1.epub" "$1.md"
I get this error back
pandoc: .md: openBinaryFile: does not exist (No such file or directory)
make: *** [Makefile:2: chapter] Error 1
This is for turning sections of a book I'm working on into epubs. I'm open to other ways to do this with make seeing as tokens don't to work.
In the make file chapter is the target and 2 would be the argument
$ make chapter num=2
The assignment to variables on the make command-line overrides any definition inside the makefile
(yep, such variables effectively become read-only).
This suggests a makefile something like:
num = $(error You must set $$num to the chapter number you want (make chapter num=4))
.PHONY: chapter
chapter:
pandoc -s -o "${num}.epub" "${num}.md"
What's going on here?
Well, if you forget to set num,
when make expands the recipe for chapter the
$(error) will cause make to stop.
$ make
Makefile:5: *** You must set $num to the chapter number you want (make chapter num=4). Stop.
And your original example?
$ make chapter num=2
pandoc -s -o "2.epub" "2.md"
Tips
I rarely recommend using the # prefix — Users can use make's -s if they don't want to see the shell commands
Don't lie to make — In particular, your rule does not produce a file called chapter, so please tell make that by marking the target .PHONY
The natural way to say this in Make is to enumerate all the chapters as targets, typically as dependencies for make all.
So basically
src := $(wildcard *.md)
epubs := $(patsubst %.md,%.epub,$(src))
.PHONY: all
all: $(epubs)
%.epub: %.md
pandoc -s -o $# $<
You can say make ch4.epub if you have a chapter whose source is ch4.md. You can't really pass in an argument which isn't a file name or a target name, and these cannot contain spaces.
I suppose you could add a phony like
.PHONY: 2
2: ch2.epub
to be able to say make 2 and have it mean make ch2.epub. If file names are systematically named like this, you could generalize to
short := $(patsubst ch%.md,%,$(src))
.PHONY: $(short)
$(short): %: ch%.epub
Don't use # in front, it just makes things harder. You can use make -s if you don't want to see the output and not wreck your Makefile.
I've just begun to study Porting Android. And I come across a new type of file which is .mk file. It's is an extension of Makefile but I don't know what it is different from a Makefile ? So, can somebody help you clarify them. Thanks very much !
A make file can have any name. The -f option of make is used to specify which file to use:
make -f foobar
You can even use -f several times:
make -f foo -f bar
In which case make processes the files in order (or, equivalently, concatenates the files and processes the result).
makefile and Makefile are special names because if make is called without the -f option it automatically searches for them, in this order, and use the first it finds. Note that GNU make also considers GNUmakefile, and prefers it over makefile and Makefile. Other make implementations can have other default names.
The .mk extension is a more or less standard extension for make files that have other names than the defaults. It is a reasonable extension if you want humans to quickly understand what these files are: convert.mk is more informative than foobar. Some editors use this extension to identify the file type and apply syntax coloring. They usually apply the same syntax coloring to makefile and Makefile.
I have a project that involves sub-directories with sub-makefiles. I'm aware that I can pass variables from a parent makefile to a sub-makefile through the environment using the export command. Is there a way to pass variables from a sub-makefile to its calling makefile? I.e. can export work in the reverse? I've attempted this with no success. I'm guessing once the sub-make finishes its shell is destroyed along with its environment variables. Is there another standard way of passing variables upward?
The short answer to your question is: no, you can't [directly] do what you want for a recursive build (see below for a non-recursive build).
Make executes a sub-make process as a recipe line like any other command. Its stdout/stderr get printed to the terminal like any other process. In general, a sub-process cannot affect the parent's environment (obviously we're not talking about environment here, but the same principle applies) -- unless you intentionally build something like that into the parent process, but then you'd be using IPC mechanisms to pull it off.
There are a number of ways I could imagine for pulling this off, all of which sound like an awful thing to do. For example you could write to a file and source it with an include directive (note: untested) inside an eval:
some_target:
${MAKE} ${MFLAGS} -f /path/to/makefile
some_other_target : some_target
$(eval include /path/to/new/file)
... though it has to be in a separate target as written above because all $(macro statements) are evaluated before the recipe begins execution, even if the macro is on a later line of the recipe.
gmake v4.x has a new feature that allows you to write out to a file directly from a makefile directive. An example from the documentation:
If the command required each argument to be on a separate line of the
input file, you might write your recipe like this:
program: $(OBJECTS)
$(file >$#.in) $(foreach O,$^,$(file >>$#.in,$O))
$(CMD) $(CMDFLAGS) #$#.in
#rm $#.in
(gnu.org)
... but you'd still need an $(eval include ...) macro in a separate recipe to consume the file contents.
I'm very leery of using $(eval include ...) in a recipe; in a parallel build, the included file can affect make variables and the timing for when the inclusion occurs could be non-deterministic w/respect to other targets being built in parallel.
You'd be much better off finding a more natural solution to your problem. I would start by taking a step back and asking yourself "what problem am I trying to solve, and how have other people solved that problem?" If you aren't finding people trying to solve that problem, there's a good chance it's because they didn't start down a path you're on.
edit You can do what you want for a non-recursive build. For example:
# makefile1
include makefile2
my_tool: ${OBJS}
# makefile2
OBJS := some.o list.o of.o objects.o
... though I caution you to be very careful with this. The build I maintain is extremely large (around 250 makefiles). Each level includes with a statement like the following:
include ${SOME_DIRECTORY}/*/makefile
The danger here is you don't want people in one tree depending on variables from another tree. There are a few spots where for the short term I've had to do something like what you want: sub-makefiles append to a variable, then that variable gets used in the parent makefile. In the long term that's going away because it's brittle/unsafe, but for the time being I've had to use it.
I suggest you read the paper Recursive Make Considered Harmful (if that link doesn't work, just google the name of the paper).
Your directory structure probably looks like this:
my_proj
|-- Makefile
|-- dir1
| `-- Makefile
`-- dir2
`-- Makefile
And what you are doing in your parent Makefile is probably this:
make -C ./dir1
make -C ./dir2
This actually spawns/forks a new child process for every make call.
You are asking for updating the environment of the parent process from its children, but that's not possible by design (1, 2).
You still could work around this by:
using a file as shared memory between two processes (see Brian's answer)
using the child's exit error code as a trigger for different actions [ugly trick]
I think the simplest solution is using standard out from a sub Makefile.
Parent Makefile
VAR := $(shell $(MAKE) -s -C child-directory)
all:
echo $(VAR)
Child Makefile
all:
#echo "MessageToTheParent"
Is there an option to output the "preprocessed" makefile, something equivalent to the GCC's -E option?
I have a project comprised of an hierarchy of dozens of modules, each with its makefile. The build is invoked from a master makefile. That master makefile contains includes, variable definitions, command line option dependent variables, etc.
So, essentially, I am looking for the processed makefile, including all substitutions.
Not that I'm aware of. The closest thing you can get to this is the output from make -qp (or similar) which will dump the make database out at you.
Part of the problem with this request is that many of the substitutions/etc. happen as targets are processed and the list of targets isn't necessarily known without actually attempting a build (at least to an extent) so it isn't necessarily possible to fully expand/etc. a makefile in-place.
The make -d output is also useful for certain incidental information related to how make has processed the makefiles but doesn't contain makefile contents directly.
Remake might also be able to provide some extra useful information.
If you are looking for the computed value of some assembled/etc. global make variable then this blog post by Eric Melski is likely to be very helpful.
tl;dr It adds a target like this to the Makefile (though there's more magic in the blog post so I suggest reading it).
print-%:
#echo '$*=$($*)'
#echo ' origin = $(origin $*)'
#echo ' flavor = $(flavor $*)'
#echo ' value = $(value $*)'
Though in personal use I replaced that first line with something more like this
#echo '$*=$(subst ','\'',$($*))'
to keep the quoting of the result correct.
This is related to my previous question: Why does .PHONY not work in this situation?.
I have a makefile system that I wrote to make it easy for developers who are not familiar with make, to do their tasks. In short, there is a generic portion which would be the same for all projects, and a set of makefiles that are specific for a given project. The project specific ones include the generic ones. It worked great on make 3.80 for some reason, but when I tried it out on make 3.81 I ran into a few problems. That forced me to make changes that are mentioned in the above post. Now I have some new problems, so I decided to make another post. Like in that post, I made a much smaller and simpler set of makefiles that show the problem. Unfortunatly, the "simple" case consists of 6 files. Sorry about that. First I'll start with the "project specific" ones (these are meant to be simple):
makefile:
TARGETS:=\
Lib1.mk \
Lib2.mk \
my_prog.mk \
include generic/top.mk
Lib1.mk:
BINARY:=Lib1
TYPE:=LIB
LOCATION:=a/location
include generic/rules.mk
Lib2.mk:
BINARY:=Lib2
TYPE:=LIB
LOCATION:=another/location
LIBS:=Lib1
include generic/rules.mk
my_prog.mk:
BINARY:=my_prog
TYPE:=EXE
LOCATION:=some/location
LIBS:=Lib1 Lib2
include generic/rules.mk
A quick description: Makefile simply lists the names of all the targets. A target is either a executable or a library. BINARY is the name of the library or executable (extensions are added by the generic part). TYPE is either EXE or LIB. LOCATION is where the binary should go. LIBS is whatever libraries this binary depends on. The real ones handles creating all the -L, rpath, etc. stuff for the user (as well as their equivalents for visual studio). Now for the generic ones (these do the REAL work):
generic/top.mk:
ALL_BINS:=
.PHONY: all
all:
include $(TARGETS)
all: $(ALL_BINS)
%.so %.exe:
mkdir -p $(dir $#)
touch $#
clean:
rm -rf out
and finally..
generic/rules.mk:
ifeq (EXE,$(TYPE))
$(BINARY).FULL_FILE_NAME:=out/$(LOCATION)/$(BINARY).exe
else
$(BINARY).FULL_FILE_NAME:=out/$(LOCATION)/lib$(BINARY).so
endif
$(BINARY).DEP_LIBS:=$(foreach a,$(LIBS),$($(a).FULL_FILE_NAME))
ALL_BINS+=$(BINARY)
$(BINARY): $($(BINARY).FULL_FILE_NAME)
$($(BINARY).FULL_FILE_NAME): $($(BINARY).DEP_LIBS)
BINARY:=
LOCATION:=
LIBS:=
Ok, in this state, things work fine. Make handles all the dependencies correctly, and if I touch any of the files, it will correctly "build" only the ones that it has to, and nothing more. The problem happens when you take the m_prog.mk line from makefile and move it to the top of the list, like so:
TARGETS:=\
my_prog.mk \
Lib1.mk \
Lib2.mk \
The problem seems to be that while its is going through rules.mk for my_prog.mk it does not yet know what the full library path for Lib1 and Lib2 (they are empty strings). So in the end, it considers my_prog to be dependent on nothing and it tries to build it out of order. In this example, you just see it "touch" my_prog first and then the other 2. Of course, when I have real compiler and linker commands in there, it throws an error.
Back when I simply had the .PHONY targets depend on each other (so my_prog depended on Lib1 and Lib2) life was easy and harmonious. Now that I can't do that, life became more difficult.
You may say, "heck just put it in the right order!". Well up to now, this has been handled automatically through make for the end users. In fact, most customers have been putting things in alphabetical order. They don't know or care what order they depend on each other. It would stink to have to tell them to re-order all of that now. Sorry for the length of this post. I'd appreciate any answers!
If you set variables using the := assignment operator, the assignment is evaluated immediately.
If you set variables using just = as the assignment operator, they're evaluated lazily, as late as possible (at the time of actual use).
See http://www.gnu.org/software/automake/manual/make/Flavors.html.
There are several ways to do what you want. The cleanest is probably by using vpath. Just modify rules.mk:
$(BINARY).DEP_LIBS:=$(foreach a,$(LIBS),$(a).so)
ALL_BINS+=$(BINARY)
vpath %.so out/$(LOCATION)