I have a Makefile which I use to call different sub-Makefiles.
I have several rules:
all
clean
fclean
re
I can already use those rules, which will call every sub makefile with the same rule.
I have several project, and I would like to generate rules with that format:
$(PROJECT_NAME)-$(RULES)
With that, I would like to have each rule for each project:
project1-all
project1-clean
...
project2-all
project2-clean
...
This way, I would be able to call a specific rule, for a specific project, like project1-fclean.
I tried that:
RULES= all clean fclean re
PROJECTS= project1 project2
define NEWLINE
endef
$(foreach _rule, $(RULES), \
$(foreach _proj, $(PROJECTS), \
$(_proj)-$(_rule): $(NEWLINE) \
$(MAKE) $(ARGS) $(PROJECT_DIR)$(_proj) $(_rule) $(NEWLINE) \
) \
)
But it doesn't seems to work. I have searched, but I haven't found advanced makefile techniques to achieve that. Plz help.
The problem is that when you combine lines together with line continuations like that, it compresses out all the newlines and other extraneous whitespace (including those newlines you're trying to insert with $(NEWLINE)) resulting in a huge mess on a single line, rather than multiple lines with multiple patterns. To do this correctly, you need to write your rule as a macro with arguments and then call it:
define PROJ_RULE
$(1)-$(2):
$(MAKE) $(ARGS) $(PROJECT_DIR)$(1) $(2)
endef
$(foreach _rule, $(RULES),
$(foreach _proj, $(PROJECTS),
$(eval $(call PROJ_RULE, $(_proj), $(_rule)))))
note that all this define and foreach stuff in GNU make specific -- other make flavors do not support it.
Okay, so I finally managed to do it this way:
$(foreach _rule, $(RULES), $(addsuffix -$(_rule),$(PROJECTS))):
$(eval _rule := $(lastword $(subst -, ,$#)))
$(eval _proj := $(#:%-$(_rule)=%))
#$(MAKE) $(ARGS) $(PROJECT_DIR)$(_proj) $(_rule)
I will decompose it for a better explanation:
$(foreach _rule, $(RULES), ...)):
We loop on every RULES and store it in _rule.
$(addsuffix -$(_rule),$(PROJECTS))
We add that rule as a prefix to each of our project. This part generate a rule with every "composed rules". With projet1 and project2 it should result in:
project1-all project2-all project1-clean project2-clean project1-fclean project2-fclean project1-re project2-re:
This way, for any of those rules name, it will be the same rule executed.
$(eval _rule := $(lastword $(subst -, ,$#)))
Here we take the target (if I call project2-clean, $# will be project2-clean), we replace - by a space to obtain project2 clean and take the last work, wich will be clean here. We then evaluate it to store that into _rule.
$(eval _proj := $(#:%-$(_rule)=%))
We use the same technique to store the project name into _proj. We just use a pattern replacement, to remove the rule name and the dash.
#$(MAKE) $(ARGS) $(PROJECT_DIR)$(_proj) $(_rule)
Finally, we call our submakefile we the right path and right rule!
Related
I have, for example, the following Makefile to generate PDF files from Markdown files in subdirectories:
FOLDERS = f1 f2 f3
.PHONY: $(FOLDERS)
f1: f1/f1.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
f2: f2/f2.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
f3: f3/f3.md
cd $# && pandoc $(notdir $^) -o $(patsubst %.md,%.pdf,$(notdir $^))
The expected result is that make f1 requires the existence of f1/f1.md, and generates the resulting PDF as f1/f1.pdf. The same for f2 and f3. This works, but the declarations seem unnecessarily repetitive.
Is there any way to combine these three rules into one, generic rule? That is, without needing to explicitly write out all of the paths to the PDF files or Markdown files, as I may be dynamically adding subfolders and I'd prefer to just change the definition of FOLDERS in the first line. I've googled around and tried a few things, but I feel like either I can't find the right incantation to use, or I'm missing a piece of knowledge about how Makefiles work. Could someone please point me in the right direction?
First, note that there's no good reason to use PHONY targets here, since these rules appear to be building files whose names are known beforehand. Targets like f1/f1.pdf would be much better.
Unfortunately we can't use a pattern rule when the stem (e.g. f1) is repeated in a prerequisite. But a "canned recipe" can do the trick:
define pdf_template
$(1): $(1)/$(1).md
cd $$# && pandoc $$(notdir $$^) -o $$(patsubst %.md,%.pdf,$$(notdir $$^))
endef
$(eval $(call pdf_template,f1))
$(eval $(call pdf_template,f2))
$(eval $(call pdf_template,f3))
(Note how you must escape the $ signs in the template.)
If those $(eval...) lines look too repetitive, you can replace them with a loop:
$(foreach folder,$(FOLDERS),$(eval $(call pdf_template,$(folder))))
EDIT: Come to think of it, there's another way. You can't construct a pattern rule that uses the stem more than once:
$(FOLDERS): %: %/%.md
cd $# && ... this won't work
And you can't use the automatic variables in the prerequisite list, because they aren't yet defined when they're needed:
$(FOLDERS): $#/$#.md
cd $# && ... this won't work either
But you can use them there if you use Secondary Expansion, which causes Make to expand the prereq list a second time:
.SECONDEXPANSION:
$(FOLDERS): $$#/$$#.md
cd $# && ... this works
Again, note the escaped $ symbols.
I have a directory structure like
packages/
foo/
lib/
a.js
b.js
bar/
lib/
c.js
d.js
and I'm trying to write a simple Makefile to run the source files through a compiler and output the result of compiling packages/foo/lib/a.js to packages/foo/build/a.js. In the past, I've done something like
JS = $(shell find packages/foo/lib -name "*.js")
BUILD = $(patsubst packages/foo/lib/%.js, packages/foo/build/%.js, $(JS))
.PHONY: all
all: $(BUILD)
packages/foo/lib/%.js: packages/foo/build/%.js
# Compile $^ and output to $#
which works great. However, I'm now doing
JS = $(shell find packages/*/lib -name "*.js")
BUILD = $(patsubst ...)
The problem here is that patsubst doesn't seem to like multiple % wildcards (ie packages/%/lib/%.js). However, if I use packages/%.js, then I can't know which directory to use in the substitution. I'm convinced there's a very simple way to do this that I can't find in the make docs.
To define BUILD, you could use the stupid but simple solution:
BUILD = $(subst /lib/,/build/,$(JS))
as long as you aren't worried about additional "lib" and "build" pathname components in your source.
However, this leaves you with the problem of how to define the actual Makefile rules, unless you want to manually specify a rule for each directory foo, bar, etc.
Instead, maybe try something like this (remember to replace spaces with tabs again where appropriate, if you cut and paste):
PACKAGES = $(wildcard packages/*)
ALL :=
define add_package
JS := $$(shell find $(1)/lib -name "*.js")
BUILD := $$(patsubst $(1)/lib/%.js, $(1)/build/%.js, $$(JS))
$(1)/build/%.js:: $(1)/lib/%.js
echo COMPILE $$^ '->' $$#
cp $$^ $$# # super fast compiler!
ALL += $$(BUILD)
endef
$(foreach p, $(PACKAGES), \
$(eval $(call add_package,$(p))))
all: $(ALL)
This evaluates a template for each package directory, so you can put anything in there, as long as you get the fiddly doubling of dollar signs correct.
I have 2 types of files. Scripts and datasets. I want to write a makefile to run each script with each dataset.
My idea was to create a target for each output, where the target has a name containing the script and the model name. These targets would match a "target-pattern"-rule (of type: some_folder/%.eval). Now the "target-pattern"-rule would need to figure out from its name which files are needed. But this seems to be a hard job.
Is there a better, more elegant way?
Example:
Files: ScriptA, ScriptB, InputA, InputB
Targets/Outputfiles: InputA_ScriptA, InputA_ScriptB, InputB_ScriptA, InputB_ScriptB
# generate all combinations
RT_HW = $(foreach script,$(RT_SCRIPT_HW),$(foreach input, $(RT_INPUTS), $(input)_$(script)))
$(SANDBOX)%.eval: <requires Script X and input X>
You've run into one of the major shortcomings of Make: it isn't very good with wildcards.
You can get the effect you want by generating rules with eval, either by iterating over both variables:
define template
$(1)_$(2).eval: $(1) $(2)
#echo target is $$#
#echo running $(2) on $(1)
endef
$(foreach script,$(RT_SCRIPT_HW),$(foreach input, $(RT_INPUTS), $(eval $(call template,$(input),$(script)))))
or by generating pattern rules, iterating over only one, such as the input:
define template
$(1)_%.eval: $(1) %
#echo target is $$#
#echo running $$* on $(1)
endef
$(foreach input, $(RT_INPUTS), $(eval $(call template,$(input))))
I'm writing a makefile which builds several platforms (windows, iOS, and so on) for different configurations (debug, release). I managed to make it work, it's building properly, but I'd like to make some of my rules more generic so that I don't have to repeat them for each configuration.
Each file is kept in a given folder structure. For instance, if building for iOS with a debug configuration, the resulting file is kept in ios/debug. As an example, I have rules for creating those directories:
define libdir
$(BINDIR)/$(strip $1)/$(strip $2)
endef
$(call libdir, ios, debug):
mkdir -p $(call libdir, ios, debug)
$(call libdir, ios, release):
mkdir -p $(call libdir, ios, release)
I'll have to repeat these rules for each configuration. It's not a lot, but it could become a pain to maintain. I would like to make these more generic, something like this (I know it doesn't work):
define libdir
$(BINDIR)/$(strip $1)/$(strip $2)
endef
$(call libdir, %, %):
mkdir -p $(call libdir, ??, ??)
Which would create any directory with the given structure. Any ideas?
Looks like you could use a simple pattern rule instead.
%/debug %/release: %:
mkdir -p $#
And then probably make each platform depend on these directories:
ios android osx windows linux: %: $(LIBDIR)/%/debug $(LIBDIR)/%/release
A statement like:
define libdir
$(BINDIR)/$(strip $1)/$(strip $2)
endef
$(call libdir, %, %):
mkdir -p $(call libdir, ??, ??)
simply strips the literal character %; the strip function is expanded when the pattern rule is parsed not when the rule matches.
But that's OK, because it doesn't make any sense the other way. First of all you can't have multiple patterns in a target, so a rule like $(BINDIR)/%/% is not a legal pattern rule anyway. Second, a pattern matches a set of characters; it doesn't matter what those characters are. If you want to strip then you have to strip the prerequisite used to match that pattern rule, not strip the pattern rule. E.g., somewhere else in your makefile you'll have a rule like:
foo: $(BINDIR)/$(FOO)/$(BAR)
that you intend to match that pattern: that's what you need to strip, not the target.
And just a note, but you don't need to use the function inside the recipe. Why not just use $#?
$(BINDIR)/%:
mkdir -p $#
I am VERY new to makefiles. I have discovered a flaw in a make file that causes files in a list to be copied from a single source file instead of each file in the list.
First, there is a sub model variable SUB_MODEL_LIST that contains 0 1 2 3 separated by white space.
Here is the segment that does the copy:
$(TARGET_BIN_LIST_NEW) : $(TARGET_BIN_LIST)
#echo copying from $< to $#
$(call COPY, $(firstword $(TARGET_BIN_LIST)), $#)
TARGET_BIN_LIST_NEW contains new file names separated by white space and is composed of something like this:
file001.200 file001.201 file001.202 file001.203
and TARGET_BIN_LIST contains the existing file names and is composed of something like this:
file001c.200 file001c.201 file001c.202 file001c.203
The last digit in the file extension is the model number.
As I read this, the makefile runs:
#echo copying from $< to $#
$(call COPY, $(firstword $(TARGET_BIN_LIST)), $#)
four times, however, it always use the first file name in the TARGET_BIN_LIST due to the firstword function. This results in file001.200, file001.201, file001.202, file001.203 being created, but they are all copies of file001c.200 when they should be copies of their respective files in the list. Each file relates to a sub model version of the code.
My thought to solve this was to use the word function. Something like this:
$(TARGET_BIN_LIST_NEW) : $(TARGET_BIN_LIST)
#echo copying from $< to $#
$(call COPY, $(word $(sub), $(TARGET_BIN_LIST)), $#)
where sub is an element of SUB_MODEL_LIST, but I am not sure how that will work. Does the above roll out into 4 separate calls, or can it be looked at as a loop that can have an increment value for sub??
I also thought about using a foreach loop:
$(foreach sub,$(SUB_MODEL_LIST),$(call COPY, $(word $(sub), $(TARGET_BIN_LIST)), $(word $(sub), $(TARGET_BIN_LIST_NEW)))
But I get the error:
*** first argument to `word' function must be greater than 0. Stop.
Ok, so I tried:
$(foreach sub,$(SUB_MODEL_LIST),$(call COPY, $(word $(sub)+1, $(TARGET_BIN_LIST)), $(word $(sub)+1, $(TARGET_BIN_LIST_NEW)))
But then I got the error:
*** non-numeric first argument to `word' function. Stop.
Now I'm stuck. I would like to keep the existing implementation in tact at much as possible, but can adopt a loop method if needed.
Thanks for the help!
You have to step back. You're misunderstanding how this works. In make an explicit rule with multiple targets is EXACTLY THE SAME as writing the same rule multiple times, once for each target. So this:
$(TARGET_BIN_LIST_NEW) : $(TARGET_BIN_LIST)
#echo copying from $< to $#
$(call COPY, $(firstword $(TARGET_BIN_LIST)), $#)
If TARGET_BIN_LIST_NEW is file001.200 file001.201 file001.202 file001.203 and TARGET_BIN_LIST is file001c.200 file001c.201 file001c.202 file001c.203, is identical to writing this:
file001.200 : file001c.200 file001c.201 file001c.202 file001c.203
...
file001.201 : file001c.200 file001c.201 file001c.202 file001c.203
...
file001.202 : file001c.200 file001c.201 file001c.202 file001c.203
...
file001.203 : file001c.200 file001c.201 file001c.202 file001c.203
...
So you can clearly see that when each rule is run, the value of $< and $(firstword $(TARGET_BIN_LIST)) will be the same thing (file001c.200).
Is it really the case that whenever ANY of the fileXXXc.YYY files change, you want to rebuild ALL the fileXXX.YYY files? That's what your rule does, but based on the recipe it doesn't seem like that's what you want.
Make is mostly about writing one rule to build one target from zero or more prerequisites. If you use a pattern rule you can do this pretty easily:
all: $(TARGET_BIN_LIST_NEW)
file001.% : file001c.%
#echo copying from $< to $#
$(call COPY,$<,$#)
If your filenames may have a more complex naming convention then you'll need something more complicated.
ETA:
Since your naming convention doesn't fit into make's pattern rule capabilities you'll have to do something fancier. You can use eval to generate the rules, like this:
all: $(TARGET_BIN_LIST_NEW)
define TARGET_BIN_COPY
$(1) : $(basename $(1))c$(suffix $(1))
#echo copying from $$< to $$#
$$(call COPY,$$<,$$#)
endef
$(foreach T,$(TARGET_BIN_LIST_NEW),$(eval $(call TARGET_BIN_COPY,$T)))
# uncomment this for debugging
#$(foreach T,$(TARGET_BIN_LIST_NEW),$(info $(call TARGET_BIN_COPY,$T)))
First off, thank you to MadScientist for your help in clarifying how this works.
This implementation worked for me:
$(TARGET_BIN_LIST_NEW) : $(TARGET_BIN_LIST)
#echo copying from $(filter %$(suffix $#), $(TARGET_BIN_LIST)) to $#
$(call COPY, $(filter %$(suffix $#), $(TARGET_BIN_LIST)), $#)