I want to create a makefile to be included in others, kind of as a "library", let's call it library.mak:
TARGET_FILES = a.txt b.txt c.txt d.txt
SRC_FOLDER = somewhere/else
.PHONY: all
all:
for target in ${TARGET_FILES} ; do \
echo -e "\nMaking $${target}:" ;\
${MAKE} ${MFLAGS} --no-print-directory $${target} ;\
done
.PHONY: ${TARGET_FILES}
${TARGET_FILES}:
cp ${SRC_FOLDER}/$# $#
The two rules are there to make all, as well as one specific target, respectively; using a default rule (which is the purpose of library.mak).
In my "user makefile" called Makefile, I want to then do this:
include library.mak
# special handling of c.txt
c.txt:
grep -v 'all except this' ${SRC_FOLDER}/$# > $#
As you can see, the user wants to be able to override the behaviour for some special cases. While this works, it always greets the user with the dreaded warning: overriding recipe for target and warning: ignoring old commands for target messages, even though the behaviour is as intended.
So here's the question: Can this be done in a different way that avoids these warnings, or is there a means to suppress them?
The warning says you overwrite a recipe for an explicit rule. And this is really wrong. It makes sense to use a pattern rule instead, like: %.txt: ${SRC_FOLDER}/%.txt. Then it's ok to have some explicit rule (c.txt: ${SRC_FOLDER}/c.txt) overwriting a recipe. Of course, it's not 100% the same, but, I think that should not be a problem. Anyway, putting an explicit rule into a reusable file is a crime.
Next, you constantly use "phonies" and rules w/o prerequisites even when you definitely should have them. This is bad. You're trying to make "make" work as a non-branching shell script. That's not only inefficient but also is a misuse. Basically, make can be viewed as "shell extension" capable of "branching on file timestamps in a sophisticated way". If you don't need it, don't use make at all.
Next, I see absolutely no reason to go into recursive make. Whatever, you're going to do, I believe, you can do without it. And even if you really need recursion, then write simply $(MAKE). $(MFLAGS) shoud not be used anymore (read this).
P.S. There's yet another (more flexible) option to define parameterized variables (macros). For example:
define nl :=
endef
define myrule.text
$1: $2/$1
#cp $$< $$#
endef
myrule = $(eval $(foreach foo$0,$1,$(call myrule.text,$(foo$0),$2)$(nl)))
...
# user makefile
# for everything except c.txt call predefined macro
$(call myrule,$(filter-out c.txt,$(TARGET_FILES)),$(SRC_FOLDER))
# for c.txt only
c.txt: $(SRC_FOLDER)/c.txt
grep -v 'all except this' $< >$#
But as you see, it's a bit of a "cryptic" stuff, and I don't feel like recommending it for a beginner.
Related
In a Makefile I'm writing I had an interest in cleaning up some of the CC prints and centralizing some of the build preparations (like creating directories in the build tree). I figured macros would be a good fit for this task. This is effectively what im trying to do, used all over various Makefiles:
define func
#mkdir -p $$(dir $(1))
#printf "%-5s $(2)\n" $(3)
endef
test:
#echo Run
$(eval $(call func,a,b,c))
My thought was that after first expansion I'd get something like (less any tabs maybe, I'm not exactly sure how the expansion works within eval):
test:
#echo Run
$(eval #mkdir -p $(dir a)\n#printf "%-5s b\n" c
and of course finally the commands would be executed. However, what I get is this:
# make
Makefile:7: *** recipe commences before first target. Stop.
I changed eval to info and got this:
#mkdir -p $(dir a)
#printf "%-5s b\n" c
Run
So I thought maybe my explicit tabs in the macro definition were causing trouble, so I removed them and tried again:
# make
Makefile:7: *** missing separator. Stop.
So it still does not quite work. If it is indeed possible at all, it seems some function of indentions in the macro, or maybe I'm defining the macros incorrectly. I thought perhaps the two commands in the macro was causing trouble (since the complaint is regarding a separator), but reducing the macro to a single line did not help either.
You don't want eval here. Eval is used to evaluate makefile syntax. That is, the thing you're evaluation has to be a valid, complete makefile. You can see that what info prints is not a valid makefile. If you put that into a file and ran make -f <file>, you'd get a syntax error.
You are just trying to expand a variable for shell syntax. Just remove the eval.
I have two lists of files as prerequisites
input_i.xx
config_j.yy
and I need to run all of their combinations. A single one looks like this:
input1_config3.output: input1.xx config3.yy
run_script $^
Also in reality, their names are not numbered, but I already have their stems defined in INPUTS and CONFIGS. With that, I can generate all the targets together
TARGETS:=$(foreach input,$(INPUTS),$(foreach config,$(CONFIGS),$(input)_$(config).output))
But I have difficulty with the prerequisites. It seems I need to
get basename
split on _
add the extensions .xx and .yy
.SECONDEXPANSION
$(TARGETS): $(basename $#)
run_script $^
Can someone show me how to do that? Not sure if this the proper way, maybe a bottom-up way is easier?
make is not really suitable for keeping track of an M x N matrix of results. The fundamental problem is that you can't have two stems in a rule, so you can't say something like
# BROKEN
input%{X}_config%{Y}.output: input%{X}.xx config%{Y}.yy
As a rough approximation, you could use a recursive make rule to set a couple of parameters, and take it from there, but this is rather clumsy.
.PHONY: all
all:
$(MAKE) -$(MAKEFLAGS) X=1 Y=6 input1_config6.output
$(MAKE) -$(MAKEFLAGS) X=1 Y=7 input1_config7.output
$(MAKE) -$(MAKEFLAGS) X=2 Y=6 input2_config6.output
:
input$X_config$Y.output: input$X.xx config$Y.yy
run_script $^
It would be a lot easier if you provided a complete sample example with a complete set of targets and prerequisites and exactly what you wanted to happen.
Using .SECONDEXPANSION might work, but you're not using it correctly; please re-read the documentation. The critical aspect of .SECONDEXPANSION is that you have to escape the variables that you want to avoid expanding until the second pass. In your example you've not escaped anything, so .SECONDEXPANSION isn't actually doing anything at all here. However, as #tripleee points out it's not easy to use multiple variable values in a single target.
To do this more easily you'll probably want to use eval. Something like this:
define DECLARE
$1_$2.output: $1.xx $2.yy
TARGETS += $1_$2.output
endef
TARGETS :=
$(foreach input,$(INPUTS),$(foreach config,$(CONFIGS),$(eval $(call DECLARE,$(input),$(config)))))
$(TARGETS):
run_script $^
I have another solution using include and bash for loop.
include trees.mk
trees.mk:
#for input in $(INPUTS); do \
for config in $(CONFIGS); do \
echo $${input}_$$config.output : $${input}.xx $$config.yy; \
echo -e '\t run_scipt $$^ ';\
done \
done > $#
At the beginning, trees.mk doesn't exist. The double for loops write out the rule to the target using file redirection >$#.
I got this idea from Managing Projects with GNU Make, Third Edition By Robert Mecklenburg, on
page 56
I want to check whether a string variable contains a specified substring in the Makefile. The purpose is to clean the sub folders.
I used the below code, but it did not work.
SERVICES_LIST = A_Service B_Service C_Service #example
SPECIFIC_SERVICE_LIST = A_Service B_Service
clean:
#list='$(SERVICES_LIST)';for subdir in $$list;do \
echo "clean in $$subdir";\
if [[ "*$$subdir*" == "$(SPECIFIC_SERVICE_LIST)" ]];then\
make $$subdir clean;\
fi;\
done;\
This hasn't much to do with make, because substantially all the logic involved is expressed in the shell language. In particular, you seem to be assuming bash.
The problem is here:
if [[ "*$$subdir*" == "$(SPECIFIC_SERVICE_LIST)" ]];then\
You seem to by trying to match (make's expansion of) "$(SPECIFIC_SERVICE_LIST)" against a glob pattern formed as (make's expansion of) "*$$subdir*". But the left-hand side is quoted, so it is not interpreted as a pattern, and the == operator performs (exact) string matching, not pattern matching.
One of the main ways to apply such pattern-matching tests in the shell language is with a case construct, because the selection expressions used with it are always interpreted as globs. That might look like so in your makefile:
case "$(SPECIFIC_SERVICE_LIST)" in *$$subdir*) make $$subdir clean ;; esac
But the whole thing seems pretty non-idiomatic. Generally speaking, a makefile is tuned to the project. Even if it is dynamically generated in part or in whole, it is reasonable and appropriate to design your build system so that the clean target can do something more like this:
clean:
for subdir in $(SPECIFIC_SERVICE_LIST); do make -C $$subdir clean; done
... or maybe like this:
clean: clean_services
...
clean_services:
for subdir in $(SPECIFIC_SERVICE_LIST); do make -C $$subdir clean; done
I would make it more make way by defining a target for cleaning up any supported service and then call all required clean targets as a prerequisite to clean. This has additional advantage to make clean in parallel when running with -j option as opposed to strictly sequential shell loop.
$ cat Makefile
SERVICES_LIST = A_Service B_Service C_Service #example
SPECIFIC_SERVICE_LIST = A_Service B_Service
.PHONY: $(addsuffix -clean, $(SERVICES_LIST))
$(addsuffix -clean, $(SERVICES_LIST)): %-clean:
$(MAKE) -C $* clean
.PHONY: clean
clean: $(addsuffix -clean, $(SPECIFIC_SERVICE_LIST))
I have a bunch of different source files in my static HTML blog. The outermost extensions explain the format to be processed next.
Example: Source file article.html.md.gz (with target article.html) should be processed by gunzip, then by my markdown processor.
Further details:
The order of the extensions may vary
Sometimes an extension is not used (article.html.gz)
I know how to process all different extensions
I know that the final form is always article.html
Ideally I would have liked to just write rules as follows:
...
all-articles: $(ALL_HTML_FILES)
%: %.gz
gunzip ...
%: %.md
markdown ...
%: %.zip
unzip ...
And let make figure out the path to take based on the sequence of extensions.
From the documentation however, I understand that there are constraints on match-all rules, and the above is not possible.
What's the best way forward? Can make handle this situation at all?
Extensions are made up examples. My actual source files make more sense :-)
I'm on holiday so I'll bite.
I'm not a fan of pattern rules, they are too restricted and yet too arbitrary at the same time for my tastes. You can achieve what you want quite nicely in pure make:
.DELETE_ON_ERROR:
all: # Default target
files := a.html.md.gz b.html.gz
cmds<.gz> = gzip -d <$< >$#
cmds<.md> = mdtool $< -o $#
define rule-text # 1:suffix 2:basename
$(if $(filter undefined,$(flavor cmds<$1>)),$(error Cannot handle $1 files: [$2$1]))
$2: $2$1 ; $(value cmds<$1>)
all: $2
endef
emit-rule = $(eval $(call rule-text,$1,$2))# 1:suffix 2:basename
emit-hierachy = $(if $(suffix $2),$(call emit-rule,$1,$2)$(call emit-hierachy,$(suffix $2),$(basename $2)))# 1:suffix 2:basename
emit-rules = $(foreach _,$1,$(call emit-hierachy,$(suffix $_),$(basename $_)))# 1:list of source files
$(call emit-rules,${files})
.PHONY: all
all: ; : $# Success
The key here is to set $files to your list of files.
This list is then passed to emit-rules.
emit-rules passes each file one-at-a-time to emit-hierachy.
emit-hierachy strips off each extension in turn,
generates the appropriate make syntax, which it passes to $(eval …).
emit-hierachy carries on until the file has only one extension left.
Thus a.html.md.gz becomes this make syntax:
a.html.md: a.html.md.gz ; gunzip <$< >$#
a.html: a.html.md ; mdtool $< -o $#
all: a.html
Similarly, b.html.gz becomes:
b.html: b.html.gz ; gunzip <$< >$#
all: b.html
Neato, or what?
If you give emit-rules a file with an unrecognised extension (c.html.pp say),
you get a compile-time error:
1:20: *** Cannot handle .pp files: [c.html.pp]. Stop.
Compile-time? Yeah, before any shell commands are run.
You can tell make how to handle .pp files by defining cmds<.pp> :-)
For extra points it's also parallel safe. So you can use -j9 on your 8 CPU laptop, and -j33 on your 32 CPU workstation. Modern life eh?
I've been learning make from the software carpentry tutorial (make patterns) and it says that we can use $^ to get the dependency list for our rule but have extra dependencies for our target by writing extra empty rules. For example
all:
touch f1.txt f2.txt a.txt
result: a.txt
result: f*.txt
#echo $^
I thought that this would print f1.txt f2.txt but instead I see f1.txt f2.txt a.txt. Am I missing something or is the tutorial wrong?
Clearly, I should have been explicit, in the example I run make to create the files for the test, then I run make result to get the result shown.
Not sure I understand your problem and what you would like to do but if you type just make it is like if you were typing make all because all is the first target and thus the default goal. It should create the 3 files (or update their last modification date if they already exist), echo the recipe and you should see:
touch f1.txt f2.txt a.txt
If you then type make result you should see the result of #echo $^, that is the list of all dependencies of result:
f1.txt f2.txt a.txt
It is absolutely normal as you declared them all as dependencies of result.
Finally, if you type make result before make or make all and if a.txt does not exist or if there is no file matching f*.txt, you will get an error because make needs a.txt and at least one file matching f*.txt to make result and it does not know how to make them.
I cannot imagine a use case where the behaviour you expected ($^ being expanded as dependencies of the current rule only) would be useful, but if really you need this feature you can use the rather obscure Double-Colon Rules (DCR):
all:
touch f1.txt f2.txt a.txt
result:: a.txt
#echo $^
result:: f*.txt
#echo $^
Note that if there is a DCR for a target, all rules of this target must also be DCR. Note also that all DCR must have a recipe. A DCR recipe is applied if one of the dependencies of this rule is newer than the target. If several DCR apply, they are executed in order of appearance in the Makefile. And finally, what you were interested in: the $^ automatic variable logically expands as the dependencies of the particular DCR.
No, $^ refers to the prerequisites for the current target, not the current rule.
Also, the results you are reporting are not correct; for make result I get make: *** No rule to make target `f*.txt', needed by `result'. Stop. like I very much expected.