Apply gnu-make rule as a function - makefile

I want a rule to behave as a function, that could for instance gzip all my temporary targets. I wrote those rules in a makefile:
%.file0:
touch $#
%.file1: %
touch $#
%.file2: %
touch $#
%.gz: %
echo "zipping to $#"
touch $#
I can call
$ make -n dotted.file.file0.file1.file2.gz -f makefile
touch dotted.file.file0
touch dotted.file.file0.file1
touch dotted.file.file0.file1.file2
echo "zipping to dotted.file.file0.file1.file2.gz"
rm dotted.file.file0 dotted.file.file0.file1.file2 dotted.file.file0.file1
My last target will successfully be gzipped. I can then gzip dotted.file.file0.file1 before:
$ make -n dotted.file.file0.file1.gz.file2 -f makefile
touch dotted.file.file0
touch dotted.file.file0.file1
echo "zipping to dotted.file.file0.file1.gz"
touch dotted.file.file0.file1.gz.file2
rm dotted.file.file0 dotted.file.file0.file1.gz dotted.file.file0.file1
That file will also be gzipped before being given to the rule %.file2. But I can't gzip several targets:
$ make -n dotted.file.file0.file1.gz.file2.gz -f makefile
make: *** No rule to make target `dotted.file.file0.file1.gz.file2.gz'. Stop.
How can I do that, i.e. applying the %.gz rule on several targets?
EDIT 1
From another point of view, I tried rewriting my rules this way:
%.file0:
touch $#
%.file1: %.gz
touch $#
%.file2: %.gz
touch $#
%.gz: %
echo "zipping to $#"
touch $#
Then I call make:
$ make -n dotted.file.file0.file1.file2 -f makefile
make: *** No rule to make target `dotted.file.file0.file1.file2'. Stop.
I expect/wish gnu-make to execute the rules this order:
- ask for the file dotted.file.file0.file1.file2
- go to rule %.file2
- ask for the dependency dotted.file.file0.file1.gz
- go to rule %.gz
- as for the dependency dotted.file.file0.file1
- go to rule %.file1
- as for the dependency dotted.file.file0.gz
- go to rule %.gz
- as for the dependency dotted.file.file0
- go to rule %.file0
- create file dotted.file.file0
However if I ask only one rule to apply %.gz, it will work this single time.
The makefile:
%.file0:
touch $#
%.file1: %.gz # !!! Notice I left this single dependency to the rule %.gz
touch $#
%.file2: % ## !!! Notice I removed the .gz here
touch $#
%.gz: %
echo "zipping to $#"
touch $#
The command:
$make -n dotted.file.file0.file1.file2 -f makefile
touch dotted.file.file0
touch dotted.file.file0.file1
echo "zipping to dotted.file.file0.file1.gz"
touch dotted.file.file0.file1.gz
touch dotted.file.file0.file1.file2
rm dotted.file.file0.file1.gz dotted.file.file0 dotted.file.file0.file1

The problem with your first version is that you would need static pattern rules, not implicit rules. The following does the job with GNU make, using shell calls to awk to build explicit lists of *.file1, *.file2 and *.gz targets from the specified make goals. It also uses foreach-eval-call combinations to instantiate as many rules (without recipes) as needed to express explicitly all dependencies. I must admit that this style of Makefile is rather unusual but your problem is unusual too...
SUFFIXES := file1 file2 gz
# For goal $(1) and suffix $(2), build the list of all possible stems <S> such
# that $(1) matches the ^<S>\.$(2)(\..*)? regular expression. Concatenate this
# list to the $(2) make variable.
define SPLIT_rule
$(2) += $$(shell echo $(1) | awk -F.$(2) '{for(i=1;i<=NF;i++){for(j=1;j<=i;j++)printf("%s",$$$$j);printf(".$(2) ")}}')
endef
# For goal $(1), instantiate SPLIT_rule for each suffix.
define GOAL_rule
$(foreach suffix,$(SUFFIXES),$(eval $(call SPLIT_rule,$(1),$(suffix))))
endef
$(foreach goal,$(MAKECMDGOALS),$(eval $(call GOAL_rule,$(goal))))
# For suffix $(1), instantiate the static pattern rule.
define SUFFIX_rule
$$($(1)): %.$(1): %
endef
$(foreach suffix,$(SUFFIXES),$(eval $(call SUFFIX_rule,$(suffix))))
%.file0:
touch $#
%.file1: %
touch $#
%.file2: %
touch $#
%.gz: %
echo "zipping to $#"
touch $#
And:
$ goal1=dotted.file.file0.file1.file2.gz
$ make -n $goal1
touch dotted.file.file0
touch dotted.file.file0.file1
touch dotted.file.file0.file1.file2
echo "zipping to dotted.file.file0.file1.file2.gz"
touch dotted.file.file0.file1.file2.gz
$ goal2=dotted.file.file0.file1.gz.file2
$ make -n $goal2
touch dotted.file.file0
touch dotted.file.file0.file1
echo "zipping to dotted.file.file0.file1.gz"
touch dotted.file.file0.file1.gz
touch dotted.file.file0.file1.gz.file2
$ goal3=dotted.file.file0.file1.gz.file2.gz
$ make -n $goal3
touch dotted.file.file0
touch dotted.file.file0.file1
echo "zipping to dotted.file.file0.file1.gz"
touch dotted.file.file0.file1.gz
touch dotted.file.file0.file1.gz.file2
echo "zipping to dotted.file.file0.file1.gz.file2.gz"
touch dotted.file.file0.file1.gz.file2.gz
$ goal4=dotted.file.file0.file1.file2
$ make -n $goal4
touch dotted.file.file0
touch dotted.file.file0.file1
touch dotted.file.file0.file1.file2
$ all="$goal1 $goal2 $goal3 $goal4"
$ make -n $all
touch dotted.file.file0
touch dotted.file.file0.file1
touch dotted.file.file0.file1.file2
echo "zipping to dotted.file.file0.file1.file2.gz"
touch dotted.file.file0.file1.file2.gz
echo "zipping to dotted.file.file0.file1.gz"
touch dotted.file.file0.file1.gz
touch dotted.file.file0.file1.gz.file2
echo "zipping to dotted.file.file0.file1.gz.file2.gz"
touch dotted.file.file0.file1.gz.file2.gz
make: 'dotted.file.file0.file1.file2' is up to date.
Note: as all intermediates are not implicit any more, they are not automatically deleted, just as if they were declared PRECIOUS in your first version. This is the reason why there is no rm ... any more in the outputs.

Related

Is there any way to make multiple targets as series of single make invocations?

I have the following Makefile:
ifneq ($(MAKECMDGOALS),clean)
-include generated.mk
endif
FOO ?= foo
all: a.txt
a.txt:
echo $(GEN_FOO) > $#
generated.mk: Makefile
echo GEN_FOO = $(FOO) > $#
.PHONY: clean
clean:
$(RM) a.txt
$(RM) generated.mk
It works OK when building single targets:
$ make clean
rm -f a.txt
rm -f generated.mk
$ make all
echo GEN_FOO = foo > generated.mk
echo foo > a.txt
However when I try to build multiple targets at once things go not so smooth:
$ make clean all
rm -f a.txt
rm -f generated.mk
echo foo > a.txt
$ make all
echo GEN_FOO = foo > generated.mk
make: Nothing to be done for 'all'.
It gets even worse if variables were provided:
$ make clean
rm -f a.txt
rm -f generated.mk
$ make FOO=bar clean all
echo GEN_FOO = bar > generated.mk
rm -f a.txt
rm -f generated.mk
echo bar > a.txt
$ make all
echo GEN_FOO = foo > generated.mk
make: Nothing to be done for 'all'.
$ make FOO=bar clean all
rm -f a.txt
rm -f generated.mk
echo foo > a.txt
Are there any ways to fix such incorrect behavior?
Make is doing exactly what you told it to do, and you haven't told us what you want it to do that's different than what you told it to do (saying fix such incorrect behavior doesn't really help us when you don't define what's incorrect about the behavior), so we can't help you very much.
You are probably getting confused about the interaction between included makefiles and comparing $(MAKECMDGOALS). Please note:
ifneq ($(MAKECMDGOALS),clean)
this will not match unless you specify exactly one target: clean. In situations where you specify multiple targets, one of which is clean, that will match because clean all is not equal to clean. So, when you run make clean all make will include the generated makefile, and will generate it if it doesn't exist.
Because generated include files are only rebuilt once, when the makefile is first parsed, it's not possible to say something like: "first run rule X (e.g., clean) then rebuild the included makefiles, then reinvoke make".
However, it's pretty much always a bad idea to invoke make with clean all. This is because if you were to ever try to add -j for parallelism, the clean and the build would be running in parallel and corrupt everything.
One semi-common option is to provide a different rule that will do both, something like this:
rebuild:
$(MAKE) clean
$(MAKE) all
then run make rebuild instead.
You can certainly force the behavior with the help of the shell. For instance, in bash you could use
for target in "clean" "all";
do
make $target;
done
and if you were going to re-do the procedure a lot you could either make it an executable script or wrap it in a shell function.

Dynamic include directive in a Makefile

Let's consider this Makefile:
.SUFFIXES:
.DEFAULT_GOAL := all
out=foo
clean:
rm -f vars.mk
rm -f $(out)
vars.mk: vars.mk.default
#echo "Regenerating $#..."
cp $< $# # Let's assume the translation is much complex than a cp
-include vars.mk
ifeq ($(filter foo,$(FOO)),)
$(error FOO undefined)
endif
all: $(out)
$(out): vars.mk
echo "Cow says: I am not a $(FOO)." > $#
And the file vars.mk.default
FOO = foo bar
I would like to regenerate my targets if vars.mk.default is updated. Furthermore, as double check, one must check that foo exists in $(FOO).
How to force make to regenerate vars.mk if vars.mk.default is updated?
In other words, I would like this output:
$ make clean
$ sed 's/dog/cat/' vars.mk.default
$ make foo
Regenerating vars.mk...
echo "Cow says: I am not a cat" > all
$ make foo
make: Nothing to be done for 'all'.
$ sed 's/cat/dog/' vars.mk.default
$ make
Regenerating vars.mk...
echo "Cow says: I am not a dog" > all
$ rm vars.mak
$ make
Regenerating vars.mk...
echo "Cow says: I am not a dog" > all
To avoid failing if vars.mk doesn't exist, just check for it first:
ifeq ($(wildcard vars.mk),vars.mk)
ifeq ($(filter foo,$(FOO)),)
$(error FOO undefined)
endif
endif
My goal is to regenerate my targets if vars.mk.default is updated.
In this case make your targets depend on that file, but filter it out in the recipes, e.g.
foo.o : foo.cc vars.mk.default
$(COMPILE) $(filter-out vars.mk.default,$^)
In the case vars.mk does not exist, make fails on the ifeq and do not generates vars.mk.
Make is going to build vars.mk and restart, see How Makefiles Are Remade for more details.
So, to avoid that error, check first if FOO is defined with ifdef FOO.
A couple of things. First, you should put a - in front of the include to prevent a warning from popping up if the file does not exist:
-include vars.mk
This will not cause a fatal error if vars.mk is not generated, but because the vars.mk rule would fail in this case, you would get your error from there.
You can then check if $(FOO) contains foo from within a recipe:
checkForFoo: vars.mk
#[[ $(FOO) =~ .*foo.* ]] || false
all:checkForFoo
The recipe is only run after the vars.mk was generated and included, so it should only fail in the conditions you want.

Makefile: rule that match multiple patterns

I have this rule in my Makefile, that responds to flags I pass:
$(BUILD_DIR)/disable_%:
mkdir -p $(BUILD_DIR)
touch $(BUILD_DIR)/disable_$*
rm -f $(BUILD_DIR)/enable_$*
cd $(BUILD_DIR) && rm -f Makefile
$(BUILD_DIR)/enable_%:
mkdir -p $(BUILD_DIR)
touch $(BUILD_DIR)/enable_$*
rm -f $(BUILD_DIR)/disable_$*
cd $(BUILD_DIR) && rm -f Makefile
What this means is that when changing the flags by which I invoke the makefile, I can trigger some recompilations that could depend on these flags.
The code presented above is a bit redundant: you see that I remove a file, touch another and remove a Makefile in both cases. The only thing that changes is the name of the files that I touch/remove, and they are related.
For instance,
make clean
make enable_debug=yes enable_video=no # will compile from zero
make enable_debug=no enable_video=no # flag change detected -> recompile some submodules that depend on this flag
Provided that the only thing that changes between the two rules ( [en|dis]able ), what I would like is to only have 1 generic rule, something like that:
# match 2 parts in the rule
$(BUILD_DIR)/%ble_%:
mkdir -p $(BUILD_DIR)
touch $(BUILD_DIR)/(???)ble_$* # should be $#
rm -f $(BUILD_DIR)/(???)able_$* # should be disable if $# is enable and inverse
cd $(BUILD_DIR) && rm -f Makefile
Is this possible ?
PS: Sorry if I didn't get the title correctly, I couldn't figure how to explain it better.
$(BUILD_DIR)/enable_% $(BUILD_DIR)/disable_%:
mkdir -p $(BUILD_DIR)
rm -f $(BUILD_DIR)/*able_$*
touch $#
cd $(BUILD_DIR) && rm -f Makefile
Not literally what you wanted (multi-wildcards are forbidden in make), but does quite the same.

Finding makefile dependencies

I have several widgets denoted by a config.xml in their root in a directory layout.
The GNUmakefile I have here is able to build them. Though if I update the folders, the dependencies aren't tracked. I don't want to depend on a clean target obviously, so how do I track the contents of each folder?
WGTS := $(shell find -name 'config.xml' | while read wgtdir; do echo `dirname $$wgtdir`.wgt; done )
all: $(WGTS)
%.wgt:
#cd $* && zip -q -r ../$(shell basename $*).wgt .
#echo Created $#
clean:
rm -f $(WGTS)
I hoped something like:
%.wgt: $(shell find $* -type f)
Would work, but it doesn't. Help.
Combining Beta's idea with mine:
WGTS := $(shell find -name config.xml)
WGTS := $(WGTS:/config.xml=.wgt)
WGTS_d := $(WGTS:.wgt=.wgt.d)
all: $(WGTS)
clean:
rm -f $(WGTS) $(WGTS_d)
-include $(WGTS_d)
define WGT_RULE
$(1): $(shell find $(1:.wgt=))
$(1:.wgt=)/%:
#
endef
$(foreach targ,$(WGTS),$(eval $(call WGT_RULE,$(targ))))
%.wgt:
#echo Creating $#
#(echo -n "$#: "; find $* -type f | tr '\n' ' ') > $#.d
#cd $* && zip -q -r ../$(shell basename $*).wgt .
Example:
$ mkdir -p foo bar/nested
$ touch {foo,bar/nested}/config.xml
$ make
Creating bar/nested.wgt
Creating foo.wgt
$ make
make: Nothing to be done for `all'.
$ touch foo/a
$ make
Creating foo.wgt
$ rm foo/a
$ make
Creating foo.wgt
$ make
make: Nothing to be done for `all'.
The only potential problem here is the dummy rule that lets make ignore targets it doesn't know how to build which are nested inside the directories. (foo/a in my example.) If those are real targets that make needs to know how to build, the duplicate recipe definition may be a problem.
Probably the best way to do this is to create the prerequisite lists explicitly, beforehand:
define WGT_RULE
$(1).wgt: $(wildcard $(1)/*)
endef
$(foreach targ,$(WGTS),$(eval $(call WGT_RULE,$(targ))))
There is another way that's very clever (a phrase that makes a good programmer wary). Years ago I came up with a left-handed kludge for treating a directory as a prerequisite. I'll see if I can dig up my old notebooks if the above isn't good enough.
EDIT:
Sorry, I didn't consider subdirectories. Here's a complete makefile (I left out the clean rule) that should do the trick.
WGTS := $(shell find -name 'config.xml' | while read wgtdir; do echo `dirname $\
$wgtdir`.wgt; done )
all: $(WGTS)
# This constructs a rule without commands ("foo.wgt: foo/bar.txt foo/baz.dat...").
define WGT_RULE
$(1).wgt: $(shell find $(1))
endef
# This invokes the above to create a rule for each widget.
$(foreach targ,$(WGTS),$(eval $(call WGT_RULE,$(targ))))
%.wgt:
#cd $* && zip -q -r ../$(shell basename $*).wgt .
#echo Created $#

Explaining makefile % as well as $< and $#

xpi_built := $(build_dir)/$(install_rdf) \
$(build_dir)/$(chrome_manifest) \
$(chrome_jar_file) \
$(default_prefs)
xpi_built_no_dir := $(subst $(build_dir)/,,$(xpi_built))
$(xpi_file): $(build_dir) $(xpi_built)
#echo "Creating XPI file."
cd $(build_dir); $(ZIP) ../$(xpi_file) $(xpi_built_no_dir)
#echo "Creating XPI file. Done!"
$(build_dir)/%: %
cp -f $< $#
$(build_dir):
#if [ ! -x $(build_dir) ]; \
then \
mkdir $(build_dir); \
fi
can anyone explain me this makefile part? particularly interested in
$(build_dir)/%: % as well as $< and $# directives
two labels $(build_dir) exists, I guess both are executed, but in which order?
$(build_dir)/%: %
cp -f $< $#
This is a static pattern rule which uses automatic variables in its command; $< expands to the leftmost prerequisite, $# expands to the target. If you try to make $(build_dir)/foo (whatever $(build_dir) is), Make will treat this rule as
$(build_dir)/foo: foo
cp -f foo $(build_dir)/foo
The next rule,
$(build_dir):
#if [ ! -x $(build_dir) ]; \
then \
mkdir $(build_dir); \
fi
is for $(build_dir) itself, and is unnecessarily complicated. It says "if $(build_dir) doesn't exist, then mkdir it", and it could be written this way:
$(build_dir):
mkdir $#
It looks as if your primary target is $(xpi_file):
$(xpi_file): $(build_dir) $(xpi_built)
So Make will first make $(build_dir) (if necessary), then the members of the list %(xpi_built), which includes a couple of things of the form $(build_dir)/%. Once those are done, it will execute the commands of this rule: it will cd into $(build_dir), zip some things up, and echo a couple of messages.
See Pattern Rules and Automatic Variables in the GNU make documentation. The first rule matches files inside $(build_dir), not $(build_dir) itself. $< expands to the list of prerequisites of the current rule, $# is the target for the current rule.

Resources