In the GNU-Make manual the How make Reads a Makefile https://www.gnu.org/software/make/manual/make.html#Reading-Makefiles sections says
GNU make does its work in two distinct phases. During the first phase it reads all the makefiles, included makefiles, etc. and internalizes all the variables and their values, implicit and explicit rules, and constructs a dependency graph of all the targets and their prerequisites. During the second phase, make uses these internal structures to determine what targets will need to be rebuilt and to invoke the rules necessary to do so.
I am not able to understand the difference b/w the two phases clearly. May be looking at an example will help understand. Are there any links or tutorials which clarifies what exactly happens in the first phase and second phase.
Take this trivial makefile:
var := some_other_file
some_file: $(var)
some_command $^ $#
After phase one the file will look like the following
var := some_other_file
some_file: some_other_file
some_command $^ $#
Notice how $^ and $# haven't been expanded yet, recipes are only expanded and invoked as part of phase 2.
In phase 2, make will use the rules resulting from phase 1 and determine which targets need to be remade, you can see how make "thinks" by running make with the -d flag (warning: lots of output).
During phase 2 of the above case, after having checked all of some_other_file's dependencies and remade it if necessary, it then considers whether some_other_file is newer then some_file.
If this is the case then (and only then) are the recipes' variables expanded, and the contents of each line is passed to the shell, which here would be some_command some_other_file some_file.
It follows that you can't use $# etc. anywhere except as part of a recipe because automatic variables are only set during phase 2.
foo: $#bar
some_command $^
This will be expanded in phase 1 to:
foo: bar
some_command $^
Which during phase 2 will result in:
foo: bar
some_command bar
Probably not what you wanted.
Some makes have ways of working around this limitation. GNU make for instance has .SECONDEXPANSION, and the following will work as expected:
.SECONDEXPANSION:
foo: $$#bar
some_command $^
Anything after .SECONDEXPANSION will be expanded both during phase 1:
.SECONDEXPANSION:
foo: $#bar
some_command $^
and phase 2:
.SECONDEXPANSION:
foo: foobar
some_command foobar
Related
GNU make allows 1) parallel execution and 2) specifying several goals in the same invocation:
make -j4 clean all
But, as GNU make parallelizes the goals, some race conditions can occur. Illustration:
$ cat Makefile
clean:
#sleep 1 && rm -f foo
all: foo
#sleep 2 && cat foo
foo:
#echo '$#' > $#
$ make -j4 clean ; make -j4 all
foo
$ make -j4 clean all
cat: foo: No such file or directory
Makefile:5: recipe for target 'all' failed
make: *** [all] Error 1
Is there a nice way to force an order between the goals, but still benefit from the parallel acceleration for each goal? In the above example it would be nice to wait until clean completes before all starts in order to avoid race conditions.
As shown, the separate make invocations work as expected but this is not 100% satisfactory:
Some goals can be invoked simultaneously, some others cannot. Completely forbidding multiple goals can thus be considered as too restrictive. But identifying all valid and invalid combinations is tricky and error prone.
To completely avoid the problem, one could warn all potential users of such a Makefile that multiple goals invocations are not supported in parallel mode, but this warning will inevitably be overlooked by some users.
Race conditions do not always cause errors. Some could apparently work seamlessly but produce erroneous results.
IMHO this seems to root in the problem that in a programming language (in this case the shell) we are able to formulate dependencies which are of fundamentally different nature than the ones that make can handle. In your example there is a dependency of clean on the non-existence of foo, while all has the inverse dependency. If you make both targets active at the same time, this seems to surpass make's theoretical foundation - I don't know if there exists a sensible theory that can handle such relations. All that I could come up with is the explicit formulation:
.PHONY: all clean
clean:
#sleep 1
rm -f foo
all: foo $(filter clean,$(MAKECMDGOALS))
#sleep 2
cat foo
foo: $(filter clean,$(MAKECMDGOALS))
#echo Creating $#
#echo '$#' > $#
I think this is an interesting problem for sure.
I found a workaround (but I am not 100% convinced that it is the best solution and that it has no hidden drawbacks). The idea is to use the MAKECMDGOALS GNU make variable and the conditionals to force the serialization of multiple goals:
ifeq ($(words $(MAKECMDGOALS)),1)
.PHONY: all clean
clean:
#sleep 1 && rm -f foo
all: foo
#sleep 2 && cat foo
foo:
#echo '$#' > $#
else
.NOTPARALLEL:
%:
#$(MAKE) $#
endif
Of course, the condition of the conditional could be more sophisticated, like, for instance, testing if one of the goals matches clean...
Consider this makefile:
%: %.c
#echo making $#
#touch $#
.SECONDEXPANSION:
%.pid: $$(basename $$#)
$(<D)/$(<F) --pidfile=$<.pid
Here, the first rule builds a program and second rule starts it, producing a pid-file. Note: I know that secondary expansion is unnecessary in this example; the real makefile is more complex and I really need secondary expansion there.
So, typing make foo.pid, I expect Make to build foo from foo.c by the first rule and then run ./foo --pidfile=foo.pid by the second one.
However, it does not seem to work:
$ make -f Makefile.test foo.pid
make: *** No rule to make target 'foo.pid'. Stop.
This somehow relates to secondary-expanded dependencies which are provided by pattern rules. If I write either
%.pid: % in the second rule (i. e. get rid of secondary-expansion), or
foo: %: %.c in the first rule (i. e. write an explicit static pattern rule),
it all suddenly works. Why? Is this a limitation of GNU Make? In my case, I'd like to avoid pre-listing all possible programs in the first rule.
I think your situation is described in section 10.5.5 Match-Anything Pattern Rules. Whay you have here is a non-terminal match-anything rule, which is not allowed to match a file name that indicates a specific type of data.
If you had let's say
%:: %.c
#echo making $#
#touch $#
it would work.
I want to make a language depend target. In Particular: I have one source-file and I want to create different Objects which where add to the corresponding language folder. That single source file will differ in the C-Flags, the compiler will get. As long as I used it in a static way, it works quite fine.
de/info.o en/info.o es/info.o : info.c
$(ECHO) (DEP) $< for $#
Now I thought, it would be great if it is a bit more dynamic, in case i'll add a new language depending file. So I used a wildcard as followed:
de/%.o en/%.o es/%.o : %.c
$(ECHO) (DEP) $< for $#
But now it just make the first target and ignores the rest. The Make-Debug prints the following thing:
Successfully remade target file `de/info.o'.
Considering target file `en/info.o'.
File `en/info.o' was considered already.
Just in case: No, the objects do not exist. So there is no target, but an existing dependencie, so make should execute the rules.
EDIT: Found a solution for that Problem.
define FOO
$(1)/%.o : %.c
$(ECHO) $$< for $(1)
endef
$(foreach lang,$(LANGUAGE_LIST), $(eval $(call FOO,$(lang))))
Inspired by: http://www.gnu.org/software/make/manual/make.html#Eval-Function
Pattern rules work differently than implicit rules. While an implicit rule such as
a b c: d
command
is equivalent to the longer notation
a: d
command
b: d
command
c: d
command
this does NOT hold for pattern rules. Pattern rules with multiple targets are explicitly required to build all of their targets in a single invocation of command. Thus you would have to write
$ cat GNUmakefile
all: de/x.o en/x.o es/x.o
de/%.o: %.c
#echo $# from $<
en/%.o: %.c
#echo $# from $<
es/%.o: %.c
#echo $# from $<
$ gmake
de/x.o from x.c
en/x.o from x.c
es/x.o from x.c
The relevant documentation is found in 10.5.1 Introduction to Pattern Rules of the GNU make manual:
Pattern rules may have more than one target. Unlike normal rules, this does not act as many different rules with the same prerequisites and recipe. If a pattern rule has multiple targets, make knows that the rule’s recipe is responsible for making all of the targets. The recipe is executed only once to make all the targets. When searching for a pattern rule to match a target, the target patterns of a rule other than the one that matches the target in need of a rule are incidental: make worries only about giving a recipe and prerequisites to the file presently in question. However, when this file’s recipe is run, the other targets are marked as having been updated themselves.
I have the following makefile I use to generate files from some templates, the generated files have two possible extensions:
%.tex: %.tex*_tpl
./generate $#_tpl -o $#
%.xml: %.xml*_tpl
./generate $#_tpl -o $#
The dependency list will here match things like a.tex_tpl, a.tex-subpart1_tpl, a.tex-subpart2_tpl.
While this works, is there a way to avoid repetition? For example by matching *.{tex,xml} in the rule name and use the whole matched name in the dependency list? Something that would look like that:
%.{tex,xml}: $#_tpl
./generate $< -o $#
(Though I know %.{tex,xml} is not a valid rule name and you can't use $# in the dependency list)
Or any other (cleaner?) way.
Seems to me this does what you are looking for:
#
# I've assumed that files of the form:
#
# a.xml_tpl
# b.tex_tpl
#
# determine what targets you want to build
#
TARGETS:=$(patsubst %_tpl,%,$(wildcard *.xml_tpl *.tex_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $^ -o $#
The key is to use .SECONDEXPANSION to allow $$(wildcard %*_tpl) to be evaluated in a second expansion phase. The double $ is not a typo, by the way; it protects the expression from being evaluated at the time of the first expansion.
If I populate a directory with these files:
a.tex-subpart1_tpl
a.tex_tpl
a.xml-subpart1_tpl
a.xml-subpart2_tpl
a.xml_tpl
and run make -n, I get this on the console:
./generate a.xml_tpl a.xml-subpart1_tpl a.xml-subpart2_tpl -o a.xml
./generate a.tex_tpl a.tex-subpart1_tpl -o a.tex
Why the Second Expansion?
Without the second expansion, you'd have to have $(wildcard %*_tpl) in the dependency because with the $$ the wildcard function would never execute. Instead, make would treat $$(wildcard..) literally as the dependency, which is obviously wrong.
Ok, so $(wildcard %*_tpl) would be evaluated at the time make first runs across that line (this is the "first expansion"). At that time % has no value yet so wildcard would roughly be doing something like what would be ls %*_tpl at the command line.
For reasons of speed, make does not by default give you the opportunity to do any evaluation later than during the first expansion. If you want a later opportunity you have to specify .SECONDEXPANSION, which turns on the second expansion processing. Make still performs the firts expansion as usual. This is why you need to have $$(wildcard: it is transformed to $(wildcard during the first expansion. At the time of the second expansion make sees $(wildcard %*_tpl), replaces % with the actual stem and then executes the wildcard function with the actual stem rather than with a literal %.
Why $(TARGETS) in the Pattern Rule?
The pattern rule could be written:
%: $$(wildcard %*_tpl)
./generate $^ -o $#
without $(TARGETS). However, this rule would do nothing, as it would be a "match-anything rule". Basically, if make took such a rule at face value, then the computation cost would be significant, and most likely it is not the case that the author of the Makefile really means to apply this rule to any file whatsoever. So such a rule comes with restrictions, which in the Makefile here make it useless.
Adding $(TARGETS) makes it into a static pattern rule, which is not a match-anything rule. The addition of $(TARGETS) in front of the target pattern tells make that the rule applies only to these targets, and nothing else.
have one time consuming step that flattens a bunch of files.
basically i'd like this to be valid syntax
%.out: %.input1 %.input2
merge $<1 $<2 $#
doit: project.out
# project.out merged
i'm far from a makefile expert, but i like to use .SULFFIXS to do that when only have two files, would be great to adapt that to two inputs, or two outputs for future use...
.SULFFIX: .in.out
.in.out:
dosomething $< $#
doit: project.out
GNU Make 3.81
I've found several discussions on how to have a rule with several outputs, but none with several inputs.
%.out: %.input1 %.input2
merge $<1 $<2 $#
There are no $1 and $2 automatic variables in makefile rules, however, there is $^ which is the list of all prerequisites. As the rule only has two prerequisites it expands to %.input1 %.input2, or, more precisely, to $*.input1 $*.input2. Hence:
%.out: %.input1 %.input2
merge $^ $#
Should suffice.
.SULFFIXES, IMO, is not very useful, since it uses implicit rule definitions which one can't see and change.
[update]
With regards to doit target, to prevent it from executing every time you might like to change the rule commands to:
doit: project.out
# project.out merged
touch $#