Understanding Makefile Syntax and Variables - makefile

I'm reading a big Makefile, part of which I don't understand:
$(IREJECTION): $(IREJECTION:%$(MACH64).o=%.cpp)
$(CPP) $(CPPDLIBOPTS) -c $(#:%$(MACH64).o=%.cpp) -o $#
In this script (note that I removed unnecessary variables to avoid verbosity), what does the following mean
$(IREJECTION:%$(MACH64).o=%.cpp) (on the first line)
$(#:%$(MACH64).o=%.cpp) ? (on the second line)
Also, what does this form A: B mean? As in :
$(IREJECTION): $(IREJECTION:%$(MACH64).o=%.cpp) #on the first line
<---- A ----> <------------ B ---------------> #first instance of A: B
<--- A ----> <-------- B ------> #second instance of A: B
As you see, there are two instances of the form A:B — the second one is inside B itself. I see similar thing here as well:
$(#:%$(MACH64).o=%.cpp) #on the second line
<A> <---------B------->
Please help me understand this.

You got quite complex example I think. It has many things in it.
Rule / Recipe
A: B
<command>
A - target
B - dependency
<command> - command to be executed to build A ("Recipe")
target: dependency is called a 'rule'.
So all in all, above example is 'rule' with appropriate 'recipe'.
(to make A, which depends on B, one needs to execute <command>)
make compares modifications dates of B and A. If B is newer, then it executes <command>
Variables
$(IREJECTION) is make's variable (it should be defined somewhere before in the file, like IREJECTION:=somefile.o)
During make execution $(IREJECTION) is replaced to actual value of the variable.
Variable substitution
From link below:
$(var:a=b) <..> take the value of the variable var, replace every 'a' at the end of a word with 'b' in that value, and substitute the resulting string.
And:
For example:
foo := a.o b.o c.o
bar := $(foo:%.o=%.c)
sets bar to a.c b.c c.c.
In your case $(IREJECTION:%$(MACH64).o=%.cpp), it takes variable named IREJECTION, tries to find $(MACH64).o (which also references variable MACH64) at the end of the word and replace it with .cpp.
Automatic variable
$# is called automatic variable.
It is reference to the 'target'.
Futher reading
http://www.gnu.org/software/make/manual/make.html

A: B meant that you add a make target A that depends on B. That means when B was changed, it has to be run before A is done.
You can use the target A by calling make A.

I'll go tep by step:
lets say you have some object file irejection.mach64.o and the source file irejection.cpp
to generate object from source you typically write a rule smth like
irejection.mach64.o : irejection.cpp # means target : dependencies
$(CC) irejection.cpp -o $# # $# is a special variable - the target (output)
Now lets say $(MACH64) is .mach64 and $(IREJECTION) is irejection$(MACH64).o, ie irejection.mach64.o
$(IREJECTION:%$(MACH64).o=%.cpp) will expand to irejection.cpp
$(#:%$(MACH64).o=%.cpp) will expand to the same, since $# is $(IREJECTION)
In essence, given object file target with a architecture extensions, rewrite the file name to its source file counterpart.
Seems unncessearily convoluted to me tho. Cleaner way would be smth like:
%$(MACH64).o : %.cpp
$(CC) -c $# $<
% are "wildcards", $< is the first dependency, $# is the output
see: http://www.gnu.org/software/make/manual/make.html#Automatic-Variables
http://www.gnu.org/software/make/manual/make.html#Pattern-Rules

Related

How do you use (GNU) make's "simply expanded" variables in build rules and not get the last definition?

I have a complicated set of rules I need to write to generate a rather large number of "parameterised" output files and thought that, rather than expand them all out by hand, I could repeatedly "include" a template file with sets of rules and use (GNU)make's facility for allowing "simply expanded" variables to avoid the pain.
(In the past I've always been using the "recursively expanded" variable approach, so this is new to me)
As a trivial example of what I thought would work, I tried putting the following in a Makefile
Targ:=A
Param1:=Pa
Param2:=Qa
$(Targ):
#echo expect A, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
Targ:=B
Param1:=Pb
Param2:=Qb
$(Targ):
#echo expect B, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
Targ:=C
Param1:=Pc
Param2:=Qc
$(Targ):
#echo expect C, get $(Targ), Target is $#. Params are $(Param1) and $(Param2)
The eventual plan was to replace the rules with an include file containing dozens of different rules, each referencing the various "parameter" variables.
However, what I get is...
prompt> make A
expect A, get C, Target is A. Params are Pc and Qc
prompt> make B
expect B, get C, Target is B. Params are Pc and Qc
Essentially, unlike each rule's target, which is picking up the intended definition, the $(Targ), $(Param1), and $(Param2) in each rule's command is instead being run with the final definition.
Does anyone know how to prevent this, i.e. how do you force the command to use the definition at the time it is encountered in the Makefile?
Simple vs recursive expansion makes no difference here; regardless of which you use you'll see the same behavior. A GNU make variable is global and obviously can have only one value.
You have to understand when variables are expanded. The documentation provides a detailed description of this. Targets and prerequisites are expanded when the makefile is read in, so the value of Targ as the makefile is being parsed is used.
Recipes are expanded when the recipe is to be invoked, which is not until after all makefiles are parsed and make starts to build targets. At that time of course the variable Targ has its last set value.
Without knowing what your makefile really does it's hard to suggest an alternative. One option is to use target-specific variables:
Targ := A
$(Targ): LocalTarg := $(Targ)
$(Targ):
#echo expect A, get $(LocalTarg), Target is $#
Another option is to use constructed variable names:
Targ := A
Targ_$(Targ) := $(Targ)
$(Targ):
#echo expect A, get $(Targ_$#), Target is $#
Apologies for answering my own question, but I now realised it is possible to solve the issue I was having by running make recursively.
E.g. if the parameter variables for the rules are Targ, Param1 and Param2 then
#Set up "default" values for the parameters (As #madscientist points out,
#these will safely be overridden by the defs on the #(make) commands below
Targ=XXXXXXXXXX
Param=XXXXXXXXXX
Param2=XXXXXXXXXX
Recursing=
#
# define N (==3) templated rule(s)
#
$(Targ)%a:
#echo Run Combo_a $(Targ) $(Param1) $(Param2) $#
$(Targ)%b:
#echo Run Combo_b $(Targ) $(Param2) $(Param1) reversed $#
$(Targ)%c:
#echo Run Combo_c $(Param1) $(Targ) $(Param2) mixed again $#
#
#Enumerate "M" (==2) sets of parameters,
# (Except if we are already recursing because unrecognised targets may cause
# it to descend forever)
#
ifneq ($(Recursing), Yes)
Set1%:
#$(MAKE) Targ=Set1 Param1=foo Param2=bar Recursing=Yes $#
Set2%:
#$(MAKE) Targ=Set2 Param1=ray Param2=tracing Recursing=Yes $#
endif
This then allows N*M different combos for N+M typing cost.
eg. (removing messages from make re recursion)
>make Set1.a
Run Combo_a Set1 foo bar Set1.a
>make Set2.c
Run Combo_c ray Set2 tracing mixed again Set2.c

How to limit Makefile pattern matching to a list of options

I would like to use an implicit rule (of the %.final: %.intermediate type) in a makefile, but only for certain files (only a.final and b.final, but not x.final)
I've tried to use
targets=a b c
${targets} %.intermediate: %.original
process1 $< > $#
${targets}: %.final: %.intermediate
process2 $< > $#
all: ${targets}
but I'm just not making make happy. I'm not sure exactly what these implicit rules are doing, and if targets needs to include '.final', or how to really make it come together.
If I do
a.intermediate: a.original
process1 a.original > a.intermediate
a.final: a.intermediate
process2 a.intermediate > a.final
#and continue for b, c, etc.
Then things are just fine and dandy.
Here's the general idea. It's not in good form because I'm not used to writing makefiles with pattern substitution and so forth.
#Reference data
CHIP_WCE_ACCESSION=SRR713343
CHIP_BLANK_ACCESSION=SRR713344
#Experiment data
CHIP_NANOG_ACCESSION=SRR713342
CHIP_SOX2_ACCESSION=SRR713341
CHIP_OCT4_ACCESSION=SRR713340
CHIP_TARGETS=${CHIP_NANOG_ACCESSION} ${CHIP_SOX2_ACCESSION} ${CHIP_OCT4_ACCESSION}
#Note that CHIP_WCE_ACCESSION is *not* processed with these.
CHIP_REFERENCE=${CHIP_BLANK_ACCESSION} ${CHIP_WCE_ACCESSION}
BAM_COVERAGE=bamCoverage --numberOfProcessors 16 --binSize 10
#This is a specific bioinformatics program.
${CHIP_REFERENCE}: %.sort.bam
echo hello > $# #Not the same way the CHIP_TARGETS are created.
${CHIP_TARGETS}: %.sort.bam
touch $# #There's actually a long chain of processing, but touch works for the minimal example.
${CHIP_TARGETS}: %.bw: %.sort.bam ${CHIP_BLANK_ACCESSION}.sort.bam
${BAM_COVERAGE} --use-reference ${CHIP_BLANK_ACCESSION}.sort.bam --bam $< -o $#
chip: ${CHIP_NANOG_ACCESSION}.bw ${CHIP_SOX2_ACCESSION}.bw ${CHIP_OCT4_ACCESSION}.bw
Expected result: ${BAM_COVERAGE} is executed on all of the CHIP_TARGETS, with appropriate execution of CHIP_REFERENCE rules for the $CHIP_BLANK_ACCESSION.
Current results: I don't know how to write this rule.
This:
${targets} %.intermediate: %.original
is not valid because all targets in a pattern rule must contain the pattern.
This:
${targets}: %.final: %.intermediate
is a static pattern rule, which is what you want, BUT every target in the static pattern rule must match the target. So you want this:
$(targets:%=%.final) : %.final : %.intermediate

Parameterized recipe in makefile?

I have a file "ORIGINAL", which, if updated, I would like to copy, modify, and distribute to a few places on the drive. The modification is made by a little bash script which takes one parameter, a parameter unique for each spawned remote file.
In my Makefile, I can do this with a separate rule/recipe for each parameter, like so:
parameters = AWK BAT CAT DOG
$(DEST_FILE_AWK) : $(ORIGINAL)
./copyAndModify "AWK" ## Creates $(ORIGINAL)_AWK, substed copy of ORIGINAL
mv - f $(ORIGINAL)_AWK $(DEST_FILE_AWK)
$(DEST_FILE_BAT) : $(ORIGINAL)
./copyAndModify "BAT" ## Creates $(ORIGINAL)_BAT, substed copy of ORIGINAL
mv - f $(ORIGINAL)_BAT $(DEST_FILE_BAT)
The dereferenced values of DEST_FILE_AWK and DEST_FILE_BAT have nothing to do with each other, but other than that, the two recipes above are exactly the same with the only difference the parameter, so I can't help but want to merge them into one super rule/recipe with a multiple target rule line.
But I just can't make it happen. I've tried all kinds of foreach() and other stuff in the target section of the rule, but the problem is that no matter what, I can't get the value of the parameter into the recipe part.
Is there a way?
With the information provided here the best you can do (assuming you're using GNU make) is an eval/call combination. As anishsane suggests, depending on the value of the DEST_FILE_* variables it might be possible to do something simpler.
But this should work:
define COPY_TO_DEST
$$(DEST_FILE_$1) : $$(ORIGINAL)
./copyAndModify "$1"
mv - f $$(ORIGINAL)_$1 $$#
endef
parameters = AWK BAT CAT DOG
$(foreach P,$(parameters),$(eval $(call COPY_TO_DEST,$P)))
It can be done without $(eval), at least in gnu make :)
Start with one recipe that specifies all of the targets, i.e. the list of targets is on the left side of the recipe. Let's assume we have a variable that holds the names of all these targets.
Now observe that both functions and variables will be evaluated separately for a given recipe as it gets expanded for each of the targets. Recall that, say $# is just a variable, and will be substituted separately for each target. Function calls behave the same.
Provide a list of types, and a list of type:target pairs. I presume that there's no need to put the targets into separate variables like you did ($(DEST_FILE_AWK) etc).
The TARGET_FOR_TYPE function takes the pairs and the types and generates a list of destination files.
The TYPE variable is assigned once for each target, computed by the TYPE_FOR_TARGET function. That way the repeated function call doesn't pollute the recipe :)
Note that the DESTINATIONS list contains plain filenames, without any further indirection.
types = AWK BAT
ORIGINAL = an_original
DESTINATIONS = \
AWK:dest_for_awk \
BAT:dest_for_bat
TARGET_FOR_TYPE = $(patsubst $(1):%,%,$(filter $(1):%,$(DESTINATIONS)))
TYPE_FOR_TARGET = $(patsubst %:$(1),%,$(filter %:$(1),$(DESTINATIONS)))
# Usage example for the functions above:
$(info type: $(call TYPE_FOR_TARGET,dest_for_awk))
$(info target: $(call TARGET_FOR_TYPE,AWK))
$(info $())
DEST_FILES = $(foreach type,$(types),$(call TARGET_FOR_TYPE,$(type)))
all: $(DEST_FILES)
$(DEST_FILES) : TYPE=$(call TYPE_FOR_TARGET,$#)
$(DEST_FILES) : $(ORIGINAL)
#echo ./copyAndModify $(TYPE)
#echo mv - f $(ORIGINAL)_$(TYPE) $#

.PHONY in linux makefile

SUBDIRS = foo bar baz
.PHONY: dirs $(SUBDIRS)
dirs: $(SUBDIRS)
$(SUBDIRS):
#echo $#
#ls $#
Anybody can just help me out to understand this make-file?
If possible explain me each statement (why do we need it?, what is the purpose? etc.)
And how exactly this make-file works?
It have wrong formatting, so i can only guess what it was before... Well, so it is:
first line assignes list "foo bar baz" to variable named SUBDIRS
second line is special command that makes all specified targets 'phonetical' - you can invoke "make dirs" or "make foo", and it will find target with that name and execute it, but it's no actual file with this name (like usual non-phony targets)
third one - creates target named 'dirs' which depends on value of SUBDIRS variable. space-separated list. this target have no real actions
fourth line creates rules for SUBRIDS variable contents, with no dependencies. The rest of text is actions that have to be performed to 'make' this target (so, in your case - if you just call "make", it will call "make dirs" (because it's the first target), which depends on foo, bar and baz - so these targets will be invoked. to perform each of these targets, make will call echo and ls - so eventually you'll get these three directory names and list of their files)

Why does make not consider a target without recipe out of date when updated by a PHONY dependency?

.PHONY: b
c: a
#touch c
#echo "Changed"
a: b
b:
#date +%s > a
Running make with the sample make file causes "Changed" to be printed the 1st time it is run; but "Changed" is only printed then on the 3rd, 5th, etc execution. This is because make doesn't seem to recognize that executing the recipe for target "b" updates a.
Changing the rule with "a" as the target into an empty recipe causes "Changed" to be printed for each time make is run (as you would expect - where phony targets are always considered "out of date"). E.g.
a: b ;
Make should skip the implicit rule search for PHONY targets, but "a" is not PHONY. If no implicit rule is found for "a", is make correct to not consider that "a" may have been changed by its PHONY dependency "b"?
Make can't analyze the effects of commands, so it is the user's responsibility to organize the rules correctly.
Consider a slightly different case:
d: c b
c: a
#touch c
#echo "Changed"
a:
b:
#date +%s > a
This has the same behavior as your example; there's no way Make could be expected to know that c "really" depends on b. The author of the makefile is at fault.
Now the way it should be written:
c: a
#touch c
#echo "Changed"
.PHONY: a
a:
#date +%s > a
The a rule modifies the file a (and PHONY is there just to force the a rule to run). This is the way to tell make that the #date ... command modifies a. This makefile works correctly.
Your example is midway between these two. If a rule modifies a file which is the target of another rule, the makefile is badly organized, and Make is not at fault. Yes, Make could assume that a target that depends on a PHONY rule may have been updated when that rule is run, but it could just as well assume that any target may have been updated when any rule is run. And if Make were that paranoid, it wouldn't be very efficient.

Resources