I am having to quickly grok a Makefile, to understand some issues I'm having. I have come across these lines, and I want to make sure I'm understanding their use here correctly.
cp $< $#
sed -i -e 's#{{DEPLOY}}#https://example.com/#g' $#
To be clear, I know that sed is being used to make a (global?) string substitution here, but the use of bash special characters make this perhaps more confusing than it should be.
Would appreciate explanation on what the two lines above are actually doing.
They are GNU Make automatic variables:
$#
The file name of the target of the rule. If the target is an archive member, then ‘$#’ is the name of the archive file. In a pattern rule that has multiple targets (see Introduction to Pattern Rules), ‘$#’ is the name of whichever target caused the rule’s recipe to be run.
$< The name of the first prerequisite. If the target got its recipe from an implicit rule, this will be the first prerequisite added by the implicit rule (see Implicit Rules).
See what targets and prerequisites are.
sed -i -e 's#{{DEPLOY}}#https://example.com/#g' $#
Replaces all occurrences of {{DEPLOY}} with https://example.com/ in $#.
Putting all together, a simpler and more robust version of
cp $< $#
sed -i -e 's#{{DEPLOY}}#https://example.com/#g' $#
is:
sed -e 's#{{DEPLOY}}#https://example.com/#g' $< > $#
Related
I have the following rules
define compile_c
$(ECHO) "CC $<"
$(Q)$(CC) $(CFLAGS) -c -MD -o $# $<
## The following fixes the dependency file.
## See http://make.paulandlesley.org/autodep.html for details.
## Regex adjusted from the above to play better with Windows paths, etc.
#$(CP) $(#:.o=.d) $(#:.o=.P); \
$(SED) -e 's/#.*//' -e 's/^.*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $(#:.o=.d) >> $(#:.o=.P); \
$(RM) -f $(#:.o=.d)
endef
vpath %.c . $(TOP)
$(BUILD)/%.o: %.c $(BUILD)/%.pp
$(call compile_c)
vpath %.c . $(TOP)
$(BUILD)/%.pp: %.c
$(ECHO) "PreProcess $<"
$(Q)$(CC) $(CFLAGS) -E -Wp,-C,-dD,-dI -o $# $<
When the build finishes, GNU make says
Removing intermediate files... and deletes all the .pp files which I do NOT want.
Why is it doing this?
How do I stop it?
Since you're using GNU Make, you can make the following adjustment to your Makefile:
.PRECIOUS: $(BUILD)/%.pp # ADD THIS LINE
$(BUILD)/%.pp: %.c
$(ECHO) "PreProcess $<"
$(Q)$(CC) $(CFLAGS) -E -Wp,-C,-dD,-dI -o $# $<
The documentation has this to say about .PRECIOUS directives:
The targets which .PRECIOUS depends on are given the following special treatment: if make is killed or interrupted during the execution of their recipes, the target is not deleted.
[...]
Also, if the target is an intermediate file, it will not be deleted after it is no longer needed, as is normally done.
[...]
You can also list the target pattern of an implicit rule (such as ‘%.o’) as a prerequisite file of the special target .PRECIOUS to preserve intermediate files created by rules whose target patterns match that file's name.
This has the benefit of not creating an unwanted additional rule. It's also clearer what you're trying to do: keep the precious intermediate files that might be expensive to recreate.
I think the best solution is to use the .SECONDARY special target. Just add this line:
.SECONDARY:
Quoting the manual:
.SECONDARY with no prerequisites causes all targets to be treated as secondary (i.e., no target is removed because it is considered intermediate).
Why is this better than making the targets prerequisites of a throw-away target? That's more clutter, and has to be done explicitly for every set of files that might be generated with pattern rules.
Why is this better than .PRECIOUS? That causes files to be retained even if their recipe fails when using .DELETE_ON_ERROR. The latter is important to avoid failing recipes leaving behind bad outputs that are then treated as current by subsequent make invocations. IMO, you always want .DELETE_ON_ERROR, but .PRECIOUS breaks it.
If you search for "gnu make intermediate files" you'll immediately find the answer as to why it's happening, in the GNU make manual section Chains of Implicit Rules.
It also tells you how to avoid it: a file cannot be intermediate if it is mentioned in the makefile as a target or prerequisite.
So, just list your .pp files as a prerequisite of some rule, somewhere. It doesn't have to be a rule that's ever invoked. You don't give enough of your makefile here for us to provide a complete answer, but it would be something like:
all_pps: $(ALL_OBJECTS:.o=.pp)
assuming you had a variable ALL_OBJECTS containing all your .o files.
Here is a detail that finally got PRECIOUS working for me. The pattern that you give to PRECIOUS has to be exactly the pattern that is being used in the rule that creates the intermediate file. I want to save all files prefixed by moc_. At first I used
.PRECIOUS: moc_%
to no avail. Then I tried
.PRECIOUS: moc_%.cpp
and this did the trick.
Consider this makefile:
%: %.c
#echo making $#
#touch $#
.SECONDEXPANSION:
%.pid: $$(basename $$#)
$(<D)/$(<F) --pidfile=$<.pid
Here, the first rule builds a program and second rule starts it, producing a pid-file. Note: I know that secondary expansion is unnecessary in this example; the real makefile is more complex and I really need secondary expansion there.
So, typing make foo.pid, I expect Make to build foo from foo.c by the first rule and then run ./foo --pidfile=foo.pid by the second one.
However, it does not seem to work:
$ make -f Makefile.test foo.pid
make: *** No rule to make target 'foo.pid'. Stop.
This somehow relates to secondary-expanded dependencies which are provided by pattern rules. If I write either
%.pid: % in the second rule (i. e. get rid of secondary-expansion), or
foo: %: %.c in the first rule (i. e. write an explicit static pattern rule),
it all suddenly works. Why? Is this a limitation of GNU Make? In my case, I'd like to avoid pre-listing all possible programs in the first rule.
I think your situation is described in section 10.5.5 Match-Anything Pattern Rules. Whay you have here is a non-terminal match-anything rule, which is not allowed to match a file name that indicates a specific type of data.
If you had let's say
%:: %.c
#echo making $#
#touch $#
it would work.
I want to make a language depend target. In Particular: I have one source-file and I want to create different Objects which where add to the corresponding language folder. That single source file will differ in the C-Flags, the compiler will get. As long as I used it in a static way, it works quite fine.
de/info.o en/info.o es/info.o : info.c
$(ECHO) (DEP) $< for $#
Now I thought, it would be great if it is a bit more dynamic, in case i'll add a new language depending file. So I used a wildcard as followed:
de/%.o en/%.o es/%.o : %.c
$(ECHO) (DEP) $< for $#
But now it just make the first target and ignores the rest. The Make-Debug prints the following thing:
Successfully remade target file `de/info.o'.
Considering target file `en/info.o'.
File `en/info.o' was considered already.
Just in case: No, the objects do not exist. So there is no target, but an existing dependencie, so make should execute the rules.
EDIT: Found a solution for that Problem.
define FOO
$(1)/%.o : %.c
$(ECHO) $$< for $(1)
endef
$(foreach lang,$(LANGUAGE_LIST), $(eval $(call FOO,$(lang))))
Inspired by: http://www.gnu.org/software/make/manual/make.html#Eval-Function
Pattern rules work differently than implicit rules. While an implicit rule such as
a b c: d
command
is equivalent to the longer notation
a: d
command
b: d
command
c: d
command
this does NOT hold for pattern rules. Pattern rules with multiple targets are explicitly required to build all of their targets in a single invocation of command. Thus you would have to write
$ cat GNUmakefile
all: de/x.o en/x.o es/x.o
de/%.o: %.c
#echo $# from $<
en/%.o: %.c
#echo $# from $<
es/%.o: %.c
#echo $# from $<
$ gmake
de/x.o from x.c
en/x.o from x.c
es/x.o from x.c
The relevant documentation is found in 10.5.1 Introduction to Pattern Rules of the GNU make manual:
Pattern rules may have more than one target. Unlike normal rules, this does not act as many different rules with the same prerequisites and recipe. If a pattern rule has multiple targets, make knows that the rule’s recipe is responsible for making all of the targets. The recipe is executed only once to make all the targets. When searching for a pattern rule to match a target, the target patterns of a rule other than the one that matches the target in need of a rule are incidental: make worries only about giving a recipe and prerequisites to the file presently in question. However, when this file’s recipe is run, the other targets are marked as having been updated themselves.
I have the following makefile I use to generate files from some templates, the generated files have two possible extensions:
%.tex: %.tex*_tpl
./generate $#_tpl -o $#
%.xml: %.xml*_tpl
./generate $#_tpl -o $#
The dependency list will here match things like a.tex_tpl, a.tex-subpart1_tpl, a.tex-subpart2_tpl.
While this works, is there a way to avoid repetition? For example by matching *.{tex,xml} in the rule name and use the whole matched name in the dependency list? Something that would look like that:
%.{tex,xml}: $#_tpl
./generate $< -o $#
(Though I know %.{tex,xml} is not a valid rule name and you can't use $# in the dependency list)
Or any other (cleaner?) way.
Seems to me this does what you are looking for:
#
# I've assumed that files of the form:
#
# a.xml_tpl
# b.tex_tpl
#
# determine what targets you want to build
#
TARGETS:=$(patsubst %_tpl,%,$(wildcard *.xml_tpl *.tex_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $^ -o $#
The key is to use .SECONDEXPANSION to allow $$(wildcard %*_tpl) to be evaluated in a second expansion phase. The double $ is not a typo, by the way; it protects the expression from being evaluated at the time of the first expansion.
If I populate a directory with these files:
a.tex-subpart1_tpl
a.tex_tpl
a.xml-subpart1_tpl
a.xml-subpart2_tpl
a.xml_tpl
and run make -n, I get this on the console:
./generate a.xml_tpl a.xml-subpart1_tpl a.xml-subpart2_tpl -o a.xml
./generate a.tex_tpl a.tex-subpart1_tpl -o a.tex
Why the Second Expansion?
Without the second expansion, you'd have to have $(wildcard %*_tpl) in the dependency because with the $$ the wildcard function would never execute. Instead, make would treat $$(wildcard..) literally as the dependency, which is obviously wrong.
Ok, so $(wildcard %*_tpl) would be evaluated at the time make first runs across that line (this is the "first expansion"). At that time % has no value yet so wildcard would roughly be doing something like what would be ls %*_tpl at the command line.
For reasons of speed, make does not by default give you the opportunity to do any evaluation later than during the first expansion. If you want a later opportunity you have to specify .SECONDEXPANSION, which turns on the second expansion processing. Make still performs the firts expansion as usual. This is why you need to have $$(wildcard: it is transformed to $(wildcard during the first expansion. At the time of the second expansion make sees $(wildcard %*_tpl), replaces % with the actual stem and then executes the wildcard function with the actual stem rather than with a literal %.
Why $(TARGETS) in the Pattern Rule?
The pattern rule could be written:
%: $$(wildcard %*_tpl)
./generate $^ -o $#
without $(TARGETS). However, this rule would do nothing, as it would be a "match-anything rule". Basically, if make took such a rule at face value, then the computation cost would be significant, and most likely it is not the case that the author of the Makefile really means to apply this rule to any file whatsoever. So such a rule comes with restrictions, which in the Makefile here make it useless.
Adding $(TARGETS) makes it into a static pattern rule, which is not a match-anything rule. The addition of $(TARGETS) in front of the target pattern tells make that the rule applies only to these targets, and nothing else.
I have a.csv,b.csv, ... in a my docs/csv directory, I need convert each of this file to a json file.
I follow this question to write a Makefile like this.
SRCS = $(wildcard docs/csv/*.csv)
DESTS = $(patsubst docs/csv/%.csv, scripts/data/%.lua, $(SRCS))
all: $(DESTS)
$(DESTS): $(SRCS)
echo $#
echo $<
but every time I ran make all, the echo $# show every file as expected, but echo $< always show the single file, called items.csv in my csv folder.
The trouble is that in this rule:
$(DESTS): $(SRCS)
...
every lua file depends on all csv files, which is not what I think you intend. And since $< expands to the first prerequisite, you get the same one (items.csv) for every target.
Try this:
all: $(DESTS)
scripts/data/%.lua: docs/csv/%.csv
echo $#
echo $<
$<
is the name of the FIRST dependency. Use $^ for all the dependencies
$#
is the name of the current target
The GNU make man page on Automatic Variables is extremely useful. Here's what it says:
$#
The file name of the target of the rule. If the target is an archive member, then ‘$#’ is the name of the archive file. In a
pattern rule that has multiple targets (see Introduction to Pattern
Rules), ‘$#’ is the name of whichever target caused the rule's recipe
to be run.
$<
The name of the first prerequisite. If the target got its recipe from an implicit rule, this will be the first prerequisite added by
the implicit rule (see Implicit Rules).
Incidentally, you probably want to write your make rule as a pattern rule instead:
%.lua : %.csv
<rules for making a lua from a csv>