When I try to compute the diff between two text files:
results.txt: file1.txt file2.txt
diff $(word 1,$^) $(word 2,$^) > $#
I get this weird (?) error:
$ make
diff file1.txt file2.txt > results.txt
makefile:2: recipe for target 'results.txt' failed
make: *** [results.txt] Error 1
What's wrong with my makefile?
The issue is that diff typically has a non-zero exit code if the files differ. This will cause make to infer the command failed. The easy fix would be to tell make to ignore the exit code...
results.txt: file1.txt file2.txt
-diff $(word 1,$^) $(word 2,$^) > $#
Edit: If the aim is to get rid of the diagnostic message entirely then you could just use something like...
results.txt: file1.txt file2.txt
diff $(word 1,$^) $(word 2,$^) > $# || exit 0
Related
There are a couple of kind of similar issues but I could not fit any of the proposed concepts to my case.
Just to give a little bit of context: I have a set of Julia files which create plots as PDFs which are part of a make procedure to create scientific papers, something like:
plots = $(shell find $(PLOT_PATH)/*.jl | sed 's/\.jl/\.pdf/g')
$(PLOT_PATH)/%.pdf: $(PLOT_PATH)/%.jl $(JULIA_SYSIMAGE)
$(JL) --project $< -o $(PLOT_PATH)
$(DOCUMENT_FILENAME).pdf: FORCE $(plots) $(figures)
latexmk $(DOCUMENT_FILENAME).tex
In the current setup, each XYZ.jl file is creating a XYZ.pdf file and it works absolutely fine.
Now I am dealing with cases where it would be much easier to create multiple plots from single Julia files, so a script like this:
#!/usr/bin/env julia
using PGFPlotsX
...
...
pgfsave("whatever.pdf")
pgfsave("another.pdf")
pgfsave("yetanother.pdf")
so that one could do a grep pgfsave SCRIPT | awk... to figure out the targets. However, I could not figure out how to generate dynamic targets (plots) based on the contents of the dependency file (Julia script).
An MWE for my problem is the following: I have a couple of files (dependencies) which are generating a bunch of targets, which are defined inside those files (and can be access via awk/grep/sed/whatever). For now, let's say that these are simply *.txt files and each line is a target.
file: a.txt
foo
bar
baz
file: b.txt
naarf
fjoord
A very basic (non-working) manual Makefile to demonstrate the goal would be something like this (it does not work as it cannot figure out how to make foo etc. but it shows the pattern for *.txt which needs to be repeated):
file: Makefile
all_products := $(shell find *.txt | xargs cat)
final_product: $(all_products)
echo $< > $#
(foo bar baz): a.txt
touch $(shell cat $<)
(narf fjoord): b.txt
touch $(shell cat $<)
so in principle, I need something to "process" the dependency (*.txt) to create a list of the targets, like
$(shell cat $%): %.txt
echo $< > $#
but I cannot manage to get a reference to the dependency on the target side ($% does not work).
Any ideas? Maybe the whole approach is just a bad idea ;)
A combination of GNU make foreach, eval and call functions is probably what you need. With your example:
TXT := $(wildcard *.txt)
.PHONY: all
.DEFAULT_GOAL := all
define MY_MACRO
$(1)-targets := $$(shell cat $(1))
$$($(1)-targets): $(1)
echo $$< > $$#
all: $$($(1)-targets)
endef
$(foreach t,$(TXT),$(eval $(call MY_MACRO,$(t))))
(pay attention to the $$ in the macro definition, they are needed). And then:
$ make
make
echo a.txt > foo
echo a.txt > bar
echo a.txt > baz
echo b.txt > naarf
echo b.txt > fjoord
If you want the recipe to build all targets at once you'll need a recent enough GNU make version (4.3 or later) and its new rule with grouped targets (x y z&: w):
TXT := $(wildcard *.txt)
.PHONY: all
.DEFAULT_GOAL := all
define MY_MACRO
$(1)-targets := $$(shell cat $(1))
$$($(1)-targets)&: $(1)
touch $$($(1)-targets)
all: $$($(1)-targets)
endef
$(foreach t,$(TXT),$(eval $(call MY_MACRO,$(t))))
And then:
$ make
touch foo bar baz
touch naarf fjoord
Note that in this case we could also use a simpler and less GNU make-dependent solution. Just use empty dummy files as time stamps, for instance .a.txt.tag for a.txt, and a static pattern rule:
TXT := $(wildcard *.txt)
TAG := $(patsubst %,.%.tag,$(TXT))
.PHONY: all
all: $(TAG)
$(TAG): .%.tag: %
touch `cat $<` $#
I have the following Makefile:
ifneq ($(MAKECMDGOALS),clean)
-include generated.mk
endif
FOO ?= foo
all: a.txt
a.txt:
echo $(GEN_FOO) > $#
generated.mk: Makefile
echo GEN_FOO = $(FOO) > $#
.PHONY: clean
clean:
$(RM) a.txt
$(RM) generated.mk
It works OK when building single targets:
$ make clean
rm -f a.txt
rm -f generated.mk
$ make all
echo GEN_FOO = foo > generated.mk
echo foo > a.txt
However when I try to build multiple targets at once things go not so smooth:
$ make clean all
rm -f a.txt
rm -f generated.mk
echo foo > a.txt
$ make all
echo GEN_FOO = foo > generated.mk
make: Nothing to be done for 'all'.
It gets even worse if variables were provided:
$ make clean
rm -f a.txt
rm -f generated.mk
$ make FOO=bar clean all
echo GEN_FOO = bar > generated.mk
rm -f a.txt
rm -f generated.mk
echo bar > a.txt
$ make all
echo GEN_FOO = foo > generated.mk
make: Nothing to be done for 'all'.
$ make FOO=bar clean all
rm -f a.txt
rm -f generated.mk
echo foo > a.txt
Are there any ways to fix such incorrect behavior?
Make is doing exactly what you told it to do, and you haven't told us what you want it to do that's different than what you told it to do (saying fix such incorrect behavior doesn't really help us when you don't define what's incorrect about the behavior), so we can't help you very much.
You are probably getting confused about the interaction between included makefiles and comparing $(MAKECMDGOALS). Please note:
ifneq ($(MAKECMDGOALS),clean)
this will not match unless you specify exactly one target: clean. In situations where you specify multiple targets, one of which is clean, that will match because clean all is not equal to clean. So, when you run make clean all make will include the generated makefile, and will generate it if it doesn't exist.
Because generated include files are only rebuilt once, when the makefile is first parsed, it's not possible to say something like: "first run rule X (e.g., clean) then rebuild the included makefiles, then reinvoke make".
However, it's pretty much always a bad idea to invoke make with clean all. This is because if you were to ever try to add -j for parallelism, the clean and the build would be running in parallel and corrupt everything.
One semi-common option is to provide a different rule that will do both, something like this:
rebuild:
$(MAKE) clean
$(MAKE) all
then run make rebuild instead.
You can certainly force the behavior with the help of the shell. For instance, in bash you could use
for target in "clean" "all";
do
make $target;
done
and if you were going to re-do the procedure a lot you could either make it an executable script or wrap it in a shell function.
I have some kind of make dependency problem. all_2 does not rebuild after a1.src has been touched, but all_1 does. Why? Can't I use absolute paths?
$ cat Makefile
DIR = ${HOME}/tmp
outputs = $(DIR)/dir1/a1.out $(DIR)/dir2/a2.out
all_1 : dir1/a1.out dir2/a2.out
all_2 : $(outputs)
ls -l $(outputs) # debug print
*/%.out : $(notdir %.src)
#touch $#
#echo 'Build $# from $(notdir $*.src)'
This is my directory structure:
$ ls -R
Makefile a1.src a2.src dir1 dir2
./dir1:
a1.out
./dir2:
a2.out
all_1 works fine:
$ touch a1.src
$ make all_1
Build dir1/a1.out from a1.src
$ make all_1
make: Nothing to be done for `all_1'.
but all_2 does not rebuild a1.out (although the out files exist, so I guess that the targets are ok):
$ touch a1.src
$ make all_2
ls -l /Users/eternity/tmp/dir1/a1.out /Users/eternity/tmp/dir2/a2.out # debug print
-rw-r--r-- 1 eternity staff 0 Jan 20 15:25 /Users/eternity/tmp/dir1/a1.out
-rw-r--r-- 1 eternity staff 0 Jan 20 14:46 /Users/eternity/tmp/dir2/a2.out
$
First, the * is not a "wildcard" in the target side. You need a pattern rule with %.out as target:
%.out: ...
That will match against target names with a trailing .out.
The stem (i.e.: the % part in the target) with which the pattern rule was matched against will be stored in the automatic variable $*:
%.out: $(notdir $*.src)
However, that alone won't do: you also need to enable secondary expansion to use the variable $* as part of the rule definition, because its value is empty during the first expansion. In order to enable secondary expansion, simply define the target .SECONDEXPANSION, i.e.:
.SECONDEXPANSION:
Once enabled, for delaying the expansion of a variable or function at the rule definition, you need to replace $ by $$:
%.out: $$(notdir $$*.src)
So, with all that in mind, your last rule should look like:
.SECONDEXPANSION:
%.out: $$(notdir $$*.src)
#touch $#
#echo 'Build $# from $(notdir $*.src)'
( my question is different from Force Makefile to execute script before building targets )
I've got the following Makefile:
.PHONY: dump_params all
all: tmpA
tmpA: tmpB tmpC dump_params
cat $(filter tmp%,$^) > $#
tmpB: dump_params
touch $#
tmpC: dump_params
touch $#
dump_params:
echo "Makefile was run." >> config.txt
with the target dump_params I want to create/append a file each time a new target is invoked (to keep track of the version of the tools used). However, when I call
make tmpA
all the targets are built from scratch
$ make tmpA
echo "Makefile was run " >> config.txt
touch tmpB
touch tmpC
cat tmpB tmpC > tmpA
$ make tmpA
echo "Makefile was run." >> config.txt
touch tmpB
touch tmpC
cat tmpB tmpC > tmpA
How can I prevent Make to re-build everything because of that target 'dump_params'? Is there a another way to create this kind of log file ?
EDIT: I'm using a parallel make (option -j). Defining a macro to create the file in the statements section is not an option.
Use order-only prerequisites?
.PHONY: dump_params all
all: tmpA
tmpA: tmpB tmpC | dump_params
cat $(filter tmp%,$^) > $#
tmpB: | dump_params
touch $#
tmpC: | dump_params
touch $#
dump_params:
echo "Makefile was run." >> config.txt
Another option is to use immediately expanded shell functions, like:
__dummy := $(shell echo "Makefile was run." >> config.txt)
Since it's immediately expanded the shell script will be invoked once, as the makefile is read in. There's no need to define a dump_params target or include it as a prerequisite. This is more old-school, but has the advantage that it will run for every invocation of make, without having to go through and ensure every target has the proper order-only prerequisite defined.
Non-answer, but snakemake (and there are others, I suspect) supports tracking of rules (the code), parameters, inputs, and executable versions.
https://bitbucket.org/johanneskoester/snakemake
I have 2 files, filea and fileb, and want to produce filec.
My Makefile looks like this
f%ec: f%eb
f%ec: f%ea
cat $^ > $#
Yet, the effect of typing make is only:
cat filea > filec
That is, fileb is not a prerequisite for filec. When not using the % signs, that is when the Makefile is
filec: fileb
filec: filea
cat $^ > $#
the resulting action is
cat filea fileb > filec
Why that? And how should the first Makefile be modified in order to have the expected result?
In the second makefile:
filec: fileb
filec: filea
cat $^ > $#
There are two rules for filec, only one of which has commands, so Make combines the lists of prereqs and runs the one with commands. In the first makefile:
f%ec: f%eb
f%ec: f%ea
cat $^ > $#
There are two different pattern rules which could match filec, so Make has to choose one (it doesn't realize it could apply both at the same time, and in principle they could both have commands). Since one lacks commands, it chooses the other one. The simplest way to get the desired behavior is by combining the rules:
f%ec: f%ea f%eb
cat $^ > $#