I have a makefile with a bunch of .R scripts that create .csv files as output. These files are then used in a python simulation.
all: $(FILES)
python simulation.py
$(FILES): %.csv: %.R
Rscript $<
This is straightforward. My wrinkle is that one (and only one) of the .R scripts has its own dependency, say sourcedata. It seems that placing this dependency in the pattern would be annoying
all: $(FILES)
python simulation.py
$(FILES): %.csv: %.R sourcedata
Rscript $<
sourcedata:
wget "sourcedata.zip"
But doing it like all: sourcedata $(FILES) and relying on order-of-operations would be less effective. I guess that I could also give that R file its own phony rule
problem_script.R: sourcedata
#echo R script read
but I haven't been able to test if that will work. Is there an established way to do this?
Unless I'm misunderstanding, you need to specify that the particular CSV file needs to be rebuilt whenever either the R file or the sourcedata file changes, right?
If so, just add this:
problem_script.csv: sourcedata
(no recipe needed). This declares an extra prerequisite for that particular CSV file, so whenever it's out of date with respect to sourcedata it will be rebuilt.
Related
For some reason make is recompiling the source files that are outside my
project tree every time. I don't normally have such files, but I noticed it while I was testing this Makefile and I would like to understand why. It behaves normally with respect to the source files inside my project tree. Here are my rules. First I have a function
src_to_obj = $(basename $(addprefix $(build_dir)/,$(1))).o
Then I populate the list of program_objects by calling foreach on the list of sources and applying the above function to each one.
$(bin_dir)/$(program_name): $(program_objects)
$(linker) $(link_flags) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) \
$^ $(OUTPUT_OPTION)
define build_template =
$$(call src_to_obj,$(1)) : $(1) $$(call dependency_file,$(1))
// make dependency and build directories
$$(compile) $$(OUTPUT_OPTION) $$<
// rename dependency
endef
$(foreach source,$(program_sources),\
$(eval $(call build_template,$(source))))
As I said, this works just fine for the source files in the project directory. For example, the Makefile lives in ~/Projects/foo and most of the source code is in ~/Projects/foo/src. But if I pull in a file A.cc from the home directory,then that file is recompiled every time I run make.
EDIT:
Following Mad-Scientists suggestion, I ran make -d and examined the output. That made the problem clear. Basically, if I feed this a source file ~/bad.cc, then the rule looks like:
build/~/bad.o : ~/bad.cc .deps/~/bad.d
...
However, my script for the rule is creating the file .deps/home/jim/bad.d. Therefore, every time I run make the dependency file is "not there" and the object file has to be remade. I am using the exact same expressions to refer to the file name in the script. However I am guessing that the shell is expanding the ~ to /home/jim. So somehow I have to escape the ~ in the recipe. Thanks to Mad Scientist for pointing out the -d option for make.
I don't have time to make a short self contained correct example right now, but if I can't solve this problem I will come up with one and post it here later.
EDIT EDIT:
So I solved the problem with the following hack:
program_sources = $(shell echo $(program_sources))
This pre-expands all names so that they are never different in the shell.
(Similar question here). For every .x file in the directory I want to run a command to generate an HTML file. The command already works fine, it's the Makefile that I'm having trouble with. Here's what I came up with:
all: $(OBJECTS)
OBJECTS := $(patsubst %.x,%.html,$(wildcard *.x))
%.html: %.x
generate $< > $#
The OBJECTS variable is meant to contain all html files I need to generate. Invoking make states nothing to be done for 'all', and there are no HTML files already in the directory.
Turns out make is sensitive to the order of your variable definitions. The Makefile works when the first two lines are switched!
This question is based on another question of mine here: Getting basename and notdir to work in prerequisite (dependency) list.
I'm using a Makefile to generate some figures automatically and
efficiently.
My figures are generated in ../thesis/figures using Octave .m
files that are in the current directory where the Makefile also is.
Each .m file, e.g. figure1.m, may generate several figures, e.g.
figure1.p1.tex and figure1.p2.tex (and their dependecies, which
are also generated by figure1.m). These .tex files are then to be compiled using LaTeX (a single run of pdflatex figure1.p1.tex suffices in this case; so, there is no need for latexmk or other alternatives).
The Makefile I have so far is
OCTAVE = octave --jit-compiler --no-gui --quiet
PDFLATEX = pdflatex
FIGDIR = ../thesis/figures
TEXTARGETS = $(wildcard $(FIGDIR)/*.tex)
.PHONY: figures
figures: $(TEXTARGETS)
.SECONDEXPANSION:
$(TEXTARGETS): %.tex : $$(basename $$(notdir %)).m
$(OCTAVE) $<
$(PDFLATEX) $#
A dry run with make -n shows me
octave --jit-compiler --no-gui --quiet figure1.m
pdflatex ../thesis/figures/figure1.p1.tex
octave --jit-compiler --no-gui --quiet figure1.m
pdflatex ../thesis/figures/figure1.p2.tex
make: *** No rule to make target `figure2.m', needed by `../thesis/figures/figure2.tex'. Stop.
There are two issues here:
1) Both figure1.p1.tex and figure.p2.tex are generated by the first run of figure1.m by octave. Is there a way to treat all targets with the same basename (or other pattern) as a prerequisite as one, so that there is no more than one invocation of octave per .m file?
2) figure2.tex was made using some other means than an .m file. How can I tell make to ignore a rule if its prerequisite does not exist. I know how to do that for an explicit prerequisite:
target: prereq
recipe
prereq:
But what to do in this case with the prerequisite being derived from the target's name?
GNU make can be taught that multiple targets are created by one command invocation by using a pattern rule for those targets.
From Pattern Rule Examples:
This pattern rule has two targets:
%.tab.c %.tab.h: %.y
bison -d $<
This tells make that the recipe ‘bison -d x.y’ will make both x.tab.c and x.tab.h. If the file foo depends on the files parse.tab.o and scan.o and the file scan.o depends on the file parse.tab.h, when parse.y is changed, the recipe ‘bison -d parse.y’ will be executed only once, and the prerequisites of both parse.tab.o and scan.o will be satisfied. (Presumably the file parse.tab.o will be recompiled from parse.tab.c and the file scan.o from scan.c, while foo is linked from parse.tab.o, scan.o, and its other prerequisites, and it will execute happily ever after.)
So you could use something like
figure1.%1.tex figure1.%2.tex: figure1.m
$(OCTAVE) $<
but to do that for N output files where N is variable would require generating an included makefile that pulled that information out of the .m file (or similar).
If a file exists that matches a target but no matching prerequisite file is found make will just use the file it found and ignore the rule (the rule doesn't apply). You shouldn't have to do anything for that.
If, however, the file would otherwise match the rule (but you don't want make to follow the rule for that file) then you can cancel just that application by giving that file an explicit target. Like this.
figure2.tex: ;
I'm trying to make a rule that will generate files regarding their names but regardless of the directory.
I'm starting from this makefile (see a previous question of mine):
TARGETS:=$(patsubst %_tpl,%,$(wildcard *_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): %: $$(wildcard %*_tpl)
./generate $#_tpl > $#
With this, I can do, for instance, make foo.xml. It looks if a set of foo.xml*_tpl files are there, consider them as prerequisites and call the generate script to generate the target.
What I would like to do is, for example, make ../ressources/foo.xml and have make use the rule to create foo.xml but creating it in the ../ressources/ directory, without having to explicitely specify this directory in the makefile.
What I have tried for the moment is adding this to the Makefile:
../ressources/%: $(notdir %)
mv $< $#
Which works, but I would like to avoid creating the file in the current directory before moving it to the destination folder. I would also like not having to specify the possible destination folders in the makefile (but this is less important).
But first of all, does this make any sense? Or is what I want to do just conceptually wrong?
EDIT: Some precisions regarding the _tpl files and the generate script to avoid confusions:
Each target has a main template ($#_tpl) that includes others ($#-part1_tpl, $#-part2_tpl...) and the generate script only takes the main template as argument. The templates are written with Jinja2 (the subparts included with the {% include %} jinja directive).
If you always want the targets in another directory, just say so.
TARGETS:=$(patsubst %_tpl,../resources/%,$(wildcard *_tpl))
.PHONY: all
all: $(TARGETS)
.SECONDEXPANSION:
$(TARGETS): ../resources/%: $$(wildcard %*_tpl)
./generate $#_tpl > $#
I'm not sure if you should have generate $^ >$# instead; superficially, this would make more sense.
If there are multiple *_tpl files for each target (i.e. there are more tpl files than xml files), the TARGETS definition isn't really correct; but we don't have enough information to actually fix it.
On the other hand, if the target directory can change a lot, the sane way forward might be to cd into the target directory and use make -f ../path/to/Makefile -- just make sure your VPATH is set up so that the source files can be found.
A common type of makefile has a line like this:
OBJS=something.o other.o third.o progname.o
progname: $(OBJS)
Then you would run make progname. But GNU Make can also generate the list of o-files itself from all the c-files it sees in the current directory. How is this done?
Basically, I want to be able to add C files to the directory without having to change the makefile.
(Is it for instance through some shell-magic, or is there is a built-in function for this?)
It can also be done like this:
SRCS = $(wildcard *.c)
OBJS = $(SRCS:.c=.o)
progname: $(OBJS)
Which works just fine if the object file with main() in it is "progname.o".
To view all the defined rules (include the implicit ones), issue make -p.
However the fact that make knows how to generate object files from source files, does not mean that it should do this. Make will try to do the bare minimum in order to satisfy the target you ask it to build.
If you want make to compile all the sources into object in the current directory you will need a rule that will depend on all the objects, e.g.:
all: $(patsubst %.c,%.o,$(wildcard *.c))
You can expand a shell command to give you a list of files. You can also use implicit rules.
It can be done like this:
$(ODIR)/%.o: %.cpp
$(CC) -c $(CFLAGS) -o $# $<
make generates .o file names from .cpp file names it found in the current directory.