As part of generating a PDF from a latex file, I got a makefile from tex.stackexchange.com.
# You want latexmk to *always* run, because make does not have all the info.
# Also, include non-file targets in .PHONY so they are run regardless of any
# file of the given name existing.
.PHONY: paper-1.pdf all clean
# The first rule in a Makefile is the one executed by default ("make"). It
# should always be the "all" rule, so that "make" and "make all" are identical.
all: paper-1.pdf
# CUSTOM BUILD RULES
# In case you didn't know, '$#' is a variable holding the name of the target,
# and '$<' is a variable holding the (first) dependency of a rule.
# "raw2tex" and "dat2tex" are just placeholders for whatever custom steps
# you might have.
%.tex: %.raw
./raw2tex $< > $#
%.tex: %.dat
./dat2tex $< > $#
# MAIN LATEXMK RULE
# -pdf tells latexmk to generate PDF directly (instead of DVI).
# -pdflatex="" tells latexmk to call a specific backend with specific options.
# -use-make tells latexmk to call make for generating missing files.
# -interaction=nonstopmode keeps the pdflatex backend from stopping at a
# missing file reference and interactively asking you for an alternative.
paper-1.pdf: paper-1.tex
latexmk -bibtex -pdf -pdflatex="pdflatex -interaction=nonstopmode" -use-make paper-1.tex
clean:
latexmk -bibtex -CA
My figures are .dot files that I turn into PNG files. I can make the PNGs with some basic shell commands, but that it doesn't make sense to use a shell script because you lose the advantages of make.
Here's what I've been trying after reading some documentation.
%.png: %.dot
dot -Tpng $(.SOURCE) -o $(.TARGET)
and
.dot.png:
dot -Tpng $(.SOURCE) -o $(.TARGET)
However, whenever I try to run the target directly the terminal prints is:
dot -Tpng -o
and it holds because it waits for input from STDIN because there was no input file.
If I try to invoke the rule by running make *.dot I get the output:
make: Nothing to be done for `figure-1a.dot'.
make: Nothing to be done for `figure-1b.dot'.
I'm clearly not understanding what I need to do. How do I get the makefile to take all the .dot files and create .png files every time I run through the creation of the PDF?
UPDATE: Here is another attempt I tried
graphs := $(wildcard *.dot)
.dot.png: $(graphs)
dot -Tpng $(.SOURCE) -o $(.TARGET).png
GNU make uses $< and $#, not .SOURCE and .TARGET, the recipe should be
.PHONY: all
all: $(patsubst %.dot,%.png,$(wildcard *.dot))
%.png: %.dot
dot -Tpng $< -o $#
Related
My goal is the following: I have a directory src which contains markdown files (.md). I want to run a command on each of these files so that the comments are removed and the edited files are stored in a separate directory. For this I want to use make.
This is the Makefile I have:
.PHONY: clean all
BUILD_DIR := build
SRC_DIRS := src
SRCS := $(shell find $(SRC_DIRS) -name *.md)
DSTS := $(patsubst $(SRC_DIRS)/%.md,$(BUILD_DIR)/%.md,$(SRCS))
all: $(DSTS)
# The aim of this is to remove all my comments from the final documents
$(DSTS): $(SRCS)
pandoc --strip-comments -f markdown -i $< -t markdown -o $#
clean:
rm $(BUILD_DIR)/*.md
While this works in general, I noticed that the command is executed on all files, even though I changed only one single file.
Example: I have 3 Files src/a.md, src/b.md and src/c.md. Now I run make and all files are correctly generated in the build folder. Now I only edit c.md and run make again. I would expect that make only "compiles" src/c.md anew but instead all three files are compiled again. What am I doing wrong?
Your line
$(DSTS): $(SRCS)
is saying ‘All of the DSTS depend on all of the SRCS’, so whenever any one of the $(SRCS) is newer than any of the $(DSTS), this pandoc action will be run.
That's not what you want to express. What you want is something more like
$(BUILD_DIR)/%.md: $(SRC_DIRS)/%.md
pandoc --strip-comments -f markdown -i $< -t markdown -o $#
all: $(DSTS)
That says that all of the $(DSTS) should be up to date, and the pattern rule teaches Make what each one depends on, and how to build it, if it is out of date.
(As a general point, looking your original rule, it's rarely the right thing to do to have multiple targets in a rule, as you have with $(DSTS); also note that in your original, $< always refers only to the first of the dependencies in $(SRCS))
In the following makefile, the wat rule is for producing a .wat file from a .c file.
CC=emcc
CFLAGS= -O2 -s WASM=1 -s SIDE_MODULE=1
# flags are for not generating any emscripten glue code
# makes a .wat version of a .c file of specified name
# TARGET must be specified from command line
wat: $(TARGET).c
$(CC) $(CFLAGS) -o $(TARGET).wasm $(TARGET).c && wasm2wat $(TARGET).wasm > $(TARGET).wat && rm $(TARGET).wasm
struct: TARGET = struct
struct: wat
clear:
rm -f *.wasm *.wat
Called like this, it works fine:
[user#pc]$ make wat TARGET=struct
emcc -O2 -s WASM=1 -s SIDE_MODULE=1 -o struct.wasm struct.c && wasm2wat struct.wasm >
struct.wat && rm struct.wasm
Now I want a more specific rule, struct, written as you see. In essence, I want to reuse the wat rule, and just make sure that TARGET is set to 'struct' before running. However, running make struct gives me a no input file error from emcc, as if the value of TARGET does not exist:
[user#pc]$ make struct
emcc -O2 -s WASM=1 -s SIDE_MODULE=1 -o .c
shared:ERROR: no input files
What is the correct way to do what I aim?
The problem is, that when make evaluates how to make targets and what the prerequisites are, your TARGET is undefined and hence the rule says: wat needs .c. You could try recursion and say something like this:
struct:
$(MAKE) TARGET=struct wat
It really isn't that great to start with, because nothing actually generates file wat so the target is always out of date and the Makefile really is just a somewhat glorified shell script.
You should probably consider writing a pattern rule how to build .wat from .c, something like (based on your example):
%.wat: %.c
$(CC) $(CFLAGS) -o $(*F).wasm $< \
&& wasm2wat $(*F).wasm > $#.wat \
&& rm $(*F).wasm
You then can call make struct.wat and if you still wanted (for convenience) have just struct target, you could add:
.PHONY: struct
struct: struct.wat
You appear to be attempting to use target-specific variables and in particular, their inheritance feature, to achieve your goal.
However, this cannot work because of this statement in the documentation:
As with automatic variables, these values are only available within the context of a target’s recipe (and in other target-specific assignments).
This means that the $(TARGET) in the prerequisite:
wat: $(TARGET).c
is evaluated when the makefile is parsed, not when the struct rule is invoked, and expands to:
wat: .c
I see that Ondrej provided an answer so I'll stop here :)
I have a series of directories organized like this:
foo/
foo.file1 foo.file2
bar/
bar.file1 bar.file2
baz/
baz.file1 baz.file2
Right now I'm processing these files using a script that does all the checking for file existence etc but I thought that perhaps I could use a Makefile for it (since said script is very fragile), to avoid reprocessing files that did not change.
The problem is that each directory is independent, and I'd need to do, for example:
foo.file1.processed: foo.file1
run_random_program foo.file1 -o foo.file1.processed
for each of the 71 directories that are in total in that path. This looks like being extremely tedious and I wonder if there's something that would prevent me from writing all of this by hand.
Is such a thing possible?
EDIT: Some examples that show what I have in mind, had I a single Makefile for each directory:
file1.cds.callable: file1.callable
long_script_name -i $< -o $#
file1.rds: file1.cds.callable
another_long_script_name $< additional_file_in_folder $#
file1.csv: file1.rds
yet_another_script $< $#
Seems like pattern rules are exactly what you need:
# These are the original source files (based on the example)
CALLABLE := $(wildcard */*.callable)
# These are the final targets
TARGETS := $(CALLABLE:%.callable=%.csv)
all: $(TARGETS)
%.csv : %.rds
yet_another_script $< $#
%.rds: %.cds.callable
another_long_script_name $< additional_file_in_folder $#
%.cds.callable: %.callable
long_script_name -i $< -o $#
I have a large project I'm working on, in which I want to perform include to some .mak file, but only after I make change to this file content via a command inside the original makefile. Since it's a large project it will be hard to write code, so I will give this ridiculous example instead:
I have some small C project that all it's C and header files are in the same directory, and I need to write a makefile. I'm not allowed to use clean rule in the makefile I write, but I have a file named file.mak that I can include in my makefile. Content of file.mak:
.PHONY: clean
cleam:
$(RM) $(objs) test
The problem here is that the rule is cleam and not clean. I'm also not allowed to change manually file.mak , but I'm allowed to do this with a command inside the original makefile. This can be done easily by:
sed -i 's/cleam/clean/g' file.mak
So I thought of writing the makefile like this:
CC = gcc
srcs = $(wildcard ./*.c)
objs = $(srcs:.c=.o)
test: $(objs) change_file include_file
$(CC) $^ -o $#
%.o: %.c
$(CC) -c $< -o $#
change_file:
$(shell sed -i 's/cleam/clean/g' file.mak)
include_file: change_file
include file.mak
But I get the following error:
include: Command not found
So I understand that there is a problem of using include inside a rule, so is there a way to achieve what I want?
(GNU) make has a feature Remaking Makefiles that can be used for scenarios like this, but your approach is wrong. include is a directive and can't be used in a recipe.
Instead, when you include a file, make first checks for rules creating this exact file and executes them. As in your case, the file you want to include already exists, you have to make this rule .PHONY to force its execution. It would look like this:
.PHONY: file.mak
file.mak:
sed -i 's/cleam/clean/g' file.mak
include file.mak
As a more robust alternative (without the need for a phony rule), consider creating a fixed version (copy) and include this:
file_fixed.mak: file.mak
sed -e 's/cleam/clean/g' <file.mak >file_fixed.mak
include file_fixed.mak
I am writing a GNUmakefile to create a workflow to analyse some biological sequence data. The data comes in a format called fastq, which then undergoes a number of cleaning and analysis tools. I have attached what I currently have written, which takes me all the way from quality control before cleaning and then quality control afterwards. My problem is that I'm not sure how to get the 'fastqc' commands to run, as its targets are not dependencies for any of the other steps in the workflow.
%_sts_fastqc.html %_sts_fastqc.zip: %_sts.fastq
# perform quality control after cleaning reads
fastqc $^
%_sts.fastq: %_st.fastq
# trim reads based on quality
sickle se -f $^ -t illumina -o $#
%_st.fastq: %_s.fastq
# remove contaminated reads
tagdust -s adapters.fa $^
%_s.fastq: %.fastq
# trim adapters
scythe -a <adapters.fa> -o $# $^
%_fastqc.html %_fastqc.zip: %.fastq
# perform quality control before cleaning reads
fastqc $^
%.fastq: %.sra
# convert .fastq to .sra
fastq-dump $^
I believe adding these lines to the start of your Makefile will do what you are asking for:
SOURCES:=$(wildcard *.sra)
TARGETS:=$(SOURCES:.sra=_fastqc.html) $(SOURCES:.sra=_fastqc.zip)\
$(SOURCES:.sra=_sts_fastqc.html) $(SOURCES:.sra=_sts_fastqc.zip)
.PHONY: all
all: $(TARGETS)
What this does is grab all .sra files from the file system and build a list of targets to build by replacing the extension with whatever strings are necessary to produce the targets. (Note the the html and zip targets being produced by the same command I could have one or the other but I've decided to put both, in case the rules change and the hmtl and zip targets are ever produced separately.) Then it sets the phony all target to build all the computed targets. Here is a Makefile I've modified from yours by adding #echo everywhere which I used to check that things were okay without having to run the actual commands in your Makefile. You could copy and paste it in a file to first check that everything is fine before modifying your own Makefile with the lines above. Here it is:
SOURCES:=$(wildcard *.sra)
TARGETS:=$(SOURCES:.sra=_fastqc.html) $(SOURCES:.sra=_fastqc.zip)\
$(SOURCES:.sra=_sts_fastqc.html) $(SOURCES:.sra=_sts_fastqc.zip)
.PHONY: all
all: $(TARGETS)
%_sts_fastqc.html %_sts_fastqc.zip: %_sts.fastq
# perform quality control after cleaning reads
#echo fastqc $^
%_sts.fastq: %_st.fastq
# trim reads based on quality
#echo sickle se -f $^ -t illumina -o $#
%_st.fastq: %_s.fastq
# remove contaminated reads
#echo tagdust -s adapters.fa $^
%_s.fastq: %.fastq
# trim adapters
#echo 'scythe -a <adapters.fa> -o $# $^'
%_fastqc.html %_fastqc.zip: %.fastq
# perform quality control before cleaning reads
#echo fastqc $^
%.fastq: %.sra
# convert .fastq to .sra
#echo fastq-dump $^
I tested it here by running touch a.sra b.sra and then running make. It ran the commands for both files.
instead of using patterns, I would use a 'define':
# 'all' is not a file
.PHONY: all
# a list of 4 samples
SAMPLES=S1 S2 S3 S4
#define a macro named analyzefastq. It takes one argument $(1). we need to protect the '$' for later expension using $(eval)
define analyzefastq
# create a .st.fastq from fastq for file $(1)
$(1).st.fastq : $(1).fastq
tagdust -s adapters.fa $$^
# create a .fastq from seq for file $(1)
$(1).fastq : $(1).sra
fastq-dump $$^
endef
#all : final target dependency is all samples with a suffix '.st.fastq'
all: $(addsuffix ${S}.st.fastq, ${SAMPLES} )
## loop over each sample , name of variable is 'S' call and eval the previous macro, using 'S'=sample for the argument
$(foreach S,${SAMPLES},$(eval $(call analyzefastq,$(S))) )
I also use my tool jsvelocity https://github.com/lindenb/jsvelocity to generate large Makefile for NGS:
https://gist.github.com/lindenb/3c07ca722f793cc5dd60