I wrote the following makefile:
include config.mk
DATA_FILES=$(wildcard ../*.dat)
all : plot output_data
.PHONY : plot
plot : output_data $(PLOT_SRC)
$(PLOT_EXE)
.PHONY : output_data
output_data : $(SIM_SRC) $(DATA_FILES)
$(SIM_EXE)
.PHONY : clean
clean :
rm -f output_data
rm -f plot
As i expected make clean is not working, the problem is that i don't know the name or the number of files that are going to be created during make. I know their extensions but I'd rather don't use wildcards+extension to avoid delete other files. Moreover, I know I could make the code print out the name of the files it gives as an output, but 1) the code is not mine, so I' prefer not changing it, and 2) the code is not just creating files, but a number of folders.
Any other options to delete all the files created that I didn't consider?
If you are using GNU make, the filter-out function may be a solution. Assuming you have a list of files and directories to keep in a make variable KEEP, you can:
KEEP = <list-of-files-and-directories-to-keep>
clean:
#echo rm -rf $(filter-out $(KEEP),$(wildcard *))
Try to run this and, if satisfied, remove echo. Or, instead of rm, move the files and directories you do not want to keep in a temporary directory:
clean:
#mkdir -p ../backups; \
mv $(filter-out $(KEEP),$(wildcard *)) ../backups
and manually check the content of ../backups.
Related
I'm using GNU Make to build a dynamic web site but I need to build two versions. As a net result currently I run my makefile using two command line incantations. This is inefficient and can result in errors (I don't always remember to run both versions or make a typing error and they fail) thus I want to make it one build.
The Command Line incantations are:
sudo make release build=test
sudo make release build=release
The two incantations activate ifeq blocks that set the path and modify some files.
Part of the much simplified (to help readability) top level makefile:
subs = today tomorrow
files = index.php foot.php
ifeq ($(build),test)
export path = /var/www/html/cgi-test
$(shell cp -f head-test.php head.php)
$(shell sed -i '/"DB"/c\ define("DB", "db_test");' subs.php)
else ifeq ($(build),release)
export path = /var/www/html/cgi-new
$(shell cp -f head-release.php head.php)
$(shell sed -i '/"DB"/c\ define("DB", "db_new");' subs.php)
endif
.PHONY: all go
all:
$(MAKE) go
#for ALL in $(subs);\
do $(MAKE) -C $$ALL all || exit $$?;\
done;
go:
cp $(files) $(path)/.
The sub-makefiles have a very similar structure but don't need the ifeq blocks because the files and paths have already been setup.
I think I can simply move the shell commands into the .PHONY rules but I can't do that with the exports because I get errors "export: : bad variable name".
I could do it with a batch file and call the makefile twice but that sidesteps the problem rather than cures it and I wish to learn from the process.
So can anybody show me the way to do this in a makefile?
Thanks to Tripleee here is the answer that finally worked back ported to match my starting post. The one major change is that I have gone back to 'all' as the rule I expect to start the build habits die hard! - Thanks
.PHONY: all both rel-test rel-release
cgi-test := cgi-test
db-test := db_test
cgi-release := cgi-new
db-release := db_new
subs = today tomorrow
files = index.php foot.php
all: both
both: rel-test rel-release
rel-test rel-release: rel-%:
cp -f head-$*.php head.php
sed -i '/"DB"/c\ define("DB", "$(db-$*)");' subs.php
$(MAKE) go path=/var/www/html/strutts/$(cgi-$*)
#for ALL in $(subs);\
do $(MAKE) build=$* path=/var/www/html/strutts/$(cgi-$*) -C $$ALL all || exit $$?;\
done;
Something like this?
.PHONY: both rel-test rel-release
both: rel-test rel-release
cgi-test := cgi-test
db-test := db_test
cgi-release := cgi-new
db-release := db_new
rel-%:
cp -f head-$*.php head.php
sed -i '/"DB"/c\ define("DB", "$(db-$*)")' subs.php
$(MAKE) release build=$* path=/var/www/html/$(cgi-$*)
The reason the export can't be moved into a recipe is that you are using the export command of make itself, not the shell's command with the same name.
You absolutely should not use sudo unless you specifically require the output files to be owned and only writable by root. Even then, running as much as possible as a regular user would be proper hygiene; maybe add a sudo command inside the Makefile to copy the files to their final location.
If I define the following variables in my makefile
REPORT_DIR = ./reports
REPORT_TARGETS = my_report.html another_report.html
clean:
rm -f $(REPORT_DIR)/$(REPORT_TARGETS)
When I execute make clean the report directory is only pasted to the first report target.
rm -f ./reports/my_report.html another_report.html
My goal is that executing make clean would produce
rm -f ./reports/my_report.html ./reports/another_report.html
How can I paste ./reports/ to all elements of REPORT_TARGETS?
Use the addprefix function:
TARGETS_WITH_PATHS := $(addprefix $(REPORT_DIR)/, $(REPORT_TARGETS))
I need to do a makefile for run some programs. Every time I run that script all the file are processed also if the file are not changed. I'm sure there is a problem on my code but I don't understand where I made the mistakes.
RDIR=RAW
OUTDIR=Fusion_res/kallisto
RFILES:=$(wildcard $(RDIR)/*_R1_001.fastq.gz)
DATABASE=/home/sbsuser/databases/Kallsto_hg38_87
OUTFILE=$(patsubst %_R1_001.fastq.gz,%_R2_001.fastq.gz,$(RFILES))
OUTKAL=$(patsubst $(RDIR)/%_R1_001.fastq.gz,$(OUTDIR)/%,$(RFILES))
.PHONY: clean all
all: $(OUTFILE) $(RFILES) $(OUTDIR) $(OUTKAL)
#$(OUTKAL) $(OUTFILE): $(RDIR)/%._R1_001.fastq.gz
# echo "kallisto quant -i" $(DATABASE)/transcripts.idx -b 100 -o $# --fusion $< $(OUTFILE)
$(OUTDIR)/%: $(RDIR)/%_R1_001.fastq.gz $(OUTFILE)
kallisto quant -i $(DATABASE)/transcripts.idx -b 100 --fusion --rf-stranded -o $# $(RDIR)/$*_R1_00
1.fastq.gz $(RDIR)/$*_R2_001.fastq.gz
$(OUTDIR):
mkdir -p $(OUTDIR)
clean::
$(RM) -rf $(OUTDIR)
I suppose if the found some change on the input file and on the output execute the command. I don't know why every time force re-run. In some case Is that I want but I wan to also if there is some new input execute only that.
Thanks so much
A couple of things:
1) $(OUTDIR)/% is dependent on $(OUTFILE) (which is a list of all outfiles). Therefore if you change any one of the OUTFILEs, you make everything in $(OUTDIR)/% obsolete. I believe what you want is this:
$(OUTDIR)/%_R1_001.fastq.gz: $(RDIR)/%_R2_001.fastq.gz
.... (rules to make out/R1 from raw/R2
$(RDIR)/%_R2_001.fastq.gz: $(RDIR)/%_R1_001.fastq.gz
.... (rules to make R2 from R1
This makes each file dependent only on the files that effect it.
2) you have the target all dependent on $(OUTDIR) which is a directory. If you use parallel make, it may generate the $(OUTDIR) after it generates the other dependencies of all: (some of which would depend on $(OUTDIR) being created). What you want there is to remove all's dependency on $(OUTDIR), and add the line:
$(OUTFILE) : | $(OUTDIR)
Notice the |, which means order only (don't consider $(OUTFILE) out of date if $(OUTDIR) is newer. This is important, as a directory's timestamp is updated each time a file in the directory is changed, and so it tends to be newer than its contents.
I have an odd project were a few files are generated at build time. they are among other files but have a special sulffix. (as opposed to a normal project, where all files of some type means they were auto generated)
for example:
src/fileA.js
src/fileB.js.tpl
src/fileC.css
src/fileD.css.tpl
...
then i have a pattern rule:
DATA=$(get string from template.txt)
%: %.tpl
sed 's/__TEMPLATE__/$(DATA)/g' $< > $#
templates: src/fileB.js src/fileD.css
And all is fine. Until the next build... now src/fileB.js will not get updated because there is one there already, and src/fileB.js.tpl was not changed, though the other file template.txt that i use as a data source to update it might. Which brings me to the clean step.
right now my clean step is rming each file. it is ugly.
.PHONY: clean
clean:
rm src/fileB.js
rm src/fileD.css
...
You can see how it gets ugly.
In a regular project my clean would be just rm *.o but here i can't do rm *.js as half the files are not auto-generated.
is there any way to make the rule %: %.tpl be a PHONY?
if not, is there any way to feed the file list from template into clean?
What about this?
TEMPLATES=$(wildcard src/*.tpl)
GENERATED=$(TEMPLATES:%.tpl=%)
clean:
rm -f $(GENERATED)
Well, I would backup before testing this...
I have the following folder structure
1st-grade-math-class/
common/
mystyle.sty
mysubstyle.sty
fonts/
font1.ttf
font2.ttf
font3.ttf
week01/
handout.tex
image1.pdf
image2.pdf
week02/
handout.tex
image1.pdf
image2.pdf
...
week13/
handout.tex
output/
[empty]
And I would like to create a Makefile - in the best way - to do the following:
make sure I include the common directory properly in TEXINPUTS
compile the handout.tex into a PDF (using either pdflatex or xelatex) and have it in the output directory as week01-handout-student.pdf
compile the handout.tex with a line of LaTeX prepended to the beginning of the file (that sets a flag) into a PDF and have it in the output directory as week01-handout-teacher.pdf
clean everything up (the log, aux, etc. files)
I am not sure I know how to this in any other way than manually duplicating an elementary Makefile/bash script in every subdirectory, then calling each of them one by one with a for loop.
I would appreciate help on how to build this process, ideally with a single Makefile in the root directory. Thanks.
UPDATE: I purposefully did not want to give any details about how I compile LaTeX, in case somebody has a better suggestion than my current usage. Right now I am using Latexmk (which is already a make-like wrapper of LaTeX):
latexmk -pdf file.tex generates file.pdf
to add the line of code, I do a simple echo "line of code" > temp.tex and cat handout.tex >> temp.tex, then the same latexmk command
latexmk -c file.tex in a directory cleans all temporary files used to compile file.tex
TEXINPUTS is the TeX path variable, to let TeX find (in its path) the style files: I do TEXINPUTS=full-path-to/common and then export TEXINPUTS before compiling anything.
If anybody has a better suggestion, I am a willing taker.
Something like this should do what you want I believe:
OUTDIR := output
# Tell make to export this environment variable to all recipe lines it runs.
export TEXINPUTS := $(abspath common)
# Get the list of all of our week directories.
weekdirs := $(wildcard week*)
#$(info weekdirs:$(weekdirs))
# Create student output filenames from week directory names.
STUDENT_HANDOUTS := $(patsubst %,$(OUTDIR)/%-handout-student.pdf,$(weekdirs))
#$(info STUDENT_HANDOUTS:$(STUDENT_HANDOUTS))
# Create teacher output filenames from week directory names.
TEACHER_HANDOUTS := $(patsubst %,$(OUTDIR)/%-handout-teacher.pdf,$(weekdirs))
#$(info TEACHER_HANDOUTS:$(TEACHER_HANDOUTS))
# Default target depends on all output files.
all: $(STUDENT_HANDOUTS) $(TEACHER_HANDOUTS)
# Pattern rule for building pdf files.
%.pdf:
#echo + Making $# from $^
#echo cd $(#D) && echo latexmx -pdf $(abspath $<)
#echo cd $(#D) && echo latexmk -c $(abspath $<)
# Static pattern rule mapping student output files to input files.
$(STUDENT_HANDOUTS) : $(OUTDIR)/%-handout-student.pdf : %/handout.tex
# Pattern rule to generate temporary input files from original input files.
%/handout-tmp.tex: %/handout.tex
#echo echo 'line of code' '>' $#
#echo cat $^ '>>' $#
# Static pattern rule mapping teacher output files to (temporary) input files.
$(TEACHER_HANDOUTS) : $(OUTDIR)/%-handout-teacher.pdf : %/handout-tmp.tex
Uncomment the $(info) lines to see a bit of how the variables are put together.
This uses latexmk -c to clean up auxiliary files after creating the output files.
Technique 1. The echo foo >$#; cat $< >>$# thing is something I've done before; I think it's reasonable.
Technique 2. Another technique is to have your .tex document, or a package file it uses, include a line like:
\InputIfFileExists{lecturenotes.config}{}{}
That allows you to make certain adjustments in a makefile, as in for example:
# any bits of configuration that should be in all the .config files
SWITCHES=\makeindex
%-notes.pdf: %.tex
printf '$(SWITCHES)\\ExecuteOptions{sidenotes}\n' >lecturenotes.config
TEXINPUTS=styles: pdflatex $<
mv ${<:.tex=.pdf} $#
rm lecturenotes.config
%-single.pdf: %.tex
printf '$(SWITCHES)\\ExecuteOptions{oneside}\n' >lecturenotes.config
TEXINPUTS=styles: pdflatex $<
mv ${<:.tex=.pdf} $#
rm lecturenotes.config
Technique 3. A third technique for controlling LaTeX from outside is to include in your document (or in a package file):
\newif\ifdemonstrator
\expandafter\ifx\csname demonstrator\endcsname\relax
\demonstratorfalse
\else
\demonstratortrue
\fi
...
\ifdemonstrator
\typeout{Demonstrator is TRUE}
\else
\typeout{Demonstrator is FALSE}
\fi
Then call latex with:
%-plain.pdf: %.tex
latex $<
mv ${<:.tex=.pdf} $#
%-demo.pdf: %.tex
latex --jobname ${<:.tex=} '\def\demonstrator{x}\input{$<}`
mv ${<:.tex=.pdf} $#
Technique 1 is a bit of a blunt instrument, but if that's all that's needed, it's pretty lightweight.
Technique 2 is probably the most neatly engineered, but it is slightly more effort.
Technique 3 is probably the one I use most in this sort of circumstance.