I have an odd project were a few files are generated at build time. they are among other files but have a special sulffix. (as opposed to a normal project, where all files of some type means they were auto generated)
for example:
src/fileA.js
src/fileB.js.tpl
src/fileC.css
src/fileD.css.tpl
...
then i have a pattern rule:
DATA=$(get string from template.txt)
%: %.tpl
sed 's/__TEMPLATE__/$(DATA)/g' $< > $#
templates: src/fileB.js src/fileD.css
And all is fine. Until the next build... now src/fileB.js will not get updated because there is one there already, and src/fileB.js.tpl was not changed, though the other file template.txt that i use as a data source to update it might. Which brings me to the clean step.
right now my clean step is rming each file. it is ugly.
.PHONY: clean
clean:
rm src/fileB.js
rm src/fileD.css
...
You can see how it gets ugly.
In a regular project my clean would be just rm *.o but here i can't do rm *.js as half the files are not auto-generated.
is there any way to make the rule %: %.tpl be a PHONY?
if not, is there any way to feed the file list from template into clean?
What about this?
TEMPLATES=$(wildcard src/*.tpl)
GENERATED=$(TEMPLATES:%.tpl=%)
clean:
rm -f $(GENERATED)
Well, I would backup before testing this...
Related
I need to do a makefile for run some programs. Every time I run that script all the file are processed also if the file are not changed. I'm sure there is a problem on my code but I don't understand where I made the mistakes.
RDIR=RAW
OUTDIR=Fusion_res/kallisto
RFILES:=$(wildcard $(RDIR)/*_R1_001.fastq.gz)
DATABASE=/home/sbsuser/databases/Kallsto_hg38_87
OUTFILE=$(patsubst %_R1_001.fastq.gz,%_R2_001.fastq.gz,$(RFILES))
OUTKAL=$(patsubst $(RDIR)/%_R1_001.fastq.gz,$(OUTDIR)/%,$(RFILES))
.PHONY: clean all
all: $(OUTFILE) $(RFILES) $(OUTDIR) $(OUTKAL)
#$(OUTKAL) $(OUTFILE): $(RDIR)/%._R1_001.fastq.gz
# echo "kallisto quant -i" $(DATABASE)/transcripts.idx -b 100 -o $# --fusion $< $(OUTFILE)
$(OUTDIR)/%: $(RDIR)/%_R1_001.fastq.gz $(OUTFILE)
kallisto quant -i $(DATABASE)/transcripts.idx -b 100 --fusion --rf-stranded -o $# $(RDIR)/$*_R1_00
1.fastq.gz $(RDIR)/$*_R2_001.fastq.gz
$(OUTDIR):
mkdir -p $(OUTDIR)
clean::
$(RM) -rf $(OUTDIR)
I suppose if the found some change on the input file and on the output execute the command. I don't know why every time force re-run. In some case Is that I want but I wan to also if there is some new input execute only that.
Thanks so much
A couple of things:
1) $(OUTDIR)/% is dependent on $(OUTFILE) (which is a list of all outfiles). Therefore if you change any one of the OUTFILEs, you make everything in $(OUTDIR)/% obsolete. I believe what you want is this:
$(OUTDIR)/%_R1_001.fastq.gz: $(RDIR)/%_R2_001.fastq.gz
.... (rules to make out/R1 from raw/R2
$(RDIR)/%_R2_001.fastq.gz: $(RDIR)/%_R1_001.fastq.gz
.... (rules to make R2 from R1
This makes each file dependent only on the files that effect it.
2) you have the target all dependent on $(OUTDIR) which is a directory. If you use parallel make, it may generate the $(OUTDIR) after it generates the other dependencies of all: (some of which would depend on $(OUTDIR) being created). What you want there is to remove all's dependency on $(OUTDIR), and add the line:
$(OUTFILE) : | $(OUTDIR)
Notice the |, which means order only (don't consider $(OUTFILE) out of date if $(OUTDIR) is newer. This is important, as a directory's timestamp is updated each time a file in the directory is changed, and so it tends to be newer than its contents.
I have this make rule:
dist/node/%.js: src/%.node.js yarn.lock .babelrc | $(NODE_DIST_DIRS)
$(BIN)/babel $< -o $#
It works fine when my source file ends with .node.js. e.g.,
$ make dist/node/fs.js
mkdir -p dist/node/
node_modules/.bin/babel src/fs.node.js -o dist/node/fs.js
However, I want to compile dist/node/fs.js from either src/fs.node.js or src/fs.js, whichever exists.
The reason for this is that I have some shared files with just the .js extension (work in both browser and node), and then more specific files with the .node.js extension. If there's a more specific version, I want to use that.
I don't know how to do conditional dependencies in combination with %. Is this possible? Can I give precedence to dependencies and take the best match?
There is no way to do that in a single rule. You'll have to write two rules with different prerequisites but otherwise the same:
dist/node/%.js: src/%.node.js yarn.lock .babelrc | $(NODE_DIST_DIRS)
$(BIN)/babel $< -o $#
dist/node/%.js: src/%.js yarn.lock .babelrc | $(NODE_DIST_DIRS)
$(BIN)/babel $< -o $#
I suppose you could try using a double-colon rule, but those cannot be pattern rules.
As an alternative, consider generating and including a makefile containing just those dependencies.
I have a metadata dotfile which I store alongside my code (in each dir of a large project). This metadata file has a list of files in each directory that satisfies a particular constraint (not being auto-generated). This metadata is sincluded in Makefile.
I want to dep other targets (the auto-generated files) on this metadata file. If ever the list of "real" code files in a dir changes, update this metadata file, which will in turn cause the auto-generated files (plural) to be re-made.
I have a rule for the metadata file, and when that rule fires make restarts correctly. But I can't quite figure how to describe what I want. I want the rule to run but only consider $# as having been changed iff I actually touch the file. I can't use the timestamp of the dir because the act of auto-generating file A causes the timestamp to change, which triggers the need to re-generate file B, which cause the timestamp to change ...
I feel like I am missing something obvious, but I can not put my finger on it...
all: prep my-bin
# For demonstration purposes.
prep: real-code.foobar
real-code.foobar:
#touch real-code.foobar
my-bin: meta real-code.foobar genfile-A.foobar genfile-B.foobar genfile-C.foobar
touch $#
meta: .
F=$$(ls *.foobar | grep -v genfile); \
echo "FILES := $$F" > $#
sinclude meta
genfile-A.foobar: meta
touch $#
genfile-B.foobar: meta
touch $#
genfile-C.foobar: meta
touch $#
clean:
rm -f *.foobar my-bin meta
You could update the meta in a shell script at the beginning of the Makefile:
$(shell ls *.foobar | grep -v genfile > meta.tmp; \
diff -q meta.tmp meta && mv meta.tmp meta || rm meta.tmp)
This way the timestamp of meta is only updated if it changes, and it is updated before make has decided which rules to run (meaning dependencies of meta will not automatically rerun).
John
I wrote the following makefile:
include config.mk
DATA_FILES=$(wildcard ../*.dat)
all : plot output_data
.PHONY : plot
plot : output_data $(PLOT_SRC)
$(PLOT_EXE)
.PHONY : output_data
output_data : $(SIM_SRC) $(DATA_FILES)
$(SIM_EXE)
.PHONY : clean
clean :
rm -f output_data
rm -f plot
As i expected make clean is not working, the problem is that i don't know the name or the number of files that are going to be created during make. I know their extensions but I'd rather don't use wildcards+extension to avoid delete other files. Moreover, I know I could make the code print out the name of the files it gives as an output, but 1) the code is not mine, so I' prefer not changing it, and 2) the code is not just creating files, but a number of folders.
Any other options to delete all the files created that I didn't consider?
If you are using GNU make, the filter-out function may be a solution. Assuming you have a list of files and directories to keep in a make variable KEEP, you can:
KEEP = <list-of-files-and-directories-to-keep>
clean:
#echo rm -rf $(filter-out $(KEEP),$(wildcard *))
Try to run this and, if satisfied, remove echo. Or, instead of rm, move the files and directories you do not want to keep in a temporary directory:
clean:
#mkdir -p ../backups; \
mv $(filter-out $(KEEP),$(wildcard *)) ../backups
and manually check the content of ../backups.
I'm wondering how I can avoid some echo in a Makefile :
clean:
rm -fr *.o
this rule will print:
$>make clean
rm -fr *.o
$>
How can I avoid that?
To start with: the actual command must be on the next line (or at least that is the case with GNU Make, it might be different with other Make's - I'm not sure of that)
clean:
rm -rf *.o
(note, you need a TAB before rm -rf *.o as in every rule)
Making it silent can be done by prefixing a #:
so your makefile becomes
clean:
#rm -rf *.o
If there are no *.o files to delete, you might still end up with an error message. To suppress these, add the following
clean:
-#rm -rf *.o 2>/dev/null || true
2>/dev/null pipes any error message to /dev/null - so you won't see any errors
the - in front of the command makes sure that make ignores a non-zero return code
In fact I was looking for something else, adding this line to the Makefile :
.SILENT:clean
while execute every step of the "clean" target silently.
Until someone point some drawback to this, I use this as my favourite solution!
I'm responding to this ancient topic because it comes up high in search and the answers are confusing. To do just what the user wants,all that is needed is:
clean:
#rm -f *.o
The # means that make will not echo that command.
The -f argument to rm tells rm to ignore any errors, like there being no *.o files, and to return success always.
I removed the -r from the OPs example, because it means recursive and here we are just rming .o files, nothing to recurse.
There's no need for the 2>&1 >/dev/null because with the -f there will be no errors printed.
.SILENT: clean
works in place of the #, but it isn't at the same place in the Makefile as the command that it affects, so someone maintaining the project later might be confused. That's why # is preferred. It is better locality of reference.
If you put an # in front of the command, it doesn't echo onto the shell. Try changing rm to #rm. (Reference)
From the manual: .SILENT is essentially obsolete since # is more flexible.
Much worse is that make prints far too much information. Warning/error/private messages are buried in the output. On the other hand -s (.SILENT) suppresses just anything. Especially the "nothing to be done" and "up to date" messages can be a pain. There is no option to suppress them. You have to filter them out actively or use something like colormake. Here is a solution for grep:
make | egrep -hiv 'nothing to be done|up to date'
But the output will have line numbers. The Perl solution is therefore better, because it suppresses line numbers and flushes stdout immediately:
make | perl -ne '$|=1; print unless /nothing to be done|up to date/i'
Make's a flawed tool. "What’s Wrong With GNU make?" explains this better than I can.
There's a great article on using .SILENT that explains how to conditionally activate it.
I have used that information to put this in my Makefile:
# Use `make V=1` to print commands.
$(V).SILENT:
# Example rule, only the #echo needs to be added to existing rules
*.o: %.c
#echo " [CC] $<"
gcc ...
What this does is if you run make normally, normal output is silenced and instead the echo commands work:
$ make
[CC] test.c
[CC] test2.c
But it allows you to debug problems by passing the V=1 parameter, which still shows the [CC] messages as it helps break up the output, but the traditional Makefile output is also visible:
$ make V=1
[CC] test.c
gcc ...
[CC] test2.c
gcc ...