I'm trying to simply copy files that are modified using make. Here is the entire Makefile:
FILES = www/foo.html www/bar.html www/zap.php
all: $(FILES)
$(FILES): src/$#
cp src/$# $#
clean:
rm $(FILES)
After modifying a file src/www/bar.html, make does not copy the file:
$ make
make: Nothing to be done for 'all'.
$ make www/bar.html
make: 'www/bar.html' is up to date.
Why does make not see the prerequisite has been modified and that the file needs to be copied?
If I run make clean, make it works (copies all files).
src/$# is not well-defined. You want
$(FILES): %: src/%
which declares a pattern rule, and restricts its scope to the files in $(FILES). (You might want or even need to remove this restriction.)
All I want to do is gather source files from different source directories into one folder and then do the build of those make files. After a make clean I have to run make command twice to do the build.
So first I run make clean, then i run make, which throws an error saying no -o files found. But when iIrun the make again, the build goes through and generates the build files.
My makefile looks like below
.PHONY: dirs all clean
all: dirs $(OBJ_DIR) $(OBJ_FILES)
"$(CC) -rvn fLib.a $(OBJ_FILES)
# clean build target. Remove all files without reporting errors in case they don't exist.
clean:
#rm -rf fLib.a $(OBJ_DIR)
# Build target for creating flat header file folder for SharedTrackerAPI (FLAT_INC_DIR)
# due to too long paths in Windows 7 build
dirs:
#echo 'Making flat source and header folders.'
#mkdir -p $(OBJ_DIR)
#for f in $(SRC_FILES); do cp $$f $(OBJ_DIR); done
#mkdir -p $(FLAT_INC_DIR)
#OLD_CWD=$(CURDIR)
#cd $(FLAT_INC_DIR)
#find $(STA_RADAR_TRACKER_IFACE) -name '*.h' | xargs -i cp -l {} $(FLAT_INC_DIR)
#cd $(OLD_CWD)
$(OBJ_DIR)/%.o: $(OBJ_DIR)/%.cpp
"$(TASKING_CTC_BIN)"/cctc.exe $(CXXFLAGS) -c -o $# $< $(CC_INCLUDE_PATH)
What am I doing wrong that I have to run make twice after a make clean.
The problem is that although dirs will place the source files in the flat source directory, Make doesn't know that. Before it executes the dirs rule, it has already determined that it knows no way to build the object files.
The quick and dirty solution is to tell Make "trust me, it'll be fine"; one way to do that is to modify the object rule like this:
$(OBJ_DIR)/%.o:
"$(TASKING_CTC_BIN)"/cctc.exe $(CXXFLAGS) -c -o $# $(OBJ_DIR)/$*.cpp $(CC_INCLUDE_PATH)
If you stop there, you will have a working solution.
If you want a more clean, efficient and flexible makefile, you must rethink the approach to finding source files. I see no good reason to use the flat source file approach, but if you really want to use it, here is a good way:
vpath %.cpp $(dir $(SRC_FILES))
$(OBJ_DIR)/%.cpp: %.cpp
#cp $< $#
Now you can get rid of dir and use your unmodified object rule:
$(OBJ_DIR)/%.o: $(OBJ_DIR)/%.cpp
"$(TASKING_CTC_BIN)"/cctc.exe $(CXXFLAGS) -c -o $# $< $(CC_INCLUDE_PATH)
How to handle the header files (FLAT_INC_DIR) is up to you, but I recommend vpath again.
I have a Makefile in which a couple of targets need to be made the same way, but one of the targets is a file whose basename is different from its prerequisite. Here is a minimal example:
ABOUT.html: README.md
help.html: help.md
%.html: %.md
pandoc --standalone --quiet -f gfm -H $(github-css) -o tmp.html $<
inliner -n < tmp.html > $#
rm -f tmp.html
With this Makefile, help.html gets made but ABOUT.html is never made. The reason, I presume, is because the base file name for %.html and %.md don't match up in the case of ABOUT.html because that target depends on README.md.
Is there a way to make this work without having to make a separate recipe for ABOUT.html?
One option is to create ABOUT.md symlink, so that your pattern rule works, by adding the following rule:
ABOUT.md : README.md
ln -s ${<F} ${#F}
You may like to avoid using the same temporary file in the recipe because that breaks in parallel builds. A better way is to use a unique temporary file for the target based on the target name:
%.html: %.md
pandoc --standalone --quiet -f gfm -H $(github-css) -o $#~ $<
inliner -n < $#~ > $#
rm -f $#~
I have a makefile like the following:
.PHONY: all
all: foo_1.txt foo_2.txt foo_xxx.txt
.PHONY: clean
clean:
rm -f foo_* bar_*
foo_%.txt: bar_%.txt
cp $< $#
#.PRECIOUS: bar_%.txt
bar_%.txt:
touch $#
bar_2.txt:
touch $#
The output of "make all" is folowing
touch bar_1.txt
cp bar_1.txt foo_1.txt
touch bar_2.txt
cp bar_2.txt foo_2.txt
touch bar_xxx.txt
cp bar_xxx.txt foo_xxx.txt
rm bar_xxx.txt bar_1.txt
The intermediate files created by the rule using pattern (bar_xxx.txt, bar_1.txt) are removed on the end. I have found that this behaviour can be supressed by .PRECIOUS (in the code is the line intentionally commented out).
Why are intermediate files created by rule with pattern removed by default and files created by rule without pattern are not?
By the definition of "intermediate files", you can't have an intermediate file created by an explicit rule (a rule "without a pattern").
See the section on Chains of Implicit Rules to understand this feature. If you have specific questions then please update your question.
I wish to write several rules that extract the contents of tar archives to produce a number of files that are then used as input dependencies for other rules. I wish this to work even with parallel builds. I'm not using recursive make.
First up, sorry for the marathon question, but I don't think I can explain it well in a shorter form.
Think of untarring a collection of source files and then compiling them with rules stored outside of the archive to produce various build artefacts that are then, in turn, used further. I am not seeking other arrangements that lead to the omission of this problem. Just take it for granted that I have good reason to do this. :)
I'll demonstrate my issue with a contrived example. Of course, I started with something basic:
TAR := test.tar.bz2
CONTENTS := $(addprefix out/,$(filter-out %/,$(shell tar -tf $(TAR))))
out: $(TAR)
rm -rf out
mkdir out
tar -xvf $< -C out --touch || (rm -rf out; exit 1)
$(CONTENTS): out
sums: $(CONTENTS)
md5sum $^ > $#
.DELETE_ON_ERROR:
.DEFAULT_GOAL := all
.PHONY: all clean
all: sums
clean:
rm -rf out sums
The thinking here is that since $(CONTENTS) are all of the files in the archive, and they all depend on out, then to run the sums target we need to end up extracting the archive.
Unfortunately, this doesn't (always) work if you use a parallel invocation after a previous build when only test.tar.bz2 is updated, because make may decide to check the timestamp of $(CONTENTS) before running the out rule, which means it thinks that each of the sources is older than sums, so there is nothing to do:
$ make clean
rm -rf out sums
$ make -j6
rm -rf out
mkdir out
tar -xvf test.tar.bz2 -C out --touch || (rm -rf out; exit 1)
data.txt
file
weird.file.name
dir/
dir/another.c
dir/more
md5sum out/data.txt out/file out/weird.file.name out/dir/another.c out/dir/more > sums
$ touch test.tar.bz2
$ make -j6
rm -rf out
mkdir out
tar -xvf test.tar.bz2 -C out --touch || (rm -rf out; exit 1)
data.txt
file
weird.file.name
dir/
dir/another.c
dir/more
Oops! The sums rule didn't run!
So, the next attempt was to tell make that the one untar rule actually does make all the $(CONTENTS) directly. This seems better since we're telling make what's really going on, so it knows when to forget any cached timestamps for targets when they are remade through their rule.
First, let's look at what seems to work, and then I'll get to my problem:
TAR := test.tar.bz2
CONTENTS := $(addprefix out/,$(filter-out %/,$(shell tar -tf $(TAR))))
# Here's the change.
$(addprefix %/,$(patsubst out/%,%,$(CONTENTS))): $(TAR)
rm -rf out
mkdir out
tar -xvf $< -C out --touch || (rm -rf out; exit 1)
sums: $(CONTENTS)
md5sum $^ > $#
.DELETE_ON_ERROR:
.DEFAULT_GOAL := all
.PHONY: all clean
all: sums
clean:
rm -rf out sums
In this case, we've effectively got a rule that says:
%/data.txt %/file %/weird.file.name %/dir/another.c %/dir/more: test.tar.bz2
rm -rf out
mkdir out
tar -xvf $< -C out --touch || (rm -rf out; exit 1)
Now you can see one of the reasons I forced the output into an out directory: to give me a place to use the % so I could use a pattern rule. I am forced to use a pattern rule even though there isn't a strong pattern here because it is the only way make can be told that one rule creates multiple output files from a single invocation. (Isn't it?)
This works if any of the files are touched (not important for my use case) or if the test.tar.bz2 file is touched, even in parallel builds, because make has the information it needs: running this recipe makes all these files and will change all their timestamps.
For example, after a previous successful build:
$ touch test.tar.bz2
$ make -j6
rm -rf out
mkdir out
tar -xvf test.tar.bz2 -C out --touch || (rm -rf out; exit 1)
data.txt
file
weird.file.name
dir/
dir/another.c
dir/more
md5sum out/data.txt out/file out/weird.file.name out/dir/another.c out/dir/more > sums
So, if I have a working solution, what's my problem?
Well, I have many of these archives to extract, each with their own set of $(CONTENTS). I can manage that, but the trouble comes in writing a nice pattern rule. Since each archive needs its own rule defined, the patterns for each rule must not overlap even if the archives have similar (or identical) content. That means the output paths for the extracted files must be made unique for each archive, as in:
TAR := test.tar.bz2
CONTENTS := $(addprefix out.$(TAR)/,$(filter-out %/,$(shell tar -tf $(TAR))))
$(patsubst out.$(TAR)/%,out.\%/%,$(CONTENTS)): $(TAR)
rm -rf out.$(TAR)
mkdir out.$(TAR)
tar -xvf $< -C out.$(TAR) --touch || (rm -rf out.$(TAR); exit 1)
sums: $(CONTENTS)
md5sum $^ > $#
.DELETE_ON_ERROR:
.DEFAULT_GOAL := all
.PHONY: all clean
all: sums
clean:
rm -rf out.$(TAR) sums
So, this can be made to work with the right target-specific variables, but it now means that the extraction points are all "ugly" in a way that is very specifically tied to how the makefile is constructed:
$ make -j6
rm -rf out.test.tar.bz2
mkdir out.test.tar.bz2
tar -xvf test.tar.bz2 -C out.test.tar.bz2 --touch || (rm -rf out.test.tar.bz2; exit 1)
data.txt
file
weird.file.name
dir/
dir/another.c
dir/more
md5sum out.test.tar.bz2/data.txt out.test.tar.bz2/file out.test.tar.bz2/weird.file.name out.test.tar.bz2/dir/another.c out.test.tar.bz2/dir/more > sums
The next natural step I took was to try to combine static pattern rules with the multiple-targets-via-pattern-rule approach. This would let me keep the patterns very general, but limit their application to a specific set of targets:
TAR := test.tar.bz2
CONTENTS := $(addprefix out/,$(filter-out %/,$(shell tar -tf $(TAR))))
# Same as second attempt, except "$(CONTENTS):" static pattern prefix
$(CONTENTS): $(addprefix %/,$(patsubst out/%,%,$(CONTENTS))): $(TAR)
rm -rf out
mkdir out
tar -xvf $< -C out --touch || (rm -rf out; exit 1)
sums: $(CONTENTS)
md5sum $^ > $#
.DELETE_ON_ERROR:
.DEFAULT_GOAL := all
.PHONY: all clean
all: sums
clean:
rm -rf out sums
Great! Except it doesn't work:
$ make
Makefile:5: *** multiple target patterns. Stop.
$ make --version
GNU Make 4.0
So, is there a way to use multiple target patterns with a static pattern rule? If not, is there another way to achieve what I have in the last working example above, but without the constraint on the output paths to make unique patterns? I basically need to tell make "when you unpack this archive, all of the files in this directory (which I am willing to enumerate if necessary) have new timestamps". A solution where I can force make to restart if and only if it unpacks an archive would also be acceptable, but less ideal.
The problem with your original makefile is that you have a collision in names. You have a target (non-phony) named out and a directory named out. make thinks those are the same thing and gets very confused.
(Note: I added .SUFFIXES: to your first makefile to cut down on some noise but it doesn't change anything. The -r and -R flags disable make built-in rules and variables also for noise reduction.)
$ make clean
....
$ make -j6
....
$ touch test.tar.bz2
$ make -rRd -j6
....
Considering target file 'all'.
File 'all' does not exist.
Considering target file 'sums'.
Considering target file 'out/data.txt'.
Looking for an implicit rule for 'out/data.txt'.
No implicit rule found for 'out/data.txt'.
Considering target file 'out'.
Considering target file 'test.tar.bz2'.
Looking for an implicit rule for 'test.tar.bz2'.
No implicit rule found for 'test.tar.bz2'.
Finished prerequisites of target file 'test.tar.bz2'.
No need to remake target 'test.tar.bz2'.
Finished prerequisites of target file 'out'.
Prerequisite 'test.tar.bz2' is older than target 'out'.
No need to remake target 'out'.
Finished prerequisites of target file 'out/data.txt'.
Prerequisite 'out' is older than target 'out/data.txt'.
No recipe for 'out/data.txt' and no prerequisites actually changed.
No need to remake target 'out/data.txt'.
.... # This following set of lines repeats for all the other files in the tarball.
Considering target file 'out/file'.
Looking for an implicit rule for 'out/file'.
No implicit rule found for 'out/file'.
Pruning file 'out'.
Finished prerequisites of target file 'out/file'.
Prerequisite 'out' is older than target 'out/file'.
No recipe for 'out/file' and no prerequisites actually changed.
No need to remake target 'out/file'.
....
Finished prerequisites of target file 'sums'.
Prerequisite 'out/data.txt' is older than target 'sums'.
Prerequisite 'out/file' is older than target 'sums'.
Prerequisite 'out/weird.file.name' is older than target 'sums'.
Prerequisite 'out/dir/more' is older than target 'sums'.
Prerequisite 'out/dir/another.c' is older than target 'sums'.
No need to remake target 'sums'.
Finished prerequisites of target file 'all'.
Must remake target 'all'.
Successfully remade target file 'all'.
make: Nothing to be done for 'all'.
The main details here are these two lines:
Considering target file 'out'.
Prerequisite 'out' is older than target 'out/data.txt'
The out directory doesn't matter here. We don't care about it (and make doesn't deal with directory prerequisites too well anyway because modification timestamps on directories don't mean the same thing as they do on files). Even more to the point you don't want out/data.txt not being created because the build artifact directory target already existed (and seemed older).
You can "fix" this by marking the out target as .PHONY but that is just going to get make to extract the tarball every time you run make (you already run tar -tf every time you run make so it would probably be better to just combine those two steps if you were going to do this).
That said I wouldn't do that. I think the simplest solution to this problem is the "atomic rules" idea from John Graham-Cunning built-up and explained here.
sp :=
sp +=
sentinel = .sentinel.$(subst $(sp),_,$(subst /,_,$1))
atomic = $(eval $1: $(call sentinel,$1) ; #:)$(call sentinel,$1): $2 ; touch $$# $(foreach t,$1,$(if $(wildcard $t),,$(shell rm -f $(call sentinel,$1))))
.PHONY: all
all: a b
$(call atomic,a b,c d)
touch a b
You could probably also do this with an extraction stamp file (prereq on the tarball), extracting the tarball to a "shadow" directory and copy/link to the "final" location (build/$file: shadow/$file target) if you wanted to but that's going to be a bit more complicated I think.