I am trying to use make to generate thumbnails of photos by typing "make all". If the thumbnails are not yet generated make all generates them, else make all just generate the thumbnails of modified photos. For this I need one target (thumbnail) for each dependency (photo) . My code is like this :
input = pictures/*.jpg
output = $(subst pictures,thumbs,$(wildcard $(input)))
all : $(output)
echo "Thumbnails generated !"
$(output) : $(input)
echo "Converting ..."
convert -thumbnail 100 $(subst thumbs,pictures,$#) $#
How can I modify it to get the desired result ?
Your problem is this line
$(output) : $(input)
The output variable is the list of every output file.
The input variable is the wildcard pattern.
This sets the prerequisites of every output target as the wildcard pattern which means if any file changes every output file will be seen as needing to be rebuilt.
The fix for this is to either use a static pattern rule like this
$(output) : thumbs/% : pictures/%
which says to build all the files in $(output) by matching them against the pattern thumbs/% and using the part that matches % (called the stem) in the prerequisite pattern (pictures/%).
Alternatively, you could construct a set of specific input/output matches for each file with something like
infiles = $(wildcard pictures/*.jpg)
$(foreach file,$(infiles),$(eval $(subst pictures/,thumbs/,$(file)): $(file)))
$(output):
echo "Converting ..."
convert -thumbnail 100 $(subst thumbs,pictures,$#) $#
Which uses the eval function to create explicit thumbs/file.jpg: pictures/file.jpg target/prerequisite pairs for each input file.
Related
I have a data file that is processed by a script to produce multiple output files. Each of these output files is then processed further. Which files are created depends on the contents of the input file, so I can't list them explicitly. I can't quite figure out how to refer to the various files that are generated in a makefile.
Currently, I have something like this:
final.out: *.out2
merge_files final.out $(sort $^)
%.out2: %.out1
convert_files $?
%.out1: data.in
extract_data data.in
This fails with No rule to make target '*.out2', needed by 'final.out'. I assume this is because the .out2 files don't exist yet and therefore the wildcard expression isn't replaced the way I would like it to. I have tried to use the wildcard function but that fails because the list of prerequisites ends up being empty.
Any pointers would be much appreciated.
EDIT: fixed the list of prerequisites in second pass.
You apparently cannot compute the list of intermediate files before running the extract_data command. In this case a solution consists in running make twice. One first time to generate the *.out1 files and a second time to finish the job. You can use an empty dummy file to mark whether the
extract_data command shall be run again or not:
ifeq ($(FIRST_PASS_DONE),)
final.out: .dummy
$(MAKE) FIRST_PASS_DONE=yes
.dummy: data.in
extract_data $<
else
OUT1 := $(wildcard *.out1)
OUT2 := $(patsubst %.out1,%.out2,$(OUT1))
final.out: $(OUT2)
merge_files $# $(sort $^)
%.out2: %.out1
convert_files $?
endif
Unfortunately your question is missing some details I would ask immediately if some SW developer would present this makefile for review:
does extract_files provide the list of files?
does convert_files convert one file or multiple? The example seems to imply that it converts multiple.
then I have to question the decision to break up extract, convert and merge into separate rules as you will not benefit from parallel build anyway
The following is the approach I would choose. I'm going to use a tar file as an example for an input file that results in multiple output files
generate a makefile fragment for the sorted list of files
use the tar option v to print files while they are extracted
convert each line into a makefile variable assignment
include the fragment to define $(DATA_FILES)
if the fragment needs to be regenerated, make will restart after it has generated it
use static pattern rule for the conversion
use the converted file list as dependency for the final target
.PHONY: all
all: final.out
# extract files and created sorted list of files in $(DATA_FILES)
Makefile.data_files: data.tar
set -o pipefail; tar xvf $< | sort | sed 's/^/DATA_FILES += /' >$#
DATA_FILES :=
include Makefile.data_files
CONVERTED_FILES := $(DATA_FILES:%.out1=%.out2)
$(CONVERTED_FILES): %.out2: %.out1
convert_files $< >$#
final.out: $(CONVERTED_FILES)
merge_files final.out $^
UPDATE if extract_data doesn't provide the list of files, you could modify my example like this. But of course that depends on that there are no other files that match *.out1 in your directory.
# extract files and created sorted list of files in $(DATA_FILES)
Makefile.data_files: data.in
set -o pipefail; \
extract_data $< && \
(ls *.out1 | sort | sed 's/^/DATA_FILES += /') >$#
As per the gnu make documentation, a pattern rule's "...recipe is executed only once to make all the targets." However, I have the following Makefile
.PHONY: entrypoint
entrypoint: test_1.cpp test_2.cpp
test_%.cpp:
echo $#
And running make produces:
echo test_1.cpp
test_1.cpp
echo test_2.cpp
test_2.cpp
I'm new to make, and I'm probably misunderstanding something, but the documentation seems misleading if clear.
$ make -v
GNU Make 4.0
...
You're misreading the documentation. It means, the recipe is run only one time assuming that all the target patterns in that rule will be created.
Since you have only one target pattern in your rule (test_%.cpp`) make knows that each time it runs that recipe it will create one output file matching that pattern. To create different targets that match that pattern it will run multiple instances of the recipe.
If you had a rule like this:
%.x %.y %.z :
dothings
then make would expect that a single invocation of the recipe dothings would create all the targets matching this pattern (e.g., foo.x, foo.y, and foo.z).
Contrast this with an explicit rule like this:
foo.x foo.y foo.z :
dothings
Make here treats this exactly as if you'd written this:
foo.x :
dothings
foo.y :
dothings
foo.z :
dothings
That is, to build all three of these targets it would run the recipe three times.
There's no way to tell make "please run this recipe one time and it will produce every single target that could possibly match the pattern foo_%.cpp".
The following functions construct a dynamic list of dependencies of your multiple-target where the non-existent files are named last. This is more or less the method named "Another attempt" in the link you gave, except that it doesn't trip over missing files and is able to make a missing file by giving it as target on the command line. What it does not: execute the multitarget recipe if one of the multitargets is out of date relative to the others, but I think this is more of wanted side effect than a problem. The only drawback is the syntactic ugliness as you have to write it into an eval expression which forces you to quote all variables in the recipe which shall be evaluated at execution time.
define newline :=
endef
list2rules = $(firstword $1) $(if $(word 2,$1),: $(word 2,$1)$(newline)$(call list2rules,$(wordlist 2,1000,$1)))
multitarget = $(call list2rules,$(wildcard $1) $(filter-out $(wildcard $1),$1))
.PHONY: all
targets = test1 test2 footest3
#$(info $(call multitarget,$(targets)))
all: somefile
somefile: $(targets)
touch somefile
# here we generate the dependency list on the spot. Only one recipe to update all targets.
$(eval $(call multitarget,\
$(targets)) : ; \
touch $(targets) \
)
I am processing some files and want to at one point create two categories depending on the filename so I can compare the two. Is this possible in a makefile?
%.output1: %.input
ifneq (,$(findstring filename1,$(echo $<)))
mv $<.output1 $#
endif
%.output2: %.input
ifneq (,$(findstring filename2,$(echo $<)))
mv $<.output2 $#
endif
%.output_final: %.output1 %.output2
do_something
I think there is two things wrong with this code:
There is a mistake in the ifneq line.
%.output1 %.output2 will always use the same filename - it may not be possible to do this in 'make' and this may require ruffus.
You have tab-indented the ifneq line so make doesn't consider it a make directive and is considering it a shell command and attempting to pass it to the shell to execute (hence the shell error you removed in your recent edit).
Use spaces (or no indentation) on that line to have make process it correctly. That being said having done that you cannot use $< in the comparison as it will not be set at that point.
$(echo) is also not a make function. You have mixed/confused processing times. You cannot combine make and shell operations that way. (Not that you need echo there to begin with.)
If you want the comparison to happen at shell time then do not use make constructs and instead use shell constructs:
%.output1: %.input
if [ filename1 = '$<' ]; then
mv $<.output1 $#
fi
Though that is also incorrect as $< is %.input and $# is %.output1 for whatever stem matched the %. That rule should probably look more like this (though I'm having trouble understanding what you are even trying to have this rule do so I may have gotten this wrong).
%.output1: %.input
# If the stem that matched the '%' is equal to 'filename1'
if [ filename1 = '$*' ]; then
# Then copy the prerequisite/input file to the output file name.
cp $< $#
fi
I'm not sure I understand your second question point. The % in a single rule will always match the same thing but between rules it can differ.
This %.output_final: %.output1 %.output2 target will map the target file foo.output_final to the prerequisite files foo.output1 and foo.output2. But will also map any other *.output_final file to appropriately matching prerequisite files.
I have script that takes in a filename and generates multiple files with same name but different extension. I want to write a makefile that depends on files generated with different extension but only specify the filename. I have a dummy example to explain it below.
test_output:test_input
genereate.py -i $^ -o $#
The above makefile dependency generates multiple files with same filename but different extension, but won't generate the actual target. For example, it generates
test_output.a test_output.b test_output.c
The way its written above is not the efficient way as there no actual target, so it runs this even though the output is already there.
How would i specify the makefile so that it reads in the target(test_output) but actually depends on the output file it generates like test_output.a or any of the other files.
If you use GNU make (you didn't say) you can use pattern rules to tell make about a rule that generates multiple outputs based on a single stem. So for example you can write:
%.a %.b %.c : test_input
genereate.py -i $^ -o $*
(it would work a lot better if the input filename was related to the output filenames with the same stem, but the above will work although you'll have to write a different one for each input file).
Typically that's what you want, so that other targets that need these outputs can depend on them.
If you really want to have a target without any extension as well, just create it:
test_output : test_output.a test_output.b test_output.c
%.a %.b %.c : test_input
genereate.py -i $^ -o $*
I have a makefile which looks for .txt files in a directory and for each file makes echo of it name.
pchs := $(wildcard $(OUTPUT:%=%/*.txt))
txt: $(pchs)
%.txt:
echo $#
But when I start it the make utility returns me that nothing to be done for txt. Why?
EDIT1:
After some answers I understand what I should make with my makefile. Now it looks like this:
pchs := $(wildcard $(OUTPUT:%=%/*.txt))
.PHONY : $(pchs)
txt: $(pchs)
%.txt:
#echo pch is '$<'
But .PHONY does not help me the result of making is the same.
Why does make says, that there ist nothing to do? Because make calculates dependencies of targets, usually file targets. And the "txt" target produces no file.
.PHONY is for targets, that produce no file, like the clean target.
This here should work:
pchs := $(wildcard $(OUTPUT:%=%/*.txt))
.PHONY: txt
txt: $(pchs)
echo $#
But, since you only echo the filename, I guess that you are post processing this output. Maybe you could formulate this post processing as a rule in the makefile?
Because makefiles define what you want to have built. And the .txt files already exist, so there is nothing to do.
To solve this there are a number of possibilities, but you should look into the .PHONY record if using gnu-make at least.
You can build fake-things out of the txt records and mark them as phony. But... it might just be easier to do this:
pchs := $(wildcard $(OUTPUT:%=%/*.txt))
txt:
for i in $(pchs) ; do echo $$i ; done
That's because every .txt file you've listed in $(pchs) is up-to-date and Make decides to take no action.
Add them to .PHONY target to force rebuilding them every time you run Make:
.PHONY : $(pchs)
UPD.
Also check that $(pchs) list is not empty, it could be done i.e. as follows:
txt : $(pchs)
#echo pchs is '$^'
I would use Bash to determine the *.txt files, instead of Make:
txt:
ls | grep -F '.txt'
You could also use this as a template to make a more general target, that echos any files that exist in the directory with a particular extension.
You may want the target to be PHONY, since it's not making a file.