Makefile, how to have prerequisites given by function with target as argument - makefile

I have a list of files contained in another file. I want to use this list as prerequisite for some target and for doing so I use a function that reads the list from file.
The problem is that I have different lists for different targets so I need to pass the target as argument to the function that reads the list. Something like that (that does not work):
getlist = $(shell cat $1)
tmp%: $(call getlist, %)
#cat file1 > $#
#cat file2 >> $#
file%:
#touch $#
#echo "$#" > $#
clean:
#rm file1 file2 tmp
where the list for building the tmp1 file is in the 1 file, the one for building the tmp2 file is in the 2 file and so on and so forth.
If I have instead tmp1: $(call figlist, 1) all works, but I need something capable of treating different file names.
If needed for the solution I can also change the way I named the files.

Related

Define a target that depends on a value/variable that need to be resolved

Updated my question as it seemed to be not clear enough!
I was listing when to use make over bash. One thing I like about make is its declarative way of describing necessary steps; we can write a rule by relying on other rules knowing how to provide necessary files (or other external states).
I'm wondering how I can get the same benefit for a value not a file, without changing outer world (like leaving a temporary file).
hello.txt: (here, tell that it needs to resolve person's name)
# Here, person's name is available.
echo Hello $(var_name) > $#
We can imperatively prepare a necessary value with $(call prepare_name, ...) at the beginning of a command in a rule, but that's not what I'm after here.
I posted my attempts as an answer when I opened this question. Hopefully that adds more info on what I'm trying to achieve.
It's not overly clear what you're after, however to clarify a few concepts:
A target must be dependent on other targets. It cannot be dependent on a variable name. It can be dependent on the value of a variable, if that variable resolves to a target name.
So you could do:
VAR=some_target
hello.txt: $(VAR)
echo "hello $^" > $#
some_target:
touch $#
You CANNOT do:
VAR=some_target
hello.txt: VAR
and expect it to work (make would try to build VAR which likely doesn't exist and it would fail).
I'm assuming from the question that you want make to request the variable name of a person, and put that into hello.txt. In that case you would likely want to store the name in a temporary file and use that for the output:
.getname.txt:
#read -p "enter name" name > $#
hello.txt: .getname.txt
#echo "hello $$(cat $$<)" > $#
This will update .getname.txt if it didn't previously exist (so it will not necessarily ask on every invokation of make...). You could make .getname.txt be a .PHONY target, and it will run every time.
If you do want to run every time, then you can simply do:
hello.txt:
#read -p "enter name: " name && echo "hello $$name" > $#
.PHONY: hello.txt
Which will invoke the hello.txt rule regardless of whether hello.txt already exists, and will always prompt the user for a name and rebuild hello.txt.
I can think of a way using eval function. Below suppose foo is a value obtained by a complex calculation.
hello.txt: var_name
echo Hello $($<) > $#
.PHONY: var_name
var_name:
$(eval $# = foo)
Or with .INTERMEDIATE target, this also works, but I feel it's more complicated.
var_name = var-name.txt
hello.txt: $(var_name)
echo Hello $$(< $<) > $#
.PHONY: $(var_name)
.INTERMEDIATE: $(var_name)
$(var_name):
rm -f $# # In case the var file already exists
echo bar > $#
Another way could be to use a target-specific variable. It's not listing a variable as a prerequisite, but I still don't need to think about how to get var_name when writing echo Hello ....
define get_name
echo foo
endef
hello.txt: var_name = $(call get_name)
hello.txt:
echo Hello $(var_name) > $#
As noted in other answers, make track dependencies between files, using timestamps. The regular solution for handling a value will be to store it in a file (or to generate it into a file). Assuming that there is significant work to do whenever the data is changing, you can follow one of the patterns below to implement dependency check on the file value.
The following makefile snapshot will trigger rebuild of complex-result, only when the content of var-value is modified. This is useful when the content of var-value is continuously regenerated, but does not change very frequently.
all: complex-result
last-value.txt: var-value.txt
cmp -s $< $# || cat <$^ > $#
complex-result: last-value.txt
echo Buildig for "$$(cat var-value.txt)"
touch $#
Or more realistic example: trigger a build if the value (content) of any file was modified, using md5 checksum,
all: complex-result
last-value.txt: $((wildcard *.data)
md5sum $^ > $#
last-value.txt: var-value.txt
cmp -s $< $# || cat <$^ > $#
complex-result: last-value.txt
echo Building for "$$(cat var-value.txt)"
touch $#

Processing multiple files generated from single input

I have a data file that is processed by a script to produce multiple output files. Each of these output files is then processed further. Which files are created depends on the contents of the input file, so I can't list them explicitly. I can't quite figure out how to refer to the various files that are generated in a makefile.
Currently, I have something like this:
final.out: *.out2
merge_files final.out $(sort $^)
%.out2: %.out1
convert_files $?
%.out1: data.in
extract_data data.in
This fails with No rule to make target '*.out2', needed by 'final.out'. I assume this is because the .out2 files don't exist yet and therefore the wildcard expression isn't replaced the way I would like it to. I have tried to use the wildcard function but that fails because the list of prerequisites ends up being empty.
Any pointers would be much appreciated.
EDIT: fixed the list of prerequisites in second pass.
You apparently cannot compute the list of intermediate files before running the extract_data command. In this case a solution consists in running make twice. One first time to generate the *.out1 files and a second time to finish the job. You can use an empty dummy file to mark whether the
extract_data command shall be run again or not:
ifeq ($(FIRST_PASS_DONE),)
final.out: .dummy
$(MAKE) FIRST_PASS_DONE=yes
.dummy: data.in
extract_data $<
else
OUT1 := $(wildcard *.out1)
OUT2 := $(patsubst %.out1,%.out2,$(OUT1))
final.out: $(OUT2)
merge_files $# $(sort $^)
%.out2: %.out1
convert_files $?
endif
Unfortunately your question is missing some details I would ask immediately if some SW developer would present this makefile for review:
does extract_files provide the list of files?
does convert_files convert one file or multiple? The example seems to imply that it converts multiple.
then I have to question the decision to break up extract, convert and merge into separate rules as you will not benefit from parallel build anyway
The following is the approach I would choose. I'm going to use a tar file as an example for an input file that results in multiple output files
generate a makefile fragment for the sorted list of files
use the tar option v to print files while they are extracted
convert each line into a makefile variable assignment
include the fragment to define $(DATA_FILES)
if the fragment needs to be regenerated, make will restart after it has generated it
use static pattern rule for the conversion
use the converted file list as dependency for the final target
.PHONY: all
all: final.out
# extract files and created sorted list of files in $(DATA_FILES)
Makefile.data_files: data.tar
set -o pipefail; tar xvf $< | sort | sed 's/^/DATA_FILES += /' >$#
DATA_FILES :=
include Makefile.data_files
CONVERTED_FILES := $(DATA_FILES:%.out1=%.out2)
$(CONVERTED_FILES): %.out2: %.out1
convert_files $< >$#
final.out: $(CONVERTED_FILES)
merge_files final.out $^
UPDATE if extract_data doesn't provide the list of files, you could modify my example like this. But of course that depends on that there are no other files that match *.out1 in your directory.
# extract files and created sorted list of files in $(DATA_FILES)
Makefile.data_files: data.in
set -o pipefail; \
extract_data $< && \
(ls *.out1 | sort | sed 's/^/DATA_FILES += /') >$#

How to use makefile variable outside of a rule

Right now I am learning how to use makefiles and am having trouble with variables inside rules. My goal is to parse a file containing names of other files, save that in a variable, and use the variable as a "target pattern" to run a rule on each file. Here is a simple example of my problem (I'm writing the file names in to make it simpler).
When I run this:
variable = file1.txt file2.txt file3.txt
run : $(variable)
$(variable): %.txt:
echo File is $#
I get the following, which is what I want:
$ make run
echo File is file1.txt
File is file1.txt
echo File is file2.txt
File is file2.txt
echo File is file3.txt
File is file3.txt
The problem is I want to define variable by parsing a file, and I want that file to be a dependency because, if it does not exist, I will make it using another rule in the makefile. So when I define the variable in a rule like this (which I know is there from my echo):
target1 :
$(eval variable = file1.txt file2.txt file3.txt)
echo $(variable)
run : $(variable)
$(variable): %.txt:
echo File is $#
I get the following:
$ make target1
echo file1.txt file2.txt file3.txt
file1.txt file2.txt file3.txt
And this:
$ make run
make: Nothing to be done for `run'.
So my question is how to define the variable in a rule and and use it in something line this:
$(variable): %.txt:
echo File is $#
I've tried reading through the manual and Googling but I just can't figure this out. I also imagine there is some simple answer I am missing. Thanks so much for your help! :)
If you can control the content of the target1 created file this is easy: just make the format of that file be a make variable assignment, then use the include command, like this:
run:
include target1
run: $(variable)
$(variable): %.txt:
echo File is $#
target1:
echo 'variable = file1.txt file2.txt file3.txt' > $#
Through the magic of automatically generated makefiles make will recreate that target file then re-invoke itself automatically.

multiple stems in makefile rule

I am trying to write a makefile that does something like the following:
%-foo-(k).out : %-foo-(k-1).out
# do something, e.g.
cat $< $#
i.e. there are files with arbitrary stems, then -foo-, then an integer, followed by .out. Each file depends on the one with the same name, with integer one smaller.
For instance, if the file blah/bleh-foo-1.out exists, then
make blah/bleh-foo-2.out
would work.
I could do this with multiple stems if there were such a thing... what's another way to do this sort of thing in (gnu) make?
There is no easy way to do something like this. You basically have two options: you can use auto-generated makefiles, or you can use $(eval ...). To me auto-generated makefiles are easier, so here's a solution:
SOURCELIST = blah/bleh-foo-1.out
all:
-include generated.mk
generated.mk: Makefile
for f in $(SOURCELIST); do \
n=`echo "$$f" | sed -n 's/.*-\([0-9]*\)\.out$/\1/p'`; \
echo "$${f%-foo-[0-9]*.out}-foo-`expr $$n + 1`.out: $$f ; cat $$< > $$#"; \
done > $#

makefile backreference to matched text

Suppose, I have 2 files, dependent upon each other:
./pictures/1_data.tex
|
V
./data/1.pl
So, 1_data.tex is generated from the Perl file. To do it I have the following rule in the makefile:
./pictures/1_data.tex: ./data/1.pl
perl given.pl 1 > $#
If I have multiple files with this pattern:
./data/1.pl
./data/2.pl
...
./data/n.pl
I'd like to use wildcards to process them. I tried this:
./pictures/*_data.tex: ./data/*.pl
perl given.pl $* > $#
But it generates incorrect command:
perl given.pl pictures/1_data > pictures/1_data.tex
Is it possible to have a backreference only to 1, and not to the whole target? As $* does.
Use pattern rules:
all: $(patsubst ./data/%.pl,./pictures/%_data.tex,$(wildcard ./data/*.pl))
./pictures/%_data.tex : ./data/%.pl
perl given.pl $* > $#

Resources