I want to define a munch of rules like this:
x-9: y-9 z-9 x-8
python gen-files.py --out-x=x-9 --out-y=y-9 --in-x=x-8
x-8: y-8 z-8 x-7
python gen-files.py --out-x=x-8 --out-y=y-8 --in-x=x-7
x-7: y-7 z-7 x-6
python gen-files.py --out-x=x-7 --out-y=y-7 --in-x=x-6
x-6: y-6 z-6 x-5
python gen-files.py --out-x=x-6 --out-y=y-6 --in-x=x-5
x-5: y-5 z-5 x-4
python gen-files.py --out-x=x-5 --out-y=y-5 --in-x=x-4
x-4:
touch x-4
As you can see, the basic idea is that I have a target
outputfile-NUMBER:
which has a couple of dependencies, either containing NUMBER in their name or containing NUMBER-MINUS-ONE in their name.
My goal would be that when I try to build the final target x-9 it automatically x-8, x-7... down to x-4.
I tried something like
define oprule
x-$(1): x-$(1) y-$(1) z-$(1) x-$(2)
python gen-files.py --out-x=x-$(1) --out-y=y-$(1) --in-x=x-$(2)
endef
ttt = 9 8 7 6 5
$(foreach now, $(ttt), \
$(call oprule, $(now), $(shell $$(( $(now)-1 )) ) ) )
I thought this would generate the 5 rules, but when I try
make x-9
I get the message
Makefile:93: *** multiple target patterns. Stop.
and I don't know what happens.
It should be
define oprule
x-$(1): y-$(1) z-$(1) x-$(2)
python gen-files.py --out-x=x-$(1) --out-y=y-$(1) --in-x=x-$(2)
endef
The last newline preceding endef is always eaten out (just like the first newline following define). So you must have one more for foreach to generate the right line splitting (or add some sort of $(nl) macro after calling oprule).
P.S. And also do not put space after comma. Although it won't hurt in this particular case, but in general the spaces are significant for make.
Related
I have a Makefile, trying to loop over a series of strings in a recipe and make them lower case.
My goal: I have a series of commands I would like to run on different files with the appropriate suffix.
# files in directory: test_file1.txt test_file2.txt test_file3.txt
MYLIST = file1 file2 file3
recipe:
for name in $(MYLIST) ; do \
$(eval FILENAME=`echo $($name) | tr A-Z a-z`) \
echo "Final name : test_${FILENAME}.txt" ; \
done
My problem, FILENAME always resolves to blank:
File name: test_.txt
I hope to see:
File name: test_file1.txt
File name: test_file2.txt
File name: test_file3.txt
You cannot mix make functions and shell commands like this. Make works like this: when it decides that your target is out of date and it wants to run the recipe, first it will expand the entire recipe string. Second it sends that expanded string to the shell to run.
So in your case, the $(eval ...) (which is a make operation) is expanded one time, then the resulting string is passed to the shell to run. The shell runs the for loop and all that stuff.
You have to use shell variables here to store values obtained by running your shell for loop. You cannot use make variables or make functions.
In general if you ever think about using $(shell ...) or $(eval ...) inside a recipe, you are probably going down the wrong road.
If I have a list of things such as this:
nodes = A B C
How would I generate dynamic targets that have dynamic prerequisites. For example (this isn't working but might help explain what I want).
# node.csr is a file that already exists, like a template
# this would create outputs like node-A-csr.json, node-B-csr.json
# I am basically guessing at the syntax here
node-%-csr.json: node-csr.json
sed 's/$${node}/$*' node-csr.json > $#
# this would create outputs like node-A-key.pem node-A.pem and would require node-A-csr.json
node-%-key.pem node-%.pem: node-%-csr.json
# some command that generates node-NAME-key.pem and node-NAME-csr.pem
$(nodes): node-%-key.pem node-%.pem
Id basically like to be able to run make all and have those targets run for everything in my list.
I am very new to Makefiles and I just dont see how something like this would work and the documentation and syntax of Make is extremely confusing to me.
I am willing to use any tool to do this but seems like Make is very standard.
You can use Make's Substitution References to generate the 'all' target. This will be enough to start processing of all rules.
Notice minor change: the 'node-csr.json' should have the token NODE where the actual node name is to be inserted
# First rule is the default
default: all
nodes = A B C
# use node-csr as a template, replacing NODE with current ID: A, B, C
node-%-csr.json: node-csr.json
sed 's/__NODE__/$*/' node-csr.json > $#
# this would create node-A-key.pem node-A.pem from node-A-csr.json
# Put real commands here
node-%-key.pem node-%.pem: node-%-csr.json
ls -l node-$*-csr.json > node-$*-key.csr
ls -l node-$*-csr.json > node-$*.pem
# all will be node-A-key.pem, node-B-key.pem, ... node-C.pem
all: $(nodes:%=node-%-key.pem) $(nodes:%=node-%.pem)
echo "done"
# Using pathsubst: all: $(patsubst %,node-%-key.pem,$(nodes)) $(pathsubst
Please pay attention to tab/spaces, some version are sensitive. You will have to put back tabs before all commands (sed, ls, ...)
Substitution References man: https://www.gnu.org/software/make/manual/html_node/Substitution-Refs.html
pathsubst function: https://www.gnu.org/software/make/manual/html_node/Text-Functions.html
I'm trying to use GNU make to automate my analysis pipeline. I have a script that reads files with the pattern data/sub001/sub001_input.txt and writes to data/sub001/sub001_output.txt. How could I write a rule that matches this pattern for each subject? Here is my attempt so far:
# List of all the subjects
SUBJECTS ?= sub001 sub002 sub003
/data/%/%_output.txt : process.py data/%/%_input.txt
python process.py $*
# for each $SUBJECT in $SUBJECTS
all : /data/$(SUBJECT)/$(SUBJECT)_output.txt
#echo 'Data analysis complete!'
I would like the all target to call:
python process.py sub001
python process.py sub002
python process.py sub003
And I would like a single subject to be re-processed if the corresponding sub###_input.txt file changes, and I would like all subjects to be re-processed if the process.py file changes.
You cannot use multiple pattern characters in pattern rules.
You can use a single pattern which will stand for the entire middle portion, then strip out the part you want like this:
/data/%_output.txt : process.py data/%_input.txt
python process.py $(*F)
I read this question: Makefile: $subst in dependency list, but I still can't make my shell script work correctly.
I have a makefile with a line with the contents:
##public_detailed#|test_create|Syntax: commoncmdsyntax test_create test_name=<test-name>
A target runs a multiline bash script, where the commoncmdsyntax must be replaced by a string containing words and spaces.
In the script, I use cut to assign to a variable desc the following string:
Syntax: commoncmdsyntax test_create test_name=<test-name>
The problem is that commoncmdsyntax is not replaced by new text here:
$(subst commoncmdsyntax,new text,$$desc)
I also tried to replace it by a single word, like XX, but it also does not work.
The subst function (as in $(subst commoncmdsyntax,new text,$$desc)) is a Make function, so Make will perform the substitution before running any rule and therefore before your script assigns a value to desc. So even if secondary expansion worked the way you seem to think it will, this approach would still fail.
If you want to perform a substitution within something made by a shell script (in a recipe), the sensible way is to do so within the recipe:
echo $dest | sed 's/commoncmdsyntax/new text/'
We can give you a more detailed solution if you give us a minimal complete example of the problem.
I have a list of objects in a Makefile variable called OBJECTS which is too big for the command buffer. Therefore I'm using the following method to create a file listing the objects (to pass to ar):
objects.lst:
$(foreach OBJ,$(OBJECTS),$(shell echo "$(OBJ)">>$#))
While this works it is extremely slow (on Cygwin at least) and I don't like relying on shell commands and redirection.
Additionlly foreach is not intended for this purpose - it is evaluated before any commands are run which means I can't for example rm -f objects.lst before appending.
Is there a better way? I don't want to use incremental archiving as that causes problems with multiple jobs.
The only thing I can think of is parsing the Makefile with a separate script to read the object list or storing the object list in a separate file. Both solutions have their own problems though.
Try something like:
OBJECTS:=a b c d
objects.lst:
echo > $# <<EOF $(OBJECTS)
i.e. make use of the <<EOF functionality that is built into the shell. It does not have any max-length limitations.
In the following example I also replaced echo with a simple Perl script to split the arguments onto new lines but this is the jist of it..
objects.lst:
echo $(wordlist 1,99,$(OBJECTS))>$#
echo $(wordlist 100,199,$(OBJECTS))>>$#
echo $(wordlist 200,299,$(OBJECTS))>>$#
echo $(wordlist 300,399,$(OBJECTS))>>$#
...
How about something like this:
OBJECTS_AM=$(filter a% b% c% d% e% f% g% h% i% j% k% l% m%,$(OBJECTS))
OBJECTS_NZ=$(filter-out a% b% c% d% e% f% g% h% i% j% k% l% m%,$(OBJECTS))
objects.lst:
$(shell echo "$(OBJECTS_AM)">$#)
$(shell echo "$(OBJECTS_NZ)">>$#)
You might need to split it one or two more times, but it's not that bad, especially as the distribution of file names doesn't change all that often.
Here's a patch to gnu make that lets you directly write a variable into a file.
It creates a new 'writefile' function, similar to the existing 'info' function, except it takes a filename argument and writes to the file:
https://savannah.gnu.org/bugs/?35384