If I have a list of things such as this:
nodes = A B C
How would I generate dynamic targets that have dynamic prerequisites. For example (this isn't working but might help explain what I want).
# node.csr is a file that already exists, like a template
# this would create outputs like node-A-csr.json, node-B-csr.json
# I am basically guessing at the syntax here
node-%-csr.json: node-csr.json
sed 's/$${node}/$*' node-csr.json > $#
# this would create outputs like node-A-key.pem node-A.pem and would require node-A-csr.json
node-%-key.pem node-%.pem: node-%-csr.json
# some command that generates node-NAME-key.pem and node-NAME-csr.pem
$(nodes): node-%-key.pem node-%.pem
Id basically like to be able to run make all and have those targets run for everything in my list.
I am very new to Makefiles and I just dont see how something like this would work and the documentation and syntax of Make is extremely confusing to me.
I am willing to use any tool to do this but seems like Make is very standard.
You can use Make's Substitution References to generate the 'all' target. This will be enough to start processing of all rules.
Notice minor change: the 'node-csr.json' should have the token NODE where the actual node name is to be inserted
# First rule is the default
default: all
nodes = A B C
# use node-csr as a template, replacing NODE with current ID: A, B, C
node-%-csr.json: node-csr.json
sed 's/__NODE__/$*/' node-csr.json > $#
# this would create node-A-key.pem node-A.pem from node-A-csr.json
# Put real commands here
node-%-key.pem node-%.pem: node-%-csr.json
ls -l node-$*-csr.json > node-$*-key.csr
ls -l node-$*-csr.json > node-$*.pem
# all will be node-A-key.pem, node-B-key.pem, ... node-C.pem
all: $(nodes:%=node-%-key.pem) $(nodes:%=node-%.pem)
echo "done"
# Using pathsubst: all: $(patsubst %,node-%-key.pem,$(nodes)) $(pathsubst
Please pay attention to tab/spaces, some version are sensitive. You will have to put back tabs before all commands (sed, ls, ...)
Substitution References man: https://www.gnu.org/software/make/manual/html_node/Substitution-Refs.html
pathsubst function: https://www.gnu.org/software/make/manual/html_node/Text-Functions.html
Related
I have a Makefile, trying to loop over a series of strings in a recipe and make them lower case.
My goal: I have a series of commands I would like to run on different files with the appropriate suffix.
# files in directory: test_file1.txt test_file2.txt test_file3.txt
MYLIST = file1 file2 file3
recipe:
for name in $(MYLIST) ; do \
$(eval FILENAME=`echo $($name) | tr A-Z a-z`) \
echo "Final name : test_${FILENAME}.txt" ; \
done
My problem, FILENAME always resolves to blank:
File name: test_.txt
I hope to see:
File name: test_file1.txt
File name: test_file2.txt
File name: test_file3.txt
You cannot mix make functions and shell commands like this. Make works like this: when it decides that your target is out of date and it wants to run the recipe, first it will expand the entire recipe string. Second it sends that expanded string to the shell to run.
So in your case, the $(eval ...) (which is a make operation) is expanded one time, then the resulting string is passed to the shell to run. The shell runs the for loop and all that stuff.
You have to use shell variables here to store values obtained by running your shell for loop. You cannot use make variables or make functions.
In general if you ever think about using $(shell ...) or $(eval ...) inside a recipe, you are probably going down the wrong road.
I have a make target, that first calls a CAE tool which generates reports. When this is done, the target calls a python script that shall take the content of the CAE reports (or more specific some grep'ed lines of the reports) as argument.
A minimum example is
target1:
date > ./bar.txt
echo $(shell cat ./bar.txt)
Problem is, that make expands the $(shell cat ./bar.txt) before the first command has been called and bar.txt has been updated. So for this minimum example, the echo prints the content of bar.txt before the update (the date from the previous target run).
(
I know that I simply could write this example in another way without variables and the shell function call, this is just for the sake of showing the problem where I call a tool that takes an argument from a shell call. So actually I want to do sth like this:
target1:
cae_tool_call
report_eval.py -text "$(shell cat $(generated_report) | grep 'foo')"
where cae_tool_call generates the generated_report. And this -text "argument" does not resolve the argument without an explicit call of the shell function.
)
I already tried with actual shell variables (instead of make variables), double escapes, immediate vs deferred variables but have no working solution yet. Any ideas?
#######################################
Edit to show some unexpected behavior:
I have this python script argument_example.py
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument("-r", "--reporttext", help="text string", required=True)
args=parser.parse_args()
if args.reporttext:
print(args.reporttext)
main()
It just prints the text given with argument -r.
And I have these two make targets:
####################################
#this does not work
REPORTNAME := ./bar.txt
variable_report:
date > $REPORTNAME
python3 ./argument_example.py --reporttext "`(cat $REPORTNAME)`"
####################################
#this works
static_report:
date > ./bar.txt
python3 ./argument_example.py --reporttext "`(cat ./bar.txt)`"
When calling variable_report, the python scripts prints the outdated bar.txt content. When calling static_report, the python script prints the updated content.
make recipes are already shell scripts. Never use the shell make function inside a recipe. In your first simple example use:
target1:
date > bar.txt
cat bar.txt
In your other example use:
generated_report := name-of-generated-report
target1:
cae_tool_call
report_eval.py -text "`cat $(generated_report) | grep 'foo'`"
Or even better:
generated_report := name-of-generated-report
target1:
cae_tool_call
report_eval.py -text "`grep 'foo' $(generated_report)`"
I read this question: Makefile: $subst in dependency list, but I still can't make my shell script work correctly.
I have a makefile with a line with the contents:
##public_detailed#|test_create|Syntax: commoncmdsyntax test_create test_name=<test-name>
A target runs a multiline bash script, where the commoncmdsyntax must be replaced by a string containing words and spaces.
In the script, I use cut to assign to a variable desc the following string:
Syntax: commoncmdsyntax test_create test_name=<test-name>
The problem is that commoncmdsyntax is not replaced by new text here:
$(subst commoncmdsyntax,new text,$$desc)
I also tried to replace it by a single word, like XX, but it also does not work.
The subst function (as in $(subst commoncmdsyntax,new text,$$desc)) is a Make function, so Make will perform the substitution before running any rule and therefore before your script assigns a value to desc. So even if secondary expansion worked the way you seem to think it will, this approach would still fail.
If you want to perform a substitution within something made by a shell script (in a recipe), the sensible way is to do so within the recipe:
echo $dest | sed 's/commoncmdsyntax/new text/'
We can give you a more detailed solution if you give us a minimal complete example of the problem.
I have a directory "FS2" that contains the following files:
ARGH
this
that
I have a makefile with the following contents.
Template:sh= ls ./FS2/*
#all: $(Template)
echo "Template is: $(Template)"
touch all
When I run "clearmake -C sun" and the file "all" does not exist, I get the following output:
"Template is: ./FS2/#ARGH# ./FS2/that ./FS2/this"
Modifying either "this" or "that" does not cause "all" to be regenerated. When run with "-d" for debug, the "all" target is only dependent on the directory "./FS2", not the three files in the directory. I determined that when it expands "Template", the "#" gets treated as the beginning of a comment and the rest of the line is ignored!
The problem is caused by an editor that when killed leaves around files that begin with "#". If one of those files exists, then no modifications to files in the directory causes "all" to be regenerated.
Although, I do not want to make compilation dependent on whether a temporary file has been modified or not and will remove the file from the "Template" variable, I am still curious as to how to get this to work if I did want to treat the "#ARGH#" as a filename that the rule "all" is dependent on. Is this even possible?
I have a directory "FS2" that contains the following files: #ARGH# ...
Therein lies your problem. In my opinion, it is unwise using "funny" characters in filenames. Now I know that those characters are allowed but that doesn't make them a good idea (ASCII control characters like backspace are also allowed with similar annoying results).
I don't even like spaces in filenames, preferring instead SomethingLikeThis to show independent words in a file name, but at least the tools for handling spaces in many UNIX tools is known reasonably well.
My advice would be to rename the file if it was one of yours and save yourself some angst. But, since they're temporary files left around by an editor crash, delete them before your rules start running in the makefile. You probably shouldn't be rebuilding based on an editor temporary file anyway.
Or use a more targeted template like: Template:sh= ls ./FS2/[A-Za-z0-9]* to bypass those files altogether (that's an example only, you should ensure it doesn't faslely exclude files that should be included).
'#' is a valid Makefile comment char, so the second line is ignored by the make program.
Can you filter out (with grep) the files that start with # and process them separately?
I'm not familiar with clearmake, but try replacing your template definition with
Template:sh= ls ./FS2/* | grep -v '#'
so that filenames containing # are not included in $(Template).
If clearmake follows the same rules as GNU make, then you can also re-write your target using something like Template := $(wildcard *.c) which will be a little more intelligent about files with oddball names.
If I really want the file #ARGH# to contribute to whether the target all should be rebuilt as well as be included in the artifacts produced by the rule, the Makefile should be modified so that the line
Template:sh= ls ./FS2/*
is changed to
Template=./FS2/*
Template_files:sh= ls $(Template)
This works because $(Template) will be replaced by the literal string ./FS2/* after all and in the expansion of $(Template_files).
Clearmake (and GNU make) then use ./FS2/* as a pathname containing a wildcard when evaluating the dependencies, which expands in to the filenames ./FS2/#ARGH# ./FS2/that ./FS2/this and $(Template_files) can be used in the rules where a list of filenames is needed.
I have a list of objects in a Makefile variable called OBJECTS which is too big for the command buffer. Therefore I'm using the following method to create a file listing the objects (to pass to ar):
objects.lst:
$(foreach OBJ,$(OBJECTS),$(shell echo "$(OBJ)">>$#))
While this works it is extremely slow (on Cygwin at least) and I don't like relying on shell commands and redirection.
Additionlly foreach is not intended for this purpose - it is evaluated before any commands are run which means I can't for example rm -f objects.lst before appending.
Is there a better way? I don't want to use incremental archiving as that causes problems with multiple jobs.
The only thing I can think of is parsing the Makefile with a separate script to read the object list or storing the object list in a separate file. Both solutions have their own problems though.
Try something like:
OBJECTS:=a b c d
objects.lst:
echo > $# <<EOF $(OBJECTS)
i.e. make use of the <<EOF functionality that is built into the shell. It does not have any max-length limitations.
In the following example I also replaced echo with a simple Perl script to split the arguments onto new lines but this is the jist of it..
objects.lst:
echo $(wordlist 1,99,$(OBJECTS))>$#
echo $(wordlist 100,199,$(OBJECTS))>>$#
echo $(wordlist 200,299,$(OBJECTS))>>$#
echo $(wordlist 300,399,$(OBJECTS))>>$#
...
How about something like this:
OBJECTS_AM=$(filter a% b% c% d% e% f% g% h% i% j% k% l% m%,$(OBJECTS))
OBJECTS_NZ=$(filter-out a% b% c% d% e% f% g% h% i% j% k% l% m%,$(OBJECTS))
objects.lst:
$(shell echo "$(OBJECTS_AM)">$#)
$(shell echo "$(OBJECTS_NZ)">>$#)
You might need to split it one or two more times, but it's not that bad, especially as the distribution of file names doesn't change all that often.
Here's a patch to gnu make that lets you directly write a variable into a file.
It creates a new 'writefile' function, similar to the existing 'info' function, except it takes a filename argument and writes to the file:
https://savannah.gnu.org/bugs/?35384