Extracting a substring from a variable in makefile - bash

Basically, I have to match all targets of the form : "tests/mytests-runthis"
I have a rule that does this:
life-tests/%:
./(?????) < $#.in > runthis.tmp
-diff runthis.tmp $#.out
i need everything after the dash to go in the area "(?????)", so it would run
./runthis < mytests-runthis > runthis.tmp
How can I extract everything after the dash from what i would get from $#?

You can do this with patsubst:
$ cat Makefile
life-tests/%:
#./$(patsubst life-tests/%,%,$#).sh
$ cat hello.sh
#! /bin/bash
echo world
$ make life-tests/hello
world

The automatic variable $* expands to the string that matched the pattern (%):
life-tests/%:
./$* < $#.in > runthis.tmp
-diff runthis.tmp $#.out

Related

How to concatenate variable in Makefile recipe

I get this Makefile:
LIST = foo bar
$(LIST):
echo $#
When I run make targets the outputs works as desired:
$ make foo
echo foo
foo
$ make bar
echo bar
bar
But if I concatenate a string another-, only another-foo works:
LIST = foo bar
another-$(LIST):
echo $#
$ make another-foo
echo another-foo
another-foo
$ make another-bar
make: *** No rule to make target 'another-bar'. Stop.
How can I concatenate a target to expand to all values in my variable?
The behavior you observe is standard: when writing
another-$(LIST)
make just replaces $(LIST) with its content, yielding another-foo bar.
That is, this has nothing to do with, e.g., a bash brace expression such as another-{foo,bar}.
Yet, you can achieve what you want by doing something like:
LIST = foo bar
$(addprefix another-,$(LIST)):
echo "$#"

Makefile: How to expand a variable in a shell function correctly?

I have a reoccurring piece of code in my makefile, which I want to put in a function. A simplified version of my code looks as follows:
IDS=4 5
MY_FUNC = $(shell echo "max=2; \
counter=1;\
while [ \$$counter -le \$$max ]; do\
id=$$(echo $(IDS) | cut -d" " -f \$${counter}); \
counter=\$$((counter+1)); \
done");
.PHONY: all
all:
#$(call MY_FUNC)
Unfortunately, cut interprets $counter literally and consequently throws the error message cut: invalid field value ‘$counter’.
I do not understand why this is the case, since the command \$$command accesses the value stored inside this variable. Do you may know how to properly call pass counter to cut?
Here is a fixed form of your Makefile.
IDS=4 5
MY_FUNC = $(shell echo 'max=2; \
counter=1; \
while [ $$counter -le $$max ]; do\
id=$$(echo $(IDS) | cut -d" " -f $${counter}); \
counter=$$((counter+1)); \
echo "debug: id: $$id"; \
done');
.PHONY: all
all:
#$(call MY_FUNC)
I have added a echo "debug: id: $$id"; command to help with debugging and prove that the script is behaving as intended. Here is the output:
$ make
debug: id: 4
debug: id: 5
Here are the important points worth noting in the fixed script:
The outermost delimiter for the argument to the outermost echo statement is single-quote (not double-quotes) in order to prevent the $counter, $max, etc. from expanding to empty strings when the echo statement is being executed by the $(shell echo ...) call from Makefile. This also allows proper nesting of the double-quotes used in cut -d" " within the outer single-quotes.
Now that we are using single-quotes as the outer delimiters, the $ symbols within (escaped as $$ in Makefile) need not be escaped with \ anymore.
If however you want to stick with double-quotes as the outermost delimiter, then the alternative solution with minimal changes to your code looks like this:
IDS=4 5
MY_FUNC = $(shell echo "max=2; \
counter=1;\
while [ \$$counter -le \$$max ]; do\
id=\$$(echo $(IDS) | cut -d\" \" -f \$${counter}); \
counter=\$$((counter+1)); \
echo \"debug: id: \$$id\"; \
done");
.PHONY: all
all:
#$(call MY_FUNC)
Once again the output is:
$ make
debug: id: 4
debug: id: 5
Here are the important points to note in this alternative solution:
All $ symbols for the shell (escaped as $$ in Makefile) need to be escaped carefully. They should all occur as \$$ in Makefile. In your code, this was missing for $$(echo. This has been fixed to \$$(echo.
Further all double-quotes within the outer double-quotes need to be carefully escaped as \", so cut -d" " should be written as cut -d\" \".
Susam Pal's answer explains why your use of double quotes was wrong. Use one or the other of the two proposed solutions.
And then, there are a few other aspects you could consider:
You are using the $(shell...) make function in a recipe which does not realy make sense: recipes are already shell scripts. And you do not need the call function neither. A simpler Makefile could be:
IDS = 4 5
define MY_FUNC
max=2; \
counter=1; \
while [ $$counter -le $$max ]; do \
id=$$(echo $(IDS) | cut -d" " -f $${counter}); \
counter=$$((counter+1)); \
done
endef
.PHONY: all
all:
#$(MY_FUNC)
As your recipe has no side effect it is not very useful. But I guess you know it already and this was just an example.
Hard-wiring the number of items in IDS (max=2) in your recipe is not optimal. If you are using GNU make you could use its words function:
max=$(words $(IDS)); \
There are much simpler ways to achieve what you want with the shell. Assuming you just want to print the id values:
for id in $(IDS); do; \
echo $$id; \
done
is easier. But I guess you know it already and this was just an example.
If you want to use the call function you could pass it a parameter (the current id) and iterate with the foreach make function instead of using a shell loop:
IDS = 4 5
define MY_FUNC
echo $(1)
endef
.PHONY: all
all:
$(foreach id,$(IDS),#$(call MY_FUNC,$(id)))
Note the empty last line of MY_FUNC. It is needed to obtain a true multi-line recipe. Alternate solution with a single-line recipe:
IDS = 4 5
MY_FUNC = echo $(1)
.PHONY: all
all:
#$(foreach id,$(IDS),$(call MY_FUNC,$(id));)
GNU make offers many handy functions and has many very useful features. In your case (and assuming you just want to print each id on standard output) you could use patsubst to create a list of phony targets, one per word in IDS and write a static pattern rule for all of them:
IDS = 4 5
ALLS = $(patsubst %,all-%,$(IDS))
.PHONY: all $(ALLS)
all: $(ALLS)
$(ALLS): all-%:
#echo $*
An advantage of this last solution is that your ids are distributed to as many independent rules (the all-X) and their recipes can be run in parallel by make if you allow it to do so (make -j) while with single rule solutions they necessarily run sequentially.

Removing substring from string variable in bash

I have a bash string variable:
switches="-r cc -c 1,2,3,4 -u"
where numbers 1,2,3,4 can be any integer, like:
-c 25,45,78,34.
Moreover, it can be with fewer numbers, like:
-c 1
-c 1,2
or
-c 1,2,3,4
It can't be like:
-c 1,2,3
So -c can have one, two, or four integers only.
I forgot to mention that that this pattern can appeares also at the beginning, or at the end of the string variable $switches too, like:
-r cc -u -c 1,2,3,4
-r cc -u -c 1,2,3
And one more thing: this pattern can be appeared in the $switches variable only once.
How can I remove the '-c 1,2,3,4 ' part of switches variable using just bash?
I tried with this:
switches=${switches/ -c /}
but get this:
-r cc1,2,3,4 -u
I expect this:
-r cc -u
Best, Pal
Using extglob:
shopt -s extglob # enables extended globbing
switches=${switches//-c *([^ ])}
*([^ ]): matches any number of non-spaces
This will leave you with unnecessary spaces. More complicated solution:
switches=${switches//-c *([^ ])*( )}
switches=${switches/%*( )}
*([^ ])*( ): matches any number of non-spaces and any number of spaces after
${switches/%*( )}: if the last option is also -c, the code above wouldn't remove the spaces left by it. /%*( ) removes any number of spaces from the end

Why does AWK not work correctly in a makefile? [duplicate]

This question already has answers here:
Escaping in makefile
(2 answers)
Closed 7 years ago.
NOTICE: Escaping not problem, sample from shell is only sample, in Makefile $$.
GNU Makefile man says why it's doesn't work:
Note that expansion using ‘%’ in pattern rules occurs after any
variable or function expansions, which take place when the makefile is
read.
--Orig. question:
In pure shell, the next script works correctly:
echo "test2.cpp src2/test2.cpp src1/test1.cpp src1/test.cpp" | \
awk 'BEGIN{RS=" "}{if(NR == 1) f=$0; else if(match($0, f)) print $0;}'
Filter is first: test1.cpp
And it returns: src1/test1.cpp
But in Makefile it does not work correctly (error compared to awk):
OBJ_DIR:=obj
SOURCES:=$(wildcard */*.cpp *.cpp)
OBJECTS_LOCAL:= $(addprefix $(OBJ_DIR)/, $(notdir $(SOURCES:.cpp=.o)))
LOCAL_PATHS_HEADERS:=$(sort $(dir $(wildcard *.h */*.h)))
TARGET:=libcommon.a
all:$(TARGET)
$(TARGET): $(OBJECTS_LOCAL)
ar -rcs $# $^
$(OBJECTS_LOCAL): $(OBJ_DIR)/%.o : $(shell echo %.cpp $(SOURCES) | awk 'BEGIN{RS=" "}{if(NR == 1) f=$$0; else if($$0 ~ f) print $$0;}' )
#mkdir -p $(OBJ_DIR)
#$(CC) -c $< -o $# $(addprefix -I,$(LOCAL_PATHS_HEADERS))
So I take simple in Makefile and check value f, and found some strange length of f
...%.cpp $(SOURCES) | awk '{print ("file1.cpp" ~ $$1)"."$$1"."length($$1)}' )
awk return fail in compared;
print returns "0.file1.cpp.5" to fail with length, because it has forgotten .cppvalue of %, info bellow. I attempted to correct it:
...%.cpp $(SOURCES) | awk 'BEGIN{RS=" "}{if(NR == 1) f=$$0".cpp"; print ("file1.cpp.cpp" ~ f)"."("file1.cpp" ~ f)"."f"."length(f)}' )
but awk return fail in all compared; print returns "0.0.file1.cpp.cpp.9".
I check awk in manual mode, like this:
...%.cpp $(SOURCES) : $(shell echo %.cpp $(SOURCES) | awk '{print "src/"$$1}' )
It works fine, but it isn't variant, because it will kill automatic mode.
--Add
Information about lost length from parameter % to AWK
...%.cppppppp $(SOURCES) | awk '{print ("file1.cpp" ~ $$1)"."$$1"."length($$1)}' )
print returns "0.test2.cppppppp.10"
--Upd, some problem
Above, I was printing return value from $<
But file redirect show that value % does not work in prerequisites(file redirect: "0.%.cpp.5").
Can I use any automatic variable with value in prerequisites?
Almost invariably, when a question is asked about awk in a Makefile, the solution is to properly escape the $ symbols. It's not entirely clear what your question is, but there are some substantial misunderstandings that need to be resolved. In particular, the following "works", but hardly for the reasons you think:
echo "test2.cpp src2/test2.cpp src1/test1.cpp src1/test.cpp" | \
awk 'BEGIN{RS=" "}{if(NR == 1) f=$$0; else if(match($$0, f)) print $$0;}'
You almost certainly do not want $$ in any of the cases they appear here. awk is generally looking for single dollar signs, and when they appear in a Makefile, they are doubled because Make parses the $$ and invokes awk with a single $. In the quoted sample, $$0 on the first record is equivalent to $test2.cpp, but the variable test2.cpp is uninitialized and so has value 0, so on the first pass f is set to the value of $0 (the string "test2.cpp").
In short, if you are invoking awk from the shell, use single $. In the Makefile, use $$ and awk will only see $.

automate rules in makefile from two variables at the same time

I have some search patterns that fits poorly as part of a file name for the result. I therefore split the regular expression and the corresponding file name part into two different variables like below. How can I automate the following so that I do not have to manually list files for ALL and also do not have to manually enter the rules running grep?
SEARCH_RE = test a/b/c a.*b
SEARCH_FILE = test abc ab
ALL = result.test result.abc result.ab
all: $(ALL)
result.test:
grep test inputfile > result.test
result.abc:
grep a/b/c inputfile > result.abc
result.ab
grep a.*b inputfile > result.ab
I recommend
ALL = $(addprefix result.,$(SEARCH_FILE))
As for writing the rules, the kind of lookup I think you want can be done in Make, but really shouldn't be-- it would be a horrible kludge. I'd suggest doing it this way:
result.test: TARG = test
result.abc: TARG = a/b/c
result.ab: TARG = a.*b
$(ALL):
grep $(TARG) inputfile > $#
I don't know how to create the rules, but the ALL target is easy enough:
ALL = $(patsubst %,result.%,$(SEARCH_FILE))

Resources