I'm probably being stupid, but I cannot do the most basic variable assignment in my Makefile.
TEST = $(pwd);
all:
echo $(TEST)
When I run "make all" from Bash (v4.2.42) in FreeBSD (v9.1) I get:
echo
No idea what I'm screwing up. I've also tried to assign the variable using $(shell ...) with the same result.
If I use back ticks (`) then basic assignment works, but it doesn't store the result, it stores the command. This breaks in the following example Makefile:
SERVERIP = `ifconfig em0 | grep -E 'inet.[0-9]' | awk '{ print $$2}'`
all:
echo $(SERVERIP)
sed -e 's/%LISTENIP%/${SERVERIP}/g' test.conf > work/test.conf.tmp
The result is:
[pete#pete] ~/make
echo `ifconfig em0 | grep -E 'inet.[0-9]' | awk '{ print $2}'`
10.128.28.151
sed -e 's/%LISTENIP%/`ifconfig em0 | grep -E 'inet.[0-9]' | awk '{ print $2}'`/g' test.conf > work/test.conf.tmp
sed: 1: "s/%LISTENIP%/`ifconfig ...": unescaped newline inside substitute pattern
*** [all] Error code 1
You can see that basic variable assignment appears to work, but when you insert the result into sed, it inserts the whole command rather than the result, which breaks!
Any ideas?
Pete.
Maybe the answer is too late but I hope I can help (somebody else).
If you don't want to install GNU's make you can use the !=:
TEST!=pwd
all:
echo ${TEST}
will work. The explanation is very simple, please read carefully FreeBSD's man make, especially the "Variable assignment modifiers" inside the "VARIABLE ASSIGNMENTS" section:
Variable assignment modifiers
The five operators that can be used to assign values to variables are
as follows: ...
!= Expand the value and pass it to the shell for execution and
assign the result to the variable. Any newlines in the result
are replaced with spaces.
The makefile contains:
TEST = $(pwd);
all:
echo $(TEST)
The first line assigns the value in the make variable pwd plus a semicolon to make variable TEST. The rule then echoes that. You forgot, I think, that $(...) is significant to both make and shell. Assuming you want to echo the output of the pwd command, then use a double-dollar so make expands that to one dollar:
TEST = $$(pwd)
all:
echo $(TEST)
To get the output of a shell command into a make variable, you have to use GNU make extensions. There are two extensions in use: the := immediate assignment that evaluates the value of SERVERIP at the point of definition, rather than at the point of use; and $(shell ...cmd...) that is used to run a shell command and capture its output.
SERVERIP := $(shell ifconfig em0 | grep -E 'inet.[0-9]' | awk '{ print $$2}')
all:
echo ${SERVERIP}
sed -e 's/%LISTENIP%/${SERVERIP}/g' test.conf > work/test.conf.tmp
This should work. The original error, about a newline in the sed regular expression, is odd; the back-ticks should have removed the trailing newline from the output of the ifconfig pipeline. However, if there were in fact multiple lines returned, then you'd still have a newline inside, justifying the complaint from sed.
This generates the complaint:
SERVERIP = `printf "%s\n" a b c`
all:
echo $(SERVERIP)
sed -e "s/%LISTENIP%/${SERVERIP}/g" test.conf > work/test.conf.tmp
This works:
SERVERIP = `printf "%s\n" c`
all:
echo $(SERVERIP)
sed -e "s/%LISTENIP%/${SERVERIP}/g" test.conf > work/test.conf.tmp
Since you're using FreeBSD, it is quite possible that you don't have GNU make. In that case, you will need to:
Make sure the ifconfig command pipeline only produces one line of output.
Use make commands like this:
SERVERIP = `ifconfig em0 | grep -E 'inet.[0-9]' | awk '{ print $$2}'`
all:
echo ${SERVERIP}
sed -e "s/%LISTENIP%/${SERVERIP}/g" test.conf > work/test.conf.tmp
That should work with any version of make provided condition 1 is met.
This is a test makefile, similar to yours. The ifconfig command has been placed in a $(shell ...) command and the result is stored in SERVERIP.
TOP:= $(shell pwd)
SERVERIP:= $(shell ifconfig en1 | grep -E 'inet.[0-9]' | awk '{ print $$2}')
LISTENIP:=240.12.4.63
all:
echo $(SERVERIP)
sed -e 's/$(LISTENIP)/$(SERVERIP)/g' test.conf > work/test.conf.tmp
Related
I want to turn this input_variable = 1
into input_variable = 01
From previous posts here I tried this but didn't work:
sed -e "s/\0" <<< "$input_variable"
I get:
Syntax error: redirection unexpected
What do I do wrong?
Thanks!
EDIT
Thanks to Benjamin I found a workaround (I would still like to know why the sed didn't work):
new_variable="0$input_variable"
While it can be done with sed, simple assignment in your script can do exactly what you want done. For example, if you have input_variable=1 and want input_variable=01, you can simply add a leading 0 by assignment:
input_variable="0${input_variable}"
or for additional types of numeric formatting you can use the printf -v option and take advantage of the format-specifiers provided by the printf function. For example:
printf -v input_variable "%02d" $input_variable
will zero-pad input_variable to a length of 2 (or any width you specify with the field-width modifier). You can also just add the leading zero regardless of the width with:
printf -v input_variable "0%s" $input_variable
sed is an excellent tool, but it isn't really the correct tool for this job.
You don't close the substitution command. Each substitution command must contain 3 delimiters
sed -e 's/pattern/replacement/' <<< 'text' # 3 backslashes
What you want to do could be done with:
sed -e 's/.*/0&/' <<< $input_variable
EDIT:
You are probably using Ubuntu and stumbled upon dash also known as the Almquist shell, which does not have the <<< redirection operator. The following would be a POSIX-compliant alternative, which works with dash as well:
sed -e 's/.*/0&/' <<~
$input_variable
~
And also this:
echo $input_variable | sed -e 's/.*/0&/'
To have the variable take on the new value, do this:
input_variable=$(echo $input_variable | sed -e 's/.*/0&/')
That's however not how you would write the shell script. Shell scripts usually give out some textual output, rather than setting external variables:
So, the script, let's call it append_zero.sh:
#!/bin/sh
echo $1 | sed 's/.*/0&/'
and you would execute it like this:
$ input_variable=1
$ input_variable=$(append_zero.sh input_variable)
$ echo $input_variable
01
This way you have a working shell script that you can reuse with any Unix system that has a POSIX compliant /bin/sh
I need to grep multiple strings, but i don't know the exact number of strings.
My code is :
s2=( $(echo $1 | awk -F"," '{ for (i=1; i<=NF ; i++) {print $i} }') )
for pattern in "${s2[#]}"; do
ssh -q host tail -f /some/path |
grep -w -i --line-buffered "$pattern" > some_file 2>/dev/null &
done
now, the code is not doing what it's supposed to do. For example if i run ./script s1,s2,s3,s4,.....
it prints all lines that contain s1,s2,s3....
The script is supposed to do something like grep "$s1" | grep "$s2" | grep "$s3" ....
grep doesn't have an option to match all of a set of patterns. So the best solution is to use another tool, such as awk (or your choice of scripting languages, but awk will work fine).
Note, however, that awk and grep have subtly different regular expression implementations. It's not clear from the question whether the target strings are literal strings or regular expression patterns, and if the latter, what the expectations are. However, since the argument comes delimited with commas, I'm assuming that the pieces are simple strings and should not be interpreted as patterns.
If you want the strings to be interpreted as patterns, you can change index to match in the following little program:
ssh -q host tail -f /some/path |
awk -v STRINGS="$1" -v IGNORECASE=1 \
'BEGIN{split(STRINGS,strings,/,/)}
{for(i in strings)if(!index($0,strings[i]))next}
{print;fflush()}'
Note:
IGNORECASE is only available in gnu awk; in (most) other implementations, it will do nothing. It seems that is what you want, based on the fact that you used -i in your grep invocation.
fflush() is also an extension, although it works with both gawk and mawk. In Posix awk, fflush requires an argument; if you were using Posix awk, you'd be better off printing to stderr.
You can use extended grep
egrep "$s1|$s2|$s3" fileName
If you don't know how many pattern you need to grep, but you have all of them in an array called s, you can use
egrep $(sed 's/ /|/g' <<< "${s[#]}") fileName
This creates a herestring with all elements of the array, sed replaces the field separator of bash (space) with | and if we feed that to egrep we grep all strings that are in the array s.
test.sh:
#!/bin/bash -x
a=" $#"
grep ${a// / -e } .bashrc
it works that way:
$ ./test.sh 1 2 3
+ a=' 1 2 3'
+ grep -e 1 -e 2 -e 3 .bashrc
(here is lots of text that fits all the arguments)
I'm trying to get the version of my compilation from a text file. I'm using this command
grep -w -m 1 "V1" server.h | sed $(VERSION) 's/#define V1[\t]//'
It works fine but now I'm trying to execute it from my Makefile using shell:
VERSION=$(shell grep -w -m 1 "V1" server.h | sed $(VERSION) 's/#define V1[\t]//')
but I'm not able to make it works because of the |. If I only put one command, grep for example it runs fine, there's another way to indicate the | to concatenate expressions? Or how else can I do this?
Edit:
Hi, thanks for the answer,
Reading your answer i realize that copy/paste has betrayed me jeje, this is the right expression i'm using:
VERSION:=$(shell grep -w -m 1 "V2" server.h | sed 's/#define V2[\t]//')
And this is the output of Makefile:
unterminated call to function shell': missing)'. Stop.
I have tested your recommendations and it also fails with the same Error.
Thanks.
It has nothing to do with the pipe. The reason it is not working is because you have defined it as a dynamic macro. By using '=' it will re-evaluate the macro on each reference. You have essentially created a recursive macro by trying to overload the VERSION variable as both the version and the filename. Try using different variable names and make it static:
VERSION_NUMBER:=$(shell grep -w -m 1 "V1" server.h | sed $(VERSION) 's/#define V1[\t]//')
$(error VERSION_NUMBER=$(VERSION_NUMBER))
Remove the $(error) once you have it working. Also, awk might be more efficient in this case anyway:
VERSION_NUMBER:=$(shell awk '/\<V1\>/ { print gensub(/^\#define V1[[:space:]]+/, "", ""); exit }' $(VERSION))
You also have the problem of the hash (pound for you US fellas) #. It is terminating the expression as a comment. Try this:
VERSION:=$(shell grep -w -m 1 "V2" server.h | sed 's/\#define V2[\t]//')
Or this:
VERSION:=$(shell awk '/\<V1\>/ { print gensub(/^\#define V1[[:space:]]+/, "", ""); exit }' server.h)
You will have this problem with many characters in make. The Dollar being the most problematic to escape. I have seen expressions like this:
V:=$(foreach f,$(list),$(eval $(shell create_rule $$$$(f))))
Sometimes it is better to write a shell script and call that:
script.sh:
#!/bin/sh
awk '/\<V1\>/ { print gensub(/^#define V1[[:space:]]+/, "", ""); exit }' "$#"
Makefile:
VERSION=$(shell script.sh server.h)
I wanted to print the name from the entire address by shell scripting. So user1#12.12.23.234 should give output "user1" and similarly 11234#12.123.12.23 should give output 11234
Reading from the terminal:
$ IFS=# read user host && echo "$user"
<user1#12.12.23.234>
user1
Reading from a variable:
$ address='user1#12.12.23.234'
$ cut -d# -f1 <<< "$address"
user1
$ sed 's/#.*//' <<< "$address"
user1
$ awk -F# '{print $1}' <<< "$address"
user1
Using bash in place editing:
EMAIL='user#server.com'
echo "${EMAIL%#*}
This is a Bash built-in, so it might not be very portable (it won't run with sh if it's not linked to /bin/bash for example), but it is probably faster since it doesn't fork a process to handle the editing.
Using sed:
echo "$EMAIL" | sed -e 's/#.*//'
This tells sed to replace the # character and as many characters that it can find after it up to the end of line with nothing, ie. removing everything after the #.
This option is probably better if you have multiple emails stored in a file, then you can do something like
sed -e 's/#.*//' emails.txt > users.txt
Hope this helps =)
I tend to use expr for this kind of thing:
address='user1#12.12.23.234'
expr "$address" : '\([^#]*\)'
This is a use of expr for its pattern matching and extraction abilities. Translated, the above says: Please print out the longest prefix of $address that doesn't contain an #.
The expr tool is covered by Posix, so this should be pretty portable.
As a note, some historical versions of expr will interpret an argument with a leading - as an option. If you care about guarding against that, you can add an extra letter to the beginning of the string, and just avoid matching it, like so:
expr "x$address" : 'x\([^#]*\)'
I am using GNU bash, version 4.2.20(1)-release (x86_64-pc-linux-gnu). I have a music file list I dumped into a variable: $pltemp.
Example:
/Music/New/2010s/2011;Ziggy Marley;Reggae In My Head
I wish to grep the 3rd field above, in the Master-Music-List.txt, then continue another grep for the 2nd field. If both matched, print else echo "Not Matched".
So the above will search for the Song Title (Reggae In My Head), then will make sure it has the artist "Shaggy" on the same line, for a success.
So far, success for a non-variable grep;
$ grep -i -w -E 'shaggy.*angel' Master-Music-MM-Playlist.m3u
$ if ! grep Shaggy Master-Music-MM-Playlist.m3u ; then echo "Not Found"; fi
$ grep -i -w Angel Master-Music-MM-Playlist.m3u | grep -i -w shaggy
I'm not sure how to best construct the 'entire' list to process.
I want to do this on a single line.
I used this to dump the list into the variable $pltemp...
Original: \Music\New\2010s\2011\Ziggy Marley - Reggae In My Head.mp3
$ pltemp="$(cat Reggae.m3u | sed -e 's/\(.*\)\\/\1;/' -e 's/\(.*\)\ -\ /\1;/' -e 's/\\/\//g' -e 's/\\/\//g' -e 's/.mp3//')"
If you realy want to "grep this, then grep that", you need something more complex than grep by itself. How about awk?
awk -F';' '$3~/title/ && $2~/artist/ {print;n=1;exit;} END {if(n=0)print "Not matched";}'
If you want to make this search accessible as a script, the same thing simply changes form. For example:
#!/bin/sh
awk -F';' -vartist="$1" -vtitle="$2" '$3~title && $2~artist {print;n=1;exit;} END {if(n=0)print "Not matched";}'
Write this to a file, make it executable, and pipe stuff to it, with the artist substring/regex you're looking for as the first command line option, and the title substring/regex as the second.
On the other hand, what you're looking for might just be a slightly more complex regular expression. Let's wrap it in bash for you:
if ! echo "$pltemp" | egrep '^[^;]+;[^;]*artist[^;]*;.*title'; then
echo "Not matched"
fi
You can compress this to a single line if you like. Or make it a stand-along shell script, or make it a function in your .bashrc file.
awk -F ';' -v title="$title" -v artist="$artist" '$3 ~ title && $2 ~ artist'
Well, none of the above worked, so I came up with this...
for i in *.m3u; do
cat "$i" | sed 's/.*\\//' | while read z; do
grep --color=never -i -w -m 1 "$z" Master-Music-Playlist.m3u \
| echo "#NotFound;"$z" "
done > "$i"-MM-Final.txt;
done
Each line is read (\Music\Lady Gaga - Paparazzi.mp3), the path is stripped, the song is searched in the Master Music List, if not found, it echos "Not Found", saved into a new playlist.
Works {Solved}
Thanks anyway.