Consider the following on a debian based system:
VAR=$(dpkg --get-selections | awk '{print $1}' | grep linux-image)
This will print a list of installed packages with the string "linux-image" in them on my system this output looks like:
linux-image-3.11.0-17-generic
linux-image-extra-3.11.0-17-generic
linux-image-generic
Now as we all know
echo $VAR
results in
linux-image-3.11.0-17-generic linux-image-extra-3.11.0-17-generic linux-image-generic
and
echo "$VAR"
results in
linux-image-3.11.0-17-generic
linux-image-extra-3.11.0-17-generic
linux-image-generic
I do not want to use external commands in a if clause, it seems rather dirty and not very elegant, so I wanted to use bash built in regex matching:
if [[ "$VAR" =~ ^linux-image-g ]]; then
echo "yes"
fi
however that does not work, since it does not seem to consider multiple lines here. How can I match beginnings of lines in a variable?
There's nothing wrong with using an external command as part of the if statement; I would skip the VAR variable altogether and use
if dpkg --get-selections | awk '{print $1}' | grep -q linux-image;
The -q option to grep suppresses its output, and the if statement uses the exit status of grep directly. You could also drop the grep and test $1 directly in the awk script:
if dpkg --get-selections | awk '$1 =~ "^linux-image" { exit 0; } END {exit 1}'; then
or you can skip awk, since there doesn't seem to be a real need to drop the other fields before calling grep:
if dpkg --get-selections | grep -q '^linux-image'; then
Related
I have a list of files paths that I need to compare with a string:
git_root_path=$(git rev-parse --show-toplevel)
list_of_files=.git/ForGeneratingSBConfigAlert.txt
cd $git_root_path
echo "These files needs new OSB config:"
while read -r line
do
modfied="$line"
echo "File for compare: $modfied"
if grep -qf $list_of_files" $modfied"; then
echo "Found: $modfied"
fi
done < <(git status -s | grep -v " M" | awk '{if ($1 == "M") print $2}')
$modified - is a string variable that stores path to file
Pattern file example:
SVCS/resources/
SVCS/bus/projects/busCallout/
SVCS/bus/projects/busconverter/
SVCS/bus/projects/Resources/ (ignore .jar)
SVCS/bus/projects/Teema/
SVCS/common/
SVCS/domain/
SVCS/techutil/src/
SVCS/tech/mds/src/java/fi/vr/h/service/tech/mds/exception/
SVCS/tech/mds/src/java/fi/vr/h/service/tech/mds/interfaces/
SVCS/app/cashmgmt/src/java/fi/vr/h/service/app/cashmgmt/exception/
SVCS/app/cashmgmt/src/java/fi/vr/h/service/app/cashmgmt/interfaces/
SVCS/app/customer/src/java/fi/vr/h/service/app/customer/exception/
SVCS/app/customer/src/java/fi/vr/h/service/app/customer/interfaces/
SVCS/app/etravel/src/java/fi/vr/h/service/app/etravel/exception/
SVCS/app/etravel/src/java/fi/vr/h/service/app/etravel/interfaces/
SVCS/app/hermes/src/java/fi/vr/h/service/app/hermes/exception/
SVCS/app/hermes/src/java/fi/vr/h/service/app/hermes/interfaces/
SVCS/app/journey/src/java/fi/vr/h/service/app/journey/exception/
SVCS/app/journey/src/java/fi/vr/h/service/app/journey/interfaces/
SVCS/app/offline/src/java/fi/vr/h/service/app/offline/exception/
SVCS/app/offline/src/java/fi/vr/h/service/app/offline/interfaces/
SVCS/app/order/src/java/fi/vr/h/service/app/order/exception/
SVCS/app/order/src/java/fi/vr/h/service/app/order/interfaces/
SVCS/app/payment/src/java/fi/vr/h/service/app/payment/exception/
SVCS/app/payment/src/java/fi/vr/h/service/app/payment/interfaces/
SVCS/app/price/src/java/fi/vr/h/service/app/price/exception/
SVCS/app/price/src/java/fi/vr/h/service/app/price/interfaces/
SVCS/app/product/src/java/fi/vr/h/service/app/product/exception/
SVCS/app/product/src/java/fi/vr/h/service/app/product/interfaces/
SVCS/app/railcar/src/java/fi/vr/h/service/app/railcar/exception/
SVCS/app/railcar/src/java/fi/vr/h/service/app/railcar/interfaces/
SVCS/app/reservation/src/java/fi/vr/h/service/app/reservation/exception/
SVCS/app/reservation/src/java/fi/vr/h/service/app/reservation/interfaces/
kraken_test.txt
namaker_test.txt
shmaker_test.txt
I need to compare file search pattern with a string, is it possible using grep?
I'm not sure I understand the overall logic, but a few immediate suggestions come to mind.
You can avoid grep | awk in the vast majority of cases.
A while loop with a grep on a line at a time inside the loop is an antipattern. You probably just want to run one grep on the whole input.
Your question would still benefit from an explanation of what you are actually trying to accomplish.
cd "$(git rev-parse --show-toplevel)"
git status -s | awk '!/ M/ && $1 == "M" { print $2 }' |
grep -Fxf .git/ForGeneratingSBConfigAlert.txt
I was trying to think of a way to add back your human-readable babble, but on second thought, this program is probably better without it.
The -x option to grep might be wrong, depending on what you are really hoping to accomplish.
This should work:
git status -s | grep -v " M" | awk '{if ($1 == "M") print $2}' | \
grep --file=.git/ForGeneratingSBConfigAlert.txt --fixed-strings --line-regexp
Piping the awk output directly to grep avoids the while loop entirely. In most cases you'll find you don't really need to print debug messages and the like in it.
--file takes a file with one pattern to match per line.
--fixed-strings avoids treating any characters in the patterns as special.
--line-regexp anchors the patterns so that they only match if a full line of input matches one of the patterns.
All that said, could you clarify what you are actually trying to accomplish?
here is what I want to do
while ["ps a | grep '[m]ono' | awk '{print $1}'" != ""] ; do
kill ps a | grep '[m]ono' | awk '{print $1}'
done
meaning if the grep returns nothing, don't try a kill.
The thing is I've always been lost with the expression evaluation in bash, sometime I use " " around something, sometime it's eval, sometime it's ''.
Could someone explain to me how to write the condition in my loop and explain the difference between the above? I'm used to find the working one with many tries and it feels like a huge loss of time.
Best choice: Do something else.
Utilities already exist for this purpose. Example:
killall mono
pkill mono
...or, even better, something targeted to the specific executable you want to terminate:
fuser -k /path/to/something.exe
...which would kill only programs with a file handle on that specific executable, rather than all applications running with mono on the machine.
...but, to explain the bugs:
There are two things wrong here: Missing command substitutions, and missing whitespace.
Missing whitespace:
["ps a | grep '[m]ono' | awk '{print $1}'" != ""]
...is literally trying to run a command with a name starting with [ps, as in, looking in the PATH for...
/bin/[ps\ a
/usr/bin/[ps\ a
...etc. [ is a command, and needs a space after its name like any other command. Thus:
[ "ps a | grep '[m]ono' | awk '{print $1}'" != "" ]
...fixes this problem (but leaves another one).
Missing command substitutions:
[ "ps a | grep '[m]ono' | awk '{print $1}'" != "" ]
...is comparing a string that starts with "ps a" to to ""; it does not compare the output of running a command that starts with ps a. To do that, you'd instead run:
[ "$(ps a | grep '[m]ono' | awk '{print $1}')" != "" ]
The content of $(...) is replaced with the output of the command within; thus, running your pipeline and comparing its output to an empty string.
I need to grep multiple strings, but i don't know the exact number of strings.
My code is :
s2=( $(echo $1 | awk -F"," '{ for (i=1; i<=NF ; i++) {print $i} }') )
for pattern in "${s2[#]}"; do
ssh -q host tail -f /some/path |
grep -w -i --line-buffered "$pattern" > some_file 2>/dev/null &
done
now, the code is not doing what it's supposed to do. For example if i run ./script s1,s2,s3,s4,.....
it prints all lines that contain s1,s2,s3....
The script is supposed to do something like grep "$s1" | grep "$s2" | grep "$s3" ....
grep doesn't have an option to match all of a set of patterns. So the best solution is to use another tool, such as awk (or your choice of scripting languages, but awk will work fine).
Note, however, that awk and grep have subtly different regular expression implementations. It's not clear from the question whether the target strings are literal strings or regular expression patterns, and if the latter, what the expectations are. However, since the argument comes delimited with commas, I'm assuming that the pieces are simple strings and should not be interpreted as patterns.
If you want the strings to be interpreted as patterns, you can change index to match in the following little program:
ssh -q host tail -f /some/path |
awk -v STRINGS="$1" -v IGNORECASE=1 \
'BEGIN{split(STRINGS,strings,/,/)}
{for(i in strings)if(!index($0,strings[i]))next}
{print;fflush()}'
Note:
IGNORECASE is only available in gnu awk; in (most) other implementations, it will do nothing. It seems that is what you want, based on the fact that you used -i in your grep invocation.
fflush() is also an extension, although it works with both gawk and mawk. In Posix awk, fflush requires an argument; if you were using Posix awk, you'd be better off printing to stderr.
You can use extended grep
egrep "$s1|$s2|$s3" fileName
If you don't know how many pattern you need to grep, but you have all of them in an array called s, you can use
egrep $(sed 's/ /|/g' <<< "${s[#]}") fileName
This creates a herestring with all elements of the array, sed replaces the field separator of bash (space) with | and if we feed that to egrep we grep all strings that are in the array s.
test.sh:
#!/bin/bash -x
a=" $#"
grep ${a// / -e } .bashrc
it works that way:
$ ./test.sh 1 2 3
+ a=' 1 2 3'
+ grep -e 1 -e 2 -e 3 .bashrc
(here is lots of text that fits all the arguments)
My program looks like this.
ALL=`cat $1 | grep -v '^#' | wc -l`
FINISHED="0"
for i in `cat $1 | grep -v '^#'`; do
echo "PROBE $i"
I will be doing some operation
FINISHED=`echo $FINISHED"+1"|bc`
I will run this script by giving a file name as parameter where a list of probes will be present.
I have 2 questions
What does grep -v '^#' mean. I learnt that '^ is usually used to matching a particular string. But in the file name which I give there is no #. Moreover I am getting the total number of probes for cat $1 | grep -v '^#' | wc -l.
echo $FINISHED"+1"|bc. Here any idea as to why the developer as added |bc?
^ means "start of line"
# is the literal character #
-v means "invert the match" in grep, in other words, return all non matching lines.
Put those together, and your expression is "select all lines that do not begin with #"
| is the pipe character, it takes the output of the command on the left hand side, and uses it as the input of the command on the right hand side. bc is like a command line calculator (to do basic math).
I would use this to exclude comments from the code I'm reading. So all comment lines start with # and I don't want to see them if there are too many of them.
grep -v '^#'
We have different ways for calculation. Pick the one which you like.
a=`echo 1+1 | bc`; echo $a
b=$((1+1)); echo $b
c=`expr 1 + 1`; echo $c
let d=1+1; echo $d
I am using GNU bash, version 4.2.20(1)-release (x86_64-pc-linux-gnu). I have a music file list I dumped into a variable: $pltemp.
Example:
/Music/New/2010s/2011;Ziggy Marley;Reggae In My Head
I wish to grep the 3rd field above, in the Master-Music-List.txt, then continue another grep for the 2nd field. If both matched, print else echo "Not Matched".
So the above will search for the Song Title (Reggae In My Head), then will make sure it has the artist "Shaggy" on the same line, for a success.
So far, success for a non-variable grep;
$ grep -i -w -E 'shaggy.*angel' Master-Music-MM-Playlist.m3u
$ if ! grep Shaggy Master-Music-MM-Playlist.m3u ; then echo "Not Found"; fi
$ grep -i -w Angel Master-Music-MM-Playlist.m3u | grep -i -w shaggy
I'm not sure how to best construct the 'entire' list to process.
I want to do this on a single line.
I used this to dump the list into the variable $pltemp...
Original: \Music\New\2010s\2011\Ziggy Marley - Reggae In My Head.mp3
$ pltemp="$(cat Reggae.m3u | sed -e 's/\(.*\)\\/\1;/' -e 's/\(.*\)\ -\ /\1;/' -e 's/\\/\//g' -e 's/\\/\//g' -e 's/.mp3//')"
If you realy want to "grep this, then grep that", you need something more complex than grep by itself. How about awk?
awk -F';' '$3~/title/ && $2~/artist/ {print;n=1;exit;} END {if(n=0)print "Not matched";}'
If you want to make this search accessible as a script, the same thing simply changes form. For example:
#!/bin/sh
awk -F';' -vartist="$1" -vtitle="$2" '$3~title && $2~artist {print;n=1;exit;} END {if(n=0)print "Not matched";}'
Write this to a file, make it executable, and pipe stuff to it, with the artist substring/regex you're looking for as the first command line option, and the title substring/regex as the second.
On the other hand, what you're looking for might just be a slightly more complex regular expression. Let's wrap it in bash for you:
if ! echo "$pltemp" | egrep '^[^;]+;[^;]*artist[^;]*;.*title'; then
echo "Not matched"
fi
You can compress this to a single line if you like. Or make it a stand-along shell script, or make it a function in your .bashrc file.
awk -F ';' -v title="$title" -v artist="$artist" '$3 ~ title && $2 ~ artist'
Well, none of the above worked, so I came up with this...
for i in *.m3u; do
cat "$i" | sed 's/.*\\//' | while read z; do
grep --color=never -i -w -m 1 "$z" Master-Music-Playlist.m3u \
| echo "#NotFound;"$z" "
done > "$i"-MM-Final.txt;
done
Each line is read (\Music\Lady Gaga - Paparazzi.mp3), the path is stripped, the song is searched in the Master Music List, if not found, it echos "Not Found", saved into a new playlist.
Works {Solved}
Thanks anyway.