Finding strings in a file and write to a file if found - bash

Just wondering if someone can point me in the right direction. I want to search a file for 2 strings and if they match add a YES to a csv file or NO if they don't. For now tough I'd like to just have it find the strings and print SUCCESS on the screen if found as per below (I'd rather try get the writing to a file done myself so I learn).
I do need both strings to be a match otherwise the result should be No.
I've gotten this so far:
#/usr/bin/bash
file=test.xml
string1=TEST1
string2=TEST2
if grep -e "$string1|$string2" "$file"
then
echo IT WORKS!!!
else
echo UH OH!!!
fi

Using grep you would need to read the file twice:
if grep "$string1" "$file" && grep "$string2" "$file"
However, using AWK, you can set flags and you'll only need to read the file once:
awk -v s1="$string1" -v s2="$string2" '$0 ~ s1 {f1 = 1} $0 ~ s2 {f2 = 1} END {if (f1 && f2) {print "Yes"} else {print "No"}}' "$file"

Presuming that either string matching is sufficient, try this:
file=test.xml
string1="TEST1"
string2="TEST2"
if grep -q -F "$string1"$'\n'"$string2" "$file" ; then
printf YES
else
printf NO
fi
Importantly, -F means that if the string happens to contain regex-special characters they will be treated literally. For convenience -q will suppress the output of grep so you only get the YES/NO returned.
EDIT:
Since you need both strings to match, try this:
if [ $(sed -n -e "/${string1}/p;/${string2}/p" "$file" | wc -l) -ge 2 ] ; then
printf YES
else
printf NO
fi
This is not as safe as the grep version since regex characters in the strings are not treated literally and is rather inefficient, but it gets the job done.

Since you want both strings to match, an inefficient way is to call grep once for each pattern:
if grep -q "$string1" && grep -q "$string2"
then
echo IT WORKS
else
echo UH OH
fi
I'm not aware of a way to do this with a single grep call without a fairly complicated regex. Something like
grep -qE "$string1.*$string2|$string2.*$string1"
could work, but I'm sure that would include some false positives and/or false negatives, depending on the values of string1 and string2.

Your code is in the good direction. You need to change -e with -E in order for grep to recognize the regular expression that you are using ($string1|$string2). You may also want to avoid seeing the actual matchings (use quiet -q flag for that):
file=test.xml
string1="TEST1"
string2="TEST2"
if grep -E -q "$string1|$string2" "$file"
then
echo IT WORKS!!!
else
echo UH OH!!!
fi
EDIT: Since you need to match both strings, this is not possible with grep (AFAIK) unless you execute grep multiple times (like in #chepner's answer).
An alternative solution would be to use awk:
awk '/'$string1'/{s1=1}/'$string2'/{s2=1} END { \
if (s1 && s2) print "IT WORKS!!!" ; else print "UH OH!!!" }' "$file"

Related

How to filter an ordered list stored into a string

Is it possible in bash to filter out a part of a string with another given string ?
I have a fixed list of motifs defined in a string. The order IS important and I want to keep only the parts that are passed as a parameter ?
myDefaultList="s,t,a,c,k" #order is important
toRetains="k,t,c,u" #provided by the user, order is not enforced
retained=filter $myDefaultList $toRetains # code to filter
echo $retained # will print t,c,k"
I can write an ugly method that will use IFS, arrays and loops, but I wonder if there's a 'clever' way to do that, using built-in commands ?
here is another approach
tolines() { echo $1 | tr ',' '\n'; }
grep -f <(tolines "$toRetains") <(tolines "$myDefaultList") | paste -sd,
will print
t,c,k
assign to a variable as usual.
Since you mention in your comments that you are open to sed/awk , check also this with GNU awk:
$ echo "$a"
s,t,a,c,k
$ echo "$b"
k,t,c,u
$ awk -v RS=",|\n" 'NR==FNR{a[$1];next}$1 in a{printf("%s%s",$1,RT)}' <(echo "$b") <(echo "$a")
t,c,k
#!/bin/bash
myDefaultList="s,t,a,c,k"
toRetains="s,t,c,u"
IFS=","
for i in $myDefaultList
do
echo $toRetains | grep $i > /dev/null
if [ "$?" -eq "0" ]
then
retained=$retained" "$i
fi
done
echo $retained | sed -e 's/ /,/g' -e 's/,//1'
I have checked it running for me. Kindly check.

bash: sed: unexpected behavior: displays everything

I wrote what I thought was a quick script I could run on a bunch of machines. Instead it print what looks like might be directory contents in a recursive search:
version=$(mysql Varnish -B --skip-column-names -e "SELECT value FROM sys_param WHERE param='PatchLevel'" | sed -n 's/^.*\([0-9]\.[0-9]*\).*$/\1/p')
if [[ $(echo "if($version == 6.10) { print 1; } else { print 0; }" | bc) -eq 1 ]]; then
status=$(dpkg-query -l | awk '{print $2}' | grep 'sg-status-polling');
cons=$(dpkg-query -l | awk '{print $2}' | grep 'sg-consolidated-poller');
if [[ "$status" != "" && "$cons" != "" ]]; then
echo "about to change /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm"; echo;
cp /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm.bkup;
sed -ir '184s!\x91\x93!\x91\x27--timeout=35\x27\x93!' /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm;
sed -n 183,185p /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm; echo;
else
echo "packages not found. Assumed to be not applicable";
fi
else
echo "This is 4.$version, skipping";
fi
The script is supposed to make sure Varnish is version 4.6.10 and has 2 custom .deb packages installed (not through apt-get). then makes a backup and edits a single line in a perl module from [] to ['--timeout=35']
it looks like its tripping up on the sed replace one liner.
There are two major problems (minor ones addressed in comments). The first is that you use the decimal code for [] instead of the hexa, so you should use \x5b\x5d instead of \x91\x93. The second problem is that if you do use the proper codes, sed will still interpret those syntactically as []. So you can't escape escaping. Here's what you should call:
sed -ri'.bkup' '184s!\[\]![\x27--timeout=35\x27]!' /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm
And this will create the backup for you (but you should double check).

if grep value greater than 1000 then

I want to grep "priority=" in a file where the value of "priority=" is greater than 1000.
I tried something like this:
if grep -q "priority=[ >1000 ]" file; then
echo "[!] Unnatural priority"
fi
e.g. <intent-filter android:priority="2147483647">
You could use this Perl one-liner:
perl -lne 'print "[!] Unnatural priority" if /priority="(\d+)"/ && $1 > 1000'
Capture the digits in priority="X" and print the warning if the value is greater than 1000.
You can also do this in native bash if you want:
while read -r line; do
if [[ $line =~ priority=\"([[:digit:]]+)\" ]] && (( BASH_REMATCH[1] > 1000 )); then
echo "[!] Unnatural priority"
fi
done < file
Try:
(( $(grep -oP 'priority\s*=\s*"\s*\K(\d+)' file) > 1000 )) && echo "warning"
Need a relatively new grep with -P perl regex support. The:
\K (variable look behind) matches, but kills everything before it from the result, so it prints only the capture group (\d+)
of course, you can use perl too,
perl -nlE 'say $1 if /priority="\K(\d+)/' <<< '<intent-filter android:priority="2147483647">'
prints
2147483647
or sed
sed 's/.*priority="\([0-9][0-9]*\).*/\1/' <<< '<intent-filter android:priority="2147483647">'
You could try using a regular expression to require a pattern that resembles a number greater than one thousand:
grep -q --regexp="priority=\"[1-9][0-9]\{3,\}\"" file
This should match the case where priority= is followed by at least four digits and the first digit is non-zero.
awk will make this easy:
$ cat file | awk -F '=' '$2 > 1000 {print $0}'
Assuming that there's only one = on each line of course.
I had a similar problem - Checking version string that needed to be "Version 2.32" or later. My grep (embedded BusyBox) doesn't support -P option or {n}, so using basic grep:
grep "Version 2\.3[2-9]\|2\.[4-9][0-9]\|[3-9]\.[0-9][0-9]"

Get any string between 2 string and assign a variable in bash

I cannot get this to work. I only want to get the string between 2 others in bash. Like this:
FOUND=$(echo "If show <start>THIS WORK<end> then it work" | **the magic**)
echo $FOUND
It seems so simple...
sed -n 's/.*<start>\(.*\)<end>.*/\1/p'
This can be done in bash without any external commands such as awk and sed. When doing a regex match in bash, the results of the match are put into a special array called BASH_REMATCH. The second element of this array contains the match from the first capture group.
data="If show <start>THIS WORK<end> then it work"
regex="<start>(.*)<end>"
[[ $data =~ $regex ]] && found="${BASH_REMATCH[1]}"
echo $found
This can also be done using perl regex in grep (GNU specific):
found=$(grep -Po '(?<=<start>).*(?=<end>)' <<< "If show <start>THIS WORK<end> then it work")
echo "$found"
If you have < start > and < end > in your string then this will work. Set the FS to < and >.
[jaypal:~/Temp] FOUND=$(echo "If show <start>THIS WORK<end> then it work" |
awk -v FS="[<>]" '{print $3}')
[jaypal:~/Temp] echo $FOUND
THIS WORK

sh shell script of working with for loop

I am using sh shell script to read the files of a folder and display on the screen:
for d in `ls -1 $IMAGE_DIR | egrep "jpg$"`
do
pgm_file=$IMAGE_DIR/`echo $d | sed 's/jpg$/pgm/'`
echo "file $pgm_file";
done
the output result is reading line by line:
file file1.jpg
file file2.jpg
file file3.jpg
file file4.jpg
Because I am not familiar with this language, I would like to have the result that print first 2 results in the same row like this:
file file1.jpg; file file2.jpg;
file file3.jpg; file file4.jpg;
In other languages, I just put d++ but it does not work with this case.
Would this be doable? I will be happy if you would provide me sample code.
thanks in advance.
Let the shell do more work for you:
end_of_line=""
for d in "$IMAGE_DIR"/*.jpg
do
file=$( basename "$d" )
printf "file %s; %s" "$file" "$end_of_line"
if [[ -z "$end_of_line" ]]; then
end_of_line=$'\n'
else
end_of_line=""
fi
pgm_file=${d%.jpg}.pgm
# do something with "$pgm_file"
done
for d in "$IMAGE_DIR"/*jpg; do
pgm_file=${d%jpg}pgm
printf '%s;\n' "$d"
done |
awk 'END {
if (ORS != RS)
print RS
}
ORS = NR % n ? FS : RS
' n=2
Set n to whatever value you need.
If you're on Solaris, use nawk or /usr/xpg4/bin/awk
(do not use /usr/bin/awk).
Note also that I'm trying to use a standard shell syntax,
given your question is sh related (i.e. you didn't mention bash or ksh,
for example).
Something like this inside the loop:
echo -n "something; "
[[ -n "$oddeven" ]] && oddeven= || { echo;oddeven=x;}
should do.
Three per line would be something like
[[ "$((n++%3))" = 0 ]] && echo
(with n=1) before entering the loop.
Why use a loop at all? How about:
ls $IMAGE_DIR | egrep 'jpg$' |
sed -e 's/$/;/' -e 's/^/file /' -e 's/jpg$/pgm/' |
perl -pe '$. % 2 && chomp'
(The perl just deletes every other newline. You may want to insert a space and add a trailing newline if the last line is an odd number.)

Resources