if grep value greater than 1000 then - bash

I want to grep "priority=" in a file where the value of "priority=" is greater than 1000.
I tried something like this:
if grep -q "priority=[ >1000 ]" file; then
echo "[!] Unnatural priority"
fi
e.g. <intent-filter android:priority="2147483647">


You could use this Perl one-liner:
perl -lne 'print "[!] Unnatural priority" if /priority="(\d+)"/ && $1 > 1000'
Capture the digits in priority="X" and print the warning if the value is greater than 1000.
You can also do this in native bash if you want:
while read -r line; do
if [[ $line =~ priority=\"([[:digit:]]+)\" ]] && (( BASH_REMATCH[1] > 1000 )); then
echo "[!] Unnatural priority"
fi
done < file


Try:
(( $(grep -oP 'priority\s*=\s*"\s*\K(\d+)' file) > 1000 )) && echo "warning"
Need a relatively new grep with -P perl regex support. The:
\K (variable look behind) matches, but kills everything before it from the result, so it prints only the capture group (\d+)
of course, you can use perl too,
perl -nlE 'say $1 if /priority="\K(\d+)/' <<< '<intent-filter android:priority="2147483647">'
prints
2147483647
or sed
sed 's/.*priority="\([0-9][0-9]*\).*/\1/' <<< '<intent-filter android:priority="2147483647">'


You could try using a regular expression to require a pattern that resembles a number greater than one thousand:
grep -q --regexp="priority=\"[1-9][0-9]\{3,\}\"" file
This should match the case where priority= is followed by at least four digits and the first digit is non-zero.


awk will make this easy:
$ cat file | awk -F '=' '$2 > 1000 {print $0}'
Assuming that there's only one = on each line of course.


I had a similar problem - Checking version string that needed to be "Version 2.32" or later. My grep (embedded BusyBox) doesn't support -P option or {n}, so using basic grep:
grep "Version 2\.3[2-9]\|2\.[4-9][0-9]\|[3-9]\.[0-9][0-9]"

Related

Trying to retrieve first 5 characters (only number & alphabet) from string in bash

I have a string like that
1-a-bc-dxyz
I'd want to get 1-a-bc-d ( first 5 characters, only number and alphabet)
Thanks

With gawk:
awk '{ for ( i=1;i<=length($0);i++) { if ( match(substr($0,i,1),/[[:alnum:]]/)) { cnt++;if ( cnt==5) { print substr($0,1,i) } } } }' <<< "1-a-bc-dxyz"
Read each character one by one and then if there is a pattern match for an alpha-numeric character (using the match function), increment a variable cnt. When cnt gets to 5, print the string we have seen so far (using the substr function)
Output:
1-a-bc-d

a='1-a-bc-dxyz'
count=0
for ((i=0;i<${#a};i++)); do
if [[ "${a:$i:1}" =~ [0-9]|[a-Z] ]] && [[ $((++count)) -eq 5 ]]; then
echo "${a:0:$((i+1))}"
exit
fi
done
You can further shrink this as;
a='1-a-bc-dxyz'
count=0
for ((i=0;i<${#a};i++)); do [[ "${a:$i:1}" =~ [0-9]|[a-Z] ]] && [[ $((++count)) -eq 5 ]] && echo "${a:0:$((i+1))}"; done

Using GNU awk:
$ echo 1-a-bc-dxyz | \
awk -F '' '{b=i="";while(gsub(/[0-9a-z]/,"&",b)<5)b=b $(++i);print b}'
1-a-bc-d
Explained:
awk -F '' '{ # separate each char to its own field
b=i="" # if you have more than one record to process
while(gsub(/[0-9a-z]/,"&",b)<5) # using gsub for counting (adjust regex if needed)
b=b $(++i) # gather buffer
print b # print buffer
}'

GNU sed supports an option to replace the k-th occurrence and all after that.
echo "1-a-bc-dxyz" | sed 's/[^a-zA-Z0-9]*[a-zA-Z0-9]//g6'

Using Combination of sed & AWK
echo 1-a-bc-dxyz | sed 's/[-*%$##]//g' | awk -F '' {'print $1$2$3$4$5'}
You can use for loop for printing character as well.

echo '1-a-bc-dxyz' | grep -Eo '^[[:print:]](-*[[:print:]]){4}'
That is pretty simple.
Neither sed nor awk.

bash: sed: unexpected behavior: displays everything

I wrote what I thought was a quick script I could run on a bunch of machines. Instead it print what looks like might be directory contents in a recursive search:
version=$(mysql Varnish -B --skip-column-names -e "SELECT value FROM sys_param WHERE param='PatchLevel'" | sed -n 's/^.*\([0-9]\.[0-9]*\).*$/\1/p')
if [[ $(echo "if($version == 6.10) { print 1; } else { print 0; }" | bc) -eq 1 ]]; then
status=$(dpkg-query -l | awk '{print $2}' | grep 'sg-status-polling');
cons=$(dpkg-query -l | awk '{print $2}' | grep 'sg-consolidated-poller');
if [[ "$status" != "" && "$cons" != "" ]]; then
echo "about to change /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm"; echo;
cp /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm.bkup;
sed -ir '184s!\x91\x93!\x91\x27--timeout=35\x27\x93!' /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm;
sed -n 183,185p /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm; echo;
else
echo "packages not found. Assumed to be not applicable";
fi
else
echo "This is 4.$version, skipping";
fi
The script is supposed to make sure Varnish is version 4.6.10 and has 2 custom .deb packages installed (not through apt-get). then makes a backup and edits a single line in a perl module from [] to ['--timeout=35']
it looks like its tripping up on the sed replace one liner.

There are two major problems (minor ones addressed in comments). The first is that you use the decimal code for [] instead of the hexa, so you should use \x5b\x5d instead of \x91\x93. The second problem is that if you do use the proper codes, sed will still interpret those syntactically as []. So you can't escape escaping. Here's what you should call:
sed -ri'.bkup' '184s!\[\]![\x27--timeout=35\x27]!' /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm
And this will create the backup for you (but you should double check).

Extracting a substring from a variable using bash script

I have a bash variable with value something like this:
10:3.0,16:4.0,32:4.0,39:2.0,65:3.0,95:4.0,110:4.0,111:4.0,2312:1.0
There are no spaces within value. This value can be very long or very short. Here pairs such as 65:3.0 exist. I know the value of a number from the first part of pair, say 65. I want to extract the number 3.0 or pair 65:3.0. I am not aware of the position (offset) of 65.
I will be grateful for a bash-script that can do such extraction. Thanks.

Probably awk is the most straight-forward approach:
awk -F: -v RS=',' '$1==65{print $2}' <<< "$var"
3.0
Or to get the pair:
$ awk -F: -v RS=',' '$1==65' <<< "$var"
65:3.0

Here's a pure Bash solution:
var=10:3.0,16:4.0,32:4.0,39:2.0,65:3.0,95:4.0,110:4.0,111:4.0,2312:1.0
while read -r -d, i; do
[[ $i = 65:* ]] || continue
echo "$i"
done <<< "$var,"
You may use break after echo "$i" if there's only one 65:... in var, or if you only want the first one.
To get the value 3.0: echo "${i#*:}".
Other (pure Bash) approach, without parsing the string explicitly. I'm assuming you're only looking for the first 65 in the string, and that it is present in the string:
var=10:3.0,16:4.0,32:4.0,39:2.0,65:3.0,95:4.0,110:4.0,111:4.0,2312:1.0
value=${var#*,65:}
value=${value%%,*}
echo "$value"
This will be very slow for long strings!
Same as above, but will output all the values corresponding to 65 (or none if there are none):
var=10:3.0,16:4.0,32:4.0,39:2.0,65:3.0,95:4.0,110:4.0,111:4.0,2312:1.0
tmpvar=,$var
while [[ $tmpvar = *,65:* ]]; do
tmpvar=${tmpvar#*,65:}
echo "${tmpvar%%,*}"
done
Same thing, this will be slow for long strings!
The fastest I can obtain in pure Bash is my original answer (and it's fine with 10000 fields):
var=10:3.0,16:4.0,32:4.0,39:2.0,65:3.0,95:4.0,110:4.0,111:4.0,2312:1.0
IFS=, read -ra ary <<< "$var"
for i in "${ary[#]}"; do
[[ $i = 65:* ]] || continue
echo "$i"
done
In fact, no, the fastest I can obtain in pure Bash is with this regex:
var=10:3.0,16:4.0,32:4.0,39:2.0,65:3.0,95:4.0,110:4.0,111:4.0,2312:1.0
[[ ,$var, =~ ,65:([^,]+), ]] && echo "${BASH_REMATCH[1]}"
Test of this vs awk,
where the 65:3.0 is at the end:
printf -v var '%s:3.0,' {100..11000}
var+=65:42.0
time awk -F: -v RS=',' '$1==65{print $2}' <<< "$var"
shows 0m0.020s (rough average) whereas:
time { [[ ,$var, =~ ,65:([^,]+), ]] && echo "${BASH_REMATCH[1]}"; }
shows 0m0.008s (rough average too).
where the 65:3.0 is not at the end:
printf -v var '%s:3.0,' {1..10000}
time awk -F: -v RS=',' '$1==65{print $2}' <<< "$var"
shows 0m0.020s (rough average) and with early exit:
time awk -F: -v RS=',' '$1==65{print $2;exit}' <<< "$var"
shows 0m0.010s (rough average) whereas:
time { [[ ,$var, =~ ,65:([^,]+), ]] && echo "${BASH_REMATCH[1]}"; }
shows 0m0.002s (rough average).

With grep:
grep -o '\b65\b[^,]*' <<<"$var"
65:3.0
Or
grep -oP '\b65\b:\K[^,]*' <<<"$var"
3.0
\K option ignores everything before matched pattern and ignore pattern itself. It's Perl-compatibility(-P) for grep command .

Here is an gnu awk
awk -vRS="(^|,)65:" -F, 'NR>1{print $1}' <<< "$var"
3.0

try
echo $var | tr , '\n' | awk '/65/'
where
tr , '\n' turn comma to new line
awk '/65/' pick the line with 65
or
echo $var | tr , '\n' | awk -F: '$1 == 65 {print $2}'
where
-F: use : as separator
$1 == 65 pick line with 65 as first field
{ print $2} print second field

Using sed
sed -e 's/^.*,\(65:[0-9.]*\),.*$/\1/' <<<",$var,"
output:
65:3.0
There are two different ways to protect against 65:3.0 being the first-in-line or last-in-line. Above, commas are added to surround the variable providing for an occurrence regardless. Below, the Gnu extension \? is used to specify zero-or-one occurrence.
sed -e 's/^.*,\?\(65:[0-9.]*\),\?.*$/\1/' <<<$var
Both handle 65:3.0 regardless of where it appears in the string.

Try egrep like below:
echo $myvar | egrep -o '\b65:[0-9]+.[0-9]+' |

Finding strings in a file and write to a file if found

Just wondering if someone can point me in the right direction. I want to search a file for 2 strings and if they match add a YES to a csv file or NO if they don't. For now tough I'd like to just have it find the strings and print SUCCESS on the screen if found as per below (I'd rather try get the writing to a file done myself so I learn).
I do need both strings to be a match otherwise the result should be No.
I've gotten this so far:
#/usr/bin/bash
file=test.xml
string1=TEST1
string2=TEST2
if grep -e "$string1|$string2" "$file"
then
echo IT WORKS!!!
else
echo UH OH!!!
fi

Using grep you would need to read the file twice:
if grep "$string1" "$file" && grep "$string2" "$file"
However, using AWK, you can set flags and you'll only need to read the file once:
awk -v s1="$string1" -v s2="$string2" '$0 ~ s1 {f1 = 1} $0 ~ s2 {f2 = 1} END {if (f1 && f2) {print "Yes"} else {print "No"}}' "$file"

Presuming that either string matching is sufficient, try this:
file=test.xml
string1="TEST1"
string2="TEST2"
if grep -q -F "$string1"$'\n'"$string2" "$file" ; then
printf YES
else
printf NO
fi
Importantly, -F means that if the string happens to contain regex-special characters they will be treated literally. For convenience -q will suppress the output of grep so you only get the YES/NO returned.
EDIT:
Since you need both strings to match, try this:
if [ $(sed -n -e "/${string1}/p;/${string2}/p" "$file" | wc -l) -ge 2 ] ; then
printf YES
else
printf NO
fi
This is not as safe as the grep version since regex characters in the strings are not treated literally and is rather inefficient, but it gets the job done.

Since you want both strings to match, an inefficient way is to call grep once for each pattern:
if grep -q "$string1" && grep -q "$string2"
then
echo IT WORKS
else
echo UH OH
fi
I'm not aware of a way to do this with a single grep call without a fairly complicated regex. Something like
grep -qE "$string1.*$string2|$string2.*$string1"
could work, but I'm sure that would include some false positives and/or false negatives, depending on the values of string1 and string2.

Your code is in the good direction. You need to change -e with -E in order for grep to recognize the regular expression that you are using ($string1|$string2). You may also want to avoid seeing the actual matchings (use quiet -q flag for that):
file=test.xml
string1="TEST1"
string2="TEST2"
if grep -E -q "$string1|$string2" "$file"
then
echo IT WORKS!!!
else
echo UH OH!!!
fi
EDIT: Since you need to match both strings, this is not possible with grep (AFAIK) unless you execute grep multiple times (like in #chepner's answer).
An alternative solution would be to use awk:
awk '/'$string1'/{s1=1}/'$string2'/{s2=1} END { \
if (s1 && s2) print "IT WORKS!!!" ; else print "UH OH!!!" }' "$file"

Get any string between 2 string and assign a variable in bash

I cannot get this to work. I only want to get the string between 2 others in bash. Like this:
FOUND=$(echo "If show <start>THIS WORK<end> then it work" | **the magic**)
echo $FOUND
It seems so simple...

sed -n 's/.*<start>\(.*\)<end>.*/\1/p'

This can be done in bash without any external commands such as awk and sed. When doing a regex match in bash, the results of the match are put into a special array called BASH_REMATCH. The second element of this array contains the match from the first capture group.
data="If show <start>THIS WORK<end> then it work"
regex="<start>(.*)<end>"
[[ $data =~ $regex ]] && found="${BASH_REMATCH[1]}"
echo $found
This can also be done using perl regex in grep (GNU specific):
found=$(grep -Po '(?<=<start>).*(?=<end>)' <<< "If show <start>THIS WORK<end> then it work")
echo "$found"

If you have < start > and < end > in your string then this will work. Set the FS to < and >.
[jaypal:~/Temp] FOUND=$(echo "If show <start>THIS WORK<end> then it work" |
awk -v FS="[<>]" '{print $3}')
[jaypal:~/Temp] echo $FOUND
THIS WORK

Resources