How can I handle both wildcard and variables in grep?
My goal is to grep anything that matches "string*", and my string is $i.
I've tried many options and nothing works:
max="$(nc resources | grep "$i*" | awk '{print $3}')"
available="$(nc resources | grep "$i*" | awk '{print $7}')"
max="$(nc resources | grep '$i*' | awk '{print $3}')"
available="$(nc resources | grep '$i*' | awk '{print $7}')"
also fgrep, grep with flags etc.
thank you.
string* matches stringggggg. You probably want to match string.*
You can concatenat strings by putting them next to each other.
max="$(nc resources | grep "$i"".*" | awk '{print $3}')"
is equivalent to
max="$(nc resources | grep "$i" | awk '{print $3}')"
So i suppose you would want the second one.
Related
I am attempting to parse the nodejs package.json for the version number.. this line usually works but somehow I am now getting an error of invalid block
PACKAGE_VERSION=$(cat package.json | grep version | head -1 | awk -F: '{ print $2 }' | sed 's/[",]//g' | xargs)
specifically this area is highlighted in VSCode sed 's/......
How can I repair this line? is the " needing escaping or ?
The problem is YAML-related (it seems the combination of : and | is the culprit).
The easy fix is to use a YAML block:
script:
# …
- |-
PACKAGE_VERSION=$(cat package.json | grep version | head -1 | awk -F: '{ print $2 }' | sed 's/[",]//g' | xargs)
And while you’re at it you can use a folding block to make it more readable:
script:
# …
- >-
PACKAGE_VERSION=$(
cat package.json |
grep version |
head -1 |
awk -F: '{ print $2 }' |
sed 's/[",]//g' |
xargs
)
Chaining grep, head, awk and sed can usually be replaced by a single awk. And there’s a useless use of cat. Not to mention parsing JSON without a proper JSON parser.
I need to split an IPv4 address into octets, calculate the MD5 hash of each and print as a CGI output:
IP1=$(echo ${REMOTE_ADDR} | tr "." " " | awk '{print $1'} | md5sum | cut -c1-32)
printf $IP1
In this example, REMOTE_ADDR = 192.168.20.100
But the MD5 of 192 gives me a wrong MD5 IP1=6be7de648baa9067fa3087928d5ab0b4, while it should be 58a2fc6ed39fd083f55d4182bf88826d
If I do this:
cat /tmp/test.txt | md5sum | cut -c1-32
where test.txt contains 192,
I get the correct MD5 hash, i.e 58a2fc6ed39fd083f55d4182bf88826d
What am I doing wrong?
Your awk's print is adding a newline so you're computing the md5 of "192\n", not "192". Use
IP1=$(printf "%s" "${REMOTE_ADDR%.*.*.*}" | md5sum | cut -c1-32)
instead, which uses shell parameter expansion to remove all but the first octet of the IP address, and printf to write it without the newline.
As #Shawn said, the problem was on awk print.
Adding tr -d '\n' solved the problem.
Now it is working correctly; for other octects I had to change print $2..etc on awk
IP1=$(echo ${REMOTE_ADDR} | awk -F. '{print $1'} | tr -d '\n' | md5sum | cut -c1-32 )
Hello guys I want to count how many duplicates there are in a column of a file and put the number next to them. I use awk and sort like this
awk -F '|' '{print $2}' FILE | sort | uniq -c
but the count (from the uniq -c) appears at the left side of the duplicates.
Is there any way to put the count on the right side instead of the left, using my code?
Thanks for your time!
Though I believe you shouls show us your Input_file so that we could create a single command or so for this requirement, since you have't shown Input_file so trying to solve it with your command itself.
awk -F '|' '{print $2}' FILE | sort | uniq -c | awk '{for(i=2;i<=NF;i++){printf("%s ",$i)};printf("%s%s",$1,RS)}'
You can just use awk to reverse the output like below:
awk -F '|' '{print $2}' FILE | sort | uniq -c | awk {'print $2" "$1'}
awk -F '|' '{print $2}' FILE | sort | uniq -c| awk '{a=$1; $1=""; gsub(/^ /,"",$0);print $0,a}'
You can use awk to calculate the amount of duplicates, so your command can be simplified as followed,
awk -F '|' '{a[$2]++}END{for(i in a) print i,a[i]}' FILE | sort
Check this command:
awk -F '|' '{c[$2]++} END{for (i in c) print i, c[i]}' FILE | sort
Use awk to do the counting is enough. If you do not want to sort by browser, remove the pipe and sort.
What's a more elegant way of declaring multiple variables from awk outputs? I have two files that are auto-generated. The files can have as many as three lines. Here's a sample of the variable declarations that I did so far:
$ cat OUTF1.tmp
25 16 8.33 VERY GOOD
12 17 25 OK
$ cat OUTF2.tmp
30 60 15.38 VERY GOOD
TMP1="OUTF1.tmp"
TMP2="OUTF2.tmp"
MATH_101=$(cat $TMP1 | awk 'NR==1{print $1}' )
ENGLISH_101=$(cat $TMP1 | awk 'NR==1{print $2}' )
SCIENCE_101=$(cat $TMP1 | awk 'NR==1{print $3}' )
GRADE_101=$(cat $TMP1 | awk 'NR==1{print $4}' )
MATH_102=$(cat $TMP1 | awk 'NR==2{print $1}' )
ENGLISH_102=$(cat $TMP1 | awk 'NR==2{print $2}' )
SCIENCE102=$(cat $TMP1 | awk 'NR==2{printf $3}' )
GRADE102=$(cat $TMP1 | awk 'NR==2{print $4}' )
MATH_201=$(cat $TMP2 | awk 'NR==1{print $1}' )
ENGLISH_201=$(cat $TMP2 | awk 'NR==1{print $2}' )
SCIENCE_201=$(cat $TMP2 | awk 'NR==1{printf $3}' )
GRADE_201=$(cat $TMP2 | awk 'NR==1{print $4}' )
and so on...
The variable contents should then be:
MATH_101=25
MATH_102=12
ENGLISH_101=16
ENGLISH_102=17
etc.
Are there better ways than hard-coding these variables?
It would be literally just as easy to read straight from your input file to your shell script the correct way as to read from input->awk->shell... so the sensible answer is to do the former.
First, to go the simple but straightforward approach:
{
read MATH_101 ENGLISH_101 SCIENCE_101 GRADE_101
read MATH_102 ENGLISH_102 SCIENCE_102 GRADE_102
read MATH_201 ENGLISH_201 SCIENCE_201 GRADE_201
} <infile
Getting a bit fancier:
for level in 101 102 201; do
read {MATH,ENGLISH,SCIENCE,GRADE}_"$level"
done <infile
Since you have two separate input files, if you want to concatenate them into a single stream, consider replacing <infile with < <(cat file1 file2).
Here is the text:
this is text this is text this is text this is text pattern_abc"00a"this is text this is text this is text this is textthis is text this is text pattern_def"001b"this is text this is text
in the output, I would like:
00a
001b
note: The values I look for are of random length and contents
I use 2 expressions:
exp_1 = grep -oP "(?<=pattern_abc\")[^\"]*"
exp_2 = grep -oP "(?<=pattern_def\")[^\"]*"
egrep does not work (I got "egrep: egrep can only use the egrep pattern syntax")
I try:
cat test | exp_1 && exp_2
cat test | (exp_1 && exp_2)
cat test | exp_1 | exp_2
cat test | (exp_1 | exp_2)
and lastly:
grep -oP "((?<=pattern_abc\")[^\"]* \| (?<=pattern_def\")[^\"]*)" test
grep -oP "((?<=pattern_abc\")[^\"]* | (?<=pattern_def\")[^\"]*)" test
Any idea?
thank you very much !
You can use this grep,
grep -oP "(?<=pattern_(abc|def)\")[^\"]*" file
You can use awk like this:
awk -F\" '{for (i=2;i<NF;i+=2) print $i}' file
00a
001b
If the pattern_* is important you can use this gnu awk (due to RS)
awk -v RS="pattern_(abc|def)" -F\" 'NR>1{print $2}'
00a
001b
And another method through grep with Perl-regex option,
$ grep -oP '\"\K[^\"]*(?="this)' file
00a
001b
It works only if the string you want to match is followed by "this.
OR
You could use the below command which combines the two search patterns,
$ grep -oP 'pattern_abc"\K[^"]*|pattern_def"\K[^"]*' file
00a
001b