This gives me the duplicates and the number of times it is repeated
$ awk -F "\"*,\"*" '{print $2}' file.csv | sort | uniq -c | sort -nr | head -n 2
4 12345
3 56789
What I then want to do is add the first column (3+4). I can do this if i write the output above to a file test. I can do this as follows:
$ awk -F" " '{print $1}' test
4
3
$ awk -F" " '{print $1}' test | paste -sd+
4+3
$ awk -F" " '{print $1}' test | paste -sd+ | bc
7
But I want to be able to do this in 1 line, and ideally don't want to write to a file, would like to understand why the following does not work
awk -F "\"*,\"*" '{print $2}' file.csv | sort | uniq -c | sort -nr | head -n 2 | awk -F" " '{print $1}' | paste -sd+ | bc
My 2nd awk seems to not like the input.
Can anyone advise how I do this, and what I am doing wrong?
EDIT1 - file.csv looks like:
"Date","Number"
"2015-11-01","12345"
"2015-11-01","12345"
"2015-11-01","12345"
"2015-11-01","12345"
"2015-11-01","56789"
"2015-11-01","56789"
"2015-11-01","56789"
awk to the rescue!
... | sort -nr | awk 'NR<=2{sum+=$1} END{print sum}'
you can pick the first two rows and summation in awk as well.
Related
> root# ps -ef | grep [j]ava | awk '{print $2,$9}'
> 45134 -Dapex=APEC
> 45135 -Dapex=JAAA
> 45136 -Dapex=APEC
I need to put the first APEC of first as First PID, third line of APEC and Second PID and last one as Third PID.
I've tried awk but no expected result.
> First_PID =ps -ef | grep [j]ava | awk '{print $2,$9}'|awk '{if ($0 == "[^0-9]" || $1 == "APEC:") {print $0; exit;}}'
Expected result should look like this.
> First_PID=45134
> Second_PID=45136
> Third_PID=45135
With your shown samples and attempts please try following awk code. Written and tested in GNU awk.
ps -ef | grep [j]ava |
awk '
{
val=$2 OFS $9
match(val,/([0-9]+) -Dapex=APEC ([0-9]+) -Dapex=JAAA\s([0-9]+)/,arr)
print "First_PID="arr[1],"Second_PID=",arr[3],"Third_PID=",arr[2]
}
'
How about this:
$ input=("1 APEC" "2 JAAA" "3 APEC")
$ printf '%s\n' "${input[#]}" | grep APEC | sed -n '2p'
3 APEC
Explanation:
input=(...) - input data in an array, for testing
printf '%s\n' "${input[#]}" - print input array, one element per line
grep APEC - keep lines containing APEC only
sed -n - run sed without automatic print
sed -n '2p' - print only the second line
If you just want the APECs first...
ps -ef |
awk '/java[ ].* -Dapex=APEC/{print $2" "$9; next; }
/java[ ]/{non[NR]=$2" "$9}
END{ for (rec in non) print non[rec] }'
If possible, use an array instead of those ordinally named vars.
mapfile -t pids < <( ps -ef | awk '/java[ ].* -Dapex=APEC/{print $2; next; }
/java[ ]/{non[NR]=$2} END{ for (rec in non) print non[rec] }' )
After read from everyone idea,I end up with the very simple solution.
FIRST_PID=$(ps -ef | grep APEC | grep -v grep | awk '{print $2}'| sed -n '1p')
SECOND_PID=$(ps -ef | grep APEC | grep -v grep | awk '{print $2}'| sed -n '2p')
JAWS_PID=$(ps -ef | grep JAAA | grep -v grep | awk '{print $2}')
I was looking for a single one-liner to produce a line with all data:
These four [separate] commands produce the info I need, but I don't know how to concatenate them into a single line:
$ cat /proc/cpuinfo | grep "model name" | sed -n -e 's/^.*: //p'
ARMv6-compatible processor rev 7 (v6l)
$ lscpu | grep "CPU(s):" | awk '{print $2}'
1
$ lscpu | grep "CPU min MHz:" | awk '{print $4}'
700.0000
$ lscpu | grep "CPU max MHz:" | awk '{print $4}'
1000.0000
And I wanted to get:
ARMv6-compatible processor rev 7 (v6l) x1 #700 MHz (max #1000 MHz)
just using echo
echo $(cat /proc/cpuinfo | grep "model name" | sed -n -e 's/^.*: //p') \
x$(lscpu | grep "CPU(s):" | awk '{print $2}') \
#$(lscpu | grep "CPU min MHz:" | awk '{print $4}') \
'( max' $(lscpu | grep "CPU max MHz:" | awk '{print $4}') ')'
By the way, you could use just grep "model name" /proc/cpuinfo instead of cat /proc/cpuinfo | grep "model name"
One way.
paste -d ' ' <(awk '/^model name/{sub(/.+: /,"");print;exit}' /proc/cpuinfo) <(lscpu | awk '/CPU min MHz:/{printf " (max #%d MHz) ", $4} /CPU max MHz:/{printf "#%d MHz", $4} /^CPU\(s\):/{printf "x%s ", $2}')
How can i run last command for every matched line in awk?
ypcat passwd | awk -F":" '/:John / { system("last" $1) }'
I'm trying to execute the last command for every user that is named John but It does not print anything.
Insert a whitespace after last:
ypcat passwd | awk -F":" '/:John / {system("last " $1)}'
Your approach is wrong, awk is not shell. awk is designed to manipulate text not to call other tools from, that is what a shell is for This may be what you want, depending on what last is/does:
ypcat passwd | awk -F":" '/:John /{print $1}' | xargs last
or:
ypcat passwd | awk -F":" '/:John /{print $1}' | xargs -n 1 last
I'm running a command which basically parses some JSON and then extracts an ID using awk and sed.
When I run the command on its own it give the correct output eg
cat CustomThemeProfile.json | sed -e 's/[{}]/''/g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | awk -F ":" '{print $0}' | grep id | awk -F ":" '{print $2}' | sed 's/\"//g'
2F13F732-4BCB-49DC-A0FB-C91B5DE58472
But when I want to assign the output to a variable I get nothing returned. eg
cat CustomThemeProfile.json | id=$(sed -e 's/[{}]/''/g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | awk -F ":" '{print $0}' | grep id | awk -F ":" '{print $2}' | sed 's/\"//g'); echo $id
Any ideas. I really want this to be ran from a script but for the moment the script just does nothing, sits waiting for something?
Script i'm calling from.
First script just finds all json files and then calls this script. so the file is passed
#!/bin/bash
echo "running search and replace script ..."
id="$(sed -e 's/[{}]/''/g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | awk -F ":" '{print $0}' | grep id | awk -F ":" '{print $2}' | sed 's/\"//g')"
filler="0-0000-0000-0000-000000000000"
echo $id
if [ $(#id) -ge 8]; then echo "New Profile ID in use"; exit
else idnew=$id$filler
fi
sed -i '"s/$id/$idnew/g"' $1
sed -i 's/ps_hpa/ps_hpa/g' $1
You need to rearrange your syntax a little bit:
id=$(sed -e 's/[{}]/''/g' CustomThemeProfile.json | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | awk -F ":" '{print $0}' | grep id | awk -F ":" '{print $2}' | sed 's/\"//g')
Notice I am avoiding a useless use of cat and passing the file directly to sed. This is why your script does nothing - sed is waiting for some input. It would be possible to move cat inside the command substitution but there's no advantage to doing so. If a tool is capable of reading a file itself, then you should use that capability.
The better solution would be to parse your JSON properly, using jq for example. In order for us to show you how to do that, you should edit your question to show us a sample of your input.
I am trying to open some files as awk's output; the command is:
grep "formatDate\s=" "js/components/" | awk '{print $1}' | awk -F ":" '/1/ {print $1}'
and it (seems to) work correctly.
If I try to open that output as vim's tabs, like this:
vim -p ${ grep "formatDate\s=" "js/components/" | awk '{print $1}' | awk -F ":" '/1/ {print $1}' }
then I get:
-bash: ${ grep "formatDate\s=" "js/components/" | awk '{print $1}' | awk -F ":" '/1/ {print $1}' }: bad substitution
Any help? Thanks.
The way to execute a command is $(), whereas you are using ${}.
Hence, this should work:
vim -p $(grep "formatDate\s=" "js/components/" | awk '{print $1}' | awk -F ":" '/1/ {print $1}')