grep 2 elements in a line and print them - bash

Here is my issue, i have a file with the entries, i would like to get just the date + the last command after the last "]: "
Aug 17 14:25:17 snaper[22134]: [ip:10.1.15.245 37985 10.1.15.18 22 uid:10000 sid:21680 tty: cwd:/data/www/hybris/hybris/bin/platform filename:/bin/ps]: /bin/ps -p 6763
How can i get it when i cat the file ?
I can get the date with:
awk '{print $1,$2,$3}'
and the last command with :
awk -F': ' '{print $NF}'
But how to combine them to get it in a single line ?
I'm not awk limited, any sed grep or other command is ok for me :)
Thanks in advance

Just remove everything between the date and the last command:
sed 's/^\(... .. ..:..:..\).*: /\1 /'

Simple solution using AWK
$awk '{print $1,$2,$3, $(NF-2), $(NF-1), $NF }' file
Aug 17 14:25:17 /bin/ps -p 6763

Using GNU grep
grep -oP '^.{15}|.*\]: \K.*' file | paste - -

Possible use tee for use 1 output to 2 commands:
echo 'Aug 17 14:25:17 snaper[22134]: [ip:10.1.15.245 37985 10.1.15.18 22 uid:10000 sid:21680 tty: cwd:/data/www/hybris/hybris/bin/platform filename:/bin/ps]: /bin/ps -p 6763' | tee >(awk -F': ' '{print $NF}') | awk '{print $1,$2,$3}' | tr '\n' ' '
and we have output as:
Aug 17 14:25:17 /bin/ps -p 6763

$ s="Aug 17 14:25:17 snaper[22134]: [ip:10.1.15.245 37985 10.1.15.18 22 uid:10000 sid:21680 tty: cwd:/data/www/hybris/hybris/bin/platform filename:/bin/ps]: /bin/ps -p 6763"
Use sed to achieve your goal,
$ sed -r 's/(^.*:[0-9]{2}) .*]:/\1/' <<< "$s"

Try following solutions too. Considering your Input_file will be having same data as shown sample.
Solution 1st: Using simple cut command.
cut -d" " -f1,2,3,14,15,16 Input_file
Solution 2nd: using awk command where I am making string snaper and ]: as field separators.
awk -F' snaper|]:' '{print $1,$4}' Input_file
Solution 3rd: making record separator as space and then printing only those lines which we need as per OP's request.
awk -v RS=" " 'NR<4||NR>13{printf("%s%s",$0,NR<3||NR<16?" ":"")}' Input_file
Solution 4th: Substituting everything from snap to till : and get whatever is OP's request.
awk '{sub(/snaper\[.*\]: /,"");print}' Input_file
Solution 5th: Using --re-interval here(as I have old version of awk) you could remove it if you latest awk version in your system too.
awk --re-interval '{match($0,/.*[0-9]{2}:[0-9]{2}:[0-9]{2}/);print substr($0,RSTART,RLENGTH),$(NF-2),$(NF-1),$NF}' Input_file
Solution 6th: using sed and substituting everything till snaper and then everything till colon and printing the match only.
sed 's/\(.[^s]*\)\(.*:\)\(.*\)/\1\3/' Input_file

Related

Extract a property value from a text file

I have a log file which contains lines like the following one:
Internal (reserved=1728469KB, committed=1728469KB)
I'd need to extract the value contained in "committed", so 1728469
I'm trying to use awk for that
cat file.txt | awk '{print $4}'
However that produces:
committed=1728469KB)
This is still incomplete and would need still some work. Is there a simpler solution to do that instead?
Thanks
Could you please try following, using match function of awk.
awk 'match($0,/committed=[0-9]+/){print substr($0,RSTART+10,RLENGTH-10)}' Input_file
With GNU grep using \K option of it:
grep -oP '.*committed=\K[0-9]*' Input_file
Output will be 1728469 in both above solutions.
1st solution explanation:
awk ' ##Starting awk program from here.
match($0,/committed=[0-9]+/){ ##Using match function to match from committed= till digits in current line.
print substr($0,RSTART+10,RLENGTH-10) ##Printing sub string from RSTART+10 to RLENGTH-10 in current line.
}
' Input_file ##Mentioning Input_file name here.
Sed is better at simple matching tasks:
sed -n 's/.*committed=\([0-9]*\).*/\1/p' input_file
$ awk -F'[=)]' '{print $3}' file
1728469KB
You can try this:
str="Internal (reserved=1728469KB, committed=1728469KB)"
echo $str | awk '{print $3}' | cut -d "=" -f2 | rev | cut -c4- | rev

Linux get data from each line of file

I have a file with many (~2k) lines similar to:
117 VALID|AUTHEN tcp:10.92.163.5:64127 uniqueID=nwCelerra
....
991 VALID|AUTHEN tcp:10.19.16.21:58332 uniqueID=smUNIX
I want only the IP address (10.19.16.21 shown above) and the value of the uniqueID (smUNIX shown above)
I am able to get close with:
cat t.txt|cut -f2- -d':'
10.22.36.69:46474 uniqueID=smwUNIX
...
I am on Linux using bash.
Using awk:
awk '{split($3,a,":"); split($4,b,"="); print a[2] " " b[2]}'
By default if splits on the whitespaces, with some extra code you can split the subfields
Update:
even easier overriding the default delimiter:
awk -F '[:=]' '{print $2 " "$4}'
using grep and sed :
grep -oP "^\d+ [A-Z]+\|[A-Z]+ \w+:\K(.*)" | sed "s/ uniqueID=/ /g"
outputs:
10.92.163.5:64127 nwCelerra
10.19.16.21:58332 smUNIX

awk command to change date format

I have a file that contains the below records:
**cat Date.txt**
SEPT,30-SEP-2017
MARCH,30-MAR-2018
JULY,30-JUL-2017
DECEM,30-DEC-2017
AUGS,30-AUG-2017
I want the output to be:
SEPT,20170930
MARCH,20180330
JULY,20170730
DECEM,20171230
AUGS,20170830
I have done a script with a while loop, as given below.
cat Date.txt > /tmp/exp.part
while read line
do
Dat=$(echo $line | awk -F , '{print $1}')
DatNew=$(date -d $Exp +%Y%m%d)
echo $DatNew
done < /tmp/exp.part
I want to know if I can do it from the awk command, instead of writing a while loop.
One way using system command within awk:
Input file:
$ cat file
27-SEP-2017
28-MAR-2018
27-JUL-2017
27-DEC-2017
29-AUG-2017
$ awk '{system("date -d " $0 " +%Y%m%d")}' file
20170927
20180328
20170727
20171227
20170829
For the updated input file:
$ cat file
SEP,27-SEP-2017
MAR,28-MAR-2018
JUL,27-JUL-2017
DEC,27-DEC-2017
AUG,29-AUG-2017
$ awk -F, '{system("date -d " $2 " +%Y%m%d")}' file
20170927
20180328
20170727
20171227
20170829
For the updated requirement:
$ cat file
SEPT,30-SEP-2017
MARCH,30-MAR-2018
JULY,30-JUL-2017
DECEM,30-DEC-2017
AUGS,30-AUG-2017
$ awk -F, '{cmd="date -d "q $2 q" "s"+%Y%m%d"s; cmd|getline x;print $1,x}' dq='"' sq="'" OFS=, file
SEPT,20170930
MARCH,20180330
JULY,20170730
DECEM,20171230
AUGS,20170830
Following on Guru's answer, this also seemed to work:
$ awk -F, '{cmd="date -d "$2" +%Y%m%d"; cmd|getline x;print $1,x}' OFS=',' file
SEPT,20170930
MARCH,20180330
JULY,20170730
DECEM,20171230
AUGS,20170830
Note the awk version was actually mawk:
$ awk -W version
mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan

Bash string replace on command result

I have a simple bash script which is getting the load average using uptime and awk, for example
LOAD_5M=$(uptime | awk -F'load averages:' '{ print $2}' | awk '{print $2}')
However this includes a ',' at the end of the load average
e.g.
0.51,
So I have then replaced the comma with a string replace like so:
LOAD_5M=${LOAD_5M/,/}
I'm not an awk or bash wizzkid so while this gives me the result I want, I am wondering if there is a succinct way of writing this, either by:
Using awk to get the load average without the comma, or
Stripping the comma in a single line
You can do that in same awk command:
uptime | awk -F 'load averages?: *' '{split($2, a, ",? "); print a[2]}'
1.32
The 5 min load is available in /proc/loadavg. You can simply use cut:
cut -d' ' -f2 /proc/loadavg
With awk you can issue:
awk '{print $2}' /proc/loadavg
If you are not working on Linux the file /proc/loadavg will not being present. In this case I would suggest to use sed, like this:
uptime | sed 's/.*, \(.*\),.*,.*/\1/'
uptime | awk -F'load average:' '{ print $2}' | awk -F, '{print $2}'
0.38
(My uptime output has 'load average:' singular)
The load average numbers are always the last 3 fields in the 'uptime' output so:
IFS=' ,' read -a uptime_fields <<<"$(uptime)"
LOAD_5M=${uptime_fields[#]: -2:1}

Delete text in file after a match

I have a file with the following:
/home/adversion/web/wp-content/plugins/akismet/index1.php: PHP.Mailer-7 FOUND
/home/beckydodman/web/oldshop/images/google68274020601e.php: Trojan.PHP-1 FOUND
/home/resurgence/web/Issue 272/Batch 2 for Helen/keynote_Philip Baldwin (author revise).doc: W97M.Thus.A FOUND
/home/resurgence/web/Issue 272/from Helen/M keynote_Philip Baldwin.doc: W97M.Thus.A FOUND
/home/skda/web/clients/sandbox/wp-content/themes/editorial/cache/external_dc8e1cb5bf0392f054e59734fa15469b.php: Trojan.PHP-58 FOUND
I need to clean this file up by removing everything after the colon (:).
so that it looks like this:
/home/adversion/web/wp-content/plugins/akismet/index1.php
/home/beckydodman/web/oldshop/images/google68274020601e.php
/home/resurgence/web/Issue 272/Batch 2 for Helen/keynote_Philip Baldwin (author revise).doc
/home/resurgence/web/Issue 272/from Helen/M keynote_Philip Baldwin.doc
/home/skda/web/clients/sandbox/wp-content/themes/editorial/cache/external_dc8e1cb5bf0392f054e59734fa15469b.php
Use awk:
$ awk -F: '{print $1}' input
/home/adversion/web/wp-content/plugins/akismet/index1.php
/home/beckydodman/web/oldshop/images/google68274020601e.php
/home/resurgence/web/Issue 272/Batch 2 for Helen/keynote_Philip Baldwin (author revise).doc
/home/resurgence/web/Issue 272/from Helen/M keynote_Philip Baldwin.doc
/home/skda/web/clients/sandbox/wp-content/themes/editorial/cache/external_dc8e1cb5bf0392f054e59734fa15469b.php
or cut
$ cut -d: -f1 input
or sed
$ sed 's/:.*$//' input
or perl in awk-mode
$ perl -F: -lane 'print $F[0]' input
finally, pure bash
#!/bin/bash
while read line
do
echo ${line%%:*}
done < input
This should be enough
awk -F: '{print $1}' file-name
Here a none sed/awk solution
cut -d : -f 1 [filename]
pipe that through sed:
$ echo "/home/adversion/web/wp-content/plugins/akismet/index1.php: PHP.Mailer-7 FOUND" | sed 's/: .*$//'
/home/adversion/web/wp-content/plugins/akismet/index1.php
Will work as long as ': ' doesn't appear twice. Note that the awk / cut examples above are more likely to fail as they match ':' not ': '

Resources