I have a file that contains the below records:
**cat Date.txt**
SEPT,30-SEP-2017
MARCH,30-MAR-2018
JULY,30-JUL-2017
DECEM,30-DEC-2017
AUGS,30-AUG-2017
I want the output to be:
SEPT,20170930
MARCH,20180330
JULY,20170730
DECEM,20171230
AUGS,20170830
I have done a script with a while loop, as given below.
cat Date.txt > /tmp/exp.part
while read line
do
Dat=$(echo $line | awk -F , '{print $1}')
DatNew=$(date -d $Exp +%Y%m%d)
echo $DatNew
done < /tmp/exp.part
I want to know if I can do it from the awk command, instead of writing a while loop.
One way using system command within awk:
Input file:
$ cat file
27-SEP-2017
28-MAR-2018
27-JUL-2017
27-DEC-2017
29-AUG-2017
$ awk '{system("date -d " $0 " +%Y%m%d")}' file
20170927
20180328
20170727
20171227
20170829
For the updated input file:
$ cat file
SEP,27-SEP-2017
MAR,28-MAR-2018
JUL,27-JUL-2017
DEC,27-DEC-2017
AUG,29-AUG-2017
$ awk -F, '{system("date -d " $2 " +%Y%m%d")}' file
20170927
20180328
20170727
20171227
20170829
For the updated requirement:
$ cat file
SEPT,30-SEP-2017
MARCH,30-MAR-2018
JULY,30-JUL-2017
DECEM,30-DEC-2017
AUGS,30-AUG-2017
$ awk -F, '{cmd="date -d "q $2 q" "s"+%Y%m%d"s; cmd|getline x;print $1,x}' dq='"' sq="'" OFS=, file
SEPT,20170930
MARCH,20180330
JULY,20170730
DECEM,20171230
AUGS,20170830
Following on Guru's answer, this also seemed to work:
$ awk -F, '{cmd="date -d "$2" +%Y%m%d"; cmd|getline x;print $1,x}' OFS=',' file
SEPT,20170930
MARCH,20180330
JULY,20170730
DECEM,20171230
AUGS,20170830
Note the awk version was actually mawk:
$ awk -W version
mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan
Related
I have a namelist.txt doc like:
Name:Mary Miller | Date of birth:03-01-1935
Name:Harry Jonhson | Date of birth:06-08-1955
And I have to write a document like:
ID;name
where ID is :
nut = $(awk '{print (NR)}' namelist.txt)
for i in $nut
do
echo -n $(($i+10000))
echo ":"
done
but now I can't add the names in front of the ":". And how would i put that on a new doc .txt? Using > ?
Expected output:
10001:Mary Miller
10002:Harry Jonhson
Can someone pls help?
another awk
$ awk -F'|' -v c=10000 '{sub(/^Name/,++c); print $1}' file
10001:Mary Miller
10002:Harry Jonhson
separate the record at the pipe sign, replace "Name" with pre incremented counter with starts at 10000 value. Print only the first field.
PS. Jonhson is most likely a typo.
Using all awk:
$ awk -F" [|] " -v c=10001 -v OFS=":" '{split($1,a,/:/);print c++,a[2]}' file
Output:
10001:Mary Miller
10002:Harry Jonhson
Explained:
$ awk -F" [|] " -v c=10001 -v OFS=":" '{ # some parameters
split($1,a,/:/) # split at : to get the name
print c++,a[2] # output id and name
}' file
To write the output to another file:
$ awk ... file > another_file
$ awk -F':| *[|]' -v OFS=':' -v c=10000 '{print ++c, $2}' file
10001:Mary Miller
10002:Harry Jonhson
or if you want a header line printed too:
$ awk -F':| *[|]' -v OFS=':' -v c=10000 'NR==1{print "ID", $1} {print ++c, $2}' file
ID:Name
10001:Mary Miller
10002:Harry Jonhson
How can I replace the value of a column and store the output into the same file?
example input, file:
#car|year|model
toyota|1998|corrola
toyota|2006|yaris
opel|2001|corsa
replace "corrola" with "corrolacoupe"
and store it to the input file
#car|year|model
toyota|1998|corrolacoupe
toyota|2006|yaris
opel|2001|corsa
I have tried this
awk -F '|' -v col=$column -v val=$value '/^[^#]/ FNR==NR {print $col = val }' OFS='|' $FILE >> $FILE
To simply replace the value in (row,col) with a new value:
$ awk -F'|' -v OFS='|' -v row=2 -v col=3 -v val=corollacoupe 'NR==row {$col=val} 1' file
#car|year|model
toyota|1998|corollacoupe
toyota|2006|yaris
opel|2001|corsa
This will set the value of input field col to val, but only in the input record row. The 1 in the end will ensure each record is printed by default. Input and output field separators are set via -F option and OFS variable.
If you need to make these changes in-place, create a temporary output file and then copy it over the original:
$ awk ... file >file.tmp && cp file{.tmp,}
Alternatively, in GNU awk, you can use the inplace library via -i inplace option:
$ awk -i inplace -F'|' -v OFS='|' -v row=2 -v col=3 -v val=corollacoupe 'NR==row {$col=val} 1' file
If you wish to skip the comments, and count only non-comment rows:
$ awk -F'|' -v OFS='|' -v row=1 -v col=3 -v val=x '/^[^#]/ {nr++} nr==row {$col=val} 1' file
#car|year|model
toyota|1998|x
toyota|2006|yaris
opel|2001|corsa
An ed solution that modifies the file in-place without any temporary files could be something like:
ed "$FILE" <<< $',s/|corrola$/|corrolacoupe/g\nw'
which uses an ANSI-C string to prevent special characters from being treated specially, then matches |corrola at the end of any line and replaces it with |corrolacoupe. Then we issue the w command to ed to have it write the file back out
A really simple solution.
darby#Debian:~/Scrivania$ cat file
#car|year|model
toyota|1998|corrola
toyota|2006|yaris
opel|2001|corsa
darby#Debian:~/Scrivania$ sed -ri 's#^(.+)\|(.+)\|corrola$#\1|\2|corrolacoupe#' file
darby#Debian:~/Scrivania$ cat file
#car|year|model
toyota|1998|corrolacoupe
toyota|2006|yaris
opel|2001|corsa
darby#Debian:~/Scrivania$
Here is my issue, i have a file with the entries, i would like to get just the date + the last command after the last "]: "
Aug 17 14:25:17 snaper[22134]: [ip:10.1.15.245 37985 10.1.15.18 22 uid:10000 sid:21680 tty: cwd:/data/www/hybris/hybris/bin/platform filename:/bin/ps]: /bin/ps -p 6763
How can i get it when i cat the file ?
I can get the date with:
awk '{print $1,$2,$3}'
and the last command with :
awk -F': ' '{print $NF}'
But how to combine them to get it in a single line ?
I'm not awk limited, any sed grep or other command is ok for me :)
Thanks in advance
Just remove everything between the date and the last command:
sed 's/^\(... .. ..:..:..\).*: /\1 /'
Simple solution using AWK
$awk '{print $1,$2,$3, $(NF-2), $(NF-1), $NF }' file
Aug 17 14:25:17 /bin/ps -p 6763
Using GNU grep
grep -oP '^.{15}|.*\]: \K.*' file | paste - -
Possible use tee for use 1 output to 2 commands:
echo 'Aug 17 14:25:17 snaper[22134]: [ip:10.1.15.245 37985 10.1.15.18 22 uid:10000 sid:21680 tty: cwd:/data/www/hybris/hybris/bin/platform filename:/bin/ps]: /bin/ps -p 6763' | tee >(awk -F': ' '{print $NF}') | awk '{print $1,$2,$3}' | tr '\n' ' '
and we have output as:
Aug 17 14:25:17 /bin/ps -p 6763
$ s="Aug 17 14:25:17 snaper[22134]: [ip:10.1.15.245 37985 10.1.15.18 22 uid:10000 sid:21680 tty: cwd:/data/www/hybris/hybris/bin/platform filename:/bin/ps]: /bin/ps -p 6763"
Use sed to achieve your goal,
$ sed -r 's/(^.*:[0-9]{2}) .*]:/\1/' <<< "$s"
Try following solutions too. Considering your Input_file will be having same data as shown sample.
Solution 1st: Using simple cut command.
cut -d" " -f1,2,3,14,15,16 Input_file
Solution 2nd: using awk command where I am making string snaper and ]: as field separators.
awk -F' snaper|]:' '{print $1,$4}' Input_file
Solution 3rd: making record separator as space and then printing only those lines which we need as per OP's request.
awk -v RS=" " 'NR<4||NR>13{printf("%s%s",$0,NR<3||NR<16?" ":"")}' Input_file
Solution 4th: Substituting everything from snap to till : and get whatever is OP's request.
awk '{sub(/snaper\[.*\]: /,"");print}' Input_file
Solution 5th: Using --re-interval here(as I have old version of awk) you could remove it if you latest awk version in your system too.
awk --re-interval '{match($0,/.*[0-9]{2}:[0-9]{2}:[0-9]{2}/);print substr($0,RSTART,RLENGTH),$(NF-2),$(NF-1),$NF}' Input_file
Solution 6th: using sed and substituting everything till snaper and then everything till colon and printing the match only.
sed 's/\(.[^s]*\)\(.*:\)\(.*\)/\1\3/' Input_file
Content of the file is
Feb-01-2014 one two
Mar-02-2001 three four
I'd like to format the first field (the date) to %Y%m%d format
I'm trying to use a combination of awk and date command, but somehow this is failing even though i got the feeling i'm almost there:
cat infile | awk -F"\t" '{$1=system("date -d " $1 " +%Y%m%d");print $1"\t"$2"\t"$3}' > test
this prints out date's usage pages which makes me think that the date command is triggered properly, but there is something wrong with the argument, do you see the issue somewhere?
i'm not that familiar with awk,
You don't need date for this, its simply rearranging the date string:
$ awk 'BEGIN{FS=OFS="\t"} {
split($1,t,/-/)
$1 = sprintf("%s%02d%s", t[3], (match("JanFebMarAprMayJunJulAugSepOctNovDec",t[1])+2)/3, t[2])
}1' file
20140201 one two
20010302 three four
You can use:
while read -r a _; do
date -d "$a" '+%Y%m%d'
done < file
20140201
20010302
system() returns the exit code of the command.
Instead:
cat infile | awk -F"\t" '{"date -d " $1 " +%Y%m%d" | getline d;print d"\t"$2"\t"$3}'
$ awk '{var=system("date -d "$1" +%Y%m%d | tr -d \"\\n\"");printf "%s\t%s\t%s\n", var, $2, $3}' file
201402010 one two
200103020 three four
I am working on the following bash script:
# contents of dbfake file
1 100% file 1
2 99% file name 2
3 100% file name 3
#!/bin/bash
# cat out data
cat dbfake |
# select lines containing 100%
grep 100% |
# print the first and third columns
awk '{print $1, $3}' |
# echo out id and file name and log
xargs -rI % sh -c '{ echo %; echo "%" >> "fake.log"; }'
exit 0
This script works ok, but how do I print everything in column $3 and then all columns after?
You can use cut instead of awk in this case:
cut -f1,3- -d ' '
awk '{ $2 = ""; print }' # remove col 2
If you don't mind a little whitespace:
awk '{ $2="" }1'
But UUOC and grep:
< dbfake awk '/100%/ { $2="" }1' | ...
If you'd like to trim that whitespace:
< dbfake awk '/100%/ { $2=""; sub(FS "+", FS) }1' | ...
For fun, here's another way using GNU sed:
< dbfake sed -r '/100%/s/^(\S+)\s+\S+(.*)/\1\2/' | ...
All you need is:
awk 'sub(/.*100% /,"")' dbfake | tee "fake.log"
Others responded in various ways, but I want to point that using xargs to multiplex output is rather bad idea.
Instead, why don't you:
awk '$2=="100%" { sub("100%[[:space:]]*",""); print; print >>"fake.log"}' dbfake
That's all. You don't need grep, you don't need multiple pipes, and definitely you don't need to fork shell for every line you're outputting.
You could do awk ...; print}' | tee fake.log, but there is not much point in forking tee, if awk can handle it as well.