Hello Sed/Bash/Awk experts,
I have a file full of dates in the following format:
Feb 5 2015
Nov 25 2014
Apr 16 2015
What I would like is to convert them to this format:
YYYY-MM-DD
So they should look like this:
2015-02-05
2014-11-25
2015-04-16
Thanks for your help.
You can simply use:
date -f dates.txt +%Y-%m-%d
In the -f option you can provide your input file with one date per line.
Using awk
awk 'BEGIN{x=" JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC"}
{printf "%04d-%02d-%02d\n",$3,index(x,toupper($1))/3,$2}' file
the date command is your friend here:
date --date="Feb 5 2015" +"%Y-%m-%d"
2015-02-05
so, you can say:
$ cat my_file | while read -r dt
> do
> date --date="${dt}" +"%Y-%m-%d"
> done
2015-02-05
2014-11-25
2015-04-16
paste the following:
{
month="00";
mon=toupper($1);
if(mon=="JAN") month="01";
else if(mon=="FEB") month="02";
else if(mon=="MAR") month="03";
else if(mon=="APR") month="04";
else if(mon=="MAY") month="05";
else if(mon=="JUN") month="06";
else if(mon=="JUL") month="07";
else if(mon=="AUG") month="08";
else if(mon=="SEP") month="09";
else if(mon=="OCT") month="10";
else if(mon=="NOV") month="11";
else if(mon=="DEC") month="12";
printf("%s-%s-%02d\n", $3, month, $2)
}
into a file (We'll refer to the filename as [script_filename]
execute the following:
awk -F' ' -E [script_filename] [date_filename]
Where [date_filename] refers to the file which contains the dates you wish to convert.
Related
I have been gathering data for the last 20 days using a bash script that runs every 5 minutes. I started the script with no idea how I was going to output the data. I have since found a rather cool js graph that reads from a CSV.
Only issue is my date is currently in the format of:
Fri Nov 6 07:52:02
and for the CSV I need it to be
2015-11-06 07:52:02
So I need to cat my results grep-ing for the date and convert it.
The cat/grep for the date is:
cat speeds.txt | grep 2015 | awk '{print $1" "$2" "$3" "$4}'
Any brainwaves on how I can switch this around either using bash or php?
Thanks
PS - Starting the checks again using date +%Y%m%d" "%H:%M:%S is sadly not an option :(
Assuming all of your lines contains dates:
$ cat file
Fri Nov 6 07:52:02
...
$ awk 'BEGIN {
months["Jan"] = 1;
months["Feb"] = 2;
months["Mar"] = 3;
months["Apr"] = 4;
months["May"] = 5;
months["Jun"] = 6;
months["Jul"] = 7;
months["Aug"] = 8;
months["Sep"] = 9;
months["Oct"] = 10;
months["Nov"] = 11;
months["Dec"] = 12;
}
{
month = months[$2];
printf("%s-%02d-%02d %s\n", 2015, month, $3, $4);
}' file > out
$ cat out
2015-11-06 07:52:02
...
If you only need to modify a some of the lines you can tweak the awk script a little bit, eg. match every line containing 2015:
...
# Match every line containing 2015
/2015/ {
month = months[$2];
printf("%s-%02d-%02d %s\n", 2015, month, $3, $4);
# Use next to prevent this the other print to happen for these lines
# Like 'continue' in while iterations
next;
};
# This '1' will print all other lines as well:
# Same as writing { print $0 }
1
You can use the date format to epoch time format in bash script.
date -d 'Fri Nov 6 07:52:02' +%s;
1446776522
date -d #1446776522 +"%Y-%m-%d %T "
2015-11-06 07:52:02
Since you didn't provide the input, I'll assume you have a file called speeds.txt that contains:
Fri Oct 31 07:52:02 3
Fri Nov 1 08:12:04 4
Fri Nov 2 07:43:22 5
(the 3, 4, and 5 above are just to show that you could have other data in the row, but are not necessary).
Using this command:
cat speeds.txt | cut -d ' ' -f2,3,4 | while read line ; do date -d"$line" "+2015-%m-%d %H:%M:%S" ; done;
You get the output:
2015-10-31 07:52:02
2015-11-01 08:12:04
2015-11-02 07:43:22
I want to convert 18-Aug-2015 date format to '2015-08-18' using shell script
Try this formatting:
$ date +"%Y-%m-%d"
http://www.cyberciti.biz/faq/linux-unix-formatting-dates-for-display/
The -d option is GNU specific.
Here, you don't need to do date calculation, just rewrite the string which already contains all the information:
a=$(printf '%s\n' "$Prev_date" | awk '{
printf "%04d-%02d-%02d\n", $6, \
(index("JanFebMarAprMayJunJulAugSepOctNovDec",$2)+2)/3,$3}')
Without awk, assuming your initial date is in $mydate:
IFS=- d=($mydate)
months=(Zer Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec)
z=1
while [[ ${months[$z]} != ${d[1]} ]]; do z=$((z+1)); done
printf "%s-%02d-%s\n" ${d[2]} $z ${d[0]}
In nginx access.log we have time format in GMT timezone. using awk command,converting logs to csv file.
awk '{print $4","$7,"$10","$15","$16","$17}' </usr/local/nginx/logs/access.log>access.csv
$4 - displays date in GMT (31/Jul/2015:16:03:07).
Kindly guide how to change it to IST and update in that csv file.
If you have access to GNU awk and GNU date, you could use TZ to easily get time at IST:
$ TZ='Asia/Kolkata' date -d "#1438374787"
Where the '1438374787' value is the seconds since epoch in GMT, also known as systime.
To make a systime out of a date, we could use mktime from (GNU) awk:
Convert the GMT string to epoch time directly in awk:
$ echo "a test date 31/07/2015:16:33:07" | awk '{gsub("[/:]"," ",$4); $0=$0;t=sprintf("%d %d %d %d %d %d 0",$6,$5,$4,$7,$8,$9);print(mktime(t))}'
For that to work, you need all values to be numbers (no jul, sorry). If that is not possible, you need to work out a format for an string that the command date could read correctly (sometimes not an easy task).
Placing all in one script:
#!/bin/bash --
echo "a test date 31/07/2015:16:33:07" | awk '{
split($4,A,"[:/]");
gmtdate=sprintf("%d %d %d %d %d %d",A[3],A[2],A[1],A[4],A[5],A[6]);
print gmtdate;
T1=mktime(gmtdate);
print T1;
pt = sprintf ("TZ=%s%s%s date -d %s%s%s%s","\047","Asia/Kolkata","\047","\042","#",T1,"\042");
print pt;
system(pt);
}'
And, running it:
$ ./stackoverflow-awk-time-test.sh
2015 7 31 16 33 7
1438374787
TZ='Asia/Kolkata' date -d "#1438374787"
Sat Aug 1 02:03:07 IST 2015
Several additional debug values are printed from awk (easy to remove).
sname;id;is_up;p_data
"0000";4256;100;"052263"
"006335";5228;100;"00522633"
"ABTEST";1452;100;"1522620 0"
How to i Edit the above file in unix to
Add 2 lines at the top for title and System date and time
Add ; at the end of each each row
Add the End tag at the end of the file
The final file should look like
!title
!Time: 2014-12-33
sname;id;is_up;p_data
"0000";4256;100;"052263";
"006335";5228;100;"00522633";
"ABTEST";1452;100;"1522620 0";
!End
Use sed to add ; to the end of line. If you want to skip the first line from this operation, use the address 1!.
{
echo '!title'
echo -n '!Time: '
date +%Y-%m-%d
sed '1! s/$/;/' file
echo '!End'
} > newfile
You can make use of BEGIN and END as this, together with $0=$0";" to add a ; at the end of the lines not being first one:
awk -v d="$(date)" '
BEGIN{print "!title"; print "!Time: ", d}
NR>1{$0=$0";"} 1;
END {print "!End"}
' file
See output:
$ awk -v d="$(date)" 'BEGIN{print "!title"; print "!Time: ", d} NR>1{$0=$0";"} 1; END {print "!End"}' file
!title
!Time: Wed Apr 30 15:25:53 CEST 2014
sname;id;is_up;p_data
"0000";4256;100;"052263";
"006335";5228;100;"00522633";
"ABTEST";1452;100;"1522620 0";
!End
For the date format, you should define which one you want. date alone prints everything. I would maybe go for:
$ date "+%F %T"
2014-04-30 15:41:04
I don't know too much of bash scripting and I'm trying to develop a bash script to do this operations:
I have a lot of .txt files in the same directory.
Every .txt file follows this structure:
file1.txt:
<name>first operation</name>
<operation>21</operation>
<StartTime>1292435633</StartTime>
<EndTime>1292435640</EndTime>
<name>second operation</name>
<operation>21</operation>
<StartTime>1292435646</StartTime>
<EndTime>1292435650</EndTime>
I want to search every <StartTime> line and convert it to standard date/time format (not unix timestamp) but preserving the structure <StartTime>2010-12-15 22:52</StartTime>, for example. This could be a function of search/replace, using sed? I think I could use these function that I found: date --utc --date "1970-01-01 $1 sec" "+%Y-%m-%d %T"
I want to to do the same with <EndTime> tag.
I should do this for all *.txt files in a directory.
I tried using sed but with not wanted results. As I said I don't know so much of bash scripting so any help would be appreciated.
Thank you for your help!
Regards
sed is incapable of doing date conversions; instead I would reccomend you to use a more appropriate tool like awk:
echo '<StartTime>1292435633</StartTime>' | awk '{
match($0,/[0-9]+/);
t = strftime("%F %T",substr($0,RSTART,RLENGTH),1);
sub(/[0-9]+/,t)
}
{print}'
If your input files have one tag per line, as in your structure example, it should work flawlessly.
If you need to repeat the operation for every .txt file just use a shell for:
for file in *.txt; do
awk '/^<[^>]*Time>/{
match($0,/[0-9]+/);
t = strftime("%F %T",substr($0,RSTART,RLENGTH),1);
sub(/[0-9]+/,t)
} 1' "$file" >"$file.new"
# mv "$file.new" "$file"
done
In comparison to the previous code, I have done two minor changes:
added condition /^<[^>]*Time>/ that checks if the current line starts with or
converted {print} to the shorter '1'
If the files ending with .new contain the result you were expecting, you can uncomment the line containing mv.
Using grep:
while read line;do
if [[ $line == *"<StartTime>"* || $line == *"<EndTime>"* ]];then
n=$(echo $line | grep -Po '(?<=(>)).*(?=<)')
line=${line/$n/$(date -d #$n)}
fi
echo $line >> file1.new.txt
done < file1.txt
$ cat file1.new.txt
<name>first operation</name>
<operation>21</operation>
<StartTime>Wed Dec 15 18:53:53 CET 2010</StartTime>
<EndTime>Wed Dec 15 18:54:00 CET 2010</EndTime>
<name>second operation</name>
<operation>21</operation>
<StartTime>Wed Dec 15 18:54:06 CET 2010</StartTime>
<EndTime>Wed Dec 15 18:54:10 CET 2010</EndTime>