Covert string to a date bash - bash

I am trying to convert ProcessedTime": 1545166011.794351 to a date using date function in bash script, what is the best way to convert it ?


date -d #154166011.794351
Date command should work fine.


With GNU awk:
echo 154166011.794351 | awk '{print strftime("%Y-%m-%d %H:%M:%S",$0)}'
Output:
1974-11-20 08:53:31
Source: https://unix.stackexchange.com/a/168317/74329

Related

sed extract data only between time range

I have a file with the following entries:
MySql-DataBase-2020-09-22_183748.zip
MySql-DataBase-2020-09-22_184023.zip
MySql-DataBase-2020-09-23_205331.zip
MySql-DataBase-2020-09-23_205606.zip
MySql-DataBase-2020-09-24_200123.zip
MySql-DataBase-2020-09-24_200358.zip
MySql-DataBase-2020-09-25_115839.zip
MySql-DataBase-2020-09-25_120115.zip
MySql-DataBase-2020-09-26_094608.zip
MySql-DataBase-2020-09-26_094843.zip
MySql-DataBase-2020-09-27_122523.zip
MySql-DataBase-2020-09-27_122758.zip
MySql-DataBase-2020-10-01_230024.zip
MySql-DataBase-2020-10-01_230300.zip
MySql-DataBase-2020-10-02_120944.zip
MySql-DataBase-2020-10-02_121219.zip
MySql-DataBase-2020-10-03_151414.zip
MySql-DataBase-2020-10-03_151649.zip
MySql-DataBase-2020-10-04_211059.zip
MySql-DataBase-2020-10-04_211334.zip
MySql-DataBase-2020-10-05_064049.zip
MySql-DataBase-2020-10-05_064324.zip
I want to extract Files which are between today's date & 3 days back.
For today's date 2020/10/05, 3 days back is 2020/10/02.
The output must be:
MySql-DataBase-2020-10-02_120944.zip
MySql-DataBase-2020-10-02_121219.zip
MySql-DataBase-2020-10-03_151414.zip
MySql-DataBase-2020-10-03_151649.zip
MySql-DataBase-2020-10-04_211059.zip
MySql-DataBase-2020-10-04_211334.zip
MySql-DataBase-2020-10-05_064049.zip
I tried using this command to gets 3days back date
date --date='-3 day' '+%Y/%m/%d'
And then used these command to get output between date range
sed -n '/3day=date --date='-3 day' '+%Y/%m/%d'/,/date/p' s.txt
I am getting this error:
sed: -e expression #1, char 20: unterminated address regex
Please Help me to fix this issue. I'll be using this in a bash script.

Using a process substitution to generate the dates and feed that to grep as the patterns to search for:
grep -F -f <(for d in {0..3}; do date -d "$d days ago" "+%F"; done) file

sed -n "/$(date --date='-3 day' '+%Y-%m-%d')/,/$(date +'%y-%m-%d')/p"
MySql-DataBase-2020-10-02_120944.zip
MySql-DataBase-2020-10-02_121219.zip
MySql-DataBase-2020-10-03_151414.zip
MySql-DataBase-2020-10-03_151649.zip
MySql-DataBase-2020-10-04_211059.zip
MySql-DataBase-2020-10-04_211334.zip
MySql-DataBase-2020-10-05_064049.zip
Notice, the use of double quotes at the outermost level. Also, notice the same format of date in both boundaries.

How to change second column in shell script

I have a file with below data. I want change the second column from "201710110923" format to "2017-10-11 09:23:00" format using shell script
"2017-10-16 14:03:07", "201710110923"
"2017-10-16 14:03:22", "201710110930"
Please help.
Thanks & Regards
Rajesh

Use the following:
sed -r 's/(.*, \")([0-9]{4})([0-9]{2})([0-9]{2})([0-9]{2})([0-9]{2})/\1\2-\3-\4 \5:\6:00/g' inputfile.in
Here, we are making use of regex groups to capture the matches and then replace the output. If you want to update the file, use the -i option.

Following awk may help you in same.
awk -F"\"" '{$4=substr($4,1,4)"-"substr($4,5,2)"-"substr($4,7,2) " " substr($4,9,2)":"substr($4,11,2)":"substr($4,13,2)} 1' OFS="\"" Input_file
Output will be as follows.
"2017-10-16 14:03:07", "2017-10-11 09:23:"
"2017-10-16 14:03:22", "2017-10-11 09:30:"

bash alternative solution:
IFS=','; while read d1 d2; do
d2=$(tr -d '[:space:]' <<<"$d2")
new_d=$(date -d"$(echo ${d2:1:4}/${d2:5:2}/${d2:7:2} ${d2:9:4})" +'%Y-%m-%d %H:%M:%S')
printf '%s, "%s"\n' "$d1" "$new_d"
done < yourfile
The output:
"2017-10-16 14:03:07", "2017-10-11 09:23:00"
"2017-10-16 14:03:22", "2017-10-11 09:30:00"

how to get the next hour's date string in bash

I 'm using bash
I have a date string now, for example:
2015111301
(yyyymmddHH)
how to get the date string of next hour?
that is:
2015111302

Try this:
in="2015111301"
out="$(date -d "${in:0:8} ${in:8:2}:00:00 +1hour" '+%Y%m%d%H')"
echo "$out"
Output:
2015111302
See: 3.5.3 Shell Parameter Expansion

This below script worked for me
#!/bin/bash
tempval=$(echo 2015111301 | sed 's/\(.\{8\}\)/\1 /g')
tempval=$(echo "$tempval 1 hour")
date -d "$tempval" +%Y%m%d%H

Awk ISO to epoch

I am writing a script in bash using awk (mawk) and I would like to return the output of a system command to awk.
I have a variable called created that is in ISO format: 2013-12-26T17:03:05Z and I want to convert it from ISO to epoch. I can run this line created_epoch=system("date +\%s -d " created) and it prints the output to the terminal, but the variable created_epoch is equal to 0, not the epoch time.
Does anyone know how I can convert an ISO format to epoch in awk?
Thanks!

Try using getline into a variable from a pipe instead of system
cmd="date +\%s -d "created; cmd | getline current_time
https://www.gnu.org/software/gawk/manual/html_node/Getline_002fVariable_002fPipe.html#Getline_002fVariable_002fPipe
The system command returns the exit status returned by the command that was executed as its value.
http://www.delorie.com/gnu/docs/gawk/gawk_137.html

The system command in awk, as in C, returns the exit status of a command, and not the output of said command. As others have suggested, your best bet is using getline from a pipe
I ran into a similar problem a while back and rolled my own exec function, here it is:
# get the output of a command
function exec(cmd, data) {
while ((cmd | getline data) > 0) printf("%s", data);
close(cmd);
}
Then using the function defined above, you should do something like this:
epoch = exec("date +%s -d \"<ISO DATE>\"")
where <ISO DATE> is an ISO conformant date/timestamp.
Example:
# helper.awk is where `exec` is defined
awk '#include "helper.awk"; BEGIN
{
epoch = exec("gdate -d \"2014-02-14T12:30\" +%s");
print epoch
}'
# 1392406200

Do it like this:
created_epoch=$(date +\%s -d "$created")
echo $created_epoch
1388077385

Bash script to convert a date and time column to unix timestamp in .csv

I am trying to create a script to convert two columns in a .csv file which are date and time into unix timestamps. So i need to get the date and time column from each row, convert it and insert it into an additional column at the end containing the timestamp.
Could anyone help me? So far i have discovered the unix command to convert any give time and date to unixstamp:
date -d "2011/11/25 10:00:00" "+%s"
1322215200
I have no experience with bash scripting could anyone get me started?
Examples of my columns and rows:
Columns: Date, Time,
Row 1: 25/10/2011, 10:54:36,
Row 2: 25/10/2011, 11:15:17,
Row 3: 26/10/2011, 01:04:39,
Thanks so much in advance!

You don't provide an exerpt from your csv-file, so I'm using this one:
[foo.csv]
2011/11/25;12:00:00
2010/11/25;13:00:00
2009/11/25;19:00:00
Here's one way to solve your problem:
$ cat foo.csv | while read line ; do echo $line\;$(date -d "${line//;/ }" "+%s") ; done
2011/11/25;12:00:00;1322218800
2010/11/25;13:00:00;1290686400
2009/11/25;19:00:00;1259172000
(EDIT: Removed an uneccessary variable.)
(EDIT2: Altered the date command so the script actually works.)

this should do the job:
awk 'BEGIN{FS=OFS=", "}{t=$1" "$2; "date -d \""t"\" +%s"|getline d; print $1,$2,d}' yourCSV.csv
note
you didn't give any example. and you mentioned csv, so I assume that the column separator in your file should be "comma".
test
kent$ echo "2011/11/25, 10:00:00"|awk 'BEGIN{FS=OFS=", "}{t=$1" "$2; "date -d \""t"\" +%s"|getline d; print $1,$2,d}'
2011/11/25, 10:00:00, 1322211600

Now two imporvements:
First: No need for cat foo.csv, just stream that via < foo.csv into the while loop.
Second: No need for echo & tr to create the date stringformat. Just use bash internal pattern and substitute and do it inplace
while read line ; do echo ${line}\;$(date -d "${line//;/ }" +'%s'); done < foo.csv

Resources