Looping with an specific step in a long datetime string in bash - bash

I have a list of files with the substring YYYYMMDDHH in them (example: 2016112200 means 2016 November 22th at 00 hours). These files are: temp_2016102200.data, temp_2016102212.data, temp_2016102300.data, temp_2016102312.data, ..., temp_20170301.data. And I also have other family of files substituting temp by wind.
For each string YYYYMMDDHH I want to create a tar with the temp and its correspondent wind file. I don't want this process to stop if one or both files are missing.
My idea was to loop from 12 hours to 12 hours, but I am having some problems because to specify the date I did: b=$(date -d '2016111400' +'%Y%m%d%H') but bash informs me that that is not a valid date...
Thanks.

It's not bash telling you the date format is wrong: date is telling you. Not everything you type is a bash command.
As Kamil comments, you have to split it up so that date can parse it. The YYYY-mm-dd HH:MM:SS format is parsable. Using bash parameter expansion to extract the relevant substrings:
$ d=2016111400
$ date -d "${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00"
Mon Nov 14 00:00:00 EST 2016
Now, when you want to add 12 hours, you have to be careful to do it in the right place in the datetime string: if you add a + character after the time, it will be parsed as a timezone offset, so put the relative part either first or between the date and the time.
$ date -d "+12 hours ${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00"
Mon Nov 14 12:00:00 EST 2016
As a loop, you could do:
d=2016111400
for ((i=1; i<=10; i++)); do
# print this datetime
date -d "${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00"
# add 12 hours
d=$( date -d "+12 hours ${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00" "+%Y%m%d%H" )
done
outputs:
Mon Nov 14 00:00:00 EST 2016
Mon Nov 14 12:00:00 EST 2016
Tue Nov 15 00:00:00 EST 2016
Tue Nov 15 12:00:00 EST 2016
Wed Nov 16 00:00:00 EST 2016
Wed Nov 16 12:00:00 EST 2016
Thu Nov 17 00:00:00 EST 2016
Thu Nov 17 12:00:00 EST 2016
Fri Nov 18 00:00:00 EST 2016
Fri Nov 18 12:00:00 EST 2016
OK, a "nicer" way to loop
start=2019043000
end=2019050300
plus12hours() {
local d=$1
date -d "+12 hours ${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00" "+%Y%m%d%H"
}
for (( d = start; d <= end; d = $(plus12hours "$d") )); do
printf "%d\t%s\n" "$d" "$(date -d "${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00")"
done
2019043000 Tue Apr 30 00:00:00 EDT 2019
2019043012 Tue Apr 30 12:00:00 EDT 2019
2019050100 Wed May 1 00:00:00 EDT 2019
2019050112 Wed May 1 12:00:00 EDT 2019
2019050200 Thu May 2 00:00:00 EDT 2019
2019050212 Thu May 2 12:00:00 EDT 2019
2019050300 Fri May 3 00:00:00 EDT 2019

Related

sort by datetime format in bash

this is an example of my data
ip=1.2.3.4, setup_time=05:58:38.617 GMT Tue Mar 16 2021, foo=moshe, bar=haim
ip=2.3.4.5, setup_time=05:59:30.260 GMT Tue Mar 16 2021, foo=moshe2, bar=haim2
i would like to be able to sort by the setup_time column in bash. I know that I can't use sort because sort allow only sort by string matching and this is not a format of YYYY-MM-DD HH:mm:ss so string sorting is not possible.
so any ideas would be greatly appreciated.
thank you
#update
ok to better understand what i'm trying to achieve i created the folowing file named 1:
ip=1.2.3.4, setup_time=06:58:38.617 GMT Tue Mar 16 2021, foo=moshe, bar=haim
ip=2.3.4.5, setup_time=05:59:30.260 GMT Tue Mar 17 2021, foo=moshe2, bar=haim2
ip=2.3.4.5, setup_time=06:50:30.260 GMT Tue Mar 18 2021, foo=moshe2, bar=haim2
so I executed this:
cat 1 | sed 's/, /!/g' | sort -t '!' -k2,2
what i did here is replaced , with ! so i can use a delimiter in sort, the problem is that sort is doing string sorting and not timestamp kind of sorting so the output is:
ip=2.3.4.5!setup_time=05:59:30.260 GMT Tue Mar 17 2021!foo=moshe2!bar=haim2
ip=2.3.4.5!setup_time=06:50:30.260 GMT Tue Mar 18 2021!foo=moshe2!bar=haim2
ip=1.2.3.4!setup_time=06:58:38.617 GMT Tue Mar 16 2021!foo=moshe!bar=haim
Sort is able to deal with month names, thanks to the option M
No need to change , into !. Use the white space as delimiter and just issue:
LC_ALL=en sort -k7nr -k5Mr -k6nr -k2r sample
If you use this as content of the file sample:
ip=2.3.4.5, setup_time=05:59:30.260 GMT Tue Apr 1 2021, foo=moshe2, bar=haim2
ip=2.3.4.5, setup_time=05:59:30.260 GMT Tue Mar 17 2021, foo=moshe2, bar=haim2
ip=1.2.3.4, setup_time=06:58:38.617 GMT Tue Mar 16 2021, foo=moshe, bar=haim
ip=1.2.3.4, setup_time=06:58:38.617 GMT Tue Feb 28 2021, foo=moshe, bar=haim
ip=2.3.4.5, setup_time=06:50:30.260 GMT Tue Mar 18 2020, foo=moshe2, bar=haim2
ip=2.3.4.5, setup_time=06:50:30.260 GMT Tue Mar 18 2021, foo=moshe2, bar=haim2
you will get this as output:
ip=2.3.4.5, setup_time=05:59:30.260 GMT Tue Apr 1 2021, foo=moshe2, bar=haim2
ip=2.3.4.5, setup_time=06:50:30.260 GMT Tue Mar 18 2021, foo=moshe2, bar=haim2
ip=2.3.4.5, setup_time=05:59:30.260 GMT Tue Mar 17 2021, foo=moshe2, bar=haim2
ip=1.2.3.4, setup_time=06:58:38.617 GMT Tue Mar 16 2021, foo=moshe, bar=haim
ip=1.2.3.4, setup_time=06:58:38.617 GMT Tue Feb 28 2021, foo=moshe, bar=haim
ip=2.3.4.5, setup_time=06:50:30.260 GMT Tue Mar 18 2020, foo=moshe2, bar=haim2
Specifying -k7 means to sort on the seventh field. The r option reverses the order of sorting to descending. The M option sorts according the name of the month. The n option sorts numerically. To sort on the time, just consider the whole second field (beginning with the string setup_time=) as a fixed length string using -k2.
LC_ALL=en in the begin of the command line tells the system to use the English names of the months.
A solution involving awk:
awk '
{
year = substr($7, 1, length($7)-1)
cmd ="date --date=\""$3" "$4" "$5" "$6" "$year"\" +%s"
cmd | getline var
print var, $0
close(cmd)
}' file | sort -k 1 | cut -f 1- -d' '
The trick is that date --date="GMT Tue Mar 18 2021" will parse the date heuristically (meaning it will also work with gdate --date="GMT Tue 18 Mar 2021"), and then you can print only the seconds since epoch.
awk will output the seconds as first column, you sort by it, then you remove the first column from the result.
Biggest advantage of this solution is that it will work for other types of date formats (within reason of course).
Note1: for this to work you need GNU date (on Mac OS gdate, for example)
Note2: instead of awk you could use also bash with while/read (as in Read a file line by line assigning the value to a variable), but awk is rather standard, so not sure if it is a big difference for you.
If you have a sort with month name support -- use that. Pierre's solution is elegant!
If you don't, convert the date to ISO 8601 (which sorts lexicographically) and use a Schwartzian transform or a Decorate / Sort / Undecorate pattern.
The easiest, since the date you have is non standard, is use Perl to decorate, sort to sort on the first field, then cut to undecorate (remove the added field):
perl -lnE '
BEGIN{
%m2n = qw(Jan 01 Feb 02 Mar 03 Apr 04 May 05 Jun 06
Jul 07 Aug 08 Sep 09 Oct 10 Nov 11 Dec 12
);}
m/setup_time=([\d:]+).*?(\w\w\w) (\d\d?) (\d\d\d\d),/;
$mon=$m2n{$2};
say "$4$mon$3$1\t$_"' YourFile | sort -t $'\t' -r -k1,1 | cut -d $'\t' -f2-
Using pierre's data, prints:
ip=2.3.4.5, setup_time=05:59:30.260 GMT Tue Apr 1 2021, foo=moshe2, bar=haim2
ip=2.3.4.5, setup_time=06:50:30.260 GMT Tue Mar 18 2021, foo=moshe2, bar=haim2
ip=2.3.4.5, setup_time=05:59:30.260 GMT Tue Mar 17 2021, foo=moshe2, bar=haim2
ip=1.2.3.4, setup_time=06:58:38.617 GMT Tue Mar 16 2021, foo=moshe, bar=haim
ip=1.2.3.4, setup_time=06:58:38.617 GMT Tue Feb 28 2021, foo=moshe, bar=haim
ip=2.3.4.5, setup_time=06:50:30.260 GMT Tue Mar 18 2020, foo=moshe2, bar=haim2

Cron Job in Spring Boot

I want to automate a task quarterly in a year.
Task should execute
Jan 1st
April 1st
July 1st
October 1st
etc
As I tried #Scheduled(cron = "0 0 6 1 1/3 ?") in Spring Boot application but its not working currently and mail didn't trigger quarterly.
Try this
#scheduled(cron = "0 0 6 1 */3 *")
So read the first day of each 3 month at 6:00
Try this one -
#scheduled(cron ="0 0 0 1 JAN,APR,JUL,OCT ? *")
Next execution:
Wed Jul 01 00:00:00 UTC 2020
Thu Oct 01 00:00:00 UTC 2020
Fri Jan 01 00:00:00 UTC 2021
Thu Apr 01 00:00:00 UTC 2021
Thu Jul 01 00:00:00 UTC 2021
Fri Oct 01 00:00:00 UTC 2021
Sat Jan 01 00:00:00 UTC 2022
Fri Apr 01 00:00:00 UTC 2022
Fri Jul 01 00:00:00 UTC 2022
Sat Oct 01 00:00:00 UTC 2022

Add seconds to current time in Bash [duplicate]

This question already has answers here:
Bash script/command to print out date 5 min before/after
(4 answers)
Closed 5 years ago.
I want to add 10 seconds 10 times. But I don't know well how to add times to the value.
This is my code.
./time.sh
time=$(date)
counter=1
while [ $counter -le 10 ]
do
echo "$time"
time=$('$time + 10 seconds') //error occurred.
((counter++))
done
echo All done
Using GNU Date
Assuming GNU date, replace:
time=$('$time + 10 seconds')
with:
time=$(date -d "$time + 10 seconds")
Putting it all together, try:
$ cat a.sh
t=$(date)
counter=1
while [ "$counter" -le 10 ]
do
echo "$t"
t=$(date -d "$t + 10 seconds")
((counter++))
done
echo All done
(I renamed time to t because time is also a bash built-in command and it is best to avoid potential confusion.)
When run, the output looks like:
$ bash a.sh
Tue Jan 16 19:19:44 PST 2018
Tue Jan 16 19:19:54 PST 2018
Tue Jan 16 19:20:04 PST 2018
Tue Jan 16 19:20:14 PST 2018
Tue Jan 16 19:20:24 PST 2018
Tue Jan 16 19:20:34 PST 2018
Tue Jan 16 19:20:44 PST 2018
Tue Jan 16 19:20:54 PST 2018
Tue Jan 16 19:21:04 PST 2018
Tue Jan 16 19:21:14 PST 2018
All done
Using Bash (>4.2)
Recent versions of bash support date calculations without external utilities. Try:
$ cat b.sh
#!/bin/bash
printf -v t '%(%s)T' -1
counter=1
while [ "$counter" -le 10 ]
do
((t=t+10))
printf '%(%c)T\n' "$t"
((counter++))
done
echo All done
Here, t is time since epoch in seconds.
When run, the output looks like:
$ bash b.sh
Tue 16 Jan 2018 07:31:44 PM PST
Tue 16 Jan 2018 07:31:54 PM PST
Tue 16 Jan 2018 07:32:04 PM PST
Tue 16 Jan 2018 07:32:14 PM PST
Tue 16 Jan 2018 07:32:24 PM PST
Tue 16 Jan 2018 07:32:34 PM PST
Tue 16 Jan 2018 07:32:44 PM PST
Tue 16 Jan 2018 07:32:54 PM PST
Tue 16 Jan 2018 07:33:04 PM PST
Tue 16 Jan 2018 07:33:14 PM PST
All done

How to fetch 5th and 6th day of every month in Unix?

Is it possible to fetch 5th and/or 6th day of every month in Unix?
I've tried this but it does not give me the desired output:
echo $((($(date +%-d)-1)/5))
Use this:
for month in {1..12}; do date -d "$month/5/2017" +"%c - is a %A"; done
Output:
Thu 05 Jan 2017 12:00:00 AM CET - is a Thursday
Sun 05 Feb 2017 12:00:00 AM CET - is a Sunday
Sun 05 Mar 2017 12:00:00 AM CET - is a Sunday
Wed 05 Apr 2017 12:00:00 AM CEST - is a Wednesday
Fri 05 May 2017 12:00:00 AM CEST - is a Friday
Mon 05 Jun 2017 12:00:00 AM CEST - is a Monday
Wed 05 Jul 2017 12:00:00 AM CEST - is a Wednesday
Sat 05 Aug 2017 12:00:00 AM CEST - is a Saturday
Tue 05 Sep 2017 12:00:00 AM CEST - is a Tuesday
Thu 05 Oct 2017 12:00:00 AM CEST - is a Thursday
Sun 05 Nov 2017 12:00:00 AM CET - is a Sunday
Tue 05 Dec 2017 12:00:00 AM CET - is a Tuesday
For the 6th of the month, it's done in a similar way.
I am not sure if this is what you are looking for: i am assuming you want to get the 5th(or 6th) day of a month sitting on any day. If so, you can make use of the current date and get the 5th day of current month like below:
dd=`date '+%d'`
if(( $dd > 5 )); then
(( diff = dd - 5 ))
myDate=`date -d "-$diff days"`
else
(( diff = 5 - dd ))
myDate=`date -d "+$diff days"`
fi
echo $myDate
For 6th day you can do similarly. Above code should work on Linux distro.

How to format bash output depending on string length?

Right now, I am running the following command:
rpm -qa --queryformat '%{name}\t%{installtime:date}\n' | sort -nr
and getting some output like this:
dhclient Fri 07 Feb 2014 01:37:47 PM EST
device-mapper-persistent-data Fri 07 Feb 2014 01:27:37 PM EST
device-mapper-libs Fri 07 Feb 2014 01:34:44 PM EST
device-mapper Fri 07 Feb 2014 01:34:46 PM EST
device-mapper-event-libs Fri 07 Feb 2014 01:34:48 PM EST
device-mapper-event Fri 07 Feb 2014 01:34:50 PM EST
dbus-libs Fri 07 Feb 2014 01:25:28 PM EST
dbus-glib Fri 07 Feb 2014 01:33:48 PM EST
db4-utils Fri 07 Feb 2014 01:30:05 PM EST
db4 Fri 07 Feb 2014 01:24:58 PM EST
dash Fri 07 Feb 2014 01:30:19 PM EST
cyrus-sasl-lib Fri 07 Feb 2014 01:25:48 PM EST
(note the odd tabs)
How do I tell the command I want it to output it into a table with common spacing instead of specifying the number of tabs?
Extra Question:
What I'm trying to do is just find out what has been installed and when so I can uninstall everything that I installed recently. How do I do that better than what I'm doing?
rpm -qa --queryformat '%-40{name} %{installtime:date}\n' | sort -nr
^^^
This will left-align the name and pad it to 40 characters.
If you want to order by time, you could print the numeric time first so it's easy to sort by.
$ rpm -qa --queryformat '%-10{installtime} %{installtime:date} %{name}\n' | sort -n
...
1375369678 Thu 01 Aug 2013 11:07:58 AM EDT xorg-x11-util-macros
1375886901 Wed 07 Aug 2013 10:48:21 AM EDT libdc1394
1378148462 Mon 02 Sep 2013 03:01:02 PM EDT gnome-system-monitor
1384526666 Fri 15 Nov 2013 09:44:26 AM EST perl-File-Next
1384526667 Fri 15 Nov 2013 09:44:27 AM EST ack
1385065567 Thu 21 Nov 2013 03:26:07 PM EST trousers
1385065568 Thu 21 Nov 2013 03:26:08 PM EST tpm-tools
1387405750 Wed 18 Dec 2013 05:29:10 PM EST libusb1

Resources