BASH looping though time - bash

Using BASH I want to loop from a start to end date at ten-minute intervals.
I tried
begin_date="2015-01-01 00:00:00"
end_date="2015-02-20 00:00:00"
d=$begin_date
while [ "$d" != "$end_date" ]; do
echo $d
d=$(date -d "${d} + 10 min" +"%Y-%m-%d %H:%M")
done
But it didn't work. Looking at Bash:Looping thru dates
#This works
d=$(date -I -d "${d} + 1 day")
#This doesn't work
d=$(date -d "${d} + 1 day" +"%Y-%m-%d")
What am I missing in the format string?

The expression date -d "${d} + 10 min" seems not to produce a date with an offset of 10 minutes. In fact, when I run your code, I see a date counter going backwards. (Posting this diagnostic as part of your question would help others see where the problem is; you should not require others to run your code just to see what it does.)
Anyway, the sane way to do this is to convert the dates to Unix epoch, then take it from there.
for ((d=$(date -d "$begin_date" +%s); d <= $(date -d "$end_date" +%s); d += 600))
do
date -d #$d +"%F %H:%M"
done
Doing date arithmetic in the shell is probably going to be rather inefficient; converting this to e.g. Awk or Perl might be worth your time if you find it's too sluggish, or need to run it lots of times.

The example you linked to just needs to be adjusted slightly:
#!/bin/bash
## User-specified dates.
# See [GNU Coreutils: Date] for more info
# [GNU Coreutils: Date]: https://www.gnu.org/software/coreutils/manual/html_node/Combined-date-and-time-of-day-items.html#Combined-date-and-time-of-day-items
begin_date="2015-01-01T00:00"
end_date="2015-01-01T00:40"
# Run through `date` to ensure iso-8601 consistency
startdate=$(date --iso-8601='minutes' --date="${begin_date}")
enddate=$(date --iso-8601='minutes' --date="${end_date}")
# Do loop
d="$startdate"
while [ "$d" != "$enddate" ]; do
echo $d
d=$(date --iso-8601='minutes' --date="$d + 10 minutes")
done
Note that the options -I and -d are equivalent to --iso-8601 and --date respectively.

Related

Bash on macOS - Get a list of dates for every Saturday on a given year

In bash on macOS, I would like to write a small script with dates (or any other program that would do) that gives me a list of dates in the format yyyymmdd of every Saturday of a given year and saves it to a variable.
For example, if I wanted to have a list of dates for all Saturdays of the year 1850, it should somehow look like this:
var = [ 18500105, 18500112, 18500119, …, 18501228 ]
with the below code:
list=()
for month in `seq -w 1 12`; do
for day in `seq -w 1 31`; do
list=( $(gdate -d "1850$month$day" '+%A %Y%m%d' | grep 'Saturday' | egrep -o '[[:digit:]]{4}[[:digit:]]{2}[[:digit:]]{2}' | tee /dev/tty) )
done
done
However, the above command does not write anything in the array list although it gives me the right output with tee.
How can I fix these issues?
Modifying Dennis Williamson's answer slightly to suit your requirement and to add the results into the array. Works on the GNU date and not on FreeBSD's version.
#!/usr/bin/env bash
y=1850
for d in {0..6}
do
# Identify the first day of the year that is a Saturday and break out of
# the loop
if (( $(date -d "$y-1-1 + $d day" '+%u') == 6))
then
break
fi
done
array=()
# Loop until the last day of the year, increment 7 days at a
# time and append the results to the array
for ((w = d; w <= $(date -d "$y-12-31" '+%j'); w += 7))
do
array+=( $(date -d "$y-1-1 + $w day" '+%Y%m%d') )
done
Now you can just print the results as
printf '%s\n' "${array[#]}"
To set up the GNU date on MacOS you need to do brew install coreutils and access the command as gdate to distinguish it from the native version provided.
Argh, just realised you need it for MacOS date.
I will leave the answer for others that do not have that restriction, but it will not work for you.
This is not quite what you want, but close:
year=1850
firstsat=$(date -d $year-01-01-$(date -d $year-01-01 +%w)days+6days +%Y%m%d)
parset a 'date -d '$firstsat'+{=$_*=7=}days +%Y%m%d' ::: {0..52}
echo ${a[#]}
It has the bug, that it finds the next 53 Saturdays, and the last of those may not be in current year.
parset is part of GNU Parallel.
I didn't do much error checking, but here's another implementation.
Takes day of week and target year as arguments.
Gets the julian day of the first matching weekday requested -
gets the epoch seconds of noon on that day -
as long as the year matches what was requested, adds that date to the array and adds a week's worth of seconds to the tracking variable.
Lather, rinse, repeat until no longer in that year.
$: typeset -f alldays
alldays () { local dow=$1 year=$2 julian=1;
until [[ "$dow" == "$( date +%a -d $year-01-0$julian )" ]]; do (( julian++ )); done;
es=$( date +%s -d "12pm $year-01-0$julian" );
allhits=( $( while [[ $year == "$( date +%Y -d #$es )" ]]; do date +%Y%m%d -d #$es; (( es+=604800 )); done; ) )
}
$: time alldays Sat 1850
real 0m9.931s
user 0m1.025s
sys 0m6.695s
$: printf "%s\n" "${allhits[#]}"
18500105
18500112
18500119
18500126
18500202
18500209
18500216
18500223
18500302
18500309
18500316
18500323
18500330
18500406
18500413
18500420
18500427
18500504
18500511
18500518
18500525
18500601
18500608
18500615
18500622
18500629
18500706
18500713
18500720
18500727
18500803
18500810
18500817
18500824
18500831
18500907
18500914
18500921
18500928
18501005
18501012
18501019
18501026
18501102
18501109
18501116
18501123
18501130
18501207
18501214
18501221
18501228

How to generate a list of all dates in a range using the tools available in bash?

I want to download a bunch of files named with ISO-8601 dates. Is there a simple way to do this using bash+GNU coreutils? (Or some trick to make wget/curl to generate the list automatically, but I find that unlikely)
Similar to this question, but not restricted to weekdays: How to generate a range of nonweekend dates using tools available in bash?.
I guess that there is a simpler way to do it without that restriction.
Also related to How to generate date range for random data on bash, but not restricted to a single year.
If you have GNU date, you could do use either a for loop in any POSIX-compliant shell:
# with "for"
for i in {1..5}; do
# ISO 8601 (e.g. 2020-02-20) using -I
date -I -d "2014-06-28 +$i days"
# custom format using +
date +%Y/%m/%d -d "2014-06-28 +$i days"
done
or an until loop, this time using Bash's extended test [[:
# with "until"
d="2014-06-29"
until [[ $d > 2014-07-03 ]]; do
echo "$d"
d=$(date -I -d "$d + 1 day")
done
Note that non-ancient versions of sh will also do lexicographical comparison if you change the condition to [ "$d" \> 2014-07-03 ].
Output from either of those loops:
2014-06-29
2014-06-30
2014-07-01
2014-07-02
2014-07-03
For a more portable way to do the same thing, you could use a Perl script:
use strict;
use warnings;
use Time::Piece;
use Time::Seconds;
use File::Fetch;
my ($t, $end) = map { Time::Piece->strptime($_, "%Y-%m-%d") } #ARGV;
while ($t <= $end) {
my $url = "http://www.example.com/" . $t->strftime("%F") . ".log";
my $ff = File::Fetch->new( uri => $url );
my $where = $ff->fetch( to => '.' ); # download to current directory
$t += ONE_DAY;
}
Time::Piece, Time::Seconds and File::Fetch are all core modules. Use it like perl wget.pl 2014-06-29 2014-07-03.
Using GNU date and bash:
start=2014-12-29
end=2015-01-03
while ! [[ $start > $end ]]; do
echo $start
start=$(date -d "$start + 1 day" +%F)
done
2014-12-29
2014-12-30
2014-12-31
2015-01-01
2015-01-02
2015-01-03
If you are on macOS, then date works a bit differently from GNU date. Here's a variant to Tom Fenech's date invocation that supports both GNU and Darwin:
if [ $(uname) = 'Darwin' ]; then
d=$(date -j -v+1d -f %Y-%m-%d $d +%Y-%m-%d)
elif [ $(uname) = 'Linux' ]; then
d=$(date -I -d "$d + 1 day")
fi
I use this handy function to work with log files in the format yyyymmdd.log.gz:
function datelist { for dt in $(seq -w $1 $2) ; do date -d $dt +'%Y%m%d' 2>/dev/null ; done ; }
It accepts dates in the format yyyymmdd.
Since I didn’t know the exact date range, only that the last file was today, I ended up using this variation of the top answer, which starts at today’s date and iterates backwards one day at a time until a download fails (presumably with a 404):
d=$(date -I);
while wget "http://www.example.com/$d.log"; do
d=$(date -I -d "$d - 1 day");
done

Generate a date+hour sequence with a given date

Given a date in format 20130522, I need to generate a sequence of date+hour as below:
2013052112,2013052113,2013052114,...,2013052122,2013052123,
2013052200,2013052201,2013052202,...,2013052222,2013052223,
2013052300
in which the first date+hour is 12 hours before the given date and the last date+hour is the midnight in the next day of the given date.
I tried several ways but none of them is ideal. How to generate such a sequence in a clean way using shell script? Thanks!
--Edit--
Per your request, this is what I have so far:
day=20130522
begin=`date --date "$day -12 hours"`
begin=`date -d "${begin:0:8} ${begin:8:2}" +%s`
end=`date --date "$day +1 day"`
end=`date -d "${end:0:8} ${end:8:2}" +%s`
datestr=`date -d #${begin} +%Y%m%d%H`
let begin=$begin+3600
while [ $begin -le $end ]
do
hr=`date -d #${begin} +%Y%m%d%H`
datestr="$datestr,$hr"
let begin=$begin+3600
done
and this is what I got from above:
2013052100,2013052101,2013052102,...,2013052123,
2013052200,2013052201,2013052202,...,2013052223,
2013052300
You can use date and brace expansion:
date=20130522
echo $(date -d "-1 day $date" +%Y%m%d){12..23} \
"$date"{00..23} \
$(date -d "+1 day $date" +%Y%m%d)00
Output (wrapped):
2013052112 2013052113 2013052114 2013052115 2013052116 2013052117 2013052118 2013052119 2013052120
2013052121 2013052122 2013052123 2013052200 2013052201 2013052202 2013052203 2013052204 2013052205
2013052206 2013052207 2013052208 2013052209 2013052210 2013052211 2013052212 2013052213 2013052214
2013052215 2013052216 2013052217 2013052218 2013052219 2013052220 2013052221 2013052222 2013052223
2013052300
Your code was quite well. What I think is that you used so much bash conversion, while date is very powerful and handles in an easier way.
I rewrited something and now I get this:
day=20130522
begin=$(date --date "$day -12 hours" "+%s")
end=$(date --date "$day +1 day" "+%s")
hr=$(date --date "#$begin" "+%s")
while [[ $hr -lt $end ]]
do
hr=$(($hr + 3600))
echo $(date -d "#$hr" "+%Y%m%d %H")
done
$ ./script
20130521 13
20130521 14
.../...
20130522 22
20130522 23
20130523 00

command date script shell bsh/ksh

I need help.
I made a shell script that you pass a date earlier than 3 days in YYYYMMDD format and tell me if it is correct or not.
My question is. Can i subtract the date command 3 days?
thanks.
you can test :
DATE="20120803"
date -d #$(( `date -d "$DATE" +%s` - (3*24*60*60) ))
for the fancy solution:
INPUT="20120803"
INPUT_SECONDS=$(date -d "$INPUT" +%s)
THREEDAYSAGO_SECONDS=$(date -d "3 days ago" "+%s")
if [ $INPUT_SECONDS -lt $THREEDAYSAGO_SECONDS ]; then
echo "too early :("
fi
Although you can use the date command to do this (see Guillame's excellent answer) it may be worth considering a scripting language such as Perl to do more complex stuff more efficiently.
e.g. see this SO answer, using Perl and the DateTime.pm module:
use DateTime;
my $date = DateTime->now;
$date->subtract(days => 3);
print $date->ymd;

Test a file date with bash

I am trying to test how old ago a file was created (in seconds) with bash in an if statement. I need creation date, not modification.
Do you have any idea how to do this, without using a command like find with grep?
I'm afraid I cann't answer the question for creation time, but for last modification time you can use the following to get the epoch date, in seconds, since filename was last modified:
date --utc --reference=filename +%s
So you could then so something like:
modsecs=$(date --utc --reference=filename +%s)
nowsecs=$(date +%s)
delta=$(($nowsecs-$modsecs))
echo "File $filename was modified $delta secs ago"
if [ $delta -lt 120 ]; then
# do something
fi
etc..
Update
A more elgant way of doing this (again, modified time only): how do I check in bash whether a file was created more than x time ago?
Here is the best answer I found at the time being, but it's only for the modification time :
expr `date +%s` - `stat -c %Y /home/user/my_file`
If your system has stat:
modsecs=$(stat --format '%Y' filename)
And you can do the math as in Joel's answer.
you can use ls with --full-time
file1="file1"
file2="file2"
declare -a array
i=0
ls -go --full-time "$file1" "$file2" | { while read -r a b c d time f
do
time=${time%.*}
IFS=":"
set -- $time
hr=$1;min=$2;sec=$3
hr=$(( hr * 3600 ))
min=$(( min * 60 ))
totalsecs=$(( hr+min+sec ))
array[$i]=$totalsecs
i=$((i+1))
unset IFS
done
echo $(( ${array[0]}-${array[1]} ))
}

Resources