I trying to print date between 2 dates using while loop in a bash script.
But when i execute i am getting below error:
test.sh: line 8: [: 02-12-14: integer expression expected
Below is my code, can anyone help me out
#!/bin/bash
sdate=02-12-14
edate=02-25-14
while [ "$sdate" -le "$edate" ]
do
echo $sdate
sdate=$(date +%m-%d-%y -d "$sdate + 1 day")
done
You should store them as timestamps:
#!/bin/bash
sdate=$(date -d '2014-02-12' +%s)
edate=$(date -d '2014-02-25' +%s)
while [[ sdate -le edate ]]; do
date -d "#$sdate" '+%m-%d-%y'
sdate=$(date -d "$(date -d "#${sdate}" ) + 1 day" +%s)
done
Output:
02-12-14
02-13-14
02-14-14
02-15-14
02-16-14
02-17-14
02-18-14
02-19-14
02-20-14
02-21-14
02-22-14
02-23-14
02-24-14
02-25-14
Always prefer [[ ]] over [ ] when it comes to conditional expressions in Bash. (( )) may also be a preference.
It requires GNU date. e.g. date --version = date (GNU coreutils) 8.21 ...
mm-dd-yy is not a format acceptable by date for input so I used yyyy-mm-dd which is acceptable.
Related
I'm trying to make a script that is working from 12/24/2020 through 06/01/2021. For now I olny have it working only for 2 dates with this:
if [ "$(date +'%m%d')" != "1224" ] && [ "$(date +'%m%d')" != "1226" ];
So this script should be working from 12/24 - 01/06.
Any suggestions? Thanks
You can use the +%s option in date to convert the start/finish dates to epoch time. You supply the dates using the -d option.
Then, you retrieve the current date using +%s again and compare using standard integer comparison, like this:
start=$(date +%s -d '12/24/2020')
finish=$(date +%s -d '06/02/2021')
now=$(date +%s)
if [ $now -ge $start ] && [ $now -lt $finish ]
then
echo "Do something"
else
echo "Skip"
fi
Edit: For the finish date, its necessary to use the day after your intended finishing date and compare using -lt. This is because date will return the timestamp for the start of the specified date. So doing this, you end up having the comparison succeed until midnight on 06/01/2021. Thanks to #gordon-davisson for pointing this out.
beg='20201224'
end='20210601'
now=$(date +'%Y%m%d')
if (( beg <= now )) && (( now <= end)); then
I am writing a code in a shell script to load data from specific range but it does not stops at the data I want and instead goes past beyond that. Below is my code of shell script.
j=20180329
while [ $j -le 20180404]
do
i have problem that my loop run after the date 20180331 till 20180399 then it go to 20180401.
i want it to go from 20180331 to 20180401. not 20180332 and so on
One simple question, 3+ not so short answer...
As your request stand for shell
1. Compatible answer first
j=20180329
while [ "$j" != "20180405" ] ;do
echo $j
j=`date -d "$j +1 day" +%Y%m%d`
done
Note I used one day after as while condition is based on equality! Of course interpreting YYYYMMDD date as integer will work too:
Note 2 Care about timezone set TZ=UTC see issue further...
j=20180329
while [ $j -le 20180404 ] ;do
echo $j
j=`TZ=UTC date -d "$j +1 day" +%Y%m%d`
done
But I don't like this because if time format change, this could become an issue.
Tested under bash and shell as dash and busybox.
(using date (GNU coreutils) 8.26.
1.2 Minimize fork under POSIX shell
Before using bash bashisms, here is a way of doing this under any POSIX shell:
The power of POSIX shell is that we could use very simple converter like date and do condition over result:
#!/usr/bin/env sh
tempdir=$(mktemp -d)
datein="$tempdir/datein"
dateout="$tempdir/dateout"
mkfifo "$datein" "$dateout"
exec 5<>"$datein"
exec 6<>"$dateout"
stdbuf -i0 -o0 date -f - <"$datein" >"$dateout" +'%Y%m%d' &
datepid=$!
echo "$2" >&5
read -r end <&6
echo "$1" >&5
read -r crt <&6
while [ "$crt" -le "$end" ];do
echo $crt
echo "$crt +1 day" >&5
read -r crt <&6
done
exec 5>&-
exec 6<&-
kill "$datepid"
rm -fR "$tempdir"
Then
daterange.sh 20180329 20180404
20180329
20180330
20180331
20180401
20180402
20180403
20180404
2. bash date via printf
Under bash, you could use so-called bashisms:
Convert date to integer Epoch (Unix time), but two dates via one fork:
{
read start;
read end
} < <(date -f - +%s <<eof
20180329
20180404
eof
)
or
start=20180329
end=20180404
{ read start;read end;} < <(date -f - +%s <<<$start$'\n'$end)
Then using bash builtin printf command (note: there is $[24*60*60] -> 86400 seconds in a regular day)
for (( i=start ; i<=end ; i+=86400 )) ;do
printf "%(%Y%m%d)T\n" $i
done
3. Timezone issue!!
Warning there is an issue around summer vs winter time:
As a function
dayRange() {
local dR_Start dR_End dR_Crt
{
read dR_Start
read dR_End
} < <(date -f - +%s <<<${1:-yesterday}$'\n'${2:-tomorrow})
for ((dR_Crt=dR_Start ; dR_Crt<=dR_End ; dR_Crt+=86400 )) ;do
printf "%(%Y%m%d)T\n" $dR_Crt
done
}
Showing issue:
TZ=CET dayRange 20181026 20181030
20181026
20181027
20181028
20181028
20181029
Replacing printf "%(%Y%m%d)T\n" $dR_Crt by printf "%(%Y%m%dT%H%M)T\n" $dR_Crt could help:
20181026T0000
20181027T0000
20181028T0000
20181028T2300
20181029T2300
In order to avoid this issue, you just have to localize TZ=UTC at begin of function:
local dR_Start dR_End dR_Crt TZ=UTC
Final step for function: Avoiding useless forks
In order to improve performances, I try to reduce forks, avoiding syntax like:
for day in $(dayRange 20180329 20180404);do ...
# or
mapfile range < <(dayRange 20180329 20180404)
I use ability of function to directly set submited variables:
There is my purpose:
dayRange() { # <start> <end> <result varname>
local dR_Start dR_End dR_Crt dR_Day TZ=UTC
declare -a dR_Var='()'
{
read dR_Start
read dR_End
} < <(date -f - +%s <<<${1:-yesterday}$'\n'${2:-tomorrow})
for ((dR_Crt=dR_Start ; dR_Crt<=dR_End ; dR_Crt+=86400 )) ;do
printf -v dR_Day "%(%Y%m%d)T\n" $dR_Crt
dR_Var+=($dR_Day)
done
printf -v ${3:-dRange} "%s" "${dR_Var[*]}"
}
Then quick little bug test:
TZ=CET dayRange 20181026 20181030 bugTest
printf "%s\n" $bugTest
20181026
20181027
20181028
20181029
20181030
Seem fine. This could be used like:
dayRange 20180329 20180405 myrange
for day in $myrange ;do
echo "Doing something with string: '$day'."
done
2.2 Alternative using shell-connector
There is a shell function for adding background command in order to reduce forks.
wget https://f-hauri.ch/vrac/shell_connector.sh
. shell_connector.sh
Initiate background date +%Y%m%d and test: #0 must answer 19700101
newConnector /bin/date '-f - +%Y%m%d' #0 19700101
Then
j=20190329
while [ $j -le 20190404 ] ;do
echo $j; myDate "$j +1 day" j
done
3.++ Little bench
Let's try little 3 years range:
j=20160329
time while [ $j -le 20190328 ] ;do
echo $j;j=`TZ=UTC date -d "$j +1 day" +%Y%m%d`
done | wc
1095 1095 9855
real 0m1.887s
user 0m0.076s
sys 0m0.208s
More than 1 second on my system... Of course, there are 1095 forks!
time { dayRange 20160329 20190328 foo && printf "%s\n" $foo | wc ;}
1095 1095 9855
real 0m0.061s
user 0m0.024s
sys 0m0.012s
Only 1 fork, then bash builtins -> less than 0.1 seconds...
And with newConnector function:
j=20160329
time while [ $j -le 20190328 ] ;do echo $j
myDate "$j +1 day" j
done | wc
1095 1095 9855
real 0m0.109s
user 0m0.084s
sys 0m0.008s
Not as quick than using builtin integer, but very quick anyway.
Store the max and min dates using seconds since epoch. Don't use dates - they are not exact (GMT? UTC? etc.). Use seconds since epoch. Then increment your variable with the number of seconds in a day - ie. 24 * 60 * 60 seconds. In your loop, you can convert the number of seconds since epoch back to human readable date using date --date=#<number>. The following will work with POSIX shell and GNU's date utlity:
from=$(date --date='2018/04/04 00:00:00' +%s)
until=$(date --date='2018/04/07 00:00:00' +%s)
counter="$from"
while [ "$counter" -le "$until" ]; do
j=$(date --date=#"$counter" +%Y%m%d)
# do somth with j
echo $j
counter=$((counter + 24 * 60 * 60))
done
GNU's date is a little strange when parsing it's --date=FORMAT format string. I suggest to always feed it with %Y/%m/%d %H/%M/%S format string so that it always knows how to parse it.
Using BASH I want to loop from a start to end date at ten-minute intervals.
I tried
begin_date="2015-01-01 00:00:00"
end_date="2015-02-20 00:00:00"
d=$begin_date
while [ "$d" != "$end_date" ]; do
echo $d
d=$(date -d "${d} + 10 min" +"%Y-%m-%d %H:%M")
done
But it didn't work. Looking at Bash:Looping thru dates
#This works
d=$(date -I -d "${d} + 1 day")
#This doesn't work
d=$(date -d "${d} + 1 day" +"%Y-%m-%d")
What am I missing in the format string?
The expression date -d "${d} + 10 min" seems not to produce a date with an offset of 10 minutes. In fact, when I run your code, I see a date counter going backwards. (Posting this diagnostic as part of your question would help others see where the problem is; you should not require others to run your code just to see what it does.)
Anyway, the sane way to do this is to convert the dates to Unix epoch, then take it from there.
for ((d=$(date -d "$begin_date" +%s); d <= $(date -d "$end_date" +%s); d += 600))
do
date -d #$d +"%F %H:%M"
done
Doing date arithmetic in the shell is probably going to be rather inefficient; converting this to e.g. Awk or Perl might be worth your time if you find it's too sluggish, or need to run it lots of times.
The example you linked to just needs to be adjusted slightly:
#!/bin/bash
## User-specified dates.
# See [GNU Coreutils: Date] for more info
# [GNU Coreutils: Date]: https://www.gnu.org/software/coreutils/manual/html_node/Combined-date-and-time-of-day-items.html#Combined-date-and-time-of-day-items
begin_date="2015-01-01T00:00"
end_date="2015-01-01T00:40"
# Run through `date` to ensure iso-8601 consistency
startdate=$(date --iso-8601='minutes' --date="${begin_date}")
enddate=$(date --iso-8601='minutes' --date="${end_date}")
# Do loop
d="$startdate"
while [ "$d" != "$enddate" ]; do
echo $d
d=$(date --iso-8601='minutes' --date="$d + 10 minutes")
done
Note that the options -I and -d are equivalent to --iso-8601 and --date respectively.
This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 4 years ago.
please bear with me and my questions, but I just started programming in bash yesterday effectively.
I have a script that does a lot of stuff so far. And I came to a point where I need to iterate through variables. I created these variables before using a while loop. The $Time variable for example looks like this:
2016-01-29 17:07:00Z
2016-01-29 17:26:20Z
2016-01-29 17:26:20Z
2016-01-29 00:07:00Z
The Grabinterval variable like this:
hour
minute
minute
day
The first step for me is to check if a different variable is not empty. If it is not I go on with checking line per line inside the $Grabinterval variable what kind it is day, hour or minute.
That is done by a while loop as well and works good. But no the problems are rising. Now I want to calculate a time difference between a time that was created earlier outside of the while loop and the time given in the first line of the $Time variable. I tried this using the following code:
while read -r line; do
if [[ ! -z "$Filelocation" ]]
then
if [[ $line = "day" || $line = "days" ]]
then
Interval="Days"
GrabTimeNew=$(date +'%Y-%m-%d 00:0'$UPOFFSET':00Z')
default=$(date --date "-1 day" +'%Y-%m-%d 00:0'$UPOFFSET':00Z')
start=$(date -d"$line$Time" +%s)
end=$(date -d"$GRABtime" +%s)
TimeDiff=$(( $start - $end ))
fi
fi
done <<< $Grabinterval
This is only on part of the bigger while loop, the other two parts looking for hour and minuteare pretty much the same.
The way I tried it here using $line$Time gives me following error message:
date: invalid date ‘day2016-01-29 17:07:00Z\n2016-01-29 17:26:20Z\n2016-01-29 17:26:20Z\n2016-01-29 00:07:00Z’
So it goes over all lines instead of only the dayline I want it to go through. Is there a way to use the first line of the $Timevariable inside the $Grabinterval variable?
I would love to use a for loop, but I have no idea how to use it later in the command block to have the wanted line read in the command block of the if statement.
Thanks,
BallerNacken
EDIT: Tried something like this now, but not working either:
while read -r GI TI; do
if [[ ! -z "$Filelocation" ]]
then
if [[ $GI = "day" || $GI = "days" ]]
then
Interval="Days"
GrabTimeNew=$(date +'%Y-%m-%d 00:0'$UPOFFSET':00Z')
default=$(date --date "-1 day" +'%Y-%m-%d 00:0'$UPOFFSET':00Z')
start=$(date -d"$TI" +%s)
end=$(date -d"$GRABtime" +%s)
TimeDiff=$(( $start - $end ))
fi
if [[ $GI = "hours" || $GI = "hour" ]]
then
Interval="Hours"
GrabTimeNew=$(date +'%Y-%m-%d %H:0'$UPOFFSET':00Z')
default=$(date --date "-1 hour" +'%Y-%m-%d %H:0'$UPOFFSET':00Z')
start=$(date -d"$TI" +%s)
end=$(date -d"$GRABtime" +%s)
TimeDiff2=$(( $start - $end ))
fi
if [[ $GI = "min" || $GI = "minutes" || $GI = "minute" ]]
then
Interval="Minutes"
GrabTimeNew=$(date +'%Y-%m-%d %H:%M:20Z')
default=$(date --date "-1 minute" +'%Y-%m-%d %H:%M:00Z')
start=$(date -d"$TI" +%s)
end=$(date -d"$GRABtime" +%s)
TimeDiff3=$(( $start - $end ))
fi
fi
done < <(paste <(echo "$Grabinterval") <(echo "$Time"))
I don't get any error messages, but no variables were created inside the if statement/while loop.
You might want to try something like this:
while read -r interval datetime; do
#...
done < <(paste <(echo "$Grabinterval") <(echo "$Time"))
That will read one line from Grabinterval and the corresponding line from Time
You need to quote the variable to keep the newlines in the <<< $var.
list=$'a\nb\nc'
while read a ; do echo $a ; done <<< $list
while read a ; do echo $a ; done <<< "$list"
I want to download a bunch of files named with ISO-8601 dates. Is there a simple way to do this using bash+GNU coreutils? (Or some trick to make wget/curl to generate the list automatically, but I find that unlikely)
Similar to this question, but not restricted to weekdays: How to generate a range of nonweekend dates using tools available in bash?.
I guess that there is a simpler way to do it without that restriction.
Also related to How to generate date range for random data on bash, but not restricted to a single year.
If you have GNU date, you could do use either a for loop in any POSIX-compliant shell:
# with "for"
for i in {1..5}; do
# ISO 8601 (e.g. 2020-02-20) using -I
date -I -d "2014-06-28 +$i days"
# custom format using +
date +%Y/%m/%d -d "2014-06-28 +$i days"
done
or an until loop, this time using Bash's extended test [[:
# with "until"
d="2014-06-29"
until [[ $d > 2014-07-03 ]]; do
echo "$d"
d=$(date -I -d "$d + 1 day")
done
Note that non-ancient versions of sh will also do lexicographical comparison if you change the condition to [ "$d" \> 2014-07-03 ].
Output from either of those loops:
2014-06-29
2014-06-30
2014-07-01
2014-07-02
2014-07-03
For a more portable way to do the same thing, you could use a Perl script:
use strict;
use warnings;
use Time::Piece;
use Time::Seconds;
use File::Fetch;
my ($t, $end) = map { Time::Piece->strptime($_, "%Y-%m-%d") } #ARGV;
while ($t <= $end) {
my $url = "http://www.example.com/" . $t->strftime("%F") . ".log";
my $ff = File::Fetch->new( uri => $url );
my $where = $ff->fetch( to => '.' ); # download to current directory
$t += ONE_DAY;
}
Time::Piece, Time::Seconds and File::Fetch are all core modules. Use it like perl wget.pl 2014-06-29 2014-07-03.
Using GNU date and bash:
start=2014-12-29
end=2015-01-03
while ! [[ $start > $end ]]; do
echo $start
start=$(date -d "$start + 1 day" +%F)
done
2014-12-29
2014-12-30
2014-12-31
2015-01-01
2015-01-02
2015-01-03
If you are on macOS, then date works a bit differently from GNU date. Here's a variant to Tom Fenech's date invocation that supports both GNU and Darwin:
if [ $(uname) = 'Darwin' ]; then
d=$(date -j -v+1d -f %Y-%m-%d $d +%Y-%m-%d)
elif [ $(uname) = 'Linux' ]; then
d=$(date -I -d "$d + 1 day")
fi
I use this handy function to work with log files in the format yyyymmdd.log.gz:
function datelist { for dt in $(seq -w $1 $2) ; do date -d $dt +'%Y%m%d' 2>/dev/null ; done ; }
It accepts dates in the format yyyymmdd.
Since I didn’t know the exact date range, only that the last file was today, I ended up using this variation of the top answer, which starts at today’s date and iterates backwards one day at a time until a download fails (presumably with a 404):
d=$(date -I);
while wget "http://www.example.com/$d.log"; do
d=$(date -I -d "$d - 1 day");
done