bash loop between two given dates - bash

Question

I'm trying to create a script that will loop through files that have their filenames written in the following format: yyyymmdd.hh.filename.
The script is called with:
./loopscript.sh 20091026.00 23
./loopscript.sh 20091026.11 15
./loopscript.sh 20091026.09 20091027.17
The need is for the script to check each hour between those two given dates/hours.
e.g.
cat 20091026.00.filename |more
cat 20091026.01.filename |more
...
cat 20091026.23.filename |more
cat 20091027.01.filename |more
cat 20091027.02.filename |more
...
and so on.
any idea how to go about this? I don't have any difficulty with standard 0 - x loops. or simple for loops. Just not sure how to go about the above.

Answer 1

How about this:
#!/bin/bash
date1=$1
date2=$2
#verify dates
if ! date -d "$date1" 2>&1 > /dev/null ;
then echo "first date is invalid" ; exit 1
fi
if ! date -d "$date2" 2>&1 > /dev/null ;
then echo "second date is invalid" ; exit 1
fi
#set current and end date
current=$(date -d "$date1")
end=$(date -d "$date2 +1 hours")
#loop over all dates
while [ "$end" != "$current" ]
do
file=$(date -d "$current" +%Y%m%d.%H)
cat $file."filename" | more
current=$(date -d "$current +1 hours")
done

Answer 2

To process each file between two given date/hours, you can use the following:
#!/usr/bin/bash
#set -x
usage() {
echo 'Usage: loopscript.sh <from> <to>'
echo ' <from> MUST be yyyymmdd.hh or empty, meaning 00000000.00'
echo ' <to> can be shorter and is affected by <from>'
echo ' e.g., 20091026.00 27.01 becomes'
echo ' 20091026.00 20091027.01'
echo ' If empty, it is set to 99999999.99'
echo 'Arguments were:'
echo " '${from}'"
echo " '${to}'"
}
# Check parameters.
from="00000000.00"
to="99999999.99"
if [[ ! -z "$1" ]] ; then
from=$1
fi
if [[ ! -z "$2" ]] ; then
to=$2
fi
## Insert this to default to rest-of-day when first argument
## but no second argument. Basically just sets second
## argument to 23 so it will be transformed to end-of-day.
#if [[ ! -z "$1"]] ; then
# if [[ -z "$2"]] ; then
# to=23
# fi
#fi
if [[ ${#from} -ne 11 || ${#to} -gt 11 ]] ; then
usage
exit 1
fi
# Sneaky code to modify a short "to" based on the start of "from".
# ${#from} is the length of ${from}.
# $((${#from}-${#to})) is the length difference between ${from} and ${to}
# ${from:0:$((${#from}-${#to}))} is the start of ${from} long enough
# to make ${to} the same length.
# ${from:0:$((${#from}-${#to}))}${to} is that with ${to} appended.
# Voila! Easy, no?
if [[ ${#to} -lt ${#from} ]] ; then
to=${from:0:$((${#from}-${#to}))}${to}
fi
# Process all files, checking that they're inside the range.
echo "From ${from} to ${to}"
for file in [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].[0-9][0-9].* ; do
if [[ ! ( ${file:0:11} < ${from} || ${file:0:11} > ${to} ) ]] ; then
echo " ${file}"
fi
done
When you create the files 20091026.00.${RANDOM} through 20091028.23.${RANDOM} inclusive, this is a couple of sample runs:
pax> ./loopscript.sh 20091026.07 9
From 20091026.07 to 20091026.09
20091026.07.21772
20091026.08.31390
20091026.09.9214
pax> ./loopscript.sh 20091027.21 28.02
From 20091027.21 to 20091028.02
20091027.21.22582
20091027.22.30063
20091027.23.29437
20091028.00.14744
20091028.01.6827
20091028.02.10366
pax> ./loopscript.sh 00000000.00 99999999.99 # or just leave off the parameters.
20091026.00.25772
20091026.01.25964
20091026.02.21132
20091026.03.3116
20091026.04.6271
20091026.05.14870
20091026.06.28826
: : :
20091028.17.20089
20091028.18.13816
20091028.19.7650
20091028.20.20927
20091028.21.13248
20091028.22.9125
20091028.23.7870
As you can see, the first argument must be of the correct format yyyymmdd.hh. The second argument can be shorter since it inherits the start of the first argument to make it the correct length.
This only attempts to process files that exist (from ls) and of the correct format, not every date/hour within the range. This will be more efficient if you have sparse files (including at the start and the end of the range) since it doesn't need to check that the files exist.
By the way, this is the command that created the test files, if you're interested:
pax> for dt in 20091026 20091027 20091028 ; do
for tm in 00 01 02 ... you get the idea ... 21 22 23 ; do
touch $dt.$tm.$RANDOM
done
done
Please don't type that in verbatim and then complain that it created files like:
20091026.you.12345
20091028.idea.77
I only trimmed down the line so it fits in the code width. :-)

Answer 3

One possible solution: convert dates into standard Unix representation of "Seconds passed since the epoch" and loop, increasing this number by 3600 (number of seconds in an hour) each iteration. Example:
#!/bin/bash
# Parse your input to date and hour first, so you get:
date_from=20090911
hour_from=10
date_to=20091026
hour_to=01
i=`date --date="$date_from $hour_from:00:00" +%s`
j=`date --date="$date_to $hour_to:00:00" +%s`
while [[ $i < $j ]]; do
date -d "1970-01-01 $i sec" "+%Y%m%d.%H"
i=$[ $i + 3600 ]
done