I am looking for a way to test the following korn shell date logic for different days and times. I do not have root access to change the actual date on the system.
CUR_DAY=$(TZ="US/Eastern" date +%a)
typeset -i CUR_HOUR=$(TZ="US/Eastern" date +%H)
# Set the start and end hour given in eastern time
typeset -i START_HOUR=22
typeset -i END_HOUR=7
case $CUR_DAY in
Sun)
if [[ $CUR_HOUR -ge $START_HOUR ]]
then
echo "Send message1"
fi;;
Mon|Tue|Wed|Thu)
if [[ $CUR_HOUR -ge $START_HOUR || $CUR_HOUR -lt $END_HOUR ]]
then
echo "Send message2"
fi;;
Fri)
if [[ "$CUR_HOUR" -lt $END_HOUR ]]
then
echo "Send message3"
fi;;
esac
If you want to test just the logic (and not that date is working correctly), then allow your script to accept CUR_DAY and CUR_HOUR as arguments or via the environment, instead of always running date.
Via the environment (CUR_DAY=9 CUR_HOUR=22 myscript)
: ${CUR_DAY:=$(date +%a)}
: ${CUR_HOUR:=$(date +%H)}
Via arguments (myscript 9 22)
CUR_DAY=${1:-$(date +%a)}
CUR_HOUR=${2:-$(date +%H)}
Either approach will work in any POSIX-compliant shell.
The easiest way to test this is to conditionally add the -d argument to date (assuming the GNU implementation) to pass it a specific date and time for which you intend to test the function's behavior.
The below passes arguments straight through to date, allowing the invocation to be adapted as appropriate:
check_dates() {
typeset cur_day cur_hour start_hour end_hour
# call external date command only once to minimize external process overhead
read -r cur_day cur_hour < <(TZ="US/Eastern" date "$#" +'%a %H')
# trim leading 0s -- otherwise, values will be seen as octal when performing math
cur_day=${cur_day#0}; cur_hour=${cur_hour#0}
start_hour=22
end_hour=8
case $cur_day in
Sun)
if (( cur_hour <= start_hour )); then
echo "Send message1"
fi
;;
Mon|Tue|Wed|Thu)
if (( cur_hour >= start_hour )) || (( cur_hour < end_hour )); then
echo "Send message2"
fi
;;
Fri)
if (( cur_hour < end_hour )); then
echo "Send message3"
fi
;;
esac
}
Thereafter:
check_dates # check for current date and time
check_dates -d '2015-01-06 03:00:00 UTC' # check for a specific date and time
If you're inclined not to use "$#", and don't mind hardcoding behavior specific to GNU date, consider:
check_dates() {
typeset cur_day cur_hour for_date
for_date=$1
read -r cur_day cur_hour < <(TZ="US/Eastern" date ${for_date:+ -d "$for_date"} +'%a %H')
...
}
...which passes -d "$for_date" only if for_date is set to a non-empty value.
Related
I am writing a code in a shell script to load data from specific range but it does not stops at the data I want and instead goes past beyond that. Below is my code of shell script.
j=20180329
while [ $j -le 20180404]
do
i have problem that my loop run after the date 20180331 till 20180399 then it go to 20180401.
i want it to go from 20180331 to 20180401. not 20180332 and so on
One simple question, 3+ not so short answer...
As your request stand for shell
1. Compatible answer first
j=20180329
while [ "$j" != "20180405" ] ;do
echo $j
j=`date -d "$j +1 day" +%Y%m%d`
done
Note I used one day after as while condition is based on equality! Of course interpreting YYYYMMDD date as integer will work too:
Note 2 Care about timezone set TZ=UTC see issue further...
j=20180329
while [ $j -le 20180404 ] ;do
echo $j
j=`TZ=UTC date -d "$j +1 day" +%Y%m%d`
done
But I don't like this because if time format change, this could become an issue.
Tested under bash and shell as dash and busybox.
(using date (GNU coreutils) 8.26.
1.2 Minimize fork under POSIX shell
Before using bash bashisms, here is a way of doing this under any POSIX shell:
The power of POSIX shell is that we could use very simple converter like date and do condition over result:
#!/usr/bin/env sh
tempdir=$(mktemp -d)
datein="$tempdir/datein"
dateout="$tempdir/dateout"
mkfifo "$datein" "$dateout"
exec 5<>"$datein"
exec 6<>"$dateout"
stdbuf -i0 -o0 date -f - <"$datein" >"$dateout" +'%Y%m%d' &
datepid=$!
echo "$2" >&5
read -r end <&6
echo "$1" >&5
read -r crt <&6
while [ "$crt" -le "$end" ];do
echo $crt
echo "$crt +1 day" >&5
read -r crt <&6
done
exec 5>&-
exec 6<&-
kill "$datepid"
rm -fR "$tempdir"
Then
daterange.sh 20180329 20180404
20180329
20180330
20180331
20180401
20180402
20180403
20180404
2. bash date via printf
Under bash, you could use so-called bashisms:
Convert date to integer Epoch (Unix time), but two dates via one fork:
{
read start;
read end
} < <(date -f - +%s <<eof
20180329
20180404
eof
)
or
start=20180329
end=20180404
{ read start;read end;} < <(date -f - +%s <<<$start$'\n'$end)
Then using bash builtin printf command (note: there is $[24*60*60] -> 86400 seconds in a regular day)
for (( i=start ; i<=end ; i+=86400 )) ;do
printf "%(%Y%m%d)T\n" $i
done
3. Timezone issue!!
Warning there is an issue around summer vs winter time:
As a function
dayRange() {
local dR_Start dR_End dR_Crt
{
read dR_Start
read dR_End
} < <(date -f - +%s <<<${1:-yesterday}$'\n'${2:-tomorrow})
for ((dR_Crt=dR_Start ; dR_Crt<=dR_End ; dR_Crt+=86400 )) ;do
printf "%(%Y%m%d)T\n" $dR_Crt
done
}
Showing issue:
TZ=CET dayRange 20181026 20181030
20181026
20181027
20181028
20181028
20181029
Replacing printf "%(%Y%m%d)T\n" $dR_Crt by printf "%(%Y%m%dT%H%M)T\n" $dR_Crt could help:
20181026T0000
20181027T0000
20181028T0000
20181028T2300
20181029T2300
In order to avoid this issue, you just have to localize TZ=UTC at begin of function:
local dR_Start dR_End dR_Crt TZ=UTC
Final step for function: Avoiding useless forks
In order to improve performances, I try to reduce forks, avoiding syntax like:
for day in $(dayRange 20180329 20180404);do ...
# or
mapfile range < <(dayRange 20180329 20180404)
I use ability of function to directly set submited variables:
There is my purpose:
dayRange() { # <start> <end> <result varname>
local dR_Start dR_End dR_Crt dR_Day TZ=UTC
declare -a dR_Var='()'
{
read dR_Start
read dR_End
} < <(date -f - +%s <<<${1:-yesterday}$'\n'${2:-tomorrow})
for ((dR_Crt=dR_Start ; dR_Crt<=dR_End ; dR_Crt+=86400 )) ;do
printf -v dR_Day "%(%Y%m%d)T\n" $dR_Crt
dR_Var+=($dR_Day)
done
printf -v ${3:-dRange} "%s" "${dR_Var[*]}"
}
Then quick little bug test:
TZ=CET dayRange 20181026 20181030 bugTest
printf "%s\n" $bugTest
20181026
20181027
20181028
20181029
20181030
Seem fine. This could be used like:
dayRange 20180329 20180405 myrange
for day in $myrange ;do
echo "Doing something with string: '$day'."
done
2.2 Alternative using shell-connector
There is a shell function for adding background command in order to reduce forks.
wget https://f-hauri.ch/vrac/shell_connector.sh
. shell_connector.sh
Initiate background date +%Y%m%d and test: #0 must answer 19700101
newConnector /bin/date '-f - +%Y%m%d' #0 19700101
Then
j=20190329
while [ $j -le 20190404 ] ;do
echo $j; myDate "$j +1 day" j
done
3.++ Little bench
Let's try little 3 years range:
j=20160329
time while [ $j -le 20190328 ] ;do
echo $j;j=`TZ=UTC date -d "$j +1 day" +%Y%m%d`
done | wc
1095 1095 9855
real 0m1.887s
user 0m0.076s
sys 0m0.208s
More than 1 second on my system... Of course, there are 1095 forks!
time { dayRange 20160329 20190328 foo && printf "%s\n" $foo | wc ;}
1095 1095 9855
real 0m0.061s
user 0m0.024s
sys 0m0.012s
Only 1 fork, then bash builtins -> less than 0.1 seconds...
And with newConnector function:
j=20160329
time while [ $j -le 20190328 ] ;do echo $j
myDate "$j +1 day" j
done | wc
1095 1095 9855
real 0m0.109s
user 0m0.084s
sys 0m0.008s
Not as quick than using builtin integer, but very quick anyway.
Store the max and min dates using seconds since epoch. Don't use dates - they are not exact (GMT? UTC? etc.). Use seconds since epoch. Then increment your variable with the number of seconds in a day - ie. 24 * 60 * 60 seconds. In your loop, you can convert the number of seconds since epoch back to human readable date using date --date=#<number>. The following will work with POSIX shell and GNU's date utlity:
from=$(date --date='2018/04/04 00:00:00' +%s)
until=$(date --date='2018/04/07 00:00:00' +%s)
counter="$from"
while [ "$counter" -le "$until" ]; do
j=$(date --date=#"$counter" +%Y%m%d)
# do somth with j
echo $j
counter=$((counter + 24 * 60 * 60))
done
GNU's date is a little strange when parsing it's --date=FORMAT format string. I suggest to always feed it with %Y/%m/%d %H/%M/%S format string so that it always knows how to parse it.
I am trying the below command in Solaris. I am trying all this in bash
START_DT="$(date --date='2013-04-01' +'%Y-%m-%d')"
and its giving me the error
-bash-3.2$ START_DT="$(date --date='2013-04-01' +'%Y-%m-%d')"
date: illegal option -- date=2013-04-01
usage: date [-u] mmddHHMM[[cc]yy][.SS]
date [-u] [+format]
date -a [-]sss[.fff]
-bash-3.2$ START_DT=$(date -d '2013-04-01' +%Y-%m-%d)
date: illegal option -- d
usage: date [-u] mmddHHMM[[cc]yy][.SS]
date [-u] [+format]
date -a [-]sss[.fff]
When I try the same in RedHat it works.
$ START_DT="$(date --date='2013-04-01' +'%Y-%m-%d')"
$ echo $START_DT
'2013-04-01'
In the very same way certain arithmetic operation doesn't work. By Arithmetic operations I mean adding 10 days to a give date etc.
I have tried multiple references - 1, 2, 3 but nothing seems to work
But all this works on Redhat. May be I have a very old version of Solaris and I have to use only this server for this problem statement
I don't ahve a Solaris 10 box to test with, but it should come with perl:
perl -MTime::Piece -sle '
($fmt = shift #ARGV) =~ s/^\+//;
print Time::Piece->strptime($date, $datefmt)->strftime($fmt)
' -- -date='2017-03-15' -datefmt='%Y-%m-%d' '+%d/%m/%y'
15/03/17
With the Time::Piece::strptime function, you need to specify the datetime format of the datetime string.
And you can wrap that up in a bash function like:
mydate () {
local OPTIND OPTARG date datefmt fmt opt
while getopts ":d:f:" opt; do
case $opt in
d) date=$OPTARG ;;
f) datefmt=$OPTARG ;;
:) echo "error: missing argument for -$OPTARG; return 1 ;;
*) echo "error: unknown argument: -$OPTARG; return 1 ;;
esac
done
shift $((OPTIND-1))
if [[ -z "$date" ]]; then
command date "$#"
elif [[ -z "$datefmt" ]]; then
echo "error: missing -f date format for input date $date"
return 1
else
if [[ -z "$1" ]]; then
fmt="%c"
else
fmt=${1#+}
fi
perl -MTime::Piece -sle 'print Time::Piece->strptime($d, $df)->strftime($f)' -- \
-d="$date" \
-df="$datefmt" \
-f="$fmt"
fi
}
I am making an adjustment a bash script of mine to output the ordinal part (st, nd, rd, th) of a day of month. I am trying to adjust it to be used within a date pattern in the date command.
Here is a simple test to determine if day of month from date command is an int:
dayOfMonth=$(date +"%d") && [[ ${dayOfMonth} =~ ^[0-9]+$ ]] && echo ${dayOfMonth} is an int || echo ${dayOfMonth} is NOT an int
Output is as I expect
01 is an int
Now I put that code into a script:
#!/bin/bash
[[ ${1} =~ ^[0-9]+$ ]] && echo ${1} is an int || echo ${1} is NOT an int
Seems OK:
dateTest.sh 12
12 is an int
But when I try to use it within a date command, the script is giving me conflicting output. echo sees 12 but the test is seeing %2
date --date='06/12/2012 07:21:22' +"`dateTest.sh %d`"
12 is NOT an int
To clarify my purpose, the end result of the script will be something like this:
#!/bin/bash
n=$(date +"%d")
if [ $# -ge 1 ] ; then
n=$1
if ! [[ $n =~ ^[0-9]+$ ]] ; then
echo Arg dateInteger [$dateInteger] must be an integer.
exit 1
fi
fi
if [ $n -ge 11 -a $n -le 13 ] ; then
echo "th"
else
case $(( $n%10 )) in
1)
echo st
;;
2)
echo nd
;;
3)
echo rd
;;
*)
echo th
;;
esac
fi
So that I can it like this:
date --date='06/12/2012 07:21:22' +"%A, %d`dateTest.sh %d` of %B %Y, %I:%M:%S %p"
which would output
Tuesday, 12th of June 2012, 07:21:22 AM
Finished script re-write. Thanks to input from #GordonDavisson, I completed my script re-write: http://pastebin.com/xZ1afqqC. Now it either outputs just the ordinal from an integer, or will output a fully formatted date where you can use standard format strings from date with the addition of "%O" for the ordinal.
You're doing things in the wrong order -- you have to turn "%d" into an integer (the day of the month) before passing it to your script, not after. Consider the command:
date --date='06/12/2012 07:21:22' +"`dateTest.sh %d`"
What this does is run dateTest.sh %d, i.e. it passes "%d" as the argument to your script. The script naturally outputs "%d is not an int". This is then used as the format string for the date command, i.e. date --date='06/12/2012 07:21:22' +"%d is not an int". The date command replaces the "%d" part with the day number, and leaves the rest alone, giving "12 is not an int".
In order to make this work, you have to get the day number first, then pass that to your script. Something like this:
dateTest.sh "$(date --date='06/12/2012 07:21:22' +"%d")"
Unfortunately, your end result script also wants a bunch of other date formatting done that can't be passed to the dateTest script. I think in that case it'd be best to do it in stages:
dayWithSuffix="$(dateTest.sh "$(date --date='06/12/2012 07:21:22' +"%d")")"
date --date='06/12/2012 07:21:22' +"%A, $dayWithSuffix of %B %Y, %I:%M:%S %p"
BTW, several general scripting suggestions:
Send error/debug output to stderr, not stdout, so it doesn't get confused with the script's regular output (part of the problem here). For example, echo "${1} is NOT an int" >&2
Speaking of which, put strings that contain variables in double-quotes (as I did in that last example) to avoid weird misparsing of whitespace, wildcards, etc. Your result script, for example, contains echo Arg dateInteger [$dateInteger] must be an integer. -- you probably don't realize that under certain circumstances the [$dateInteger] part will be replaced by a list of filenames.
Finally, use $( ... ) instead of backquotes. In most cases they're equivalent, but the parenthesis version is easier to read and avoids some weird parsing oddities of the contents. Notice how I nested two such expressions in the assignment to dayWithSuffix? That's much trickier to get right with backquotes.
I have some operations to do on files last modified on a specific date. I would like to get the date, stock it in a string, then split it to test if the day corresponds to what I want.
So far, I've been trying things like that:
#!/bin/bash
for i in {45..236}; do
nom=M$i
chem=/Users/nfs/helene/soft/metAMOS-1.5rc3/$nom.fastq/Assemble/out
if [ -e $chem ]; then
IN= $(date -r $chem)
arr=(${IN//\ / })
if [[ ${arr[1]} == 'juin' && ${arr[2]} == '10' ]]; then
echo $nom
#cp $chem/proba.faa /Users/nfs/helene/metagenomes/DB/$nom.faa
fi
fi
done
exit 0
But it seems like the date isn't well stocked in $IN, and I'm not sure about the space-spliting either..
Perhaps the simple mistake is that you didn't place your assignment adjacent to =. It must be:
IN=$(date -r $chem)
And here's a simplified suggestion:
#!/bin/bash
for i in {45..236}; do
nom="M${i}"
chem="/Users/nfs/helene/soft/metAMOS-1.5rc3/${nom}.fastq/Assemble/out"
if [[ -e $chem ]]; then
read month day < <(exec date -r "$chem" '+%b %d')
if [[ $month == 'Jun' && $day == 10 ]]; then
echo "$nom"
# cp "$chem/proba.faa" "/Users/nfs/helene/metagenomes/DB/$nom.faa"
fi
fi
done
exit 0
* See date --help for a list of formats.
* <() is a form of Process Substitution. Check Bash's manual for it.
* Always place your arguments around double quotes when they have variables to avoid word splitting.
I'm trying to create a script that will loop through files that have their filenames written in the following format: yyyymmdd.hh.filename.
The script is called with:
./loopscript.sh 20091026.00 23
./loopscript.sh 20091026.11 15
./loopscript.sh 20091026.09 20091027.17
The need is for the script to check each hour between those two given dates/hours.
e.g.
cat 20091026.00.filename |more
cat 20091026.01.filename |more
...
cat 20091026.23.filename |more
cat 20091027.01.filename |more
cat 20091027.02.filename |more
...
and so on.
any idea how to go about this? I don't have any difficulty with standard 0 - x loops. or simple for loops. Just not sure how to go about the above.
How about this:
#!/bin/bash
date1=$1
date2=$2
#verify dates
if ! date -d "$date1" 2>&1 > /dev/null ;
then echo "first date is invalid" ; exit 1
fi
if ! date -d "$date2" 2>&1 > /dev/null ;
then echo "second date is invalid" ; exit 1
fi
#set current and end date
current=$(date -d "$date1")
end=$(date -d "$date2 +1 hours")
#loop over all dates
while [ "$end" != "$current" ]
do
file=$(date -d "$current" +%Y%m%d.%H)
cat $file."filename" | more
current=$(date -d "$current +1 hours")
done
To process each file between two given date/hours, you can use the following:
#!/usr/bin/bash
#set -x
usage() {
echo 'Usage: loopscript.sh <from> <to>'
echo ' <from> MUST be yyyymmdd.hh or empty, meaning 00000000.00'
echo ' <to> can be shorter and is affected by <from>'
echo ' e.g., 20091026.00 27.01 becomes'
echo ' 20091026.00 20091027.01'
echo ' If empty, it is set to 99999999.99'
echo 'Arguments were:'
echo " '${from}'"
echo " '${to}'"
}
# Check parameters.
from="00000000.00"
to="99999999.99"
if [[ ! -z "$1" ]] ; then
from=$1
fi
if [[ ! -z "$2" ]] ; then
to=$2
fi
## Insert this to default to rest-of-day when first argument
## but no second argument. Basically just sets second
## argument to 23 so it will be transformed to end-of-day.
#if [[ ! -z "$1"]] ; then
# if [[ -z "$2"]] ; then
# to=23
# fi
#fi
if [[ ${#from} -ne 11 || ${#to} -gt 11 ]] ; then
usage
exit 1
fi
# Sneaky code to modify a short "to" based on the start of "from".
# ${#from} is the length of ${from}.
# $((${#from}-${#to})) is the length difference between ${from} and ${to}
# ${from:0:$((${#from}-${#to}))} is the start of ${from} long enough
# to make ${to} the same length.
# ${from:0:$((${#from}-${#to}))}${to} is that with ${to} appended.
# Voila! Easy, no?
if [[ ${#to} -lt ${#from} ]] ; then
to=${from:0:$((${#from}-${#to}))}${to}
fi
# Process all files, checking that they're inside the range.
echo "From ${from} to ${to}"
for file in [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].[0-9][0-9].* ; do
if [[ ! ( ${file:0:11} < ${from} || ${file:0:11} > ${to} ) ]] ; then
echo " ${file}"
fi
done
When you create the files 20091026.00.${RANDOM} through 20091028.23.${RANDOM} inclusive, this is a couple of sample runs:
pax> ./loopscript.sh 20091026.07 9
From 20091026.07 to 20091026.09
20091026.07.21772
20091026.08.31390
20091026.09.9214
pax> ./loopscript.sh 20091027.21 28.02
From 20091027.21 to 20091028.02
20091027.21.22582
20091027.22.30063
20091027.23.29437
20091028.00.14744
20091028.01.6827
20091028.02.10366
pax> ./loopscript.sh 00000000.00 99999999.99 # or just leave off the parameters.
20091026.00.25772
20091026.01.25964
20091026.02.21132
20091026.03.3116
20091026.04.6271
20091026.05.14870
20091026.06.28826
: : :
20091028.17.20089
20091028.18.13816
20091028.19.7650
20091028.20.20927
20091028.21.13248
20091028.22.9125
20091028.23.7870
As you can see, the first argument must be of the correct format yyyymmdd.hh. The second argument can be shorter since it inherits the start of the first argument to make it the correct length.
This only attempts to process files that exist (from ls) and of the correct format, not every date/hour within the range. This will be more efficient if you have sparse files (including at the start and the end of the range) since it doesn't need to check that the files exist.
By the way, this is the command that created the test files, if you're interested:
pax> for dt in 20091026 20091027 20091028 ; do
for tm in 00 01 02 ... you get the idea ... 21 22 23 ; do
touch $dt.$tm.$RANDOM
done
done
Please don't type that in verbatim and then complain that it created files like:
20091026.you.12345
20091028.idea.77
I only trimmed down the line so it fits in the code width. :-)
One possible solution: convert dates into standard Unix representation of "Seconds passed since the epoch" and loop, increasing this number by 3600 (number of seconds in an hour) each iteration. Example:
#!/bin/bash
# Parse your input to date and hour first, so you get:
date_from=20090911
hour_from=10
date_to=20091026
hour_to=01
i=`date --date="$date_from $hour_from:00:00" +%s`
j=`date --date="$date_to $hour_to:00:00" +%s`
while [[ $i < $j ]]; do
date -d "1970-01-01 $i sec" "+%Y%m%d.%H"
i=$[ $i + 3600 ]
done