I am trying to test how old ago a file was created (in seconds) with bash in an if statement. I need creation date, not modification.
Do you have any idea how to do this, without using a command like find with grep?
I'm afraid I cann't answer the question for creation time, but for last modification time you can use the following to get the epoch date, in seconds, since filename was last modified:
date --utc --reference=filename +%s
So you could then so something like:
modsecs=$(date --utc --reference=filename +%s)
nowsecs=$(date +%s)
delta=$(($nowsecs-$modsecs))
echo "File $filename was modified $delta secs ago"
if [ $delta -lt 120 ]; then
# do something
fi
etc..
Update
A more elgant way of doing this (again, modified time only): how do I check in bash whether a file was created more than x time ago?
Here is the best answer I found at the time being, but it's only for the modification time :
expr `date +%s` - `stat -c %Y /home/user/my_file`
If your system has stat:
modsecs=$(stat --format '%Y' filename)
And you can do the math as in Joel's answer.
you can use ls with --full-time
file1="file1"
file2="file2"
declare -a array
i=0
ls -go --full-time "$file1" "$file2" | { while read -r a b c d time f
do
time=${time%.*}
IFS=":"
set -- $time
hr=$1;min=$2;sec=$3
hr=$(( hr * 3600 ))
min=$(( min * 60 ))
totalsecs=$(( hr+min+sec ))
array[$i]=$totalsecs
i=$((i+1))
unset IFS
done
echo $(( ${array[0]}-${array[1]} ))
}
Related
I'm a bash enthusiast. I've written many scripts, but this time I think this one is too big for me. I'm working on a script that would check the number of hours based on the schedule from the file and sum them up.
I have a file with weekly schedule. By using grep it returns me time schedule (for example 13:00 - 20:30 or 13:30 - 21:30). Those files will come always in the same format, only the hours will be different so my grep query will always work. The only difference is that sometimes it will return four lines (with hours), sometimes less, sometimes more, dependes what's in the given file (I can always use grep -c if the number of those lines matters).
When I subtract the finishing time from the starting time, the result is correct, although it won't work if the hours are a bit odd (for example 14:00 - 16:17). How can I achive that (2h and 17minutes)?
And then I think I should use a loop. I tried a couple of things and got super confused and decided to come here.
Here's what I've got so far (26to31August is obviously the file with the schedule).
#!/bin/bash
#COUNT=$(grep -c '[0-9][0-9]:[0-9][0-9]' 26to31August)
FINISH=$(grep '[0-9][0-9]:[0-9][0-9]' 26to31August |head -1 |awk '{ print $3 }')
START=$(grep '[0-9][0-9]:[0-9][0-9]' 26to31August |head -1 |awk '{ print $1 }')
VAR1=$(date -d "${FINISH}" +%s)
VAR2=$(date -d "${START}" +%s)
HOURS=$((VAR1-VAR2))
SECS2HOURS=$(expr $HOURS / 3600)
echo "That day it's $SECS2HOURS hours"
I want the script to echo each of those lines telling me how many hours it is per line and then add them all together.
Should I use For loop? Should I use a function in the loop?
I'll apprecieate your answers.
This may not be what you want to hear, but this is definitely a case where a different programming language will work infinitely better than Bash arithmetic. In Python, for example, you would simply do duration = finish - start. That will do a timezone-aware, leap day-aware, sane calculation resulting in an object from which you can get everything from years to fractional seconds.
Yes you would use a loop. You should maintain the running total in seconds and convert to hours:minutes in the final output
First thought (warning, this code contains a bug)
total=0
grep '[0-9][0-9]:[0-9][0-9]' 26to31August | while read -r start x finish x; do
a=$( date -d "$start" "+%s" )
b=$( date -d "$finish" "+%s" )
(( total += b - a ))
done
# output
hours=$(( total / 3600 ))
minutes=$(( (total % 2600) / 60 ))
printf "%d:%02d\n" $hours $minutes
bash can parse a whitespace-separated string into words with the read command.
Where's the bug? With bash's default configuration, each command in a pipeline is executed in a separate subshell. Therefore, the total variable's contents will be lost when the subshell running the while loop ends.
Here are two ways to get around that:
group the output commands into the same subshell
total=0
grep '[0-9][0-9]:[0-9][0-9]' 26to31August | {
while read -r start x finish x; do
a=$( date -d "$start" "+%s" )
b=$( date -d "$finish" "+%s" )
(( total += b - a ))
done
hours=$(( total / 3600 ))
minutes=$(( (total % 2600) / 60 ))
printf "%d:%02d\n" $hours $minutes
}
don't use a pipe, redirect the output of a process substitution
into the while loop
total=0
while read -r start x finish x; do
a=$( date -d "$start" "+%s" )
b=$( date -d "$finish" "+%s" )
(( total += b - a ))
done < <( grep '[0-9][0-9]:[0-9][0-9]' 26to31August )
hours=$(( total / 3600 ))
minutes=$(( (total % 2600) / 60 ))
printf "%d:%02d\n" $hours $minutes
Note, don't use ALLCAPS variable names. It's too easy to accidentally overwrite crucial shell variables.
I am writing a code in a shell script to load data from specific range but it does not stops at the data I want and instead goes past beyond that. Below is my code of shell script.
j=20180329
while [ $j -le 20180404]
do
i have problem that my loop run after the date 20180331 till 20180399 then it go to 20180401.
i want it to go from 20180331 to 20180401. not 20180332 and so on
One simple question, 3+ not so short answer...
As your request stand for shell
1. Compatible answer first
j=20180329
while [ "$j" != "20180405" ] ;do
echo $j
j=`date -d "$j +1 day" +%Y%m%d`
done
Note I used one day after as while condition is based on equality! Of course interpreting YYYYMMDD date as integer will work too:
Note 2 Care about timezone set TZ=UTC see issue further...
j=20180329
while [ $j -le 20180404 ] ;do
echo $j
j=`TZ=UTC date -d "$j +1 day" +%Y%m%d`
done
But I don't like this because if time format change, this could become an issue.
Tested under bash and shell as dash and busybox.
(using date (GNU coreutils) 8.26.
1.2 Minimize fork under POSIX shell
Before using bash bashisms, here is a way of doing this under any POSIX shell:
The power of POSIX shell is that we could use very simple converter like date and do condition over result:
#!/usr/bin/env sh
tempdir=$(mktemp -d)
datein="$tempdir/datein"
dateout="$tempdir/dateout"
mkfifo "$datein" "$dateout"
exec 5<>"$datein"
exec 6<>"$dateout"
stdbuf -i0 -o0 date -f - <"$datein" >"$dateout" +'%Y%m%d' &
datepid=$!
echo "$2" >&5
read -r end <&6
echo "$1" >&5
read -r crt <&6
while [ "$crt" -le "$end" ];do
echo $crt
echo "$crt +1 day" >&5
read -r crt <&6
done
exec 5>&-
exec 6<&-
kill "$datepid"
rm -fR "$tempdir"
Then
daterange.sh 20180329 20180404
20180329
20180330
20180331
20180401
20180402
20180403
20180404
2. bash date via printf
Under bash, you could use so-called bashisms:
Convert date to integer Epoch (Unix time), but two dates via one fork:
{
read start;
read end
} < <(date -f - +%s <<eof
20180329
20180404
eof
)
or
start=20180329
end=20180404
{ read start;read end;} < <(date -f - +%s <<<$start$'\n'$end)
Then using bash builtin printf command (note: there is $[24*60*60] -> 86400 seconds in a regular day)
for (( i=start ; i<=end ; i+=86400 )) ;do
printf "%(%Y%m%d)T\n" $i
done
3. Timezone issue!!
Warning there is an issue around summer vs winter time:
As a function
dayRange() {
local dR_Start dR_End dR_Crt
{
read dR_Start
read dR_End
} < <(date -f - +%s <<<${1:-yesterday}$'\n'${2:-tomorrow})
for ((dR_Crt=dR_Start ; dR_Crt<=dR_End ; dR_Crt+=86400 )) ;do
printf "%(%Y%m%d)T\n" $dR_Crt
done
}
Showing issue:
TZ=CET dayRange 20181026 20181030
20181026
20181027
20181028
20181028
20181029
Replacing printf "%(%Y%m%d)T\n" $dR_Crt by printf "%(%Y%m%dT%H%M)T\n" $dR_Crt could help:
20181026T0000
20181027T0000
20181028T0000
20181028T2300
20181029T2300
In order to avoid this issue, you just have to localize TZ=UTC at begin of function:
local dR_Start dR_End dR_Crt TZ=UTC
Final step for function: Avoiding useless forks
In order to improve performances, I try to reduce forks, avoiding syntax like:
for day in $(dayRange 20180329 20180404);do ...
# or
mapfile range < <(dayRange 20180329 20180404)
I use ability of function to directly set submited variables:
There is my purpose:
dayRange() { # <start> <end> <result varname>
local dR_Start dR_End dR_Crt dR_Day TZ=UTC
declare -a dR_Var='()'
{
read dR_Start
read dR_End
} < <(date -f - +%s <<<${1:-yesterday}$'\n'${2:-tomorrow})
for ((dR_Crt=dR_Start ; dR_Crt<=dR_End ; dR_Crt+=86400 )) ;do
printf -v dR_Day "%(%Y%m%d)T\n" $dR_Crt
dR_Var+=($dR_Day)
done
printf -v ${3:-dRange} "%s" "${dR_Var[*]}"
}
Then quick little bug test:
TZ=CET dayRange 20181026 20181030 bugTest
printf "%s\n" $bugTest
20181026
20181027
20181028
20181029
20181030
Seem fine. This could be used like:
dayRange 20180329 20180405 myrange
for day in $myrange ;do
echo "Doing something with string: '$day'."
done
2.2 Alternative using shell-connector
There is a shell function for adding background command in order to reduce forks.
wget https://f-hauri.ch/vrac/shell_connector.sh
. shell_connector.sh
Initiate background date +%Y%m%d and test: #0 must answer 19700101
newConnector /bin/date '-f - +%Y%m%d' #0 19700101
Then
j=20190329
while [ $j -le 20190404 ] ;do
echo $j; myDate "$j +1 day" j
done
3.++ Little bench
Let's try little 3 years range:
j=20160329
time while [ $j -le 20190328 ] ;do
echo $j;j=`TZ=UTC date -d "$j +1 day" +%Y%m%d`
done | wc
1095 1095 9855
real 0m1.887s
user 0m0.076s
sys 0m0.208s
More than 1 second on my system... Of course, there are 1095 forks!
time { dayRange 20160329 20190328 foo && printf "%s\n" $foo | wc ;}
1095 1095 9855
real 0m0.061s
user 0m0.024s
sys 0m0.012s
Only 1 fork, then bash builtins -> less than 0.1 seconds...
And with newConnector function:
j=20160329
time while [ $j -le 20190328 ] ;do echo $j
myDate "$j +1 day" j
done | wc
1095 1095 9855
real 0m0.109s
user 0m0.084s
sys 0m0.008s
Not as quick than using builtin integer, but very quick anyway.
Store the max and min dates using seconds since epoch. Don't use dates - they are not exact (GMT? UTC? etc.). Use seconds since epoch. Then increment your variable with the number of seconds in a day - ie. 24 * 60 * 60 seconds. In your loop, you can convert the number of seconds since epoch back to human readable date using date --date=#<number>. The following will work with POSIX shell and GNU's date utlity:
from=$(date --date='2018/04/04 00:00:00' +%s)
until=$(date --date='2018/04/07 00:00:00' +%s)
counter="$from"
while [ "$counter" -le "$until" ]; do
j=$(date --date=#"$counter" +%Y%m%d)
# do somth with j
echo $j
counter=$((counter + 24 * 60 * 60))
done
GNU's date is a little strange when parsing it's --date=FORMAT format string. I suggest to always feed it with %Y/%m/%d %H/%M/%S format string so that it always knows how to parse it.
In bash on macOS, I would like to write a small script with dates (or any other program that would do) that gives me a list of dates in the format yyyymmdd of every Saturday of a given year and saves it to a variable.
For example, if I wanted to have a list of dates for all Saturdays of the year 1850, it should somehow look like this:
var = [ 18500105, 18500112, 18500119, …, 18501228 ]
with the below code:
list=()
for month in `seq -w 1 12`; do
for day in `seq -w 1 31`; do
list=( $(gdate -d "1850$month$day" '+%A %Y%m%d' | grep 'Saturday' | egrep -o '[[:digit:]]{4}[[:digit:]]{2}[[:digit:]]{2}' | tee /dev/tty) )
done
done
However, the above command does not write anything in the array list although it gives me the right output with tee.
How can I fix these issues?
Modifying Dennis Williamson's answer slightly to suit your requirement and to add the results into the array. Works on the GNU date and not on FreeBSD's version.
#!/usr/bin/env bash
y=1850
for d in {0..6}
do
# Identify the first day of the year that is a Saturday and break out of
# the loop
if (( $(date -d "$y-1-1 + $d day" '+%u') == 6))
then
break
fi
done
array=()
# Loop until the last day of the year, increment 7 days at a
# time and append the results to the array
for ((w = d; w <= $(date -d "$y-12-31" '+%j'); w += 7))
do
array+=( $(date -d "$y-1-1 + $w day" '+%Y%m%d') )
done
Now you can just print the results as
printf '%s\n' "${array[#]}"
To set up the GNU date on MacOS you need to do brew install coreutils and access the command as gdate to distinguish it from the native version provided.
Argh, just realised you need it for MacOS date.
I will leave the answer for others that do not have that restriction, but it will not work for you.
This is not quite what you want, but close:
year=1850
firstsat=$(date -d $year-01-01-$(date -d $year-01-01 +%w)days+6days +%Y%m%d)
parset a 'date -d '$firstsat'+{=$_*=7=}days +%Y%m%d' ::: {0..52}
echo ${a[#]}
It has the bug, that it finds the next 53 Saturdays, and the last of those may not be in current year.
parset is part of GNU Parallel.
I didn't do much error checking, but here's another implementation.
Takes day of week and target year as arguments.
Gets the julian day of the first matching weekday requested -
gets the epoch seconds of noon on that day -
as long as the year matches what was requested, adds that date to the array and adds a week's worth of seconds to the tracking variable.
Lather, rinse, repeat until no longer in that year.
$: typeset -f alldays
alldays () { local dow=$1 year=$2 julian=1;
until [[ "$dow" == "$( date +%a -d $year-01-0$julian )" ]]; do (( julian++ )); done;
es=$( date +%s -d "12pm $year-01-0$julian" );
allhits=( $( while [[ $year == "$( date +%Y -d #$es )" ]]; do date +%Y%m%d -d #$es; (( es+=604800 )); done; ) )
}
$: time alldays Sat 1850
real 0m9.931s
user 0m1.025s
sys 0m6.695s
$: printf "%s\n" "${allhits[#]}"
18500105
18500112
18500119
18500126
18500202
18500209
18500216
18500223
18500302
18500309
18500316
18500323
18500330
18500406
18500413
18500420
18500427
18500504
18500511
18500518
18500525
18500601
18500608
18500615
18500622
18500629
18500706
18500713
18500720
18500727
18500803
18500810
18500817
18500824
18500831
18500907
18500914
18500921
18500928
18501005
18501012
18501019
18501026
18501102
18501109
18501116
18501123
18501130
18501207
18501214
18501221
18501228
Using BASH I want to loop from a start to end date at ten-minute intervals.
I tried
begin_date="2015-01-01 00:00:00"
end_date="2015-02-20 00:00:00"
d=$begin_date
while [ "$d" != "$end_date" ]; do
echo $d
d=$(date -d "${d} + 10 min" +"%Y-%m-%d %H:%M")
done
But it didn't work. Looking at Bash:Looping thru dates
#This works
d=$(date -I -d "${d} + 1 day")
#This doesn't work
d=$(date -d "${d} + 1 day" +"%Y-%m-%d")
What am I missing in the format string?
The expression date -d "${d} + 10 min" seems not to produce a date with an offset of 10 minutes. In fact, when I run your code, I see a date counter going backwards. (Posting this diagnostic as part of your question would help others see where the problem is; you should not require others to run your code just to see what it does.)
Anyway, the sane way to do this is to convert the dates to Unix epoch, then take it from there.
for ((d=$(date -d "$begin_date" +%s); d <= $(date -d "$end_date" +%s); d += 600))
do
date -d #$d +"%F %H:%M"
done
Doing date arithmetic in the shell is probably going to be rather inefficient; converting this to e.g. Awk or Perl might be worth your time if you find it's too sluggish, or need to run it lots of times.
The example you linked to just needs to be adjusted slightly:
#!/bin/bash
## User-specified dates.
# See [GNU Coreutils: Date] for more info
# [GNU Coreutils: Date]: https://www.gnu.org/software/coreutils/manual/html_node/Combined-date-and-time-of-day-items.html#Combined-date-and-time-of-day-items
begin_date="2015-01-01T00:00"
end_date="2015-01-01T00:40"
# Run through `date` to ensure iso-8601 consistency
startdate=$(date --iso-8601='minutes' --date="${begin_date}")
enddate=$(date --iso-8601='minutes' --date="${end_date}")
# Do loop
d="$startdate"
while [ "$d" != "$enddate" ]; do
echo $d
d=$(date --iso-8601='minutes' --date="$d + 10 minutes")
done
Note that the options -I and -d are equivalent to --iso-8601 and --date respectively.
This question pertains to taking action if a file has a modification date older than so many days. I'm sure it would be similar for creation date or access date, but for modification date, if I have:
file=path-name-to-some-file
N=100 # for example, N is number of days
How would I do:
if file modification time is older than N days
then
fi
Several approaches are available. One is just to ask find to do the filtering for you:
if [[ $(find "$filename" -mtime +100 -print) ]]; then
echo "File $filename exists and is older than 100 days"
fi
Another is to use GNU date to do the math:
# collect both times in seconds-since-the-epoch
hundred_days_ago=$(date -d 'now - 100 days' +%s)
file_time=$(date -r "$filename" +%s)
# ...and then just use integer math:
if (( file_time <= hundred_days_ago )); then
echo "$filename is older than 100 days"
fi
If you have GNU stat, you can ask for a file's timestamp in seconds-since-epoch, and do some math yourself (though this will potentially be a bit off on the boundary cases, since it's counting seconds -- and not taking into account leap days and such -- and not rounding to the beginning of a day):
file_time=$(stat --format='%Y' "$filename")
current_time=$(( date +%s ))
if (( file_time < ( current_time - ( 60 * 60 * 24 * 100 ) ) )); then
echo "$filename is older than 100 days"
fi
Another option, if you need to support non-GNU platforms, is to shell out to Perl (which I'll leave it to others to demonstrate).
If you're interested more generally in getting timestamp information from files, and portability and robustness constraints surrounding same, see also BashFAQ #87.
I'm surprised no one mentioned this method - it's hiding right there in man test, implied by -nt and -ot, so I suspect it's been there for a long time:
N_DAYS_AGO=/tmp/n-days-ago.$$
touch -d "$N days ago" $N_DAYS_AGO
if [ "$myfile" -ot "$N_DAYS_AGO" ]; then
...
fi
I found another great answer to the question. We use find with the -newerXY option. Here is an example:
if [[ ! $(find "$file" -newermt "8 hours ago") ]]; then
echo "File can't be older than 8 hours: $file"
exit 1
fi
Found it here: https://superuser.com/a/1106706/1090383
Create a temporary file, then set it's time to N days ago. Compare your file with the temporary file.
N=100
rf=/tmp/temp1234
echo "temporary file" &> $rf
touch -d "$N days ago" $rf
for myfile in $(find . -type f); do
if [[ $rf -nt $myfile ]]; then
echo "$myfile is older than $N days."
else
echo "$myfile is not too old."
fi
done