I am looking for a bash script to who check time difference between two directory? But in $oldtime i can`t get only 20:09:24 i get this "20:09:24.660157390".
How can i get only hours:min only.
#!/bin/bash
oldtime=$(stat -c %y /mnt/dir1| awk '{print $2}' )
ctime=$(date | awk '{print $4}')
DIFF=$($oldtime-$ctime)
if [ $DIFF > 600 ]; then
echo "This directory have more that 10 min"
fi
You could shave off the extra suffix,
but that won't really help you.
20:09:24 - $ctime is not going to work,
because "20:09:24" is not a number.
You have several other syntax errors as well.
To calculate the difference between dates,
you need to work with seconds.
oldseconds=$(stat -c %Y /mnt/dir1)
newseconds=$(date +%s)
if (((newseconds - oldseconds) / 60)); then
echo "This directory have more than 10 min"
fi
Use epoch timestamp and date for converting to that format.
$ date +%H:%M -d #`stat -c %Y /tmp/`
19:19
But it's not recommended to use this format for comparing. Use unix timestamps instead:
if [ $(( $(date +%s) - $(stat -c %Y /tmp) )) -gt 600 ]; then
echo "This directory have more that 10 min"
fi
Related
I am trying to write a bash script, to merge 24 files in a given day. The requirement changes during Day light saving time changes, where I get 23 or 25 files.
So, with further research I realized that day-light savings begins on the second Sunday of March(23) of every year and ends on first sunday of Novemeber(25).
I need more inputs to get second sunday in a given month to do the check of finding 23 or 25 files for March and November respectively.
Any inputs to help me with this will be really appreciated.
Thank you
Here is the sample code to find 24 files in a day-
if [ -z "$1" ];then
now=$(date -d "-1 days" +%Y-%m-%d);
else now=$1;
fi
load_date='load_date='$now
singlePath="$newPath/$load_date"
fileCount=$(hdfs dfs -ls -R $hdfsPath/$load_date/ | grep -E '^-' | wc -l)
path=$hdfsPath/$load_date
if [ $fileCount -eq 24 ]; then
echo "All files are available for "$load_date;
hadoop fs -cat $path/* | hadoop fs -put - $singlePath/messages.txt
else echo $fileCount" files are available for "$load_date"! Please note, few files are being missed";
fi
I wouldn't hardcode the dates of DST transistions. I would just count "how many hours did today have":
a "normal" day:
$ diff=$(( $(date -d now +%s) - $(date -d yesterday +%s) ))
$ echo $(( diff / 3600 ))
24
"spring forward"
$ diff=$(( $(date -d "2019-03-10 23:59:59" +%s) - $(date -d "2019-03-09 23:59:59" +%s) ))
$ echo $(( diff / 3600 ))
23
"fall back"
$ diff=$(( $(date -d "2019-11-03 23:59:59" +%s) - $(date -d "2019-11-02 23:59:59" +%s) ))
$ echo $(( diff / 3600 ))
25
One thing to note: since bash only does integer arithmetic, if the difference is not 86400 but 86399, you get:
$ echo $((86399 / 3600))
23
So, better to query yesterday's time first in the tiny-but-non-zero chance that the seconds tick over between the 2 date calls:
diff=$(( -$(date -d yesterday +%s) + $(date -d now +%s) ))
Here, $diff will be 86400 or 86401 (for non DST transition days), and dividing by 3600 will give 24 not 23.
I have two dates as follows:
2019-01-06 00:02:10 | END
2019-01-05 23:52:00 | START
How could I calculate and print the difference between START and END dates in seconds?
For above case I would like to get something like:
610
Assuming GNU implementation based OS, you can use date's option %s and -d to calculate the time difference in seconds using command substitution and arithmetic operations.
START="2019-01-05 23:52:00"
END="2019-01-06 00:02:10"
Time_diff_in_secs=$(($(date -d "$END" +%s) - $(date -d "$START" +%s)))
echo $Time_diff_in_secs
Output:
610
Hope this helps!!!
With bash and GNU date:
while read d t x x; do
[[ $x == "END" ]] && end="$d $t"
[[ $x == "START" ]] && start="$d $t"
done < file
end=$(date -u -d "$end" '+%s')
start=$(date -u -d "$start" '+%s')
diff=$(($end-$start))
echo "$diff"
Output:
610
See: man date
What you're asking for is difficult verging on impossible using pure bash. Bash doesn't have any date functions of its own. For date processing, most recommendations you'll get will be to use your operating system's date command, but the usage of this command varies by operating system.
In BSD (including macOS):
start="2019-01-05 23:52:00"; end="2019-01-06 00:02:10"
printf '%d\n' $(( $(date -j -f '%F %T' "$end" '+%s') - $(date -j -f '%F %T' "$start" '+%s') ))
In Linux, or anything using GNU date (possibly also Cygwin):
printf '%d\n' $(( $(date -d "$end" '+%s') - $(date -d "$start" '+%s') ))
And just for the fun of it, if you can't (or would prefer not to) use date for some reason, you might be able to get away with gawk:
gawk 'END{ print mktime(gensub(/[^0-9]/," ","g",end)) - mktime(gensub(/[^0-9]/," ","g",start)) }' start="$start" end="$end" /dev/null
The mktime() option parses a date string in almost exactly the format you're providing, making the math easy.
START="2019-01-05 23:52:00"
END="2019-01-06 00:02:10"
parse () {
local data=(`grep -oP '\d+' <<< "$1"`)
local y=$((${data[0]}*12*30*24*60*60))
local m=$((${data[1]}*30*24*60*60))
local d=$((${data[2]}*24*60*60))
local h=$((${data[3]}*60*60))
local mm=$((${data[4]}*60))
echo $((y+m+d+h+mm+${data[5]}))
}
START=$(parse "$START")
END=$(parse "$END")
echo $((END-START)) // OUTPUT: 610
Was trying to solve the same problem on a non-GNU OS, i.e. macOS. I couldn't apply any of the solutions above, although it inspired me to come up with the following solution. I am using some in-line Ruby from within my shell script, which should work out of the box on macOS.
START="2019-01-05 23:52:00"
END="2019-01-06 00:02:10"
SECONDS=$(ruby << RUBY
require 'date'
puts ((DateTime.parse('${END}') - DateTime.parse('${START}')) * 60 * 60 * 24).to_i
RUBY)
echo ${SECONDS}
# 610
I have date and time format like this(yearmonthday):
20141105 11:30:00
I need assignment year, month, day, hour and minute values to variable.
I can do it year, day and hour like this:
year=$(awk '{print $1}' log.log | sed 's/^\(....\).*/\1/')
day=$(awk '{print $1}' log.log | sed 's/^.*\(..\).*/\1/')
hour=$(awk '{print $2}' log.log | sed 's/^\(..\).*/\1/')
How can I do this for month and minute?
--
And I need that every line of my log file:
20141105 11:30:00 /bla/text.1
20141105 11:35:00 /bla/text.2
20141105 11:40:00 /bla/text.3
....
I'm trying read line by line this log file and do this:
mkdir -p "/bla/backup/$year/$month/$day/$hour/$minute"
mv $file "/bla/backup/$year/$month/$day/$hour/$minute"
Here is my not working code:
#!/bin/bash
LOG=/var/log/LOG
while read line
do
year=${line:0:4}
month=${line:4:2}
day=${line:6:2}
hour=${line:9:2}
minute=${line:12:2}
file=$(awk '{print $3}')
if [ -f "$file" ]; then
printf -v path "%s/%s/%s/%s/%s" $year $month $day $hour $minute
mkdir -p "/bla/backup/$path"
mv $file "/bla/backup/$path"
fi
done < $LOG
You don't need to call out to awk to date at all, use bash's substring operations
d="20141105 11:30:00"
yr=${d:0:4}
mo=${d:4:2}
dy=${d:6:2}
hr=${d:9:2}
mi=${d:12:2}
printf -v dir "/bla/%s/%s/%s/%s/%s/\n" $yr $mo $dy $hr $mi
echo "$dir"
/bla/2014/11/05/11/30/
Or directly, without all the variables.
printf -v dir "/bla/%s/%s/%s/%s/%s/\n" ${d:0:4} ${d:4:2} ${d:6:2} ${d:9:2} ${d:12:2}
Given your log file:
while read -r date time file; do
d="$date $time"
printf -v dir "/bla/%s/%s/%s/%s/%s/\n" ${d:0:4} ${d:4:2} ${d:6:2} ${d:9:2} ${d:12:2}
mkdir -p "$dir"
mv "$file" "$dir"
done < filename
or, making a big assumption that there are no whitespace or globbing characters in your filenames:
sed -r 's#(....)(..)(..) (..):(..):.. (.*)#mv \6 /blah/\1/\2/\3/\4/\5#' | sh
date command also do this work
#!/bin/bash
year=$(date +'%Y' -d'20141105 11:30:00')
day=$(date +'%d' -d'20141105 11:30:00')
month=$(date +'%m' -d'20141105 11:30:00')
minutes=$(date +'%M' -d'20141105 11:30:00')
echo "$year---$day---$month---$minutes"
You can use only one awk
month=$(awk '{print substr($1,5,2)}' log.log)
year=$(awk '{print substr($1,0,4)}' log.log)
minute=$(awk '{print substr($2,4,2)}' log.log)
etc
I guess you are processing the log file, which each line starts with the date string. You may have already written a loop to handle each line, in your loop, you could do:
d="$(awk '{print $1,$2}' <<<"$line")"
year=$(date -d"$d" +%Y)
month=$(date -d"$d" +%m)
day=$(date -d"$d" +%d)
min=$(date -d"$d" +%M)
Don't repeat yourself.
d='20141105 11:30:00'
IFS=' ' read -r year month day min < <(date -d"$d" '+%Y %d %m %M')
echo "year: $year"
echo "month: $month"
echo "day: $day"
echo "min: $min"
The trick is to ask date to output the fields you want, separated by a character (here a space), to put this character in IFS and ask read to do the splitting for you. Like so, you're only executing date once and only spawn one subshell.
If the date comes from the first line of the file log.log, here's how you can assign it to the variable d:
IFS= read -r d < log.log
eval "$(
echo '20141105 11:30:00' \
| sed 'G;s/\(....\)\(..\)\(..\) \(..\):\(..\):\(..\) *\(.\)/Year=\1\7Month=\2\7Day=\3\7Hour=\4\7Min=\5\7Sec=\6/'
)"
pass via a assignation string to evaluate. You could easily adapt to also check the content by replacing dot per more specific pattern like [0-5][0-9] for min and sec, ...
posix version so --posix on GNU sed
I wrote a function that I usually cut and paste into my script files
function getdate()
{
local a
a=(`date "+%Y %m %d %H %M %S" | sed -e 's/ / /'`)
year=${a[0]}
month=${a[1]}
day=${a[2]}
hour=${a[3]}
minute=${a[4]}
sec=${a[5]}
}
in the script file, on a line of it's own
getdate
echo "year=$year,month=$month,day=$day,hour=$hour,minute=$minute,second=$sec"
Of course, you can modify what I provided or use answer [6] above.
The function takes no arguments.
I have log file in which I'm trying to delete all entries older than a specified date. Though I haven't succeeded with this yet. What I've tested so far is having an input for what the entries must be older than to be deleted and then loop like this:
#!/bin/bash
COUNTER=7
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -#${DATE} "+%Y-%m-%d")
while [ -n "$(grep $DATE test.txt)" ]; do
sed -i "/$DATE/d" test.txt
COUNTER=$((${COUNTER}+1))
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d #${DATE} +"%Y-%m-%d")
done
This kind of works except when a log entry doesn't exist for date. When it doesn't find a match it aborts the loop and the even older entries are kept.
Update
This was how I solved it:
#!/bin/bash
COUNTER=$((7+1))
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -#${DATE} "+%Y-%m-%d")
if [ -z "$(grep $DATE test.txt)" ]; then
exit 1
fi
sed -i "1,/$DATE/d" test.txt
Sorry for answering my own question but I went with Martin Frost's suggestion in the comments. It was much easier than the other suggestions.
This was my implementation:
#!/bin/bash
# requirements for script script
COUNTER=$((7+1))
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -#${DATE} "+%Y-%m-%d")
sed -i "1,/$DATE/d" test.txt
Thanks for all the help!
Depending on your logfile format, assuming that the timestamp is the first column in the file you can do it like this with (g)awk.
awk 'BEGIN { OneWeekEarlier=strftime("%Y-%m-%d",systime()-7*24*60*60) }
$1 <= OneWeekEarlier { next }
1' INPTUTLOG > OUTPUTLOG
This computes the date - surprise, surprise - one week earlier, then checks if the first column (white space separated columns by default) is less than or equal, and if true, skips the line, otherwise prints.
The hard part is doing the "in place" editing with awk. But it can be done:
{ rm LOGFILE && awk 'BEGIN { OneWeekEarlier=strftime("%Y-%m-%d",systime()-7*24*60*60) }
$1 <= OneWeekEarlier { next }
1' > LOGFILE ; } < LOGFILE
HTH
I deleted log records in syslog-ng files before 60 days ago with following code.
#!/bin/bash
LOGFILE=/var/log/syslog
DATE=`date +"%b %e" --date="-60days"`
sed -i "/$DATE/d" $LOGFILE
Is it possible to format the time output of stat? I am using
stat -c '%n %A %z' $filename
in a bash script, but its time format is not what I want. Is it possible to change this format in the command, or would I have to manually do it later?
An example output follows:
/lib drwxr-xr-x 2010-11-15 04:02:38.000000000 -0800
You can simply strip of the decimal portion like this:
stat -c '%n %A %z' "$filename" | sed 's/\(:[0-9]\{2\}\)\.[0-9]* /\1 /'
Edit:
Here's another way to truncate the decimal portion:
stat -c '%n %A %.19z' "$filename"
This depends on the date being 19 characters long: 2010-11-15 04:02:38
You could try something like:
date -d "1970-01-01 + $(stat -c '%Z' $filename ) secs"
Which gives you only the date. You can format the date using date's formatting options (see man date), for example:
date -d "1970-01-01 + $(stat -c '%Z' $filename ) secs" '+%F %X'
This doesn't give you the name and permissions but you may be able to do that like:
echo "$(stat -c '%n %A' $filename) $(date -d "1970-01-01 + $(stat -c '%Z' $filename ) secs" '+%F %X')"