I use DATE_TIME=$(date +"%Y-%m-%d_%T") to calculate actual date. I want to format this date in order to get %d.%m.%Y, %T and save it as another variable. Unfortunately DATE_TIME_2=$(date -d $DATE_TIME +"%d.%m.%Y, %T") does not work. Any tips?
Thank you.
First, save the date in a canonical form that you can use as input. A Unix timestamp is a good choice.
DATE_TIME=$(date +%s)
DATE_TIME is now the number of seconds that have elapsed since midnight Jan 1, 1970.
Then use that to compute both of your desired forms.
DATE_TIME_1=$(date -d #$DATE_TIME +"%Y-%m-%d_%T")
DATE_TIME_2=$(date -d #$DATE_TIME +"%d.%m.%Y, %T")
Note the # prefixed to the canonical form; that lets date know that the argument to d is to be interpreted as a integer timestamp.
Or, just compute a single string with both formats, separated by an unambiguous character:
DT=$(date +"%Y-%m-%d_%T=%d.%m.%Y, %T")
Then, you can split that easily with the read command.
IFS="=" read DATE_TIME DATE_TIME_2 <<< "$DT"
You could use string manipulations to break your original date into components, then rearrange them:
foo=$(date +"%Y-%m-%d_%T")
parts=(${foo//[-_]/ })
echo "${parts[2]}.${parts[1]}.${parts[0]}, ${parts[3]}"
whole lot of syntax, where there's a much simpler solution. e.g never fuss with an array. this will do what you need:
data.$ set -- $(date "+%Y %m %d %T")
data.$ echo $*
2015 04 26 19:32:55
data.$ DATE_TIME="$1-$2-$3_$4"
data.$ echo $DATE_TIME
2015-04-26_19:32:55
data.$ DATE_TIME_1="$1.$2.$3, $4"
data.$ echo $DATE_TIME_1
2015.04.26, 19:32:55
data.$
there might be some value in learning the syntax, but, ... if you can avoid it, by all means, do. at this point, with positional parameters, and you need fancier formatting, use "printf" rather than echo.
Related
I have 4 different files with different fileName.date formats, having a date embedded as part of the name. I want to identify the files older than 3 months based on their name only because the files would be edited/changed later as well. I want to create a shell script and run it as a cron.
Here below are the file under the same directory:
fileone.log.2018-03-23
file_two_2018-03-23.log
filethree.log.2018-03-23
file_four_file_four_2018-03-23.log
I have checked the existing example but have not found what I am actually looking for!
Working on the premise that you mean 90 days - if you need specifically months, we can check that too, but it's different logic.
here's some code you could work from -
(you said you don't want to work from a list, so I edited to use the current directory.)
$: cat chkDates
# while read f # replaced with -
for f in *[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]*
do # first get the epoch timestamp of the file based on the sate string embedded in the name
filedate=$(
date +%s -d $(
echo $f | sed -E 's/.*([0-9]{4}-[0-9]{2}-[0-9]{2}).*/\1/'
) # this returns the date substring
) # this converts it to an epoch integer of seconds since 1/1/70
# now see if it's > 90 days ( you said 3 months. if you need *months* we have to do some more...)
daysOld=$(( ( $(date +%s) - $filedate ) / 86400 )) # this should give you an integer result, btw
if (( 90 < $daysOld ))
then echo $f is old
else echo $f is not
fi
done # < listOfFileNames # not reading list now
You can pass date a date to report, and a format to present it.
sed pattern explanation
Note the sed -E 's/.*([0-9]{4}-[0-9]{2}-[0-9]{2}).*/\1/' command. This assumes the date format will be consistently YYYY-MM-DD, and does no validations of reasonableness. It will happily accept any 4 digits, then 2, then 2, delimited by dashes.
-E uses expanded regexes, so parens () can denote values to be remembered, without needing \'s. . means any character, and * means any number (including zero) of the previous pattern, so .* means zero or more characters, eating up all the line before the date. [0-9] means any digit. {x,y} sets a minimum(x) and maximum(y) number of consecutive matches - with only one value {4} means only exactly 4 of the previous pattern will do. So, '.*([0-9]{4}-[0-9]{2}-[0-9]{2}).*' means ignore as many characters as you can until seeing 4 digits, then a dash, 2 digits, then a dash, then 2 digits; remember that pattern (the ()'s), then ignore any characters behind it.
In a substitution, \1 means the first remembered match, so
sed -E 's/.*([0-9]{4}-[0-9]{2}-[0-9]{2}).*/\1/'
means find and remember the date pattern in the filenames, and replace the whole name with just that part in the output. This assumes the date will be present - on a filename where there is no date, the pattern will not match, and the whole filename will be returned, so be careful with that.
(hope that helped.)
By isolating the date string from the filenames with sed (your examples were format-consistent, so I used that) we pass it in and ask for the UNIX Epoch timestamp of that date string using date +%s -d $(...), to represent the file with a math-handy number.
Subtract that from the current date in the same format, you get the approximate age of the file in seconds. Divide that by the number of seconds in a day and you get days old. The file date will default to midnight, but the math will drop fractions, so it sorts out.
here's the file list I made, working from your examples
$: cat listOfFileNames
fileone.log.2018-03-23
fileone.log.2018-09-23
file_two_2018-03-23.log
file_two_2018-08-23.log
filethree.log.2018-03-23
filethree.log.2018-10-02
file_four_file_four_2018-03-23.log
file_four_file_four_2019-03-23.log
I added a file for each that would be within the 90 days as of this posting - including one that is "post-dated", which can easily happen with this sort of thing.
Here's the output.
$: ./chkDates
fileone.log.2018-03-23 is old
fileone.log.2018-09-23 is not
file_two_2018-03-23.log is old
file_two_2018-08-23.log is not
filethree.log.2018-03-23 is old
filethree.log.2018-10-02 is not
file_four_file_four_2018-03-23.log is old
file_four_file_four_2019-03-23.log is not
That what you had in mind?
An alternate pure-bash way to get just the date string
(You still need date to convert to the epoch seconds...)
instead of
filedate=$(
date +%s -d $(
echo $f | sed -E 's/.*([0-9]{4}-[0-9]{2}-[0-9]{2}).*/\1/'
) # this returns the date substring
) # this converts it to an epoch integer of seconds since 1/1/70
which doesn't seem to be working for you, try this:
tmp=${f%[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]*} # unwanted prefix
d=${f#$tmp} # prefix removed
tmp=${f#*[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]} # unwanted suffix
filedate=${d%$tmp} # suffix removed
filedate=$( date +%s --date=$filedate ) # epoch time
This is hard to read, but doesn't have to spawn as many subprocesses to get the work done. :)
If that doesn't work, then I'm suspicious of your version of date. Mine:
$: date --version
date (GNU coreutils) 8.26
UPDATE:
Simple Version:
Method for using the date inside of the file's name :
typeset stamp=$(date --date="90 day ago" +%s)
for file in /directory/*.log; do
fdate="$(echo "$file" | sed 's/[^0-9-]*//g')"
fstamp=$(date -d "${fdate} 00:00:00" +"%s")
if [ ${fstamp} -le ${stamp} ] ; then
echo "${file} : ${fdate} (${fstamp})"
fi
done
A More Complete Version:
This version will look at all files, if it fails to make a date value from the file it moves on.
typeset stamp=$(date --date="90 day ago" +%s)
for file in /tmp/* ; do
fdate="$(echo "$file" | sed 's/[^0-9-]*//g')"
fstamp=$(date -d "${fdate} 00:00:00" +"%s" 2> /dev/null)
[[ $? -ne 0 ]] && continue
if [ ${fstamp} -le ${stamp} ] ; then
echo "${file} : ${fdate} (${fstamp})"
fi
done
output:
/tmp/file_2016-05-23.log : 2016-05-23 (1463976000)
/tmp/file_2017-05-23.log : 2017-05-23 (1495512000)
/tmp/file_2018-05-23.log : 2018-05-23 (1527048000)
/tmp/file_2018-06-23.log : 2018-06-23 (1529726400)
/tmp/file_2018-07-23.log : 2018-07-23 (1532318400)
in this example the following were ignored :
/tmp/file_2018-08-23.log : 2018-08-23 (1534996800)
/tmp/file_2018-10-18.log : 2018-10-18 (1539835200)
I am trying to use bash to produce a list of dates and times between a starting point and an end point.
I would like the output to be in mm/dd/yyyy hh:mm format.
On the command line, the command:
date +"%m/%d/%Y %H:%M"
Produces the output that I am looking for.
When I use the line that is presently commented out, I get an error.
date: extra operand ‘%H:%M’
Try 'date --help' for more information.
I am not sure how to alter the script to produce the output that I am looking for.
DATE=date
#FORMAT="%m/%d/%Y %H:%M"
FORMAT="%m/%d/%Y"
start=`$DATE +$FORMAT -d "2013-05-06"`
end=`$DATE +$FORMAT -d "2013-09-16"`
now=$start
while [[ "$now" < "$end" ]] ; do
now=`$DATE +$FORMAT -d "$now + 1 day"`
echo "$now"
done
I have played around with adding an 00:00 after the start and end times, but that did not work.
Any ideas where I am getting the output format wrong?
Code from:
https://ocroquette.wordpress.com/2013/04/21/how-to-generate-a-list-of-dates-from-the-shell-bash/
When you use the FORMAT="%m/%d/%Y %H:%M" you need quotes because it contains a space, so:
now=`$DATE +"$FORMAT" -d "$now + 1 day"`
Also, I do not think that you can compare dates like that. You might need timestamp:
date +%s
I am working on bash. I have to subtract current date from a given date to get number of days as a difference. The given date is in format m/d/yyyy so instead of 09/26/2015 it is 9/26/2015. So even if I try to convert both dates into same format and subtract it says invalid date format.
date1=$(date +"%F")
date2=$(date -d 11/2/2015 +"%F")
diff=$(date "--date=${date2} -${date1}" +%F)
echo $diff days remaining
This is what I had tried with some variations, but doesn't work. What am I doing wrong? Thanks in advance.
Try this:
let diff=(`date +%s -d 11/2/2015` - `date +%s`)/86400
echo $diff days remaining
there are two problems: converting the user-provided date into a normalized form and calculcating the difference in days.
normalizing date
how date interprets a date-string depends on the current locale.
Try to find a locale that uses your special formatting (%m/%d/%Y):
$ LC_TIME=en_US.UTF-8 date -d 1/2/2015
Fri Jan 2 00:00:00 CET 2015
calculating the difference
bash only can only do integer arithmetic, so convert your date first to some integer representation, do the subtraction and convert the representation to days (if needed).
$ LC_TIME=en_US.UTF-8 \
echo $(( ( $(date -d 11/2/2015 +%s) - $(date +%s)) / (3600*24) ))
32
This uses $(...) instead of ... to function substitution.
It also uses $(( ... )) for evalution of math expression instead of the bashism let x=(), so you can use it in POSIX-conformant shell-scripts (e.g. interpreted by /bin/dash)
I want to add the code below to my script but it's not showing the total_time value although CurrentTime is showing correctly. In this I want to change epoch time to current system time and then add 20 minutes to it.
CurrentTime=`date -d #$2`
echo "CurrentTime : $CurrentTime " >> ${LOGFILE}
Total_time=`"$CurrentTime" -d "+20 min"`
How can I do it?
Change your totaltime assignment like this:
Total_time=`date -d "$CurrentTime +20 mins"`
The reason this isn't working is that by the time you're trying to assign the value to $Total_time, your $CurrentTime variable has already been set to a time. It's not a command anymore, it's a string that is the result of a command.
Each time you want to calculate a new date, you need a new invocation of the `date` command. That's what Guru's answer provides you with, though he didn't explain why.
If what you need is to make a "base" date to which you apply modifiers, you can still do this, but I'd recommend a slightly different notation:
#!/bin/bash
start=$(date '+%s')
# do stuff
sleep 20
duration=$((`date '+%s'` - $start))
You can then use your $duration as an easier basis for other calculations, AND you can use it with relative dates:
printf "[%s] Start of job\n" "$(date -d #$start '+%Y-%m-%d %T')"
...
printf "[%s] End of job\n" "$(date -d #"$((start + duration))" '+%Y-%m-%d %T')"
Probably better to format your log files in a more standard format than date's default.
I'm trying to work on a logfile, and I need to be able to specify the range of dates. So far (before any processing), I'm converting a date/time string to timestamp using date --date "monday" +%s.
Now, I want to be able to iterate over each line in a file, but check if the date (in a human readable format) is within the allowed range. To do this, I'd like to do something like the following:
echo `awk '{if(`date --date "$3 $4 $5 $6 $7" +%s` > $START && `date --date "" +%s` <= $END){/*processing code here*/}}' myfile`
I don't even know if thats possible... I've tried a lot of variations, plus I couldn't find anything understandable/usable online.
Thanks
Update:
Example of myfile is as follows. Its logging IPs and access times:
123.80.114.20 Sun May 01 11:52:28 GMT 2011
144.124.67.139 Sun May 01 16:11:31 GMT 2011
178.221.138.12 Mon May 02 08:59:23 GMT 2011
Given what you have to do, its really not that hard AND it is much more efficient to do your date processing by converting to strings and comparing.
Here's a partial solution that uses associative arrays to convert the month value to a number. Then you rely on the %02d format specifier to ensure 2 digits. You can reformat the dateTime value with '.', etc or leave the colons in the hr:min:sec if you really need the human readability.
The YYYYMMDD format is a big help in these sort of problems, as LT, GT, EQ all work without any further formatting.
echo "178.221.138.12 Mon May 02 08:59:23 GMT 2011" \
| awk 'BEGIN {
mons["Jan"]=1 ; mons["Feb"]=2; mons["Mar"]=3
mons["Apr"]=4 ; mons["May"]=5; mons["Jun"]=6
mons["Jul"]=7 ; mons["Aug"]=8; mons["Sep"]=9
mons["Oct"]=10 ; mons["Nov"]=11; mons["Dec"]=12
}
{
# 178.221.138.12 Mon May 02 08:59:23 GMT 2011
printf("dateTime=%04d%02d%02d%02d%02d%02d\n",
$NF, mons[$3], $4, substr($5,1,2), substr($5,4,2), substr($5,7,2) )
} ' -v StartTime=20110105235959
The -v StartTime is ilustrative of how to pass in (and the matching format) your starTime value.
I hope this helps.
Here's an alternative approach using awk's built-in mktime() function. I've never bothered with the month parsing until now - thanks to shelter for that part (see accepted answer). It always feels time to switch language around that point.
#!/bin/bash
# input format:
#(1 2 3 4 5 6 7)
#123.80.114.20 Sun May 01 11:52:28 GMT 2011
awk -v startTime=1304252691 -v endTime=1306000000 '
BEGIN {
mons["Jan"]=1 ; mons["Feb"]=2; mons["Mar"]=3
mons["Apr"]=4 ; mons["May"]=5; mons["Jun"]=6
mons["Jul"]=7 ; mons["Aug"]=8; mons["Sep"]=9
mons["Oct"]=10 ; mons["Nov"]=11; mons["Dec"]=12;
}
{
hmsSpaced=$5; gsub(":"," ",hmsSpaced);
timeInSec=mktime($7" "mons[$3]" "$4" "hmsSpaced);
if (timeInSec > startTime && timeInSec <= endTime) print $0
}' myfile
(I've chosen example time thresholds to select only the last two log lines.)
Note that if the mktime() function were a bit smarter this whole thing would reduce to:
awk -v startTime=1304252691 -v endTime=1306000000 't=mktime($7" "$3" "$4" "$5); if (t > startTime && t <= endTime) print $0}' myfile
I'm not sure of the format of the data you're parsing, but I do know that you can't use the backticks within single quotes. You'll have to use double quotes. If there are too many quotes being nested, and it's confusing you, you can also just save the output of your date command to a variable beforehand.