Bash Script for deleting old backup files - bash

every night my server creates backups for every mysql database. All of these files are saved in a folder like /backup/mysql/2014-11-28. Over the last few months a lot of folders and files have been stored in that directory and I would like to reduce this.
Therefore I would need a bash script which deletes every folder in the given directory, except every folder created this month (not in the last 30 days, but the actual month) and except one backup from every week (for example the backup from sunday). Since I have no clue how to do the sunday party I decided it would be simpler to keep the backups from the 07th, the 14th, the 21st and the the 28th.
#!/bin/bash
in_array() {
local haystack=${1}[#]
local needle=${2}
for i in ${!haystack}; do
if [[ ${i} == ${needle} ]]; then
return 0
fi
done
return 1
}
YEAR=`date +%Y`
LASTYEAR=`date +%Y -d "1 year ago"`
MONTH=`date +%m`
DAYS="07 14 21 28"
for i in $( ls ); do
BACKUP_YEAR=$(echo "${i}" | cut -d'-' -f1)
BACKUP_MONTH=$(echo "${i}" | cut -d'-' -f2)
BACKUP_DAY=$(echo "${i}" | cut -d'-' -f3)
DELETE=false
if [[ "$BACKUP_YEAR" == "$YEAR" ]]; then
if [[ "$BACKUP_MONTH" != "$MONTH" ]]; then
if [ $(in_array $DAYS "$BACKUP_DAY") ]; then
DELETE=true
fi
fi
fi
if [[ "$BACKUP_YEAR" == "$LASTYEAR" ]]; then
if [[ "$BACKUP_DAY" != ${DAYS[0]} ]]; then
DELETE=true
fi
fi
if [ "$DELETE" = true ] ; then
#OUTPUT=`rm -v -R $i`
echo $i
fi
done
The second part (backups from the year before) works perfectly fine, but the first part (for backups from this year) doesn't work and I don't know why. I found the function on this site, but I guess somehow my call is wrong.
EDIT: The code I am now using:
#!/bin/bash
read YEAR MONTH <<<$(date "+%Y %m")
LASTYEAR=$(( YEAR-1 ))
DAYS=" 07 14 21 28 "
for fn in $( ls )
do
if ([ "${fn:0:4}" = "$YEAR" ] &&
[ "${fn:5:2}" != "$MONTH" ] &&
[ "${DAYS/ ${fn:8:2} /}" = "$DAYS" ]) || ([ "${fn:0:4}" = "$LASTYEAR" ] &&
[ "${fn:8:2}" != ${DAYS:1:2} ])
then
#OUTPUT=`rm -v -R $fn`
echo "$fn"
fi
done

#!/bin/bash
read YEAR MONTH <<<$(date "+%Y %m")
LASTYEAR=$(( YEAR-1 ))
DAYS=" 07 14 21 28 "
for fn in $( ls )
do
DELETE=false
if [ "${fn:0:7}" = "$YEAR-$MONTH" ] &&
[ "${DAYS/ ${fn:8:2} /}" != "$DAYS" ]
then
DELETE=true
elif [ "${fn:0:4}" = "$LASTYEAR" ] &&
[ "${fn:8:2}" != ${DAYS:1:2} ]
then
DELETE=true
fi
if [ "$DELETE" = true ]
then
#OUTPUT=`rm -v -R $fn`
echo "$fn"
fi
done
Assuming that a file name matches the following date format YYYY-MM-DD, for instance, 2014-03-27, the expression ${fn:0:7} would be 2014-03; ${fn:8:2} would be 27; ${fn:0:4} would be 2014.
Read the bash manual pages about the parameter expansion.
The expression ${DAYS/ ${fn:8:2} /} checks the day from the filename (wrapped by white spaces) is found in the DAYS list. It would be equivalent to echo "${DAYS}" | grep -q "${BACKUP_DAY}".
note: it is possible to reduce the commands inside the for loop (join if and elif by means of a || and remove the DELETE variable checking and the DELETE variable itself) but I decided to keep the look of the original script.

Related

shell script for date between 2 different dates

I'm struggling a bit on how to put the correct date in my shell script for a specific time range.
For instance i need to check if i'm in the correct range to generate some file based on date(12am - 3am )
so far I'm checking :
workflow_start_time=$(date -d "$(date +%Y-%m-%d) 23:00:00" +"%s")
scheduled_run_time=$workflow_start_time
if [ "$(date -d #${scheduled_run_time} +"%H")" -ge 23 ] && [ "$(date -d #${scheduled_run_time} +"%H")" -lt 2 ]; then
that is starting at 23 because i need to take into account 1 hour server difference, so this will start around 12 and should finish around 2am, which will be 3am.
any help here?
Convert date to timestamp.
toStamp() { date --date="$1" +%s; }
Check if date is between other dates.
# [$1, $2): bounds
# $3 current date
isBetween() {
[[ `toStamp $1` -le `toStamp $3` ]]&&\
[[ `toStamp $2` -gt `toStamp $3` ]]
}
or
isTimeStampBetween() {
[[ $1 -le $3 ]]&&\
[[ $2 -gt $3 ]]
}
if you have timestamps only.
An example to check if a date is between 2 other would be:
if $(isBetween 2012 2020 2013) ; then dosomething
You can include these functions in your file.

Convert 4 digits into a DateTime (Year + padded with minimum values)

I am writing a script to return a formatted date based on user input, which can be :
2016
2016/11
2016/11/10
2016/11/10 24
Basically I have to pad these input with the minimum possible values. (2016 returns 2016/01/01 00:00:00).
I've tried doing it with date -d "2016" but this returns the time 8:16pm on today's date
Many thanks!!
Here's a script that should basically do what you want to do:
#!/bin/bash
echo INPUT:$1
[[ "$1" == "" ]] && YEAR=2016
[[ "$1" =~ ^[0-9]{4}$ ]] && YEAR="$1"
[[ $YEAR == "" ]] && YEAR=`echo "$1" | grep -Po ".*(?=/[0-9]{2}/)"`
MONTH=`echo "$1" | grep -Po "(?<=[0-9]{4}/).*(?=/[0-9]{2})"`
DAY=`echo "$1" | grep -Po "(?<=/[0-9]{2}/).*(?=\ )"`
HR=`echo "$1" | grep -Po "(?<=[0-9]{2}\ ).*"`
MIN="00"
SEC="00"
[[ "$MONTH" == "" ]] && MONTH="01"
[[ "$DAY" == "" ]] && DAY="01"
[[ "$HR" == "" ]] && HR="00"
echo $YEAR/$MONTH/$DAY $HR:$MIN:$SEC
Please note that this is a basic skeleton, you will have to add your own border cases and conditions.
You can use date command to get your output like below
d1=date +%Y #Output is 2016
d2=date +%Y/%m #Output is 2016/11
d3=date +%Y/%m/%d #Output is 2016/11/10
d4=date +%Y/%m/%d\t%H #Output is 2016 -> \t-tab specified,if you want the whole time you can use "%X or %T" instead of "%H"
echo $d1
echo $d2
echo $d3
echo $d4

Bash several and in if

Hi I have a csv file with dates like this which I want to parse with bash and check against todays date.
12-Jan-2015
Checking only day and month works good
DAT=$(date '+%d %b %Y')
DAY=${DAT:0:2}
MON=$(echo ${DAT:3:3} | awk '{print toupper("$0");}')
YEAR=${DAT:6:5}
while IFS=",-" read name day month year
do
day=$(printf "%02d\n" "$day")
month=$(echo "$month" | awk '{print toupper($0);}')
year=$(printf "%04d\n" "$year")
if [ "$day" -eq "$DAY" ] && [ "$month" = "$MON" ]; then
echo "$name";
fi
done < $SNAPDB > $SNAPTMPDB
but when I try to also check the year
if [ "$day" -eq "$DAY" ] && [ "$month" = "$MON" ] && [ "$year" -eq "$YEAR" ]; then
Script ends with an error: line 119: 2015: command not found
I´ve by now tried several writings like [[ … ]] -a and so on. but nothing did work.
Thanks for help!
Use -a for 'and' and -o for 'or'. From the documentation: http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html (look specifically at Table 7-2)
Updated from comments:
Use double brackets and parentheses:
if ( [[ "$day" -eq "$DAY" ]] && [[ "$month" = "$MON" ]] && [[ "$year" -eq "$YEAR" ]] ); then`

Validate date format in a shell script

I have to create a Shell Script wherein one of the parameters will be the date in the format dd/mm/yyyy. My question is, how can I check if the Date passed as parameter really follows this Date Format? I tried to use the grep command as below:
if echo "$1" | grep -q '^[0-3][0-9]/[0-1][0-9]/[0-9]\{4\}$'
but it didn't give the correct format because the day for example can be 33, 34, (...), that is not really the correct format. Anyone know something that can really check if the date passed really follows the format dd/mm/yyyy ?
Use date
date "+%d/%m/%Y" -d "09/99/2013" > /dev/null 2>&1
is_valid=$?
The date string must be in "MM/DD/YYYY" format.
If you do not get 0 then date is in invalid format.
The simplest solution, that still works perfectly, is the following :
if [[ $1 =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] && date -d "$1" >/dev/null 2>&1
...
It consists in combining 2 checks :
the first part checks that $1 is of this format : NNNN-NN-NN
the second part checks that it is a valid date
You need the two checks because :
if you don't do the first check, date will exit with code 0 even if your variable is a valid date in another format
if you don't do the second check, then you can end up with a 0 even for variables such as 2016-13-45
This function expects 2 strings,a format string, a date string
The format string uses the codes from the date command but does not include the '+'
The function returns 0 if the provided date matches the given format, otherwise it returns 1
Code (my_script.sh)
#!/bin/bash
datecheck() {
local format="$1" d="$2"
[[ "$(date "+$format" -d "$d" 2>/dev/null)" == "$d" ]]
}
date_test="$1"
echo $date_test
if datecheck "%d %b %Y" "$date_test"; then
echo OK
else
echo KO
fi
Output
$ ./my_script.sh "05 Apr 2020"
05 Apr 2020
OK
$ ./my_script.sh "foo bar"
foo bar
KO
First, check the form of the input using the regex. Then use awk to switch to mm/dd/yyyy and use date to validate. You can use the following expression in your if statement:
echo "$1" | egrep -q '^[0-3][0-9]/[0-1][0-9]/[0-9]{4}$' && date -d "$(echo "$1" | awk 'BEGIN{FS=OFS="/"}{print $2"/"$1"/"$3}')" >/dev/null 2>&1
Simplest way for dd/mm/yyyy exactly in Bash is:
if [[ $1 == [0-3][0-9]/[0-1][0-9]/[0-9][0-9][0-9][0-9] ]]
Or
if [[ $1 =~ ^[0-3][0-9]/[0-1][0-9]/[0-9]{4}$ ]]
How about using awk:
echo "31/12/1999" | awk -F '/' '{ print ($1 <= 31 && $2 <= 12 && match($3, /^[1-9][1-9][1-9][1-9]$/)) ? "good" : "bad" }'
It prints "good" if its valid date else prints "bad"
#! /bin/bash
isDateInvalid()
{
DATE="${1}"
# Autorized separator char ['space', '/', '.', '_', '-']
SEPAR="([ \/._-])?"
# Date format day[01..31], month[01,03,05,07,08,10,12], year[1900..2099]
DATE_1="((([123][0]|[012][1-9])|3[1])${SEPAR}(0[13578]|1[02])${SEPAR}(19|20)[0-9][0-9])"
# Date format day[01..30], month[04,06,09,11], year[1900..2099]
DATE_2="(([123][0]|[012][1-9])${SEPAR}(0[469]|11)${SEPAR}(19|20)[0-9][0-9])"
# Date format day[01..28], month[02], year[1900..2099]
DATE_3="(([12][0]|[01][1-9]|2[1-8])${SEPAR}02${SEPAR}(19|20)[0-9][0-9])"
# Date format day[29], month[02], year[1904..2096]
DATE_4="(29${SEPAR}02${SEPAR}(19|20(0[48]|[2468][048]|[13579][26])))"
# Match the date in the Regex
if ! [[ "${DATE}" =~ "^(${DATE_1}|${DATE_2}|${DATE_3}|${DATE_4})$" ]]
then
echo -e "ERROR - '${DATE}' invalid!"
else
echo "${DATE} is valid"
fi
}
echo
echo "Exp 1: "`isDateInvalid '12/13/3000'`
echo "Exp 2: "`isDateInvalid '12/11/2014'`
echo "Exp 3: "`isDateInvalid '12 01 2000'`
echo "Exp 4: "`isDateInvalid '28-02-2014'`
echo "Exp 5: "`isDateInvalid '12_02_2002'`
echo "Exp 6: "`isDateInvalid '12.10.2099'`
echo "Exp 7: "`isDateInvalid '31/11/2000'`
Here's a function to do some data validation this:
# Script expecting a Date parameter in MM-DD-YYYY format as input
verifyInputDate(){
echo ${date} | grep '^[0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9]$'
if [ $? -eq 0 ]; then
echo "Date is valid"
else
echo "Date is not valid"
fi
}
`X="2016-04-21" then check for the below value being 1 or 0.
cal echo $x | cut -c 6-7 echo $x | cut -c 1-4 2>/dev/null | grep -c echo $x | cut -c 9-10
If the value is 1, then it's valid, else it's not valid.
Though the solution (if [[ $1 =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] && date -d "$1" >/dev/null 2>&1) of #https://stackoverflow.com/users/2873507/vic-seedoubleyew is best one at least for linux, but it gives error as we can not directly compare/match regex in if statement. We should put the regex in a variable and then we should compare/match that variable in if statement. Moreover second part of if condition does not return a boolean value so this part will also cause error.
So I have done slight modification in the above formula and this modification can also be customized further for various other formats or combination of them.
DATEVALUE=2019-11-12
REGEX='^[0-9]{4}-[0-9]{2}-[0-9]{2}$'
if [[ $DATEVALUE =~ $REGEX ]] ; then
date -d $DATEVALUE
if [ $? -eq 0 ] ; then
echo "RIGHT DATE"
else
echo "WRONG DATE"
fi
else
echo "WRONG FORMAT"
fi
I would like to give an extended answer for a slightly different format, but this can easily be changed to the dd/mm/YY format with the answers already given; it's tested on busybox (posix shell)
This is one of the first hits for web searches similar to "busybox posix shell script date" and "test format" or "validate" etc, so here my solution for busybox (tested with 1.29.3, 1.23.1)
#!/bin/sh
##########
#
# check if date valid in busybox
# tested in busybox 1.29.3, 1.23.1
#
# call with:
# $0 <yyyymmdd>
#
##########
mydate=$1
if echo $mydate | grep -qE '20[0-9][0-9](0[1-9]|1[0-2])([012][0-9]|3[01])'; then
printf 'may be valid\n'
date +%Y%m%d -d $mydate -D %Y%m%d > /dev/null 2>&1
is_valid=$?
if [ $is_valid -ne 0 ]; then
printf 'not valid\n'
return 1
else
mytestdate=$(date +%Y%m%d -d $mydate -D %Y%m%d)
if [ $mydate -ne $mytestdate ]; then
printf 'not valid, results in "%s"\n' "$mytestdate"
return 1
else
printf 'valid\n'
fi
fi
else
printf 'not valid (must be: <yyyymmdd>)\n'
return 1
fi
as in busybox (1.29.3 & 1.23.1) you have responds like:
lxsys:~# date +%Y%m%d -d 20110229 -D "%Y%m%d"
20110301
I had the need to validate the date in some better way but i wanted to rely mostly on the system itself
so with
mytestdate=$(date +%Y%m%d -d $mydate -D %Y%m%d)
if [ $mydate -ne $mytestdate ]; then
...
fi
there is a second test - do we have a difference between the wanted or given format (input, $mydate) and the system interpretation (output, $mytestdate) of it ... if it's not the same, discard the date
I wrote this bash script to validate date. I can accept mont as alphanumeric.
#!/bin/bash
function isDateValid {
DATE=$1
if [[ $DATE =~ ^[0-9]{1,2}-[0-9a-zA-Z]{1,3}-[0-9]{4}$ ]]; then
echo "Date $DATE is a number!"
day=`echo $DATE | cut -d'-' -f1`
month=`echo $DATE | cut -d'-' -f2`
year=`echo $DATE | cut -d'-' -f3`
if [ "$month" == "01" ] || [ "$month" == "1" ]; then
month="Jan"
elif [ "$month" == "02" ] || [ "$month" == "2" ]; then
month="Feb"
elif [ "$month" == "03" ] || [ "$month" == "3" ]; then
month="Mar"
elif [ "$month" == "04" ] || [ "$month" == "4" ]; then
month="Apr"
elif [ "$month" == "05" ] || [ "$month" == "5" ]; then
month="May"
elif [ "$month" == "06" ] || [ "$month" == "6" ]; then
month="Jun"
elif [ "$month" == "07" ] || [ "$month" == "7" ]; then
month="Jul"
elif [ "$month" == "08" ] || [ "$month" == "8" ]; then
month="Aug"
elif [ "$month" == "09" ] || [ "$month" == "9" ]; then
month="Sep"
elif [ "$month" == "10" ]; then
month="Oct"
elif [ "$month" == "11" ]; then
month="Nov"
elif [ "$month" == "12" ]; then
month="Dec"
fi
ymd=$year"-"$month"-"$day
echo "ymd: "$ymd
dmy=$(echo "$ymd" | awk -F- '{ OFS=FS; print $3,$2,$1 }')
echo "dmy: "$dmy
if date --date "$dmy" >/dev/null 2>&1; then
echo "OK"
return 0
else
echo "NOK"
return 1
fi
else
echo "Date $DATE is not a number"
return 1
fi
}
if isDateValid "15-15-2014"; then
echo "date is valid =)"
else
echo "bad format date"
fi
echo "==================="
if isDateValid "15-12-2014"; then
echo "date is valid =)"
else
echo "bad format date"
fi
echo "==================="
if isDateValid "15-Dec-2014"; then
echo "date is valid =)"
else
echo "bad format date"
fi
echo "==================="
if isDateValid "1-May-2014"; then
echo "date is valid =)"
else
echo "bad format date"
fi
echo "==================="
if isDateValid "1-1-2014"; then
echo "date is valid =)"
else
echo "bad format date"
fi
echo "==================="
if isDateValid "12-12-2014"; then
echo "date is valid =)"
else
echo "bad format date"
fi
Blockquote
DATE = "$*"
[[ "${DATE}" != #(((([123][0]|[012][1-9])|3[1])?([ \/._-])(0[13578]|1[02])?([ \/._-])(19|20)[0-9][0-9])|(([123][0]|[012][1-9])?([ \/._-])\
(0[469]|11)?([ \/._-])(19|20)[0-9][0-9])|(([12][0]|[01][1-9]|2[1-8])?([ \/._-])02?([ \/._-])(19|20)[0-9][0-9])|(29?([ \/._-])02?([ \/._-])\
(19|20(0[48]|[2468][048]|[13579][26])))) ]] && echo error || echo good)

Unix convert Month name to number

In BASH shell scripting or using gdate, given a date like "Oct 2011" how do I convert to a year-month number format? Output should be "2011-10", for example.
mydate="Oct 2011"
date --date="$(printf "01 %s" $mydate)" +"%Y-%m"
The parse_datetime interface for GNU date (which is what the example uses) has lots of rules. the Oct 2011 form of the date isn't one of them, so you prepend a "01 " to the front of it and date likes it.
read mon year <<< "Oct 2012"
date -d "$mon 1 $year" "+%Y-%m"
Result:
2012-10
You can convert the month to a number by finding the position of the name string:
#!/bin/bash
month=Oct
months="JanFebMarAprMayJunJulAugSepOctNovDec"
tmp=${months%%$month*}
month=${#tmp}
monthnumber $((month/3+1))
printf "%02d\n" $monthnumber
The output of the script above is:
10
Your specific string you could code:
#!/bin/bash
mydate="Oct 2011"
monthnumber() {
month=$1
months="JanFebMarAprMayJunJulAugSepOctNovDec"
tmp=${months%%$month*}
month=${#tmp}
monthnumber=$((month/3+1))
printf "%02d\n" $monthnumber
}
arr=(`echo ${mydate}`);
month=$(monthnumber ${arr[0]})
year=$(echo ${arr[1]})
echo "$year-$month"
The output would be:
2011-10
case "`date | awk '{print $2 }'`" in
Jan) MON="01" ;;
Feb) MON="02" ;;
Mar) MON="03" ;;
Apr) MON="04" ;;
May) MON="05" ;;
Jun) MON="06" ;;
Jul) MON="07" ;;
Aug) MON="08" ;;
Sep) MON="09" ;;
Oct) MON="10" ;;
Nov) MON="11" ;;
Dec) MON="12" ;;
esac
echo $MON
I'm not sure if there is a shorter way of doing this, but here is one way. This is by no means fool proof. You can improve this by adding other checks to input and make the comparison case insensitive.
#!/bin/ksh
### Validate input
if [ $# -eq 0 ]
then
echo "Usage: $0 InputMonYYYY"
echo "Example: $0 \"Oct 2011\""
exit 1
fi
### Read input
INPUTSTR=$1
MON_STR=`echo $INPUTSTR |cut -d' ' -f1`
YYYY_STR=`echo $INPUTSTR |cut -d' ' -f2`
if [[ "$MON_STR" = "Jan" ]] then
MON_NUM=01
elif [[ "$MON_STR" = "Feb" ]] then
MON_NUM=02
elif [[ "$MON_STR" = "Mar" ]] then
MON_NUM=03
elif [[ "$MON_STR" = "Apr" ]] then
MON_NUM=04
elif [[ "$MON_STR" = "May" ]] then
MON_NUM=05
elif [[ "$MON_STR" = "Jun" ]] then
MON_NUM=06
elif [[ "$MON_STR" = "Jul" ]] then
MON_NUM=07
elif [[ "$MON_STR" = "Aug" ]] then
MON_NUM=08
elif [[ "$MON_STR" = "Sep" ]] then
MON_NUM=09
elif [[ "$MON_STR" = "Oct" ]] then
MON_NUM=10
elif [[ "$MON_STR" = "Nov" ]] then
MON_NUM=11
elif [[ "$MON_STR" = "Dec" ]] then
MON_NUM=12
fi
echo ${YYYY_STR}-${MON_NUM}
Bash4 supports hash-tables (answer by Jim is the correct one though).
Example
#!/bin/bash
declare -A months=( ["Jan"]="01" ["Feb"]="02" )
mydate="Jan 2011"
echo ${mydate:4:8}-"${months["${mydate:0:3}"]}"
Output:
2011-01
Let's kick this dead horse.
If you don't care about invalid month names you can use this function I've written which is quite short (and only does 1 exec) but expects a month to be valid english 3-chars lower or upper case and only requires GNU sed and bash:
m2n() { echo $((-10+$(sed 's/./\U&/g;y/ABCEGLNOPRTUVY/60AC765A77ABB9/;s/./+0x&/g'<<<${1#?}) ));}
For your example I'd do:
read m y <<<"$#"; echo "$y-`m2n $m`"

Resources