I am trying to run a schedule job on centOS to .gz and delete a whole month of logs from a directory. The logs are named somelogfile_12Apr19_18_19_41.log, somelogfile_28Mar19_07_08_20.log
I can run the below script to manually do this task
tar -cvzf somezipfile_Mar19.tar.gz somelogfile_**Mar** --remove-files
The scheduled job should run every 5th day of the month to compress and delete the previous months logs. What will the automated script be like? I am stuck at how to include only the previous months logs based on the month name (Jan,Feb,Mar, etc.)
The first problem to solve is the fact that you want a more generic function that can be run any month and produce the correct result. A good tool to get the information you need(Abbreviated month and last two digits of the year) is date.
date -d "last month" +%b
When run on April 1st, 2019 will produce "Mar".
date -d "last month" +%b%y
When run on April 1st, 2019 will produce "Mar19".
Now that we know how to get the information we want, placing the date commands in the tar command will automatically produce what the result you're looking for.
tar -cvzf somezipfile_$(date -d "last month"%b%y).tar.gz somelogfile_**$(date -d "last month" +%b)** --remove-files
The last issue that exists is scheduling, which can be solved using cron. The below statement will run /bin/foobar, on the 5th day of every month when added to your crontab file. (crontab -e to edit your crontab file)
0 0 5 * * /bin/foobar
Combining everything together, you get:
0 0 5 * * /bin/tar -cvzf somezipfile_$(date -d "last month"\%b\%y).tar.gz somelogfile_**$(date -d "last month" +\%b)** --remove-files
Don't forget to escape the %'s in the crontab
Related
I m trying to get yesterday date, it's not working in hp ux server.
Prev_date=$(date +"y%m%d" -d "1 day ago")
For this I m still getting current date only.
20210811
Could you please help on the same.
You missed a percentage in front of the 'y': this is working fine for me:
echo $(date +"%y%m%d" -d "1 day ago")
You can use below command, if you want.
date --date=' 1 days ago' '+%Y-%m-%d'
It will give result like
2021-09-06
I prefer this format since most of the time my scripts include SQL queries for data fetch and hence date is required to filter out data on daily basis.
I have an backup tool that runs every day on 2 am,
The backup output saved under existed folder named "collector"
i'm trying to write script bash,
That will check if folder named "repository" exist under the "collector" directory with the current/today date. (i.e in compare to the RHEL date)
in case folder exist with older date, send email using echo.
Appreciate the help.
Find the number of seconds for 0:00h of today (start of the date) wrt a reference time (1970-01-01 00:00:00 UTC), say today.
Then find the number of seconds the the last modified time of the desired file wrt the same reference time, say mdate.
The two values can be compared to find whether the file has created today or not.
today="$(date -d "$(date "+%D")" +%s)" # seconds at the start of the sate
mdate="$(stat -c %Y src.sh)" # seconds to the last modification
if [[ $mdate -ge $today ]]; then
echo "modified today"
else
echo "modified before today"
fi
oldest_year_month_temp=201602
NUM_PART_RETAIN=20
oldest_year_month=`date --date="$(oldest_year_month_temp +%Y%m) - $NUM_PART_RETAIN month" "+%Y%m"`
Date is not coming as expected.
One easy way to do it would be to simply append a 01 to your input of yymm to provide a format date -d could read as the starting date, then simply subtract 20 months and output the resulting date in %y%m format. For example, if you provide the date 9910 (Oct. 1999), you can do:
$ date -d "991001 - 20 months" +%y%m
9802
Which returns Feb. 1998 (20 months earlier)
(note: the $ above just indicates a command by a normal user as opposed to # indicating a command by the super user (e.g. root))
Inside the $(...) there must be a command, e.g. $(date ...).
This should have been obvious from the error message you got, which was probably oldest_year_month_temp: no such command.
When reading from a variable, you must write a $ before its name.
I have a bash script that is sending me a text daily, for 100 days.
#! /bin/bash
EMAIL="my-phone-gateway#address.net"
MESSAGE="message_content.txt"
mail $EMAIL < $MESSAGE
Using crontab, I can have the static $MESSAGE sent to me every day.
Other than hard-coding 100 days of texts ;)
How could I implement a variable counter such that I can have my texts say:
"Today is Day #1" on the first day, "Today is Day #2" on the second day, etc. ?
Note: The location of the requested text within the $MESSAGE file doesn't matter. Last line, first line, middle, etc.
The only requirement for an answer here is that I know what day it is relative to the first, where the first day is the day the script was started.
Of course, bonus awesome points for the cleanest, simplest, shortest solution :)
For our nightly build systems, I wrote a C program that does the calculation (using local proprietary libraries that store dates as a number of days since a reference date). Basically, given a (non-changing) reference date, it reports the number of days since the reference date. So, the cron script would have a hard-wired first day in it, and the program would report the number of days since then.
The big advantage of this system is that the reference date doesn't change (very often), so the script doesn't change (very often), and there are no external files to store information in.
There probably are ways to achieve the same effect with standard Unix tools, but I've not sat down and worked out the portable solution. I'd probably think it terms of using Perl. (The C program only works up to 2999 CE; I left a note in the code for people to contact me about 50 years before it becomes a problem for the Y3K fix. It is probably trivial.)
You could perhaps work in terms of Unix timestamps...
Create a script 'days_since 1234567890' which treats the number as the reference date, gets the current time stamp (from date with appropriate format specification; on Linux, date '+%s' would do that job, and it works on Mac OS X too), takes the difference and divides by 86,400 (the number of seconds in a day).
refdate=1234567890
bc <<EOF
scale=0
($(date '+%s') - $refdate) / 86400
EOF
An example:
$ timestamp 1234567890
1234567890 = Fri Feb 13 15:31:30 2009
$ timestamp
1330027280 = Thu Feb 23 12:01:20 2012
$ refdate=1234567890
$ bc <<EOF
> scale=0
> ($(date '+%s') - $refdate) / 86400
> EOF
1104
$
So, if the reference date was 13th Feb 2009, today is day 1104. (The program bc is the calculator; its name has nothing to do with Anno Domini or Before Christ. The program timestamp is another homebrew of mine that prints timestamps according to a format that can be specified; it is a specialized variant of date originally written in the days before date had the functionality, by which I mean in the early 1980s.)
In a Perl one-liner (assuming you specify the reference date in your script):
perl -e 'printf "%d\n", int((time - 1234567890)/ 86400)'
or:
days=$(perl -e 'printf "%d\n", int((time - 1234567890)/ 86400)')
The only way to accomplish this would be to store the date in a file, and read from that file each day. I would suggest storing the epoch time.
today=$(date +%s)
time_file="~/.first_time"
if [[ -f $time_file ]]; then
f_time=$(< "$time_file")
else
f_time=$today
echo "$f_time" > "$time_file"
fi
printf 'This is day: %s\n' "$((($today - $f_time) / 60 / 60 / 24))"
Considering that your script is running only once a day, something like this should work:
#!/bin/bash
EMAIL="my-phone-gateway#address.net"
MESSAGE="message_content.txt"
STFILE=/tmp/start.txt
start=0
[ -f $STFILE ] && start=$(<$STFILE)
start=$((start+1))
MESSAGE=${MESSAGE}$'\n'"Today is Day #${start}"
echo "$start" > $STFILE
mail $EMAIL < $MESSAGE
A simple answer would be to export the current value to an external file, and read that back in again later.
So, for example, make a file called "CurrentDay.dat" that has the number 1 in it.
Then, in your bash script, read in the number and increment it.
e.g. your bash script could be:
#!/bin/bash
#Your stuff here.
DayCounter=$(<CurrentDay.dat)
#Use the value of DayCounter (i.e. $DayCounter) in your message.
DayCounter=$((DayCounter + 1))
echo $DayCounter > CurrentDay.dat
Of course, you may need to implement some additional checks to avoid something going wrong, but that should work as is.
I am using Bash on RedHat. I need to schedule a cron job to run at at 9:00 AM on first Sunday of every month. How can I do this?
You can put something like this in the crontab file:
00 09 * * 7 [ $(date +\%d) -le 07 ] && /run/your/script
The date +%d gives you the number of the current day, and then you can check if the day is less than or equal to 7. If it is, run your command.
If you run this script only on Sundays, it should mean that it runs only on the first Sunday of the month.
Remember that in the crontab file, the formatting options for the date command should be escaped.
It's worth noting that what looks like the most obvious approach to this problem does not work.
You might think that you could just write a crontab entry that specifies the day-of-week as 0 (for Sunday) and the day-of-month as 1-7, like this...
# This does NOT work.
0 9 1-7 * 0 /path/to/your/script
... but, due to an eccentricity of how Cron handles crontab lines with both a day-of-week and day-of-month specified, this won't work, and will in fact run on the 1st, 2nd, 3rd, 4th, 5th, 6th, and 7th of the month (regardless of what day of the week they are) and on every Sunday of the month.
This is why you see the recommendation of using a [ ... ] check with date to set up a rule like this - either specifying the day-of-week in the crontab and using [ and date to check that the day-of-month is <=7 before running the script, as shown in the accepted answer, or specifying the day-of-month range in the crontab and using [ and date to check the day-of-week before running, like this:
# This DOES work.
0 9 1-7 * * [ $(date +\%u) = 7 ] && /path/to/your/script
Some best practices to keep in mind if you'd like to ensure that your crontab line will work regardless of what OS you're using it on:
Use =, not ==, for the comparison. It's more portable, since not all shells use an implementation of [ that supports the == operator.
Use the %u specifier to date to get the day-of-week as a number, not the %a operator, because %a gives different results depending upon the locale date is being run in.
Just use date, not /bin/date or /usr/bin/date, since the date utility has different locations on different systems.
You need to combine two approaches:
a) Use cron to run a job every Sunday at 9:00am.
00 09 * * 7 /usr/local/bin/once_a_week
b) At the beginning of once_a_week, compute the date and extract the day of the month via shell, Python, C/C++, ... and test that is within 1 to 7, inclusive. If so, execute the real script; if not, exit silently.
A hacky solution: have your cron job run every Sunday, but have your script check the date as it starts, and exit immediately if the day of the month is > 7...
This also works with names of the weekdays:
0 0 1-7 * * [ "$(date '+\%a')" == "Sun" ] && /usr/local/bin/urscript.sh
But,
[ "$(date '+\%a')" == "Sun" ] && echo SUNDAY
will FAIL on comandline due to special treatment of "%" in crontab (also valid for https://stackoverflow.com/a/3242169/2919695)
Run a cron task 1st monday, 3rd tuesday, last sunday, anything..
http://xr09.github.io/cron-last-sunday/
Just put the run-if-today script in the path and use it with cron.
30 6 * * 6 root run-if-today 1 Sat && /root/myfirstsaturdaybackup.sh
The run-if-today script will only return 0 (bash value for True) if it's the right date.
EDIT:
Now with simpler interface, just one parameter for week number.
# run every first saturday
30 6 * * 6 root run-if-today 1 && /root/myfirstsaturdaybackup.sh
# run every last sunday
30 6 * * 7 root run-if-today L && /root/lastsunday.sh
There is a hacky way to do this with a classic (Vixie, Debian) cron:
0 9 1-7 * */7
The day-of-week field starts with a star (*), and so cron considers it "unrestricted" and uses the AND logic between the day-of-month and the day-of-week fields.
*/7 means "every 7 days starting from weekday 0 (Sunday)". Effectively, this means "every Sunday".
Here's my article with more details: Schedule Cronjob for the First Monday of Every Month, the Funky Way
Note – it's a hack. If you use this expression, make sure to document it to avoid confusion later.
maybe use cron.hourly to call another script. That script will then check to see if it's the first sunday of the month and 9am, and if so, run your program. Sounds optimal enough to me :-).
If you don't want cron to run your job everyday or every Sunday you could write a wrapper that will run your code, determine the next first Sunday, and schedule itself to run on that date.
Then schedule that wrapper for the next first Sunday of the month. After that it will handle everything itself.
The code would be something like (emphasis on something...no error checking done):
#! /bin/bash
#We run your code first
/path/to/your/code
#now we find the next day we want to run
nskip=28 #the number of days we want to check into the future
curr_month=`date +"%m"`
new_month=`date --date='$nskip days' +"%m"`
if [[ curr_month = new_month ]]
then
((nskip+=7))
fi
date=`date --date='$nskip days' +"09:00AM %D` #you may need to change the format if you use another scheduler
#schedule the job using "at"
at -m $date < /path/to/wrapper/code
The logic is simple to find the next first Sunday. Since we start on the first Sunday of the current month, adding 28 will either put us on the last Sunday of the current month or the first Sunday of the next month. If it is the current month, we increment to the next Sunday (which will be in the first week of the next month).
And I used "at". I don't know if that is cheating. The main idea though is finding the next first Sunday. You can substitute whatever scheduler you want after that, since you know the date and time you want to run the job (a different scheduler may need a different syntax for the date, though).
try the following
0 15 10 ? * 1#1
http://www.quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger
00 09 1-7 * 0 /usr/local/bin/once_a_week
every sunday of first 7 days of the month