I'm currently trying to run script that will run in the background when my AWS instance boots for the duration of the instance life. I'm testing it with a simple script to see if it works, before I test with my more complicated one:
#!/bin/bash
while [true]; do
sleep 1
echo "Hello World" >> "tempStorage.json"
done
And my sudo crontab -l returns:
# All the comment stuff
#reboot sh /home/ubuntu/test/testScript/test.sh
Which is the path to the script. I've also obviously run chmod +x test.sh to make sure its an executable.
The problem is when I stop and then start the AWS instance there's nothing in the tempStorage.json file. I've checked other threads and they all suggest this is what I should be doing, so I'm very confused and advice would be appreciated. Thanks.
As Mark B mentioned, the issue is the execution directory of the cron script. There are two solutions then.
A) Change the path to file as Mark B recommended so the script would look something like:
#!/bin/bash
while [true]; do
sleep 1
echo "Hello World" >> "/home/ubuntu/test/testScript/tempStorage.json"
done
B) Change the directory of the cron execution and keep the script as it was. This works better if you need to put the script in any directory. It would look like this for the crontab:
# All the comment stuff
#reboot cd /home/ubuntu/test/testScript && sh test.sh
That should work fine. I think the issue is that you aren't giving the full path to the tempSTorage.json file within your script. So it is being written to in a different folder than the one you are looking in, specifically whatever folder cron starts processes in by default. Try changing it to something like /tmp/tempSTorage.json and then rebooting the server again.
Note that if you are wanting something that starts on boot and runs forever, this probably isn't the best method. In that case I would look into running your process as a service.
Related
I'm setting a cron job that is a bash script containing the below:
#!/bin/bash
NUM_CONTAINERS=$(docker ps -q | wc -l)
if [ $NUM_CONTAINERS -lt 40 ]
then
echo "Time: $(date). Restart containers."
cd /opt
pwd
sudo docker kill $(docker ps -q)
docker-compose up -d
echo "Completed."
else
echo Nothing to do
fi
The output is appended to a log file:
>> cron.log
However the output in the cron file only shows:
Time: Sun Aug 15 10:50:01 UTC 2021. Restart containers.
/opt
Completed.
Both command do not seem to execute as I don't see any change in my containers either.
These 2 non working commands work well in a standalone .sh script without condition though.
What am I doing wrong?
User running the cron has sudo privileges, and we can see the second echo printing.
Lots of times, things that work outside of cron don't work within cron because the environment is not set up in the same way.
You should generally capture standard output and error, to see if something going wrong.
For example, use >> cron.log 2>&1 in your crontab file, this will capture both.
There's at least the possibility that docker is not in your path or, even if it is, the docker commands are not working for some other reason (that you're not seeing since you only capture standard output).
Capturing standard error should help out with that, if it is indeed the issue.
As an aside, I tend to use full path names inside cron scripts, or set up very limited environments at the start to ensure everything works correctly (once I've established why it's not working correctly).
As written in the title, I am having some problem with my cron job script not executing. I am using CentOS 7.
My crontab -e looks like this:
30 0 * * * /opt/abc/efg/cron_jobs.sh >> /opt/abc/logs/cron_jobs.log
My cron_jobs.sh looks like this:
#!/bin/bash
#keep this script in efg folder
#run this daily through crontab -e
#45 0 * * * /opt/abc/efg/cron_job.sh
cd "$(dirname "$0")"
export PYTHONPATH=$PYTHONPATH:`pwd`
#some daily jobs script for abc
date
#send email to users whose keys will expire 7 days later
/usr/local/bin/python2.7 scripts/send_expiration_reminder.py -d 7
#send email to key owners whos keys will expire
/usr/local/bin/python2.7 scripts/send_expiration_reminder.py -d -1
# review user follow status daily task
# Need to use venv due to some library dependencies
/opt/abc/virtualenv/bin/python2.7 scripts/review_user_status.py
So, what I've found is that the log for the cron jobs in /var/logs/cron states that the cron ran at 0:30 am accordingly.
Strangely, I find that /opt/abc/logs/cron_jobs.log empty, and the scripts does not seem to run at all. It used to output some log before I re-inputted the crontab (to re-install the cron jobs), and replaced cron_jobs.sh, so I think the problem might have arose from those actions.
And also, I would like to know if there are any ways to log the error from executing a python script. I have been trying to run /opt/abc/virtualenv/bin/python2.7 scripts/review_user_status.py but it never seem to work as intended (does not run the main function at all), and there is no log output whatsoever.
I tried to run this on a different machine and it works properly, so I am not sure what is wrong with the cron job.
Here is a snippet of the log I got from /var/log/cron to show that the cron called the job:
Mar 22 18:32:01 web41 CROND[20252]: (root) CMD (/opt/abc/efg/cron_jobs.sh >> /opt/abc/logs/cron_jobs.log)
There are a few areas to check if you haven't performed these already,
if your executable permissions set on the script,
chmod +x <python file>
in addition permissions for the user to access the directories.
Run the script manually to test the script works from beginning to end, as the user who will be running the script, will be more realistic.
You can test your crontab schedule by temporarily setting every minute for testing, unlike Windows where you can right, click and click Run.
First, thank you all for the suggestions and heads up. I found out that what was ruining my script is the existence of /r in the line break. Apparently, Linux in general does not accept /r and only accepts /n.
It is because I ftp my files to the machine where the script breaks. On the other hand, it works fine on another machine because I used git pull instead of ftp.
Hope that this info will also be a helpful to others!
I very very rarely use Linux and so don't have any experience with bash scripts and cron jobs.
This is in fact my first attempt. So it's probably something really simple to fix.
I have the following:
/etc/cron.d/clear-mixtape-dir.sh
permissions are: 644
#!/bin/bash
# Clears the /tmp/mixtape2 directory
rm -rf "/tmp/mixtape2/"*
My crontab file looks like so:
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
*/15 * * * * /etc/cron.d/clear-mixtape-dir.sh >/dev/null 2>&1
I'm trying to execute the .sh script every 15 minutes.
Everything i've found says this should work, but it doesn't.
Does anything like file permissions (on files within /tmp/mixtape2/) matter in this case?
Or perhaps the permissions set on the actual .sh script - maybe they need setting to executable?
Any advice appreciated.
Remove the .sh extension from the script in /etc/cron.d and it will be called.
run-parts ignores files with a period in the name, so the .sh extension is preventing your script from running.
From man cron -
Files must conform to the same naming convention as used by run-parts(8): they must consist solely of upper- and lower-case letters, digits, underscores, and hyphens.
Note: These comments refer to /etc/crontab.
Before doing anything else, which cron are you accessing crontab -e or
su -vim
<your-favorite-editor> /etc/crontab
If you are using crontab -e, then no user field exists in that form of crontab. That might be why you're not running.
In your example, your user field is *. I would make it root or a user that has proper permissions.
Before running this program, I would make a dummy crontab entry that just does
echo "Hello" and runs every minute. Get that to work on which ever crontab you're editing (crontab -e or vim /etc/crontab). Then using that as a template, get your script to run.
Next, see if cron is running:
ps -ef | grep cron
If it is not running, become root and start it by enter
/etc/init.d/cron start (Ubuntu and Red Hat).
You already have a good answer suggesting you add root as the user because of a permissions problem. I'm going to suggest more things to help you debug. I have run into a lot of cron problems over the years.
1) Set the email to a known address, unless you will continually monitor root's email
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/local/bin:/usr/bin
MAILTO=fred#somewhere.com
HOME=/
2) Until everything runs properly, take out the >/dev/null 2>&1 out of your cron entry, so you see the outputs in your email generated after the script runs.
3) Bump */15 down to an interval greater than it takes your script to run -- likr */5, so the script runs more often.
4) I do not know the exact reason, but scripts I run out of cron have to set up their own environments despite being run as that user in cron. This may include steps like cd /home/script-owner and running source .bashrc and calling other script(s) that set environment variables.
*/15 * * * * root /etc/cron.d/clear-mixtape-dir.sh >/dev/null 2>&1
Add user root because your permission seems to be only for root.
I am running Ubuntu 11.10 (Unity interface) and I created a Bash script that uses 'gnome-open' to open a series of web pages I use every morning. When I manually execute the script in the Terminal, the bash script works just fine. Here's a sample of the script (it's all the same so I've shortened it):
#!/bin/bash
gnome-open 'https://docs.google.com';
gnome-open 'https://mail.google.com';
Since it seemed to be working well, I added a job to my crontab (mine, not root's) to execute every weekday at a specific time.
Here's the crontab entry:
30 10 * * 1,2,3,4,5 ~/bin/webcheck.sh
The problem is this error gets returned for every single 'gnome-open' command in the bash script:
GConf-WARNING **: Client failed to connect to the D-BUS daemon:
Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
GConf Error: No D-BUS daemon running
Error: no display specified
I did some searching to try and figure this out. The first thing I tried was relaunching the daemon using SIGHUP:
killall -s SIGHUP gconfd-2
That didn't work so I tried launching the dbus-daemon using this code from the manpage for dbus-launch:
## test for an existing bus daemon, just to be safe
if test -z "$DBUS_SESSION_BUS_ADDRESS" ; then
## if not found, launch a new one
eval `dbus-launch --sh-syntax --exit-with-session`
echo "D-Bus per-session daemon address is: $DBUS_SESSION_BUS_ADDRESS"
fi
But that didn't do anything.
I tried adding simply 'dbus-launch' at the top of my bash script and that didn't work either.
I also tried editing the crontab to include the path to Bash, because I saw that suggestion on another thread but that didn't work.
Any ideas on how I can get this up and running?
Here is how the problem was solved. It turns out the issue was primarily caused by Bash not having access to an X window session (or at least that's how I understood it). So my problem was solved by editing my crontab like so:
30 10 * * 1,2,3,4,5 export DISPLAY=:0 && ~/bin/webcheck.sh
The "export DISPLAY=:0" statement told cron which display to use. I found the answer on this archived Ubuntu forum after searching for "no display specified" or something like that:
http://ubuntuforums.org/archive/index.php/t-105250.html
So now, whenever I'm logged in, exactly at 10:30 my system will automatically launch a series of webpages that I need to look at every day. Saves me having to go through the arduous process of typing in my three-letter alias every time :)
Glad you asked!
It depends on when it is run.
If the Gnome GDM Greeter is live, you can use the DBUS session from the logon dialog, if you will. You can, e.g., use this to send notifications to the logon screen, if no-one is logged in:
function do_notification
{
for pid in $(pgrep gnome-session); do
unset COOKIE
COOKIE="$(grep -z DBUS_SESSION_BUS_ADDRESS /proc/$pid/environ|cut -d= -f2-)"
GNUSER="$(ps --no-heading -o uname $pid)"
echo "Notifying user $GNUSER (gnome-session $pid) with '$#'"
sudo -u "$GNUSER" DBUS_SESSION_BUS_ADDRESS="$COOKIE" /usr/bin/notify-send -c "From CRON:" "$#"
done
unset COOKIE
}
As you can see the above code simply runs the same command (notify-send) on all available gnome-sessions, when called like:
do_notification "I wanted to let you guys know"
You can probably pick this apart and put it to use for your own purposes.
I have a bash script that performs several file operations. When any user runs this script, it executes successfully and outputs a few lines of text but when I try to cron it there are problems. It seems to run (I see an entry in cron log showing it was kicked off) but nothing happens, it doesn't output anything and doesn't do any of its file operations. It also doesn't appear in the running processes anywhere so it appears to be exiting out immediately.
After some troubleshooting I found that removing "set -e" resolved the issue, it now runs from the system cron without a problem. So it works, but I'd rather have set -e enabled so the script exits if there is an error. Does anyone know why "set -e" is causing my script to exit?
Thanks for the help,
Ryan
With set -e, the script will stop at the first command which gives a non-zero exit status. This does not necessarily mean that you will see an error message.
Here is an example, using the false command which does nothing but exit with an error status.
Without set -e:
$ cat test.sh
#!/bin/sh
false
echo Hello
$ ./test.sh
Hello
$
But the same script with set -e exits without printing anything:
$ cat test2.sh
#!/bin/sh
set -e
false
echo Hello
$ ./test2.sh
$
Based on your observations, it sounds like your script is failing for some reason (presumably related to the different environment, as Jim Lewis suggested) before it generates any output.
To debug, add set -x to the top of the script (as well as set -e) to show commands as they are executed.
When your script runs under cron, the environment variables and path may be set differently than when the script is run directly by a user. Perhaps that's why it behaves differently?
To test this: create a new script that does nothing but printenv and echo $PATH.
Run this script manually, saving the output, then run it as a cron job, saving that output.
Compare the two environments. I am sure you will find differences...an interactive
login shell will have had its environment set up by sourcing a ".login", ".bash_profile",
or similar script (depending on the user's shell). This generally will not happen in a
cron job, which is usually the reason for a cron job behaving differently from running
the same script in a login shell.
To fix this: At the top of the script, either explicitly set the environment variables
and PATH to match the interactive environment, or source the user's ".bash_profile",
".login", or other setup script, depending on which shell they're using.