I'm at my wits end with this. I have the following script:
#!/usr/bin/env bash
declare -a nriArray=(newrelic-cli node-newrelic nri-flex helm-charts infrastructure-agent opentelemetry-exporter-java opentelemetry-exporter-go newrelic-lambda-cli newrelic-node-apollo-server-plugin nri-kubernetes nri-prometheus nri-redis infrastructure-bundle newrelic-lambda-layers nri-jmx newrelic-winston-logenricher-node nri-cassandra micrometer-registry-newrelic newrelic-fluent-bit-output nri-kafka node-newrelic-aws-sdk nri-elasticsearch nri-mysql nri-nagios nri-snmp node-newrelic-superagent nri-kube-events nri-mssql newrelic-module-util-java newrelic-logenricher-dotnet aws-log-ingestion newrelic-lambda-tracer-java nri-docker nri-oracledb k8s-metadata-injection nri-discovery-kubernetes nri-haproxy nri-postgresql node-native-metrics nri-consul nri-rabbitmq nri-vsphere nri-winservices nri-ecs newrelic-airflow-plugin newrelic-monolog-logenricher-php node-newrelic-koa nri-f5 aws_s3_log_ingestion_lambda dropwizard-metrics-newrelic k8s-webhook-cert-manager newrelic-fluentd-output nri-couchbase nri-memcached nri-mongodb java-aws-lambda logstash-output-plugin nri-apache nri-varnish java-log-extensions nri-nginx nri-statsd newrelic-opencensus-exporter-go newrelic-opencensus-exporter-python python-agent-extension)
cd /Users/aschneider/NewRelic-OpenSource/
counter=0
while [[ "$counter" -lt "${#nriArray[#]}" ]]; do
cd "${nriArray[$counter]}"
git pull &>> /Users/aschneider/NewRelic-OpenSource/cronLog.log
gsed -i '/Already up to date./d' /Users/aschneider/NewRelic-OpenSource/cronLog.log
git clean -q -d -f
cd ..
((counter=counter+1))
done
printf "\n\n" >> /Users/aschneider/NewRelic-OpenSource/cronLog.log
holdingVar=$(cat -s /Users/aschneider/NewRelic-OpenSource/cronLog.log) && echo "$holdingVar" > /Users/aschneider/NewRelic-OpenSource/cronLog.log
/Applications/Utilities/terminal-notifier.app/Contents/MacOS/terminal-notifier -message "~/NewRelic-OpenSource/ repositories were updated."
The script recursively goes through an array of GitHub repos, pulls in any updates to them, and then moves onto the next folder. It removes any lines stating that the folder is up to date in order to only keep changes in the log file, and then it uses Terminal Notifier to alert me after each update. It works perfectly when I run it with bash ~/NewRelic-OpenSource/updateOpenSource.sh, but it doesn't even seem to execute when I put the script in crontab. My crontab -l output is below:
SHELL=/usr/local/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
*/30 8-17 * * 1-5 /usr/local/bin/bash /Users/aschneider/NewRelic-OpenSource/updateOpenSources.sh
I have it running every 30 minutes, from 8am to 5pm, on weekdays. Even if I switch to * * * * * [command], it doesn't do anything, so I don't think it's a scheduling syntax issue. I'm currently running it all on my MacBook, which uses GNU bash, version 5.1.4(1)-release (x86_64-apple-darwin20.2.0). I've tried adding the PATH definition to the crontab, as well as to the script itself. I've also verified the script is executable with chmod +x {script}. Any ideas on what I'm doing wrong here?
Cronjobs run as root by default. You may want to run as the user you would normally run the script as, if you were running the script manually. For example,
*/5 * * * * (add the username here) sample_executable
I have also read, but I may be wrong they do not use your PATH variables. So all executables should be fully refrenced before running the script.
Also check the permissions to run the script in crontab.
Following is the entry in the crontab:
MAILTO=abc#gmail.com
45 14 * * * /home/user/simple.sh
I've also done chmod +x on the simple.sh But the crontab does not run, it doesn't even send an email.
pgrep cron shows an Id. I also tried bouncing crond. But no luck!
Could someone please point out the mistake here
The simple.sh script is:
#! /bin/bash
echo hello
Thanks
Since you are doing a echo within the cron job script, you need to capture its output somewhere.
Your shebang and file mode (using chmod +x) are all right, so those aren't the issue here and running without /bin/sh should work fine.
Try using the following to see the output in cron.log file (This runs every minute)
* * * * * /home/user/simple.sh >> /home/user/cron.log
Note that cron jobs run in separate subprocess shell, with reduced environment, so its output won't be visible on your terminal.
Regarding sending of email - you need to have some mail package (like postman, mutt etc) configured for the cron daemon to send out error mails.
Do not use relative paths, but absolute ones. Also, indicate the binary running the script, that is /bin/sh (or whatever coming from which sh):
45 14 * * * /bin/sh /path/to/script/simple.sh
Maybe there shouldn't be a space in line 1 of your .sh script:
#! /bin/bash
to
#!/bin/bash
Although I could see why it would still seem to work from when invoked in an interactive shell (# could merely comment out the rest of the line).
Still, I'd guess at worst it'd merely ignore that line and inherit cron's interpreter of /bin/sh
Asterisk check script is not running only when run by crontab, but runs by ./script.sh and sh script.sh. Here is the script:
date
asterisk -rx "show channels"
asterisk -rx "zap show channels"
Then I >> into a log file. When I run manually via ./ or sh with >> log.log it works, just not as a crontab listed as
* * * * * /root/script.sh
I have tried adding #!/bash/sh at the top of the script and only the date is shown no matter what I try. I am a noob to bash scripts and I'm trying to learn.
Since feature requests to mark a comment as an answer remain declined, I copy the above solution here.
Have you checked your path? It's almost certainly different when run under cron. (You can set PATH=... in your crontab. From the command line, type "echo $PATH" to see what you're expecting.) It might be more standard to provide full paths to date, asterisk and your log file inside script.sh (e.g., "/bin/date /path/to/asterisk ....") – mjk
In my crontab file I execute a script like so (I edit the crontab using sudo crontab -e):
01 * * * * bash /etc/m/start.sh
The script runs some other scripts like so:
sudo bash -c "/etc/m/abc.sh --option=1" &
sleep 2
sudo bash -c "/etc/m/abc.sh --option=2" &
When cron runs the script start.sh, I do ps aux | grep abc.sh and I see the abc.sh script running.
After a couple of seconds, the script is no longer running, even though abc.sh should take hours to finish.
If I do sudo bash /etc/m/start.sh & from the command line, everything works fine (the abc.sh scripts run for hours in the background until they complete).
How do I debug this?
Is there something I'm doing that is preventing these scripts from running in the background until they are done?
The program(s) you're starting might be expecting a terminal to send their output to, or receive input from.
If you set the MAILTO= variable, and you have a sendmail(-like) daemon installed, you will get an email with the error message(s) it prints, if there are any:
MAILTO=your#email.address.here.com
01 * * * * bash /path/to/something.sh
Another way to debug would be to run the script from the command line, while redirecting all inputs and outputs:
$ sudo bash -c "foo.sh" > output_file 2>&1 < /dev/null
Also, the system log files (usually found in /var/log) might contain useful hints.
I normally have several problems with how cron executes scripts as they normally don't have my environment setup. Is there a way to invoke bash(?) in the same way cron does so I could test scripts before installing them?
Add this to your crontab (temporarily):
* * * * * env > ~/cronenv
After it runs, do this:
env - `cat ~/cronenv` /bin/sh
This assumes that your cron runs /bin/sh, which is the default regardless of the user's default shell.
Footnote: if env contains more advanced config, eg PS1=$(__git_ps1 " (%s)")$, it will error cryptically env: ": No such file or directory.
Cron provides only this environment by default :
HOME user's home directory
LOGNAME user's login
PATH=/usr/bin:/usr/sbin
SHELL=/usr/bin/sh
If you need more you can source a script where you define your environment before the scheduling table in the crontab.
Couple of approaches:
Export cron env and source it:
Add
* * * * * env > ~/cronenv
to your crontab, let it run once, turn it back off, then run
env - `cat ~/cronenv` /bin/sh
And you are now inside a sh session which has cron's environment
Bring your environment to cron
You could skip above exercise and just do a . ~/.profile in front of your cron job, e.g.
* * * * * . ~/.profile; your_command
Use screen
Above two solutions still fail in that they provide an environment connected to a running X session, with access to dbus etc. For example, on Ubuntu, nmcli (Network Manager) will work in above two approaches, but still fail in cron.
* * * * * /usr/bin/screen -dm
Add above line to cron, let it run once, turn it back off. Connect to your screen session (screen -r). If you are checking the screen session has been created (with ps) be aware that they are sometimes in capitals (e.g. ps | grep SCREEN)
Now even nmcli and similar will fail.
You can run:
env - your_command arguments
This will run your_command with empty environment.
Depending on the shell of the account
sudo su
env -i /bin/sh
or
sudo su
env -i /bin/bash --noprofile --norc
From http://matthew.mceachen.us/blog/howto-simulate-the-cron-environment-1018.html
Answering six years later: the environment mismatch problem is one of the problems solved by systemd "timers" as a cron replacement. Whether you run the systemd "service" from the CLI or via cron, it receives exactly the same environment, avoiding the environment mismatch problem.
The most common issue to cause cron jobs to fail when they pass manually is the restrictive default $PATH set by cron, which is this on Ubuntu 16.04:
"/usr/bin:/bin"
By contrast, the default $PATH set by systemd on Ubuntu 16.04 is:
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
So there's already a better chance that a systemd timer is going to find a binary without further hassle.
The downside with systemd timers, is there's a slightly more time to set them up. You first create a "service" file to define what you want to run and a "timer" file to define the schedule to run it on and finally "enable" the timer to activate it.
Create a cron job that runs env and redirects stdout to a file.
Use the file alongside "env -" to create the same environment as a cron job.
Don't forget that since cron's parent is init, it runs programs without a controlling terminal. You can simulate that with a tool like this:
http://libslack.org/daemon/
By default, cron executes its jobs using whatever your system's idea of sh is. This could be the actual Bourne shell or dash, ash, ksh or bash (or another one) symlinked to sh (and as a result running in POSIX mode).
The best thing to do is make sure your scripts have what they need and to assume nothing is provided for them. Therefore, you should use full directory specifications and set environment variables such as $PATH yourself.
The accepted answer does give a way to run a script with the environment cron would use. As others pointed out, this is not the only needed criteria for debugging cron jobs.
Indeed, cron also uses a non-interactive terminal, without an attached input, etc.
If that helps, I have written a script that enables painlessly running a command/script as it would be run by cron. Invoke it with your command/script as first argument and you're good.
This script is also hosted (and possibly updated) on Github.
#!/bin/bash
# Run as if it was called from cron, that is to say:
# * with a modified environment
# * with a specific shell, which may or may not be bash
# * without an attached input terminal
# * in a non-interactive shell
function usage(){
echo "$0 - Run a script or a command as it would be in a cron job, then display its output"
echo "Usage:"
echo " $0 [command | script]"
}
if [ "$1" == "-h" -o "$1" == "--help" ]; then
usage
exit 0
fi
if [ $(whoami) != "root" ]; then
echo "Only root is supported at the moment"
exit 1
fi
# This file should contain the cron environment.
cron_env="/root/cron-env"
if [ ! -f "$cron_env" ]; then
echo "Unable to find $cron_env"
echo "To generate it, run \"/usr/bin/env > /root/cron-env\" as a cron job"
exit 0
fi
# It will be a nightmare to expand "$#" inside a shell -c argument.
# Let's rather generate a string where we manually expand-and-quote the arguments
env_string="/usr/bin/env -i "
for envi in $(cat "$cron_env"); do
env_string="${env_string} $envi "
done
cmd_string=""
for arg in "$#"; do
cmd_string="${cmd_string} \"${arg}\" "
done
# Which shell should we use?
the_shell=$(grep -E "^SHELL=" /root/cron-env | sed 's/SHELL=//')
echo "Running with $the_shell the following command: $cmd_string"
# Let's route the output in a file
# and do not provide any input (so that the command is executed without an attached terminal)
so=$(mktemp "/tmp/fakecron.out.XXXX")
se=$(mktemp "/tmp/fakecron.err.XXXX")
"$the_shell" -c "$env_string $cmd_string" >"$so" 2>"$se" < /dev/null
echo -e "Done. Here is \033[1mstdout\033[0m:"
cat "$so"
echo -e "Done. Here is \033[1mstderr\033[0m:"
cat "$se"
rm "$so" "$se"
Another simple way I've found (but may be error prone, I'm still testing) is to source your user's profile files before your command.
Editing a /etc/cron.d/ script:
* * * * * user1 comand-that-needs-env-vars
Would turn into:
* * * * * user1 source ~/.bash_profile; source ~/.bashrc; comand-that-needs-env-vars
Dirty, but it got the job done for me. Is there a way to simulate a login? Just a command you could run? bash --login didn't work. It sounds like that would be the better way to go though.
EDIT: This seems to be a solid solution: http://www.epicserve.com/blog/2012/feb/7/my-notes-cron-directory-etccrond-ubuntu-1110/
* * * * * root su --session-command="comand-that-needs-env-vars" user1 -l
Answer https://stackoverflow.com/a/2546509/5593430 shows how to obtain the cron environment and use it for your script. But be aware that the environment can differ depending on the crontab file you use. I created three different cron entries to save the environment via env > log. These are the results on an Amazon Linux 4.4.35-33.55.amzn1.x86_64.
1. Global /etc/crontab with root user
MAILTO=root
SHELL=/bin/bash
USER=root
PATH=/sbin:/bin:/usr/sbin:/usr/bin
PWD=/
LANG=en_US.UTF-8
SHLVL=1
HOME=/
LOGNAME=root
_=/bin/env
2. User crontab of root (crontab -e)
SHELL=/bin/sh
USER=root
PATH=/usr/bin:/bin
PWD=/root
LANG=en_US.UTF-8
SHLVL=1
HOME=/root
LOGNAME=root
_=/usr/bin/env
3. Script in /etc/cron.hourly/
MAILTO=root
SHELL=/bin/bash
USER=root
PATH=/sbin:/bin:/usr/sbin:/usr/bin
_=/bin/env
PWD=/
LANG=en_US.UTF-8
SHLVL=3
HOME=/
LOGNAME=root
Most importantly PATH, PWD and HOME differ. Make sure to set these in your cron scripts to rely on a stable environment.
In my case, cron was executing my script using sh, which fail to execute some bash syntax.
In my script I added the env variable SHELL:
#!/bin/bash
SHELL=/bin/bash
I don't believe that there is; the only way I know to test a cron job is to set it up to run a minute or two in the future and then wait.