Related
Tried solutions found online but none worked so far
How can I further debug this ? Thank you.
I am using a M1 mac with Monterey 12.0.1, I changed from bash to zsh as I thought this might help. I touched .zprofile, copied everthing from .bash_profile
PATH is copied from echo $PATH
The command works executed in terminal but does not work via crontab
inside crontab -e
SHELL=/bin/zsh
PATH=/Users/username/opt/anaconda3/bin:/Users/username/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/Library/Apple/usr/bin
* * * * * /bin/zsh /Users/username/path/to/folder/script.sh > /tmp/result.log
/tmp/result.log is empty
script.sh
#!/bin/zsh
search_dir=/Users/username/path/to/folder
for i in {1..3}
do
echo "Number: $i"
touch $search_dir/$i.txt
done
I'm trying to get cron to call in the correct PATHs. When I run a Python script from shell the script runs fine as it uses the PATHs set in bashrc but when I use cron all the PATHs are not used from bashrc. Is there a file I can enter the PATHs into for cron like bashrc or a way to call the PATHs from bashrc?
Sorry I don't think I worded this correctly, I can get the correct script to run (meaning the PATH to the script in crontab is not the problem here), it's just when that script is running I run a build and this uses the PATHs set in .bashrc. When I run the script when I'm logged in, the .bashrc PATHs are pulled in. Since cron doesn't run in a shell per say it does not pull in .bashrc. Is there a way of pulling this in without having to write a bash script wrapper?
I used /etc/crontab. I used vi and entered in the PATHs I needed into this file and ran it as root. The normal crontab overwrites PATHs that you have set up. A good tutorial on how to do this.
The systemwide cron file looks like this:
This has the username field, as used by /etc/crontab.
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file.
# This file also has a username field, that none of the other crontabs do.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user command
42 6 * * * root run-parts --report /etc/cron.daily
47 6 * * 7 root run-parts --report /etc/cron.weekly
52 6 1 * * root run-parts --report /etc/cron.monthly
01 01 * * 1-5 root python /path/to/file.py
Most likely, cron is running in a very sparse environment. Check the environment variables cron is using by appending a dummy job which dumps env to a file like this:
* * * * * env > env_dump.txt
Compare that with the output of env in a normal shell session.
You can prepend your own environment variables to the local crontab by defining them at the top of your crontab.
Here's a quick fix to prepend $PATH to the current crontab:
# echo PATH=$PATH > tmp.cron
# echo >> tmp.cron
# crontab -l >> tmp.cron
# crontab tmp.cron
The resulting crontab will look similar to chrissygormley's answer, with PATH defined before the crontab rules.
You should put full paths in your crontab. That's the safest option.
If you don't want to do that you can put a wrapper script around your programs, and set the PATH in there.
e.g.
01 01 * * * command
becomes:
01 01 * * * /full/path/to/command
Also anything called from cron should be be very careful about the programs it runs, and probably set its own choice for the PATH variable.
EDIT:
If you don't know where the command is that you want execute which <command> from your shell and it'll tell you the path.
EDIT2:
So once your program is running, the first thing it should do is set PATH and any other required variable (e.g. LD_LIBRARY_PATH) to the values that are required for the script to run.
Basically instead of thinking how to modify the cron environment to make it more suitable for your program/script - make your script handle the environment it's given, by setting an appropriate one when it starts.
Adding a PATH definition into the user crontab with correct values will help...
I've filled mine with this line on top (after comments, and before cron jobs):
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
And it's enough to get all my scripts working... Include any custom path there if you need to.
Setting PATH right before the command line in my crontab worked for me:
* * * * * PATH=$PATH:/usr/local/bin:/path/to/some/thing
Make your variables work for you, this will allow access t
Define your PATH in /etc/profile.d/*.sh
System-wide environment variables
Files with the .sh extension in the /etc/profile.d directory get executed whenever a bash login shell is entered (e.g. when logging in from the console or over ssh), as well as by the DisplayManager when the desktop session loads.
You can for instance create the file /etc/profile.d/myenvvars.sh and set variables like this:
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0
export PATH=$PATH:$JAVA_HOME/bin
Execute crontab with login option!
CRONTAB run script or command with Environment Variables
0 9 * * * cd /var/www/vhosts/foo/crons/; bash -l -c 'php -f ./download.php'
0 9 * * * cd /var/www/vhosts/foo/crons/; bash -l -c download.sh
Problem
Your script works when you run it from the console but fails in cron.
Cause
Your crontab doesn't have the right path variables (and possibly shell)
Solution
Add your current shell and path the crontab
Script to do it for you
#!/bin/bash
#
# Date: August 22, 2013
# Author: Steve Stonebraker
# File: add_current_shell_and_path_to_crontab.sh
# Description: Add current user's shell and path to crontab
# Source: http://brakertech.com/add-current-path-to-crontab
# Github: hhttps://github.com/ssstonebraker/braker-scripts/blob/master/working-scripts/add_current_shell_and_path_to_crontab.sh
# function that is called when the script exits (cleans up our tmp.cron file)
function finish { [ -e "tmp.cron" ] && rm tmp.cron; }
#whenver the script exits call the function "finish"
trap finish EXIT
########################################
# pretty printing functions
function print_status { echo -e "\x1B[01;34m[*]\x1B[0m $1"; }
function print_good { echo -e "\x1B[01;32m[*]\x1B[0m $1"; }
function print_error { echo -e "\x1B[01;31m[*]\x1B[0m $1"; }
function print_notification { echo -e "\x1B[01;33m[*]\x1B[0m $1"; }
function printline {
hr=-------------------------------------------------------------------------------------------------------------------------------
printf '%s\n' "${hr:0:${COLUMNS:-$(tput cols)}}"
}
####################################
# print message and exit program
function die { print_error "$1"; exit 1; }
####################################
# user must have at least one job in their crontab
function require_gt1_user_crontab_job {
crontab -l &> /dev/null
[ $? -ne 0 ] && die "Script requires you have at least one user crontab job!"
}
####################################
# Add current shell and path to user's crontab
function add_shell_path_to_crontab {
#print info about what's being added
print_notification "Current SHELL: ${SHELL}"
print_notification "Current PATH: ${PATH}"
#Add current shell and path to crontab
print_status "Adding current SHELL and PATH to crontab \nold crontab:"
printline; crontab -l; printline
#keep old comments but start new crontab file
crontab -l | grep "^#" > tmp.cron
#Add our current shell and path to the new crontab file
echo -e "SHELL=${SHELL}\nPATH=${PATH}\n" >> tmp.cron
#Add old crontab entries but ignore comments or any shell or path statements
crontab -l | grep -v "^#" | grep -v "SHELL" | grep -v "PATH" >> tmp.cron
#load up the new crontab we just created
crontab tmp.cron
#Display new crontab
print_good "New crontab:"
printline; crontab -l; printline
}
require_gt1_user_crontab_job
add_shell_path_to_crontab
Source
https://github.com/ssstonebraker/braker-scripts/blob/master/working-scripts/add_current_shell_and_path_to_crontab.sh
Sample Output
The simplest workaround I've found looks like this:
* * * * * root su -l -c command
This example invokes su as root user and starts the shell with the user's full environment, including $PATH, set as if they were logged in. It works the same on different distros, is more reliable than sourcing .bashrc (which hasn't worked for me) and avoids hardcoding specific paths which can be a problem if you're providing an example or setup tool and don't know what distro or file layout on the user's system.
You can also specify the username after su if you want a different user than root, but you should probably leave the root parameter before su command since this ensures su has sufficient privileges to switch to any user you specify.
The default environment for cron jobs is very sparse and may be very different from the environment you develop your python scripts in. For a script that might be run in cron, any environment that you depend on should be set explicitly. In the cron file itself, include full paths to python executables and to your python scripts.
On my AIX cron picks up it's environmental variables from /etc/environment ignoring what is set in the .profile.
Edit: I also checked out a couple of Linux boxes of various ages and these appear to have this file as well, so this is likely not AIX specific.
I checked this using joemaller's cron suggestion and checking the output before and after editing the PATH variable in /etc/environment.
If you don't want to have to make the same edits in various places, then roughly do this:
* * * * * . /home/username/.bashrc && yourcommand all of your args
The . space and then the path to .bashrc and the && command are the magic there to get your environment changes into the running bash shell. Too, if you really want the shell to be bash, it is a good idea to have a line in your crontab:
SHELL=/bin/bash
Hope it helps someone!
#Trevino: your answer helped me solve my problem. However, for a beginner, trying to give a step by step approach.
Get your current installation of java via $ echo $JAVA_HOME
$ crontab -e
* * * * * echo $PATH - this lets you understand whats the PATH value being used by crontab at present. Run crontab and grab $PATH value used by crontab.
Now edit crontab again to set your desired java bin path: a) crontab -e; b) PATH=<value of $JAVA_HOME>/bin:/usr/bin:/bin (its a sample path); c) now your scheduled job/script like */10 * * * * sh runMyJob.sh &; d) remove echo $PATH from crontab as its not needed now.
Set the required PATH in your cron
crontab -e
Edit: Press i
PATH=/usr/local/bin:/usr/local/:or_whatever
10 * * * * your_command
Save and exit :wq
I know this has been answered already, but I thought that his would be useful to some. I had a similar issue that I recently solved (found here) and here are the highlights of the steps I took to answer this question:
make sure that you have the variables you need in PYTHONPATH (found here and here and for more info here) inside the .profile or .bash_profile for any shell you want to test your script in to make sure it works.
edit your crontab to include the directories needed to run your script in a cron job (found here and here)
a) be sure to include the root directory in the PATH variable (.) as explained here (basically if you are running an executable with your command it needs to be able to find root or the directory where the executable is stored) and probably these (/sbin:/bin:/usr/sbin:/usr/bin)
in your crontab file, create a cronjob that will change directory to the directory where you have successfully ran the script before (i.e. Users/user/Documents/foo)
a) This will look like the following:
* * * * cd /Users/user/Documents/foo; bar -l doSomething -v
Should you use webmin then these are the steps how to set the PATH value:
System
-> Scheduled Cron Jobs
-> Create a new environment variable
-> For user: <Select the user name>
-> Variable name: PATH
-> Value: /usr/bin:/bin:<your personal path>
-> Add environment variable: Before all Cron jobs for user
I have a crontab running every hour. The user running it has environment variabless in the .bash_profile that work when the user runs the job from the terminal, however, obviously these don't get picked up by crontab when it runs.
I've tried setting them in .profile and .bashrc but they still don't seem to get picked up. Does anyone know where I can put environment vars that crontab can pick up?
You can define environment variables in the crontab itself when running crontab -e from the command line.
LANG=nb_NO.UTF-8
LC_ALL=nb_NO.UTF-8
# m h dom mon dow command
* * * * * sleep 5s && echo "yo"
This feature is only available to certain implementations of cron. Ubuntu and Debian currently use vixie-cron which allows these to be declared in the crontab file (also GNU mcron).
Archlinux and RedHat use cronie which does not allow environment variables to be declared and will throw syntax errors in the cron.log. Workaround can be done per-entry:
# m h dom mon dow command
* * * * * export LC_ALL=nb_NO.UTF-8; sleep 5s && echo "yo"
I got one more solution for this problem:
0 5 * * * . $HOME/.profile; /path/to/command/to/run
In this case it will pick all the environment variable defined in your $HOME/.profile file.
Of course $HOME is also not set, you have to replace it with the full path of your $HOME.
Setting vars in /etc/environment also worked for me in Ubuntu. As of 12.04, variables in /etc/environment are loaded for cron.
Have 'cron' run a shell script that sets the environment before running the command.
Always.
# #(#)$Id: crontab,v 4.2 2007/09/17 02:41:00 jleffler Exp $
# Crontab file for Home Directory for Jonathan Leffler (JL)
#-----------------------------------------------------------------------------
#Min Hour Day Month Weekday Command
#-----------------------------------------------------------------------------
0 * * * * /usr/bin/ksh /work1/jleffler/bin/Cron/hourly
1 1 * * * /usr/bin/ksh /work1/jleffler/bin/Cron/daily
23 1 * * 1-5 /usr/bin/ksh /work1/jleffler/bin/Cron/weekday
2 3 * * 0 /usr/bin/ksh /work1/jleffler/bin/Cron/weekly
21 3 1 * * /usr/bin/ksh /work1/jleffler/bin/Cron/monthly
The scripts in ~/bin/Cron are all links to a single script, 'runcron', which looks like:
: "$Id: runcron.sh,v 2.1 2001/02/27 00:53:22 jleffler Exp $"
#
# Commands to be performed by Cron (no debugging options)
# Set environment -- not done by cron (usually switches HOME)
. $HOME/.cronfile
base=`basename $0`
cmd=${REAL_HOME:-/real/home}/bin/$base
if [ ! -x $cmd ]
then cmd=${HOME}/bin/$base
fi
exec $cmd ${#:+"$#"}
(Written using an older coding standard - nowadays, I'd use a shebang '#!' at the start.)
The '~/.cronfile' is a variation on my profile for use by cron - rigorously non-interactive and no echoing for the sake of being noisy. You could arrange to execute the .profile and so on instead. (The REAL_HOME stuff is an artefact of my environment - you can pretend it is the same as $HOME.)
So, this code reads the appropriate environment and then executes the non-Cron version of the command from my home directory. So, for example, my 'weekday' command looks like:
: "#(#)$Id: weekday.sh,v 1.10 2007/09/17 02:42:03 jleffler Exp $"
#
# Commands to be done each weekday
# Update ICSCOPE
n.updics
The 'daily' command is simpler:
: "#(#)$Id: daily.sh,v 1.5 1997/06/02 22:04:21 johnl Exp $"
#
# Commands to be done daily
# Nothing -- most things are done on weekdays only
exit 0
If you start the scripts you are executing through cron with:
#!/bin/bash -l
They should pick up your ~/.bash_profile environment variables
Expanding on #carestad example, which I find easier, is to run the script with cron and have the environment in the script.
In crontab -e file:
SHELL=/bin/bash
*/1 * * * * $HOME/cron_job.sh
In cron_job.sh file:
#!/bin/bash
source $HOME/.bash_profile
some_other_cmd
Any command after the source of .bash_profile will have your environment as if you logged in.
Whatever you set in crontab will be available in the cronjobs, both directly and using the variables in the scripts.
Use them in the definition of the cronjob
You can configure crontab so that it sets variables that then the can cronjob use:
$ crontab -l
myvar="hi man"
* * * * * echo "$myvar. date is $(date)" >> /tmp/hello
Now the file /tmp/hello shows things like:
$ cat /tmp/hello
hi man. date is Thu May 12 12:10:01 CEST 2016
hi man. date is Thu May 12 12:11:01 CEST 2016
Use them in the script run by cronjob
You can configure crontab so that it sets variables that then the scripts can use:
$ crontab -l
myvar="hi man"
* * * * * /bin/bash /tmp/myscript.sh
And say script /tmp/myscript.sh is like this:
echo "Now is $(date). myvar=$myvar" >> /tmp/myoutput.res
It generates a file /tmp/myoutput.res showing:
$ cat /tmp/myoutput.res
Now is Thu May 12 12:07:01 CEST 2016. myvar=hi man
Now is Thu May 12 12:08:01 CEST 2016. myvar=hi man
...
For me I had to set the environment variable for a php application. I resolved it by adding the following code to my crontab.
$ sudo crontab -e
crontab:
ENVIRONMENT_VAR=production
* * * * * /home/deploy/my_app/cron/cron.doSomethingWonderful.php
and inside doSomethingWonderful.php I could get the environment value with:
<?php
echo $_SERVER['ENVIRONMENT_VAR']; # => "production"
I hope this helps!
Instead of
0 * * * * sh /my/script.sh
Use bash -l -c
0 * * * * bash -l -c 'sh /my/script.sh'
You can also prepend your command with env to inject Environment variables like so:
0 * * * * env VARIABLE=VALUE /usr/bin/mycommand
Expanding on #Robert Brisita has just expand , also if you don't want to set up all the variables of the profile in the script, you can select the variables to export on the top of the script
In crontab -e file:
SHELL=/bin/bash
*/1 * * * * /Path/to/script/script.sh
In script.sh
#!/bin/bash
export JAVA_HOME=/path/to/jdk
some-other-command
I'm using Oh-my-zsh in my macbook so I've tried many things to get the crontab task runs but finally, my solution was prepending the .zshrc before the command to run.
*/30 * * * * . $HOME/.zshrc; node /path/for/my_script.js
This task runs every 30 minutes and uses .zshrc profile to execute my node command.
Don't forget to use the dot before the $HOME var.
I tried most of the provided solutions, but nothing worked at first. It turns out, though, that it wasn't the solutions that failed to work. Apparently, my ~/.bashrc file starts with the following block of code:
case $- in
*i*) ;;
*) return;;
esac
This basically is a case statement that checks the current set of options in the current shell to determine that the shell is running interactively.
If the shell happens to be running interactively, then it moves on to sourcing the ~/.bashrc file.
However, in a shell invoked by cron, the $- variable doesn't contain the i value which indicates interactivity.
Therefore, the ~/.bashrc file never gets sourced fully. As a result, the environment variables never got set.
If this happens to be your issue, feel free to comment out the block of code as follows and try again:
# case $- in
# *i*) ;;
# *) return;;
# esac
I hope this turns out useful
Unfortunately, crontabs have a very limited environment variables scope, thus you need to export them every time the corntab runs.
An easy approach would be the following example, suppose you've your env vars in a file called env, then:
* * * * * . ./env && /path/to_your/command
this part . ./env will export them and then they're used within the same scope of your command
Another way - inspired by this this answer - to "inject" variables is the following (fcron example):
%daily 00 12 \
set -a; \
. /path/to/file/containing/vars; \
set +a; \
/path/to/script/using/vars
From help set:
-a Mark variables which are modified or created for export.
Using + rather than - causes these flags to be turned off.
So everything in between set - and set + gets exported to env and is then available for other scripts, etc. Without using set the variables get sourced but live in set only.
Aside from that it's also useful to pass variables when a program requires a non-root account to run but you'd need some variables inside that other user's environment. Below is an example passing in nullmailer vars to format the e-mail header:
su -s /bin/bash -c "set -a; \
. /path/to/nullmailer-vars; \
set +a; \
/usr/sbin/logcheck" logcheck
All the above solutions work fine.
It will create issues when there are any special characters in your environment variable.
I have found the solution:
eval $(printenv | awk -F= '{print "export " "\""$1"\"""=""\""$2"\"" }' >> /etc/profile)
For me I had to specify path in my NodeJS file.
// did not work!!!!!
require('dotenv').config()
instead
// DID WORK!!
require('dotenv').config({ path: '/full/custom/path/to/your/.env' })
I found this issue while looking at a similar problem that matched the title, but I am stuck with the environment file syntax that systemd or docker use:
FOO=bar
BAZ=qux
This won't work for Vishal's excellent answer because they aren't bash scripts (note the lack of export).
The solution I've used is to read each line into xargs and export them before running the command:
0 5 * * * export $(xargs < $HOME/.env); /path/to/command/to/run
Set Globally env
sudo sh -c "echo MY_GLOBAL_ENV_TO_MY_CURRENT_DIR=$(pwd)" >> /etc/environment"
Add scheduled job to start a script
crontab -e
*/5 * * * * sh -c "$MY_GLOBAL_ENV_TO_MY_CURRENT_DIR/start.sh"
=)
what worked for me (debian based):
create a file with all the needed env var :
#!/bin/bash
env | grep VAR1= > /etc/environment
env | grep VAR2= >> /etc/environment
env | grep VAR3= >> /etc/environment
then build the crontab content, by calling the env file before calling the script that needs it, therefore start the cron service
(crontab -l ; echo '* * * * * . /etc/environment; /usr/local/bin/python /mycode.py >> /var/log/cron-1.log 2>&1') | crontab
service cron start
nb : for python use case, be sure to call the whole python path, else wrong python could be invocated, generating non-sense syntax error
I'm trying to get cron to call in the correct PATHs. When I run a Python script from shell the script runs fine as it uses the PATHs set in bashrc but when I use cron all the PATHs are not used from bashrc. Is there a file I can enter the PATHs into for cron like bashrc or a way to call the PATHs from bashrc?
Sorry I don't think I worded this correctly, I can get the correct script to run (meaning the PATH to the script in crontab is not the problem here), it's just when that script is running I run a build and this uses the PATHs set in .bashrc. When I run the script when I'm logged in, the .bashrc PATHs are pulled in. Since cron doesn't run in a shell per say it does not pull in .bashrc. Is there a way of pulling this in without having to write a bash script wrapper?
I used /etc/crontab. I used vi and entered in the PATHs I needed into this file and ran it as root. The normal crontab overwrites PATHs that you have set up. A good tutorial on how to do this.
The systemwide cron file looks like this:
This has the username field, as used by /etc/crontab.
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file.
# This file also has a username field, that none of the other crontabs do.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user command
42 6 * * * root run-parts --report /etc/cron.daily
47 6 * * 7 root run-parts --report /etc/cron.weekly
52 6 1 * * root run-parts --report /etc/cron.monthly
01 01 * * 1-5 root python /path/to/file.py
Most likely, cron is running in a very sparse environment. Check the environment variables cron is using by appending a dummy job which dumps env to a file like this:
* * * * * env > env_dump.txt
Compare that with the output of env in a normal shell session.
You can prepend your own environment variables to the local crontab by defining them at the top of your crontab.
Here's a quick fix to prepend $PATH to the current crontab:
# echo PATH=$PATH > tmp.cron
# echo >> tmp.cron
# crontab -l >> tmp.cron
# crontab tmp.cron
The resulting crontab will look similar to chrissygormley's answer, with PATH defined before the crontab rules.
You should put full paths in your crontab. That's the safest option.
If you don't want to do that you can put a wrapper script around your programs, and set the PATH in there.
e.g.
01 01 * * * command
becomes:
01 01 * * * /full/path/to/command
Also anything called from cron should be be very careful about the programs it runs, and probably set its own choice for the PATH variable.
EDIT:
If you don't know where the command is that you want execute which <command> from your shell and it'll tell you the path.
EDIT2:
So once your program is running, the first thing it should do is set PATH and any other required variable (e.g. LD_LIBRARY_PATH) to the values that are required for the script to run.
Basically instead of thinking how to modify the cron environment to make it more suitable for your program/script - make your script handle the environment it's given, by setting an appropriate one when it starts.
Adding a PATH definition into the user crontab with correct values will help...
I've filled mine with this line on top (after comments, and before cron jobs):
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
And it's enough to get all my scripts working... Include any custom path there if you need to.
Setting PATH right before the command line in my crontab worked for me:
* * * * * PATH=$PATH:/usr/local/bin:/path/to/some/thing
Make your variables work for you, this will allow access t
Define your PATH in /etc/profile.d/*.sh
System-wide environment variables
Files with the .sh extension in the /etc/profile.d directory get executed whenever a bash login shell is entered (e.g. when logging in from the console or over ssh), as well as by the DisplayManager when the desktop session loads.
You can for instance create the file /etc/profile.d/myenvvars.sh and set variables like this:
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0
export PATH=$PATH:$JAVA_HOME/bin
Execute crontab with login option!
CRONTAB run script or command with Environment Variables
0 9 * * * cd /var/www/vhosts/foo/crons/; bash -l -c 'php -f ./download.php'
0 9 * * * cd /var/www/vhosts/foo/crons/; bash -l -c download.sh
Problem
Your script works when you run it from the console but fails in cron.
Cause
Your crontab doesn't have the right path variables (and possibly shell)
Solution
Add your current shell and path the crontab
Script to do it for you
#!/bin/bash
#
# Date: August 22, 2013
# Author: Steve Stonebraker
# File: add_current_shell_and_path_to_crontab.sh
# Description: Add current user's shell and path to crontab
# Source: http://brakertech.com/add-current-path-to-crontab
# Github: hhttps://github.com/ssstonebraker/braker-scripts/blob/master/working-scripts/add_current_shell_and_path_to_crontab.sh
# function that is called when the script exits (cleans up our tmp.cron file)
function finish { [ -e "tmp.cron" ] && rm tmp.cron; }
#whenver the script exits call the function "finish"
trap finish EXIT
########################################
# pretty printing functions
function print_status { echo -e "\x1B[01;34m[*]\x1B[0m $1"; }
function print_good { echo -e "\x1B[01;32m[*]\x1B[0m $1"; }
function print_error { echo -e "\x1B[01;31m[*]\x1B[0m $1"; }
function print_notification { echo -e "\x1B[01;33m[*]\x1B[0m $1"; }
function printline {
hr=-------------------------------------------------------------------------------------------------------------------------------
printf '%s\n' "${hr:0:${COLUMNS:-$(tput cols)}}"
}
####################################
# print message and exit program
function die { print_error "$1"; exit 1; }
####################################
# user must have at least one job in their crontab
function require_gt1_user_crontab_job {
crontab -l &> /dev/null
[ $? -ne 0 ] && die "Script requires you have at least one user crontab job!"
}
####################################
# Add current shell and path to user's crontab
function add_shell_path_to_crontab {
#print info about what's being added
print_notification "Current SHELL: ${SHELL}"
print_notification "Current PATH: ${PATH}"
#Add current shell and path to crontab
print_status "Adding current SHELL and PATH to crontab \nold crontab:"
printline; crontab -l; printline
#keep old comments but start new crontab file
crontab -l | grep "^#" > tmp.cron
#Add our current shell and path to the new crontab file
echo -e "SHELL=${SHELL}\nPATH=${PATH}\n" >> tmp.cron
#Add old crontab entries but ignore comments or any shell or path statements
crontab -l | grep -v "^#" | grep -v "SHELL" | grep -v "PATH" >> tmp.cron
#load up the new crontab we just created
crontab tmp.cron
#Display new crontab
print_good "New crontab:"
printline; crontab -l; printline
}
require_gt1_user_crontab_job
add_shell_path_to_crontab
Source
https://github.com/ssstonebraker/braker-scripts/blob/master/working-scripts/add_current_shell_and_path_to_crontab.sh
Sample Output
The simplest workaround I've found looks like this:
* * * * * root su -l -c command
This example invokes su as root user and starts the shell with the user's full environment, including $PATH, set as if they were logged in. It works the same on different distros, is more reliable than sourcing .bashrc (which hasn't worked for me) and avoids hardcoding specific paths which can be a problem if you're providing an example or setup tool and don't know what distro or file layout on the user's system.
You can also specify the username after su if you want a different user than root, but you should probably leave the root parameter before su command since this ensures su has sufficient privileges to switch to any user you specify.
The default environment for cron jobs is very sparse and may be very different from the environment you develop your python scripts in. For a script that might be run in cron, any environment that you depend on should be set explicitly. In the cron file itself, include full paths to python executables and to your python scripts.
On my AIX cron picks up it's environmental variables from /etc/environment ignoring what is set in the .profile.
Edit: I also checked out a couple of Linux boxes of various ages and these appear to have this file as well, so this is likely not AIX specific.
I checked this using joemaller's cron suggestion and checking the output before and after editing the PATH variable in /etc/environment.
If you don't want to have to make the same edits in various places, then roughly do this:
* * * * * . /home/username/.bashrc && yourcommand all of your args
The . space and then the path to .bashrc and the && command are the magic there to get your environment changes into the running bash shell. Too, if you really want the shell to be bash, it is a good idea to have a line in your crontab:
SHELL=/bin/bash
Hope it helps someone!
#Trevino: your answer helped me solve my problem. However, for a beginner, trying to give a step by step approach.
Get your current installation of java via $ echo $JAVA_HOME
$ crontab -e
* * * * * echo $PATH - this lets you understand whats the PATH value being used by crontab at present. Run crontab and grab $PATH value used by crontab.
Now edit crontab again to set your desired java bin path: a) crontab -e; b) PATH=<value of $JAVA_HOME>/bin:/usr/bin:/bin (its a sample path); c) now your scheduled job/script like */10 * * * * sh runMyJob.sh &; d) remove echo $PATH from crontab as its not needed now.
Set the required PATH in your cron
crontab -e
Edit: Press i
PATH=/usr/local/bin:/usr/local/:or_whatever
10 * * * * your_command
Save and exit :wq
I know this has been answered already, but I thought that his would be useful to some. I had a similar issue that I recently solved (found here) and here are the highlights of the steps I took to answer this question:
make sure that you have the variables you need in PYTHONPATH (found here and here and for more info here) inside the .profile or .bash_profile for any shell you want to test your script in to make sure it works.
edit your crontab to include the directories needed to run your script in a cron job (found here and here)
a) be sure to include the root directory in the PATH variable (.) as explained here (basically if you are running an executable with your command it needs to be able to find root or the directory where the executable is stored) and probably these (/sbin:/bin:/usr/sbin:/usr/bin)
in your crontab file, create a cronjob that will change directory to the directory where you have successfully ran the script before (i.e. Users/user/Documents/foo)
a) This will look like the following:
* * * * cd /Users/user/Documents/foo; bar -l doSomething -v
Should you use webmin then these are the steps how to set the PATH value:
System
-> Scheduled Cron Jobs
-> Create a new environment variable
-> For user: <Select the user name>
-> Variable name: PATH
-> Value: /usr/bin:/bin:<your personal path>
-> Add environment variable: Before all Cron jobs for user
I normally have several problems with how cron executes scripts as they normally don't have my environment setup. Is there a way to invoke bash(?) in the same way cron does so I could test scripts before installing them?
Add this to your crontab (temporarily):
* * * * * env > ~/cronenv
After it runs, do this:
env - `cat ~/cronenv` /bin/sh
This assumes that your cron runs /bin/sh, which is the default regardless of the user's default shell.
Footnote: if env contains more advanced config, eg PS1=$(__git_ps1 " (%s)")$, it will error cryptically env: ": No such file or directory.
Cron provides only this environment by default :
HOME user's home directory
LOGNAME user's login
PATH=/usr/bin:/usr/sbin
SHELL=/usr/bin/sh
If you need more you can source a script where you define your environment before the scheduling table in the crontab.
Couple of approaches:
Export cron env and source it:
Add
* * * * * env > ~/cronenv
to your crontab, let it run once, turn it back off, then run
env - `cat ~/cronenv` /bin/sh
And you are now inside a sh session which has cron's environment
Bring your environment to cron
You could skip above exercise and just do a . ~/.profile in front of your cron job, e.g.
* * * * * . ~/.profile; your_command
Use screen
Above two solutions still fail in that they provide an environment connected to a running X session, with access to dbus etc. For example, on Ubuntu, nmcli (Network Manager) will work in above two approaches, but still fail in cron.
* * * * * /usr/bin/screen -dm
Add above line to cron, let it run once, turn it back off. Connect to your screen session (screen -r). If you are checking the screen session has been created (with ps) be aware that they are sometimes in capitals (e.g. ps | grep SCREEN)
Now even nmcli and similar will fail.
You can run:
env - your_command arguments
This will run your_command with empty environment.
Depending on the shell of the account
sudo su
env -i /bin/sh
or
sudo su
env -i /bin/bash --noprofile --norc
From http://matthew.mceachen.us/blog/howto-simulate-the-cron-environment-1018.html
Answering six years later: the environment mismatch problem is one of the problems solved by systemd "timers" as a cron replacement. Whether you run the systemd "service" from the CLI or via cron, it receives exactly the same environment, avoiding the environment mismatch problem.
The most common issue to cause cron jobs to fail when they pass manually is the restrictive default $PATH set by cron, which is this on Ubuntu 16.04:
"/usr/bin:/bin"
By contrast, the default $PATH set by systemd on Ubuntu 16.04 is:
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
So there's already a better chance that a systemd timer is going to find a binary without further hassle.
The downside with systemd timers, is there's a slightly more time to set them up. You first create a "service" file to define what you want to run and a "timer" file to define the schedule to run it on and finally "enable" the timer to activate it.
Create a cron job that runs env and redirects stdout to a file.
Use the file alongside "env -" to create the same environment as a cron job.
Don't forget that since cron's parent is init, it runs programs without a controlling terminal. You can simulate that with a tool like this:
http://libslack.org/daemon/
By default, cron executes its jobs using whatever your system's idea of sh is. This could be the actual Bourne shell or dash, ash, ksh or bash (or another one) symlinked to sh (and as a result running in POSIX mode).
The best thing to do is make sure your scripts have what they need and to assume nothing is provided for them. Therefore, you should use full directory specifications and set environment variables such as $PATH yourself.
The accepted answer does give a way to run a script with the environment cron would use. As others pointed out, this is not the only needed criteria for debugging cron jobs.
Indeed, cron also uses a non-interactive terminal, without an attached input, etc.
If that helps, I have written a script that enables painlessly running a command/script as it would be run by cron. Invoke it with your command/script as first argument and you're good.
This script is also hosted (and possibly updated) on Github.
#!/bin/bash
# Run as if it was called from cron, that is to say:
# * with a modified environment
# * with a specific shell, which may or may not be bash
# * without an attached input terminal
# * in a non-interactive shell
function usage(){
echo "$0 - Run a script or a command as it would be in a cron job, then display its output"
echo "Usage:"
echo " $0 [command | script]"
}
if [ "$1" == "-h" -o "$1" == "--help" ]; then
usage
exit 0
fi
if [ $(whoami) != "root" ]; then
echo "Only root is supported at the moment"
exit 1
fi
# This file should contain the cron environment.
cron_env="/root/cron-env"
if [ ! -f "$cron_env" ]; then
echo "Unable to find $cron_env"
echo "To generate it, run \"/usr/bin/env > /root/cron-env\" as a cron job"
exit 0
fi
# It will be a nightmare to expand "$#" inside a shell -c argument.
# Let's rather generate a string where we manually expand-and-quote the arguments
env_string="/usr/bin/env -i "
for envi in $(cat "$cron_env"); do
env_string="${env_string} $envi "
done
cmd_string=""
for arg in "$#"; do
cmd_string="${cmd_string} \"${arg}\" "
done
# Which shell should we use?
the_shell=$(grep -E "^SHELL=" /root/cron-env | sed 's/SHELL=//')
echo "Running with $the_shell the following command: $cmd_string"
# Let's route the output in a file
# and do not provide any input (so that the command is executed without an attached terminal)
so=$(mktemp "/tmp/fakecron.out.XXXX")
se=$(mktemp "/tmp/fakecron.err.XXXX")
"$the_shell" -c "$env_string $cmd_string" >"$so" 2>"$se" < /dev/null
echo -e "Done. Here is \033[1mstdout\033[0m:"
cat "$so"
echo -e "Done. Here is \033[1mstderr\033[0m:"
cat "$se"
rm "$so" "$se"
Another simple way I've found (but may be error prone, I'm still testing) is to source your user's profile files before your command.
Editing a /etc/cron.d/ script:
* * * * * user1 comand-that-needs-env-vars
Would turn into:
* * * * * user1 source ~/.bash_profile; source ~/.bashrc; comand-that-needs-env-vars
Dirty, but it got the job done for me. Is there a way to simulate a login? Just a command you could run? bash --login didn't work. It sounds like that would be the better way to go though.
EDIT: This seems to be a solid solution: http://www.epicserve.com/blog/2012/feb/7/my-notes-cron-directory-etccrond-ubuntu-1110/
* * * * * root su --session-command="comand-that-needs-env-vars" user1 -l
Answer https://stackoverflow.com/a/2546509/5593430 shows how to obtain the cron environment and use it for your script. But be aware that the environment can differ depending on the crontab file you use. I created three different cron entries to save the environment via env > log. These are the results on an Amazon Linux 4.4.35-33.55.amzn1.x86_64.
1. Global /etc/crontab with root user
MAILTO=root
SHELL=/bin/bash
USER=root
PATH=/sbin:/bin:/usr/sbin:/usr/bin
PWD=/
LANG=en_US.UTF-8
SHLVL=1
HOME=/
LOGNAME=root
_=/bin/env
2. User crontab of root (crontab -e)
SHELL=/bin/sh
USER=root
PATH=/usr/bin:/bin
PWD=/root
LANG=en_US.UTF-8
SHLVL=1
HOME=/root
LOGNAME=root
_=/usr/bin/env
3. Script in /etc/cron.hourly/
MAILTO=root
SHELL=/bin/bash
USER=root
PATH=/sbin:/bin:/usr/sbin:/usr/bin
_=/bin/env
PWD=/
LANG=en_US.UTF-8
SHLVL=3
HOME=/
LOGNAME=root
Most importantly PATH, PWD and HOME differ. Make sure to set these in your cron scripts to rely on a stable environment.
In my case, cron was executing my script using sh, which fail to execute some bash syntax.
In my script I added the env variable SHELL:
#!/bin/bash
SHELL=/bin/bash
I don't believe that there is; the only way I know to test a cron job is to set it up to run a minute or two in the future and then wait.