I'm trying to run a python script on a cronjob. To access environment variables in that script, I've wrapped the python script in a short shell script. However, cron does not appear to be accessing my environment correctly, as I'm still getting errors in my python script about not being able to access environment variables. If you consider the below shell script
#!/bin/bash
source /home/jfeldman/.bashrc
env
Running this script from a cronjob yields only this output
SHELL=/bin/sh
PATH=/usr/bin:/bin
PWD=/home/jfeldman
LANG=en_US.UTF-8
SHLVL=1
HOME=/home/jfeldman
LOGNAME=jfeldman
_=/usr/bin/env
Whereas running that shell script from the command line yields a much larger list of environment variables, including the API tokens that I want to access from my python script. Additionally, my PATH variable is considerably shorter than the cronjob version of env
Does anyone know why Cronjobs couldn't access my environment correctly even if the cronjob is running a script that explicitly declares source /home/jfeldman/.bashrc?
This is the line from the crontab fwiw
*/5 * * * * /home/jfeldman/crons/test_cron.sh >> /home/jfeldman/crons/test.log 2>&1$`
Edit: after some testing it seems like cron can source my virtual environment just fine, just not .bashrc, which seems incredibly bizarre.
Shell sh is not bash. sh doesn't know the command source and possibly wouldn't import .bashrc without errors when using it's source command . /home/user/.bashrc for sourcing it.
Put the line
SHELL=/bin/bash
on top of your crontab to run the jobs with bash.
Sourcing a users .bashrc to a cronjob running as root user is a bad idea, you could think about creating a job in /etc/cron.d/ instead, and specify a user other than root running the job.
5/* * * * * jfeldman /home/jfeldman/cron_runner.sh
.bashrc can change over time it makes sense to copy everyting into the cron_runner.sh or even better - create a new user with own home dir just for running the cronjob in a seperate home dir.
Related
I have a bash script where I am trying to use values of some environment variables. Those variables are defined - I see the keys and values if I run printenv.
Also, these variables are defined and exported like
export FOO="bar"
in both ~/.bash_profile and ~/.bashrc.
I am trying to execute the script via ./script-name which fails to get the environment variables. If I run sudo -E ./script-name, that somehow gets the script the variables it needs.
Confused as to why these variables aren't available to the script even when they are exported in above files.
The only thing I can think of, is that for some reason, the shell process which you are calling to run the script, does not have full read access to your current environment.
ls -al /usr/bin/bash
ls -al /bin/sh
Assuming neither of them are symlinks, make sure that your current user has read and execute priveleges. A safer (in security terms) option, would be for you to install bash in ~/opt, and use #!~/opt/bin/bash as your shebang line.
I was trying to get a crontab working on my Raspberry PI and I think I messed up my environment variables. I can execute a file from the GUI by right-clicking and choosing execute. However I cannot get the same file to run from command line. I can use ls to see the file (ChromeTab.sh), but when I type ChromeTab.sh, I get "bash: ChromeTab.sh: command not found".
I think I messed up my environment variables when I put this in the crontab.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
I followed the examples in Script doesn't run via crontab but works fine standalone.
Any idea what I'm doing wrong?
UPDATE:
OK,
Let me clarify what efforts I took on my part BEFORE posting my question on stackoverflow before getting anymore downvotes.
First of all thanks S. Adam Nissley for your suggestions.
In answer to your steps listed above.
Running this from home path, or fully qualified path does Not work as stated.
Error: bash: ChromeTab.sh: command not found
./ChromeTab.sh
I have also ensured read/write and execute permissions on the file with
chmod +x ./ChromeTab.sh
Also, my bash script starts off with the following shebang
#!/bin/sh
So, what i'm trying to say is, regardless of using crontab or not the issue at hand is that I can not even execute the script from command line. This started happening after I was messing around with the environment variables in the crontab. I'm looking for a way to revert to the situation where I can at least run/execute bash commands from the terminal.
The only way I can effectively execute this script is (right-click execute) through the GUI.
Assuming you are in the same directory as your script, you should just be able to enter
./ChromeTab.sh
If it does not execute, make sure it is executable with the command
chmod +x ./ChromeTab.sh
Or
chmod 755 ./ChromeTab.sh
And if it still won't execute, make sure it has an appropriate hashbang on the very first line of the script like #!/bin/sh or #!/bin/bash
When you add it to your crontab, make sure it has the full path like
/home/pi/bin/ChromeTab.sh <br/>
EDIT: Default PATH and SHELL for Raspbian
You can check your PATH and SHELL environmental variables from the command line as follows:
echo $SHELL
echo $PATH
The default PATH for Rasbian is:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
And the default SHELL is:
/bin/bash
So if you need to set those it is as simple as:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
SHELL=/bin/bash
If you are having other issues with your environment, you may want to disable some of your local settings to see if the problem is in your profile. You can list all files with ls -a, which includes hidden files. Raspbian typically has a .bashrc and a .profile in each user's home directory. To disable them simple rename them:
mv .bashrc .bashrc_disabled
mv .profile .profile_disabled
If that solves the problem, you can inspect the files and make the necessary corrections before renaming them back to their original names.
I have a script which runs perfectly fine while it is run in the shell.
But runs incompletely or partially while the same script is scheduled in cron job.
It gets terminated abruptly
I have scheduled the cron job to run every 15mins.
Here is the code for the cronjob.
0,15,30,45 * * * * /home/convteam/Conversion/update_revenue/RunUpdateRevenue.ksh > /home/convteam/Conversion/update_revenue/RunUpdateRevenue.log
I'm scheduling the job using a user called as 'convteam' and not as a root.
Can you help me what I'm missing here ?
This is a recurring issue. The most common reasons for cron jobs failing are:
Assumptions about relative file paths - use full absolute filenames throughout
Assumptions about the environment. You should "source" start-up files that you rely on, typically (for ksh):
. /etc/profile
. ~/.profile
. "$ENV"
$ENV is optional, but when used usually has the value ~/.kshrc.
crontab invokes shell in an non-login, non-interactive mode. Bash shell in this mode does not load the usual startup files such as /etc/profile, ~/.bash_profile, ~/.bash_login, ~/.profile, ~/.bashrc, which may loose some important environment varible definations like PATH.
You may solve your problem by sourcing particular startup files (/etc/profile, ~/.profile in ksh if they exist and are readable) before run ksh script when runing in crontab. you can "man ksh" to see the details of ksh startup process.
I'm trying to get a file to be automatically run by crontab, contents like so:
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
*/5 * * * * php /path/to/script
I've put this file called prod.cron into the /etc/cron.d folder but can't seem to get it to run on it's own. Running the script directly via command line shows that it works okay, but not sure if the files in cron.d get auto detected or do I need to do something else to init them?
Note that directory /etc/cron.d is intended to be used by packages, crontabs should be installed into an users crontab. That being said, check out the following Cron Issues:
The script is not having execute permissions. Use chmod a+x to provide permissions.
In the case of php, perl or other scripting language, the path to interpretor or program is not correct. Always provide actual path to
the interpretor while defining command to run in cron. To find out
where the program is located, use whereis command. For e.g.
for php, whereis php. for perl, whereis php, etc.
The environment variables required to run the script are unavailable. The cron doesn't use the same environment of a user. It
uses its own environment while running the commands. So the variables
defined in .login or .profile are not visible to him. The most common
issue is related to PATH variable which is not similar.
There are two flavors of cron file.
Files installed under /etc/cron.d are system cron files. They require the 6th field on each line to be the user account used to run the command. See other files under /etc/cron.d for examples.
Non-system cron files just have the usual 5 fields to specify when the job runs, followed by the command to run. They're installed by the crontab command, executed by the user account that will run the command(s). This is probably what you wnat to use.
Either add root as the 6th field of the relevant line, or run crontab prod.cron as root (or as whatever account should run the command).
And of course make sure the php command is in the specified $PATH.
I have crontab running a shell script periodically. I need the script to run in the same environment that I usually log in. Can I just simply add this line in 2nd line of the script (after shebang).
source /home/<my username>/.cshrc
Or what's the proper way to set the cron shell process to use my login environment?
PS: I am quite sure which exactly setting is needed by my script, so I can only source the whole .cshrc.
Try something like that:
sudo su - <user> -c <cmd>
Of course you have to alter the sudoers file first.
Take a look at the man page.
hth