I'm trying to run a python script on a cronjob. To access environment variables in that script, I've wrapped the python script in a short shell script. However, cron does not appear to be accessing my environment correctly, as I'm still getting errors in my python script about not being able to access environment variables. If you consider the below shell script
#!/bin/bash
source /home/jfeldman/.bashrc
env
Running this script from a cronjob yields only this output
SHELL=/bin/sh
PATH=/usr/bin:/bin
PWD=/home/jfeldman
LANG=en_US.UTF-8
SHLVL=1
HOME=/home/jfeldman
LOGNAME=jfeldman
_=/usr/bin/env
Whereas running that shell script from the command line yields a much larger list of environment variables, including the API tokens that I want to access from my python script. Additionally, my PATH variable is considerably shorter than the cronjob version of env
Does anyone know why Cronjobs couldn't access my environment correctly even if the cronjob is running a script that explicitly declares source /home/jfeldman/.bashrc?
This is the line from the crontab fwiw
*/5 * * * * /home/jfeldman/crons/test_cron.sh >> /home/jfeldman/crons/test.log 2>&1$`
Edit: after some testing it seems like cron can source my virtual environment just fine, just not .bashrc, which seems incredibly bizarre.
Shell sh is not bash. sh doesn't know the command source and possibly wouldn't import .bashrc without errors when using it's source command . /home/user/.bashrc for sourcing it.
Put the line
SHELL=/bin/bash
on top of your crontab to run the jobs with bash.
Sourcing a users .bashrc to a cronjob running as root user is a bad idea, you could think about creating a job in /etc/cron.d/ instead, and specify a user other than root running the job.
5/* * * * * jfeldman /home/jfeldman/cron_runner.sh
.bashrc can change over time it makes sense to copy everyting into the cron_runner.sh or even better - create a new user with own home dir just for running the cronjob in a seperate home dir.
I would like to use cron on my Mac. I choose it over launchd, because I want to be able to use my new knowledge on Linux as well. However, I cannot seem to get the crontab -e command to work. It fires up vim, I enter my test job:
0-59 * * * * mollerhoj3 echo "Hello World"
But after saving and quitting (:wq),
crontab -l
says:
No crontab for mollerhoj3
What am I doing wrong?
Just follow these steps:
In Terminal: crontab -e.
Press i to go into vim's insert mode.
Type your cron job, for example:
30 * * * * /usr/bin/curl --silent --compressed http://example.com/crawlink.php
Press Esc to exit vim's insert mode.
Type ZZ to exit vim (must be capital letters).
You should see the following message: crontab: installing new crontab. You can verify the crontab file by using crontab -l.
Note however that this might not work depending on the content of your ~/.vimrc file.
I've never had this problem, but I create a ~/.crontab file and edit that (which allows me to back it up, Time Machine or otherwise), then run
crontab ~/.crontab
Has worked for me for 20+ years across many flavors of unix.
NOTE: the answer that says to use the ZZ command doesn't work for me on my Mavericks system, but this is probably due to something in my vim configuration because if I start with a pristine .vimrc, the accepted answer works. My answer might work for you if the other solution doesn't.
On MacOS X, according to the crontab manpage, the crontab temporary file that gets created with crontab -e needs to be edited in-place. Vim doesn't edit in-place by default (but it might do some special case to support crontab -e), so if your $EDITOR environment variable is set to vi (the default) or vim, editing the crontab will always fail.
To get Vim to edit the file in-place, you need to do:
:setlocal nowritebackup
That should enable you to update the crontab when you do crontab -e with the :wq or ZZ commands.
You can add an autocommand in your .vimrc to make this automatically work when editing crontabs:
autocmd FileType crontab setlocal nowritebackup
Another way is to add the setlocal nowritebackup to ~/.vim/after/ftplugin/crontab.vim, which will be loaded by Vim automatically when you're editing a crontab file if you have the Filetype plugin enabled. You can also check for the OS if you're using your vim files across multiple platforms:
""In ~/.vim/after/ftplugin/crontab.vim
if has("mac")
setlocal nowritebackup
endif
The use of cron on OS X is discouraged. launchd is used instead. Try man launchctl to get started. You have to create special XML files that define your jobs and put them in a special place with certain permissions.
You'll usually just need to figure out launchctl load
https://www.unix.com/man-page/osx/1/launchctl/
http://nb.nathanamy.org/2012/07/schedule-jobs-using-launchd/
Edit
If you really do want to use cron on OS X, check out this answer: https://superuser.com/a/243944/2449
Difference between cron and launchd
As has been mentioned cron is deprecated (but supported), and launchd is recommended for OS X.
This is taken from developer.apple.com
Effects of Sleeping and Powering Off
If the system is turned off or asleep, cron jobs do not execute; they will not run until the next designated time occurs.
If you schedule a launchd job by setting the StartCalendarInterval key and the computer is asleep when the job should have run, your job will run when the computer wakes up. However, if the machine is off when the job should have run, the job does not execute until the next designated time occurs.
All other launchd jobs are skipped when the computer is turned off or asleep; they will not run until the next designated time occurs.
Consequently, if the computer is always off at the job’s scheduled time, both cron jobs and launchd jobs never run. For example, if you always turn your computer off at night, a job scheduled to run at 1 A.M. will never be run.
I did 2 things to solve this problem.
I touched the crontab file, described in this link
coderwall.com/p/ry9jwg (Thanks #Andy).
Used Emacs instead of my default vim: EDITOR=emacs crontab -e (I have no idea why vim does not work)
crontab -lnow prints the cronjobs. Now I only need to figure out why the cronjobs are still not running ;-)
In user crontab (crontab -e) do not put the user field.
Correct cron is:
0-59 * * * * echo "Hello World"
Syntax with user field is for /etc/crontab only:
0-59 * * * * mollerhoj3 echo "Hello World"
The error crontab: temp file must be edited in place is because of the way vim treats backup files.
To use vim with cron, add the following lines in your .bash_profile
export EDITOR=vim
alias crontab="VIM_CRONTAB=true crontab"
Source the file:
source .bash_profile
And then in your .vimrc add:
if $VIM_CRONTAB == "true"
set nobackup
set nowritebackup
endif
This will disable backups when using vim with cron. And you will be able to use crontab -e to add/edit cronjobs.
On successfully saving your cronjob, you will see the message:
crontab: installing new crontab
Source:
http://drawohara.com/post/6344279/crontab-temp-file-must-be-edited-in-placeenter link description here
Other option is not to use crontab -e at all. Instead I used:
(crontab -l && echo "1 1 * * * /path/to/my/script.sh") | crontab -
Notice that whatever you print before | crontab - will replace the entire crontab file, so use crontab -l && echo "<your new schedule>" to get the previous content and the new schedule.
As the previous posts didn't work for me because of some permissions issues, I found that creating a separate crontab file and adding it to the user's crontab with the -u parameter while root worked for me.
sudo crontab -u {USERNAME} ~/{PATH_TO_CRONTAB_FILE}
The above has a mix of correct answers. What worked for me for having the exact same errors are:
1) edit your bash config file
$ cd ~ && vim .bashrc
2) in your bash config file, make sure default editor is vim rather than vi (which causes the problem)
export EDITOR=vim
3) edit your vim config file
$cd ~ && vim .vimrc
4) make sure set backupcopy is yes in your .vimrc
set backupcopy=yes
5) restart terminal
6) now try crontab edit
$ crontab -e
10 * * * * echo "hello world"
You should see that it creates the crontab file correctly. If you exit vim (either ZZ or :wq) and list crontab with following command; you should see the new cron job. Hope this helps.
$ crontab -l
Use another text editor
env EDITOR=nano crontab -e
or
env EDITOR=code crontab -e
DISCLAIMER: I am not a Mac user. However, I have used crontab a great deal on Linux and similar systems. I presume that crontab on Mac works similarly.
A number of answers and comments suggest using an editor other than vim. That's fine, but there's no good reason that vim shouldn't work.
The cron system maintains a crontab for each user. The location of that file is unimportant; only the crontab implementation needs to know about it. cron provides an interface that allows users to request updates to their own crontabs.
The crontab -e command performs the following steps:
Copy the user's crontab to a temporary file.
Invoke the editor ($VISUAL or $EDITOR) on that temporary file.
When the editor command terminates, copy the temporary file back to the user's crontab (after doing some sanity checks).
Reload the user's crontab.
The vim editor, when you use it to edit a file, usually creates a new file with the same name. This shouldn't matter. The crontab -e command creates a temporary file, waits for the user to edit it, and then re-reads the temporary file by name. It doesn't keep the temporary file open while the user is editing it. If the Mac crontab implementation does this, I would argue that it's buggy. I'll also note that the original poster did not mention the "crontab: temp file must be edited in place" message; that was mentioned in a comment.
The most plausible explanation I can think of is that the OP was using gvim or something similar. gvim is a GUI version of vim, and if you invoke it from a shell prompt you'll get a new prompt immediately; it launches a GUI editor that runs in the background. If you use crontab -e and it tries to use gvim to edit the temporary file, then crontab will see the editor command terminating immediately, and it won't see any updates.
In fact, I was able to reproduce the problem by invoking
VISUAL=gvim EDITOR=gvim crontab -e
on a Linux system. (Again, I'm not a Mac user, and there might be some Mac-specific weirdness that I'm missing.)
A number of other answers and comments suggest using a different editor. That shouldn't be necessary. Any correctly working text editor should work with crontab -e -- as long as the editor command doesn't terminate until after the temporary file is updated. I've always had vim (or vi or nvi) as my default editor, and I've never had a problem using it with crontab.
The crontab line in the question:
0-59 * * * * mollerhoj3 echo "Hello World"
appears to be intended for a system crontab The first 5 fields specify the schedule, and the 6th is the user name. A normal user crontab does not have this 6th field, since the crontab command keeps track of the owner (the account that ran the crontab command). 99% of the time, you don't need to worry about system crontabs. Even root can use the crontab command to install a user crontab for the root account. However, this alone doesn't explain the problem seen by the OP; the syntax is valid for a user crontab, but it will attempt to invoke a command called mollerhoj3 with arguments echo "Hello World". That will cause an error when the command is scheduled, not when the crontab is created or updated.
Finally, a bit of advice that doesn't address the original question. Using crontab -e can be slightly dangerous, since it's easy to damage or destroy your crontab without meaning to. Instead, I suggest keeping a file in your home directory (possibly maintained in a version control system) and applying the contents of that file using crontab filename. For example, you might create and edit $HOME/.crontab, and run crontab $HOME/.crontab after updating it.
The question is over 9 years old, but the original poster is still more or less active (but may not have the same system).
#Hanzaplastique answer in this comment worked for me: export EDITOR=nano
I'm trying to get a file to be automatically run by crontab, contents like so:
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
*/5 * * * * php /path/to/script
I've put this file called prod.cron into the /etc/cron.d folder but can't seem to get it to run on it's own. Running the script directly via command line shows that it works okay, but not sure if the files in cron.d get auto detected or do I need to do something else to init them?
Note that directory /etc/cron.d is intended to be used by packages, crontabs should be installed into an users crontab. That being said, check out the following Cron Issues:
The script is not having execute permissions. Use chmod a+x to provide permissions.
In the case of php, perl or other scripting language, the path to interpretor or program is not correct. Always provide actual path to
the interpretor while defining command to run in cron. To find out
where the program is located, use whereis command. For e.g.
for php, whereis php. for perl, whereis php, etc.
The environment variables required to run the script are unavailable. The cron doesn't use the same environment of a user. It
uses its own environment while running the commands. So the variables
defined in .login or .profile are not visible to him. The most common
issue is related to PATH variable which is not similar.
There are two flavors of cron file.
Files installed under /etc/cron.d are system cron files. They require the 6th field on each line to be the user account used to run the command. See other files under /etc/cron.d for examples.
Non-system cron files just have the usual 5 fields to specify when the job runs, followed by the command to run. They're installed by the crontab command, executed by the user account that will run the command(s). This is probably what you wnat to use.
Either add root as the 6th field of the relevant line, or run crontab prod.cron as root (or as whatever account should run the command).
And of course make sure the php command is in the specified $PATH.
I need for a Ruby script to be run using an rvm-selected version. I cannot change how the script is invoked, but I can modify the script. The script starts with:
#!/usr/bin/env ruby
Now, based on some information I found (in this question, for example), I tried this:
#!/usr/bin/env rvm-shell ree-1.8.7-2012.02#gitorious
But this only gives me this error message:
/usr/bin/env: rvm-shell ree-1.8.7-2012.02#gitorious: No such file or directory
Now, rvm is available, because this works (but doesn't bring the required ruby/gemset):
#!/usr/bin/env rvm-shell
I've tried this as well:
#!/usr/local/rvm/bin/rvm-shell ree-1.8.7-2012.02#gitorious
But this doesn't bring in the environment ("gem", which is only installed inside that gemset, is not available, for example). If I run that on the command line itself, it does open a shell with the proper environment.
So, has anyone done something like this? How can I fix it?
Does this work?
#!/location/of/rvm/folder/rubies/ree-1.8.7-2012.02#gitorious/bin/ruby
I installed Ruby 1.9.3p125 via this guide (up to point #5): LINK
Now I have this problem: my script works wonderfully from my command line, but if I execute it from Cron it seems to use a default environment and defaults to /usr/bin/ruby instead of mine (~/.rvm/rubies/ruby-1.9.3-p125/bin/ruby). What is the best way to have executed commands - manually or via cron - produce the same results?
PS: It seems to skip processing ~/.bash_login for example, where rvm is loaded into PATH
In your crontab line, you can source the .bash_login before you script is run.
source ~/.bash_login && <your original command here>
That way your script will have everything you have when you run it.
The usual way recommended to do this would be to put the full path to the executable in your crontab. E.g.
crontab should show:
/Users/Poochie/.rvm/rubies/ruby-1.9.3-p125/bin/ruby /full/path/to/script.rb
or whatever the full path is. It's much more robust than trying to get rvm loading, as here an rvm script is modifying your path for you. If you want to set it to whichever is the rvm default ruby (e.g. whatever was set by rvm use x.x.x --default), You can use: /Users/Poochie/.rvm/bin/ruby as the executable instead, e.g.:
/Users/Poochie/.rvm/bin/ruby /full/path/to/script.rb
I actually found this post which helped me a lot: LINK
I managed to run my script as I wanted but the questions is theoretically still open because the issue could still affect cron usage in general.