I have a ruby script that connects to an Amazon S3 bucket and downloads the latest production backup. I have tested the script (which is very simple) and it works fine.
However, when I schedule this script to be run as a cron job it seems to fail when it loads the Amazon (aws-s3) gem.
The first few lines of my script looks like this:
#!/usr/bin/env ruby
require 'aws/s3'
As I said, when I run this script manually, it works fine. When I run it via a scheduled cron job, it fails when it tries to load the gem:
`require': no such file to load -- aws/s3 (LoadError)
The crontab for this script looks like this:
0 3 * * * ~/Downloader/download.rb > ~/Downloader/output.log 2>&1
I originally thought it might be because cron is running as a different user, but when I do a 'whoami' at the start of my ruby script it tells me it's running as the same user I always use.
I have also done a bundle init and added the gem to my gemfile, but this doesn't seem to have any affect.
Why does cron fail to load the gem? I am running Ubuntu.
As mentioned here https://coderwall.com/p/vhv8aw you can simply try
rvm cron setup # let RMV do your cron settings
Make sure that you make copy of your crontab before running this command
If you're running it manually and it works you're probably in a different shell environment than cron is executing in. Since you mention you're on Ubuntu, the cron jobs probably execute under /bin/sh, and you're manually running them under /bin/bash if you haven't changed anything.
You can debug your environment problems or you can change the shell that your job runs under.
To debug, There are several ways to figure out what shell your cron jobs are using. It can be defined in
/etc/crontab
or you can make a cron job to dump your shell and environment information, as has been mentioned in this SO answer: How to simulate the environment cron executes a script with?
To switch to that shell and see the actual errors causing your job to fail, do
sudo su
env -i <path to shell> (e.g. /bin/sh)
Then running your script you should see what the errors are and be able to fix them (rubygems?).
Option 2 is to switch shells. You can always try something like:
0 3 * * * /bin/bash -c '~/Downloader/download.rb > ~/Downloader/output.log 2>&1'
To force your job into bash. That might also clear things up.
You may also explicitly set your Gem path:
GEM_HOME="/usr/local/rvm/gems/ruby-1.9.2-p290#my-special-gemset"
in a non cron environment execute echo $PATH, copy the path and paste it into your crontab, before your command:
echo $PATH
/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin
and inside crontab:
PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin
0 3 * * * ~/Downloader/download.rb > ~/Downloader/output.log 2>&1
Add this at the beginning of your cron
PATH="/home/user/.rvm/gems/ruby-2.1.4/bin:/home/user/.rvm/gems/ruby-2.1.4#global/bin:/home/user/.rvm/rubies/ruby-2.1.4/bin:/home/user/.rvm/gems/ruby-2.1.4/bin:/home/user/.rvm/gems/ruby-2.1.4#global/bin:/home/user/.rvm/rubies/ruby-2.1.4/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/home/user/.rvm/bin:/usr/local/sbin:/usr/sbin:/home/user/.rvm/bin:/home/user/.local/bin:/home/user/bin"
GEM_HOME='/home/user/.rvm/gems/ruby-2.1.4'
GEM_PATH='/home/user/.rvm/gems/ruby-2.1.4:/home/user/.rvm/gems/ruby-2.1.4#global'
MY_RUBY_HOME='/home/user/.rvm/rubies/ruby-2.1.4'
IRBRC='/home/user/.rvm/rubies/ruby-2.1.4/.irbrc'
RUBY_VERSION='ruby-2.1.4'
I've tried all the solution above, none of them worked until I tried;
0 12 * * * /bin/bash -l -c 'ruby /Users/simon/Desktop/script.rb'
Related
I'm running a script from crontab in which I want to set the symbolic link for npx. It does some other things which are dependent on the npx command itself. Its running the script as expected on the giving time interval, but its giving me no result for command which npx or whereis npx. When I try to run the script from terminal directly these commands does generate the correct path.
Note that, crontab I'm using is under the root user privilege, i.e set with sudo crontab -e and verified with echoing whoami inside the script which generate 'root')
By default, crontab will run your cron jobs using sh which might be the reason you are getting no results.
Try to explicitly change the shell to your default shell by adjusting the crontab entry:
*/30 * * * * /bin/bash -c "/my_script.sh"
In this case I changed it to bash, you can change it to your desired shell.
I have a build server. I'm using the Azure Build Agent script. It's a shell script that will run continuously while the server is up. Problem is that I cannot seem to get it to run on startup. I've tried /etc/init.d and /etc/rc.local and the agent is not being run. Nothing concerning the build agent in the boot logs.
For /etc/init.d I created the script agent.sh which contains:
#!/bin/bash
sh ~/agent/run.sh
Gave it the proper permissions chmod 755 agent.shand moved it to /etc/init.d.
and for /etc/rc.local, I just appended the following
sh ~/agent/run.sh &
before exit 0.
What am I doing wrong?
EDIT: added examples.
EDIT 2: Just noticed that the init.d README says that shell scripts need to start with #!/bin/sh and not #!/bin/bash. Also used absolute path, but no change.
FINAL EDIT: As #ewrammer suggested, I used cron and it worked. crontab -e and then #reboot /home/user/agent/run.sh.
It is hard to see what is wrong if you are not posting what you have done, but why not add it as a cron job with #reboot as pattern? Then cron will run the script every time the computer starts.
Just in case, using a supervisor could be a good idea, In Ubuntu 14 you don't have systemd but you can choose from others https://en.wikipedia.org/wiki/Process_supervision.
If using immortal, after installing it, you just need to create a run.yml file in /etc/immortal with something like:
cmd: /path/to/command
log:
file: /var/log/command.log
This will start your script/command on every start, besides ensuring your script/app is always up and running.
I am using:
Ruby 1.9.2
whenever 0.7.2
capistrano 2.9.0
capistrano-ext 1.2.1
I am using whenever in conjunction with Capistrano on deploys to manage my crontab files.
I noticed that it completely rewrites my crontab files each time.
I'd like to be able to set environment variables in cron to control PATH and MAILTO settings, which are regular cron environment variables.
Is there a way to make whenever not overwrite the entire crontab file, so that I can add customizations to my crontab file and be sure that they will persist?
Yes, you can do this. You'll just need to assign an identifier to the task being written to crontab:
whenever --update-crontab some_identifier_name
It will generate an entry in crontab like this:
# Begin Whenever generated tasks for: some_identifier_name
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash -l -c 'cd /var/www/test/releases/20120416183153 && script/rails runner -e production '\''Model.some_method'\'' >> /tmp/cron_log.log 2>&1'
# End Whenever generated tasks for: some_identifier_name
Then whenever you call the command above it will only update where it finds the identifier you specified.
This question already has answers here:
CronJob not running
(19 answers)
Closed last month.
I have a cron job that I want to execute every 5 minutes:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /scr_temp/scheduleSpider.sh
In /var/spool/cron/crontabs/root
The cron should execute a shell script:
#!/bin/sh
if [ ! -f "sync.txt" ]; then
touch "sync.txt"
chmod 777 /scr_temp
curl someLink
fi
That works fine from command line but not from cron. However the cron itself is startet but the script does not start.
I read about the path problem but I dont really understand it. I setup a cron that writes some env data to a file. This is the output:
HOME=/root
LOGNAME=root
PATH=/usr/bin:/bin
SHELL=/bin/sh
If I execute the env command in command line I get following output for PATH
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
What path do I have to set in my shell script?
Your $PATH is fine; leave it alone. On Ubuntu, all the commands you're invoking (touch, chmod, curl) are in /bin and/or /usr/bin.
How did you set up the cron job? Did you run crontab some-file as root?
It seems that /etc/crontab is the usual mechanism for running cron commands as root. On my Ubuntu system, sudo crontab -l says no crontab for root. Running crontab as root, as you would for any non-root account, should be ok, but you might consider using /etc/crontab instead. Note that it uses a different syntax than an ordinary crontab, as explained in the comments at the top of /etc/crontab:
$ head -5 /etc/crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
Run sudo crontab -l. Does it show your command?
Temporarily modify your script so it always produces some visible output. For example, add the following right after the #!/bin/sh:
echo "Running scheduleSpider.sh at \`date\`" >> /tmp/scheduleSpider.sh.log
and see what's in /tmp/scheduleSpider.sh.log after a few minutes. (You can set the command to run every minute so you don't have to wait as long for results.) If that works (it should), you can add more echo commands to your script to see in detail what it's doing.
It looks like your script is designed to run only once; it creates the sync.txt file to prevent it from running again. That could be the root (ahem) of your problem. What that your intent? Did you mean to delete sync.txt after running the command, and just forgot to do it?
root's home directory on Ubuntu is /root. The first time your script runs, it should create /root/sync.txt. Does that file exist? If so, how old is it?
Note that curl someLink (assuming someLink is a valid URL) will just dump the content from the specified link to standard output. Was that your intent (it will show up as e-mail to root? Or did you just not show us the entire command?
First: you can substitute the first field with */5 (see man 5 crontab)
Second: have cron mail the output to your email address by entering MAILTO=your#email.address in your crontab. If the script has any output, it'll be mailed. Instead of that, you may have a local mailbox in which you can find the cron output (usually $MAIL).
A better syntax for you CRON is
*/5 * * * * /scr_temp/scheduleSpider.sh
Also, check the authority of your scheduleSpider.sh file. Cron runs under a different user than the one you are likely executing your program interactively, so it may be that cron does not have authority. Try chmod 777 for now, just to check.
I suggest to:
check that /scr_temp/scheduleSpider.sh has executable bit
set PATH properly inside your script or use absolute path to command (/bin/touch instead of touch)
specify absolute path to sync.txt file (or calculate it relatively to script)
Have you added the comand via crontab -e or just by editing the crontab file? You should use crontab -e to get it correctly updated.
Set the working directory in the cron script, it probably doesn't execute the things where you think it should.
You should add /bin/sh before the absolute path of your script.
*/5 * * * * /bin/sh /scr_temp/scheduleSpider.sh
My ruby is in /usr/local/bin. whenever can't find it, and setting PATH at the top of my cron file doesn't work either, I think because whenever is running the command inside of a new bash instance.
# this does not work
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/sbin
# Begin Whenever generated tasks for: foo
0 * * * * /bin/bash -l -c 'cd /srv/foo/releases/20110429110637 && script/rails runner -e production '\''ActiveRecord::SessionStore::Session.destroy_recent(15)'\'''
# End Whenever generated tasks for: foo
How can I tell whenever where my ruby binary is? Making a symbolic link from /usr/bin seems messy to me, but I guess that might be the only option.
This question offers env :PATH, "..." in schedule.rb as a solution, but (a) I can't find any documentation of that feature anywhere in the docs (b) it doesn't seem to have solved the asker's problem (unfortunately it takes non-trivial turnaround time for me to just try it).
update actually it is in the bottom of this page, i'll try it now.
more info
I can't modify the cron command because it's generated by whenever
i verified that if I make a new bash shell with bash -l, /usr/bin/env finds ruby just fine
I just tried the exact command in cron, starting with /bin/bash, from the command line of that user, and it worked.
so, this is very mysterious...
The solution is to put this in schedule.rb:
env :PATH, ENV['PATH']
Here's a little guide I put together on the topic.
rewrite your crontab as
0 * * * * { PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/sbin ; export PATH ;/bin/bash -l -c 'cd /srv/foo/releases/20110429110637 && script/rails runner -e production '\''ActiveRecord::SessionStore::Session.destroy_recent(15)'\''' ; }
Or you should try to figure out why your BASH shell is not picking the PATH=... that is almost certainly in your .profile or .bash_profile.
I hope this helps.
As John Bachir pointed out, you can do it via env. But let me add more input. I am deploying on AWS Opsworks. Unfortunately they do not have a ruby manager (RVM, Rbenv, etc) installed by default.
The first thing I needed to do was SSH into the instance and figure out which ruby I was using. This was easy enough by executing the which ruby command in a terminal.
$ which ruby
/usr/local/bin/ruby
Cron was using ruby located at /usr/bin/ruby. This needed to be changed.
In schedule.rb, I have:
set :env_path, ''
env :PATH, #env_path if #env_path.present?
In local, env_path doesn't need to be set. For most users, the only thing to do is execute whenever as such:
bundle exec whenever --set 'environment=development' --update-crontab
On a staging / production environment, ruby may be installed elsewhere. So running this may be more appropriate:
bundle exec whenever --set 'environment=staging&env_path=/usr/bin/local' --update-crontab
You will need to replace /usr/bin/local with the output of echo $PATH.
In Opsworks, however, I needed to create a custom Chef recipe that looked like:
node[:deploy].each do |application, deploy|
execute 'whenever' do
user 'deploy'
group 'nginx'
cwd "#{deploy[:deploy_to]}/current"
command "bundle exec whenever --set 'environment=#{deploy[:environment_variables][:RAILS_ENV]}&env_path=#{ENV['PATH']}' --update-crontab"
end
end
I hope the information here is clear enough.