I'm trying to setup a cron job on Openshift due to import emails in a Redmine application. Therefore, I prepared a minutely script like this:
#!/bin/bash
rake RAILS_ENV=production -f ${OPENSHIFT_REPO_DIR}/Rakefile redmine:email:receive_imap host=imap.googlemail.com port=993 ssl=1 username=xxx#artistii.com password=yyy ...
It runs without problems when launched by hand on a ssh connection. When run by cron, instead, rake could not be found.
Making some debugging, I found that the path is not the same as the login shell; and even if I used a full path for rake, ruby that is found is version 1.8 (not 1.9 as per the cartridge), and whenever I set the very same path as the shell, then libruby-1.9 is not found.
Following some other advice I tried to add the following line in place of setting a custom PATH:
source /usr/bin/rhcsh
but nevertheless rake is still not found. I also tries to use bundle exec.
What is the right way to set an environment for cron on Openshift so that it can run like a login shell?
You may need to cd to the directory where your bundle is installed first (where your Gemfile is) something like this maybe?
cd $OPENSHIFT_REPO_DIR && bundle exec rake .....
This is a bug in the cron cartridge. You can refer to this question in SO. It is actually a question with the Python cartridge and the cron cartridge. But it is the cron cartridge which will affect all. There is also a OpenShift Bug Report mentioned within.
The bug is as you have observed, the cron cartridge uses Ruby 1.8 instead of Ruby 1.9. Thus, the gems installed with Ruby 1.9 are not available to the cron cartridge using Ruby 1.8.
There is already a bugfix for this bug, you can refer to the OpenShift Bug Report. But not too sure if it is pushed out already.
Meanwhile, there is a temporary workaround, by exporting the PATH and LD_LIBRARY_PATH in the cron script. You can refer to the OpenShift Bug Report.
Hope this helps.
If you are using rvm, openshift may getting some problem to shift to default rvm.You can also try something like this so it will set rvm to default before running bundle and can also generate your cron log as well to get the exact status of your cron job:
https://rvm.io/rvm/install
use bundle exec to get rid from more than one version of rake
cd $OPENSHIFT_REPO_DIR && rvm gemset use "yourgemsetname" && RAILS_ENV=production bundle exec rake cron_job:cron_job --silent >> log/cron_log
Related
I have a ruby script that requires 3rd party gems, I'm using in the filter section of logstash built out with docker. I'm using the logstash base image and installing the ruby gems in the docker file .
eg
FROM .../logstash:7.16.2
RUN /usr/share/logstash/bin/ruby -S gem install *gem*
RUN echo "gem '*gem*'" >> /usr/share/logstash/Gemfil
This is the only solution I could find in order to utilize the gems in my ruby script thats being called in the filter of logstash.
From here if I run my image, I will get an error -
[Fatal] Logstash-Logstash was unable to start due to an unexpected Gemfile change.
If you are a user this is a bug.
If you are a logstash developer, please try restarting logstash with the '--enable-local-plugin-development' flag set.
If I append the noted flag to my run command I am able to run locally.
eg
docker run my-image-build --enable-local-plugin-development
But this does not work when I deploy to dev/prod.
I will receive the same error ([FATAL]) in the pod logs.
The noted flag is meant for development purposes
https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html
and I don't know of a way to append the flag to the docker run command in dev/prod if I could use it.
What is the fix for this so I can use the script in dev/prod?
Have I attempted to add the gems in a way that doesn't work?
This was an additional question to get clarity with an issue I was facing -- this issue is resolved by this answer.
How to use a gem in logstash pipeline ruby filter
Specifically the docker file is where I'm installing ruby gems for use in logstash filter ruby script.
I have to add a final line at the end of the docker file after installing gems and adding them to the gemfile..
The docker file looks like this.
FROM .../logstash:7.16.2
RUN /usr/share/logstash/bin/ruby -S gem install *gem*
RUN echo "gem '*gem*'" >> /usr/share/logstash/Gemfile
RUN /usr/share/logstash/bin/logstash-plugin update logstash-filter-grok
I have a running puppet server and want to do following steps on the puppet agent:
download rbenv from git server to /usr/local/rbenv (success)
download ruby-build from git server (success)
create file rbenv.sh in /etc/profile.d, so that on every login then rbenv-environment ist loaded (success)
install a specific ruby version using an exec-resource with rbenv install (success)
activate the specific ruby version using an exec-resource with rbenv global (fail for current login)
install a specific ruby gem (failed because of ruby version dependency)
The problem is, rbenv started from an exec-resource in puppet switches the ruby version in a subshell, and stores that information somewhere. The ruby version for the current shell, where I startet puppet agent --test is not changed. After logging out and in again, the puppet run is successful, because the environment is successfully loaded again.
So my question is: Is there a possibility to run the package provider in a new subshell? Or do I have to use another exec running gem install?
So my question is: Is there a possibility to run the package provider in a new subshell? Or do I have to use another exec running gem install?
No. Providers run in the host Ruby instance. They may launch external commands, including shells, but they themselves are not run that way.
Furthermore, you said,
After logging out and in again, the puppet run is successful, because
the environment is successfully loaded again.
If it really requires logging out and back in to make the rbenv reconfiguration active (which is unclear), then running the provider in a subshell wouldn't be effective anyway. Neither would running gem install via an Exec, unless that Exec also expressly activated the desired Ruby environment.
If you are using Puppet to swap out the system's default Ruby, out from under itself, then you need to be prepared for inconsistent behavior. You may be able to work around it by employing an appropriate Exec to perform the gem installation.
I had this same problem and wrestled for a solution. I wanted my servers to provision successfully from scratch so running provision twice wasn't going to cut it.
I fixed this by adding a symlink to the shim'ed gem. Puppet looks for the gem provider there.
This worked for me...
include rbenv
rbenv::plugin {'rbenv/ruby-build':}
-> rbenv::build { '2.5.3':
global => true,
}
-> file { '/usr/bin/gem':
ensure => link,
target => '/usr/local/rbenv/shims/gem',
}
-> package {'passenger':
ensure => '6.0.1',
provider => gem,
}
I'm currently working on a build pipeline that uses Jenkins and GitLab to trigger builds for the project. Basically, the build is triggered when someone pushes to the repository. Also, some Ruby scripts are executed as part of the build process. These scripts run some checks on the projects and perform some fixes, like synchronizing an Xcode project with added and deleted files from the source directory - in this case they are not the same.
I'm using several tools to configure the pipeline. The builds run on a machine that is physically located on the build slave. Jenkins is deployed to an AWS machine. For this reason, I used pritunl to connect the two on a virtual network. I can use local IPs to communicate between the machines and SSH is working fine both ways.
When I push to the remote the build starts correctly on the slave, but it fails to complete. However, if I manually access using SSH through the terminal, the build performs fine. This is the output I get from Jenkins:
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- xcodeproj (LoadError)
from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /Users/jenkins/workspace/Core/platform/ios/scripts/pbxsync.rb:58:in `<main>'
As you can see, it fails to require Xcodeproj, causing the build to fail. Still, this only happens if the build is triggered by Jenkins, not manually.
This makes me think that Jenkins is using some different installation of Ruby, or at least a different environment. Basically what I need is to install gems for the same Ruby environment that Jenkins is using, but I don't know which one that is. Any ideas?
Jenkins has a console that runs Groovy scripts on the remote slave. I've been playing with it a bit, but not many conclusions so far. Maybe that helps.
This may be important; this is the shebang I'm using for the Ruby scripts: #!/usr/bin/env ruby
On the terminal, I'm using the same user as Jenkins is to access the slave machine. It's called "jenkins".
One thing I forgot to mention is that the output is telling me the right version: /Users/jenkins/.rvm/rubies/ruby-2.4.0. At least that's the path it's indicating it's trying to load the gem from. So I tried the following:
: /Users/jenkins/.rvm/rubies/ruby-2.4.0/bin/ruby
require 'xcodeproj'
Then I press ctrl+D and get no output - that installation of ruby is finding the gem properly.
If you are using Jenkins Slave plugin to communicate between Jenkins Master and Jenkins Slave, every command that u specify will be run in non-interactive shell. That means that Jenkins will only have access to system ruby in your case.
So if you want to install something that needs to be installed you have to do it in system ruby. You are using rvm so: rvm use system and you can install gem to system ruby.
If you want to use different Ruby version than system ruby you need to add RVM to $PATH for non-interactive shell. Here is basic setup that should help: https://rvm.io/rvm/basics
I finally managed it. As #Cosaquee indicated in another response, it's important to distinguish between interactive and non-interactive shells. The main reason for this is because, depending on how you call SSH, it makes a difference. As the man page indicates:
If command is specified, it is executed on the remote host instead of
a login shell.
This is meaningful, because the Launch Command for the node I have set for Jenkins is this one:
ssh jenkins#x.x.x.x java -jar ~/bin/slave.jar
In the meanwhile, I was logging in with the standard ssh jenkins#x.x.x.x from the terminal, which starts a login shell. It makes sense that I was getting different results because the two shells load different initial scripts. Basically, if you use ssh jenkins#x.x.x.x to login into the machine ~/.bash_profile is loaded, while if you specify a command, such as ssh jenkins#x.x.x.x whatever, then ~/.bashrc is loaded instead. As such, I added this line to ~/.bashrc:
[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm"
Without it I got:
RVM is not a function, selecting rubies with 'rvm use ...' will not
work.
The advantage was that I could now use RVM from the same environment Jenkins was using. The rest is easy:
ssh jenkins#x.x.x.x rvm --default use 2.3
And:
ssh jenkins#x.x.x.x
rvm --default use 2.3
And both are now using the same version of ruby.
Ok, I'm making my first ruby app. Who know moving everything over to 'production' is so fugging complicated. So far I've struggled my way through configuring passenger, getting it to run on startup, then getting redis to run on startup.
My last task is on startup to add 1 worker. Right now, I have to ssh in and run my rake command rake workers:start. Obviously this is no good when I want to close ssh.. so I just dont really know how or what the next step is.
I tried copying resque default config to config.ru and it just blows up Passenger with errors. I also looked into resque-pool which some people mentioned but that is over my head.
all i have to do is add 1 worker on bootup. This isnt that serious of an app so simpler would be best at this point.
I don't use the god gem because (1) I've seen a project that was very thrashed by the complexity of setup it introduced, and (2) I'm personally really comfortable with the standard Linux (Ubuntu) tools that handle this kind of thing.
To start the Resque workers on bootup
I have this code in my /etc/rc.local file. I have a deploy user on the system:
# Start Resque
su -l deploy -c "/home/deploy/start-resque-workers"
su -l deploy -c "/home/deploy/start-resque-webui"
Then, in those scripts I set up the ruby environment and run the rake task:
# Load RVM into a shell session *as a function*
if [[ -s "$HOME/.rvm/scripts/rvm" ]] ; then
# First try to load from a user install
source "$HOME/.rvm/scripts/rvm"
elif [[ -s "/usr/local/rvm/scripts/rvm" ]] ; then
# Then try to load from a root install
source "/usr/local/rvm/scripts/rvm"
else
printf "ERROR: An RVM installation was not found.\n"
fi
# Use rvm to switch to the default ruby.
rvm use default
# Now launch the app
cd /home/deploy/app-name-here/current
nohup rake QUEUE=* RAILS_ENV=production environment resque:work &
I've been using this kind of set up for years, and it's solid. The servers don't crash. I don't yet need the overhead of installing another system (like the god gem) to watch over these other servers.
Additionally, I use a capistrano gem to handle restarting the workers on deploy.
In production you should be using god to watch your processes. Even if this project is a small one, I strongly recommend investing your time and setting it up.
Another big a must is Capistrano.
So, if you were using god, here's a config file that would help you.
You could also try scheduling rake resque:work at system startup, using a proper script in /etc/init.d/ or /etc/init/ or another (depends on what system you use). I tried this some time ago and I gave up (don't remember why).
I understand that this my answer isn't exactly what you're looking for right now. But imagine this: if everything is set up, then deploying next version is as easy as running rake deploy on your development machine. And it will take care of pulling your code from repository, running migrations, restarting workers and webservers and what not.
I'm trying to setup an automated "build" server for my rails projects using Hudson CI. SO far it's able to run specs and do metrics on the code but I have 2 different projects dependent on 2 different versions of ruby. So i'm trying to use RVM to run multiple copies of ruby then switch back and forth in a pre-build step.
I found a couple posts like this one that try and explain how to make this work, but I'm not running a startup script for hudson, it starts on boot which is how it worked out of the box when i installed it via the debian instructions.
The problem seems to be that even though hudson runs under the "hudson" account and that account has rvm installed (and working) when it tries to run a shell based prebuild step to call rvm switch 1.8.7 it fails with the error "rvm: command not found"
Not sure what I'm doing wrong. Hudson is using SH as its shell but i also tried using bash. no luck.
Has anyone gotten this working before in this setup?
edit the "/etc/init.d/hudson" (!) and change the line:
SU=/bin/su
... change to:
SU="/bin/su -"
... and add rvm setup in the /home/hudson/.profile
I had the same symptoms as you.
After a couple of hours of headbanging, check your $HOME environment variable for Hudson (viewable at http://yourserver/hudson/systemInfo).
Under Ubuntu, the Tomcat 6 start script doesn't set $HOME. Somehow, Hudson inherited my $HOME instead!
I added HOME=$CATALINA_HOME to the /etc/init.d/tomcat6 script just under the rest of the ENV declarations, and now it all works. Very annoying issue, to be sure.