Starting sidekiq with capistrano - ruby

I want to start sidekiq with capistrano. Below is code for that
namespace :sidekiq do
task :start do
run "cd #{current_path} && bundle exec sidekiq -c 10 -e production -L log/sidekiq.log &"
p capture("ps aux | grep sidekiq | awk '{print $2}' | sed -n 1p").strip!
end
end
It executes successfully but still sidekiq is not started on server.
output:
$ cap sidekiq:start
triggering load callbacks
* 2014-06-03 15:03:01 executing `sidekiq:start'
* executing "cd /home/project/current && bundle exec sidekiq -c 10 -e production -L log/sidekiq.log &"
servers: ["x.x.x.x"]
[x.x.x.x] executing command
command finished in 1229ms
* executing "ps aux | grep sidekiq | awk '{print $2}' | sed -n 1p"
servers: ["x.x.x.x"]
[x.x.x.x] executing command
command finished in 1229ms
"19291"

Your problem lies here:
cd /home/project/current && bundle exec sidekiq -c 10 -e production -L log/sidekiq.log &
When you add & at the end command is being executed in a separate process, but this process is still a child of a current process and is terminated when current process stops. Instead you need to run sidekiq as a deamon.
bundle exec sidekiq -c 10 -e production -L log/sidekiq.log -d
Note the extra -d option

Without making use of any gem, here is my solution working perfectly with Capistrano 3.4.0
namespace :sidekiq do
task :restart do
invoke 'sidekiq:stop'
invoke 'sidekiq:start'
end
before 'deploy:finished', 'sidekiq:restart'
task :stop do
on roles(:app) do
within current_path do
pid = p capture "ps aux | grep sidekiq | awk '{print $2}' | sed -n 1p"
execute("kill -9 #{pid}")
end
end
end
task :start do
on roles(:app) do
within current_path do
execute :bundle, "exec sidekiq -e #{fetch(:stage)} -C config/sidekiq.yml -d"
end
end
end
end

Just in case somehow trying to start/restart/stop environment with capistrano:
bundle exec cap production sidekiq:start
bundle exec cap production sidekiq:stop
bundle exec cap production sidekiq:restart
#staging
bundle exec cap staging sidekiq:start
bundle exec cap staging sidekiq:stop
bundle exec cap staging sidekiq:restart
#same with other dependencies
bundle exec cap production puma:restart
bundle exec cap staging puma:stop
Brief explanation
(in case you are hitting a repo online, like github remember to run your ssh agent to connect via ssh in the repo and pull latest version of the code/branch)
setup your own github ssh key locally
run ssh agent with the key eval $(ssh-agent) && ssh-add ~/.ssh/id_rsa
check agent with ssh -T git#github.com
After that i always use this to deploy
run capistrano targeting env bundle exec cap staging deploy
And these are really handy when you are already in prod and had issues but specially for staging, you could do individual exec depending on your Capfile (for instance most of the time i use puma as rack middleware server and sidekiq for scheculed-jobs)
Capfile
require "capistrano/setup"
# Include default deployment tasks
require "capistrano/deploy"
# Load the SCM plugin appropriate to your project:
#
# require "capistrano/scm/hg"
# install_plugin Capistrano::SCM::Hg
# or
# require "capistrano/scm/svn"
# install_plugin Capistrano::SCM::Svn
# or
require "capistrano/scm/git"
install_plugin Capistrano::SCM::Git
# Include tasks from other gems included in your Gemfile
#
# For documentation on these, see for example:
#
# https://github.com/capistrano/rvm
# https://github.com/capistrano/rbenv
# https://github.com/capistrano/chruby
# https://github.com/capistrano/bundler
# https://github.com/capistrano/rails
# https://github.com/capistrano/passenger
#
require "capistrano/rvm"
# require "capistrano/rbenv"
# require "capistrano/chruby"
require "capistrano/bundler"
require "capistrano/rails/assets"
require "capistrano/rails/migrations"
require "capistrano/yarn"
require "capistrano/puma"
install_plugin Capistrano::Puma # Default puma tasks
require 'capistrano/sidekiq'
require 'slackistrano/capistrano'
require_relative 'lib/capistrano/slack_deployment_message'
# Load custom tasks from `lib/capistrano/tasks` if you have any defined
Dir.glob("lib/capistrano/tasks/*.rake").each { |r| import r }
So in the end for executing start|stop|restart on these features enabled/installed/configured by capistrano
I could always restart puma with capistrano in production:
bundle exec cap production sidekiq:restart
bundle exec cap production puma:restart
As well as in staging:
bundle exec cap staging sidekiq:restart
bundle exec cap staging puma:restart
Hope this helps!
:D

Related

Run rake task on Heroku with arguments and the Heroku app specified

I have a rake task that I used to run with arguments like this:
heroku rake query:process -a myherokuapp
Now that rake task accepts arguments. If I were running it locally, I can call them like this:
rake query:process -- -d true -c false
When I try to run this process on Heroku, it doesn't like the combination of the arguments and the app.
# These don't work
heroku rake query:process -- -d true -c false -a myherokuapp
heroku -a myherokuapp rake query:process -- -d true -c false
Is it possible to run this rake task with arguments on Heroku?
This should work:
heroku -a myherokuapp run rake:process -- -d true -c false

sidekiq not starting with bundle exec

I am trying to start up sidekiq and am using:
bundle exec sidekiq
from the directory my script and gemfile are located.
This is what I am getting:
2015-11-17T19:20:48.801Z 78733 TID-owl77getk INFO: ==================================================================
2015-11-17T19:20:48.801Z 78733 TID-owl77getk INFO: Please point sidekiq to a Rails 3/4 application or a Ruby file
2015-11-17T19:20:48.801Z 78733 TID-owl77getk INFO: to load your worker classes with -r [DIR|FILE].
2015-11-17T19:20:48.801Z 78733 TID-owl77getk INFO: ==================================================================
2015-11-17T19:20:48.801Z 78733 TID-owl77getk INFO: sidekiq [options]
-c, --concurrency INT processor threads to use
-d, --daemon Daemonize process
-e, --environment ENV Application environment
-g, --tag TAG Process tag for procline
-i, --index INT unique process index on this machine
-q, --queue QUEUE[,WEIGHT] Queues to process with optional weights
-r, --require [PATH|DIR] Location of Rails application with workers or file to require
-t, --timeout NUM Shutdown timeout
-v, --verbose Print more verbose output
-C, --config PATH path to YAML config file
-L, --logfile PATH path to writable logfile
-P, --pidfile PATH path to pidfile
-V, --version Print version and exit
-h, --help Show help
The current directory must be a Rails app OR you need to use -r to load your Ruby script so it can configure Sidekiq properly.
bundle exec sidekiq -r ./script.rb

Drop database of Rails application from Ruby script, outside of application

I need to run 'rake db:drop' console command from ruby script (outside of Rails app). Generally it looks simple:
system("cd /my/path && rake db:drop")
But the problem is I'm under rvm hell, and when I'm doing cd /my/path it doesn't load correct gemset and environment, that's why as result I have
Could not find activesupport-3.1.12 in any of the sources
Run `bundle install` to install missing gems
Additionally, I have open connections problem, as Rails application is running.
How it could be done?
(my global task is to close existing connections to the database (we may have some, as I need to drop running rails app database), drop it, and then recreate again)
Thank you!
To kill active connections, I use this in a rake task. That should take care of one problem.
task :kill_postgres_connections => :environment do
db_name = "#{File.basename(Rails.root)}_#{Rails.env}"
sh = <<EOF
ps xa \
| grep postgres: \
| grep #{db_name} \
| grep -v grep \
| awk '{print $1}' \
| xargs kill
EOF
puts `#{sh}`
end
task "db:drop" => :kill_postgres_connections
To drop PostgreSQL db in Rails 4 you can also patch ActiveRecord drop method. Based on https://www.krautcomputing.com/blog/2014/01/10/how-to-drop-your-postgres-database-with-rails-4/
# config/initializers/postgresql_database_tasks.rb
module ActiveRecord
module Tasks
class PostgreSQLDatabaseTasks
def drop
establish_master_connection
connection.select_all "select pg_terminate_backend(pg_stat_activity.pid) from pg_stat_activity where datname='#{configuration['database']}' AND state='idle';"
connection.drop_database configuration['database']
end
end
end
end

Use Foreman to Start Rack App Located in Different Directory

I have a Procfile setup that is running a number of processes successfully:
# /Procfile
redis: bundle exec redis-server
sidekiq: bundle exec sidekiq -v -C ./config.yml
forward: forward 4567 mock-api
I need to add one more process - a Sinatra app that lives in a different directory on my machine. If I cd to the directory, I can start it from the Terminal with:
$ rackup -p 4567
And I can start it from a different directory using the Terminal with:
$ sh -c 'cd /Path/to/project/ && exec rackup -p 4567'
But how should I do this using foreman. I have tried adding the following, but it fails silently:
mock-api: sh -c 'cd /Path/to/project/ && exec rackup -p 4567'
Is this even possible? And if so, how?
Of all the stupid things ...
It was failing because of the hyphen in the process name.

Problem deploying Ruby+RVM and daemontools

I am using daemontools in production to keep the services alive and want to run a Ruby server, not Rails, and without RVM it works well but with RVM I have some issues.
My goal is to start a process by root, make it drop root rights to get another user rights and then spawn a Ruby process with RVM and a specified Ruby version.
Here is the run script I was using until now:
#!/bin/sh
exec 2>&1
cd /app/src
. /usr/local/rvm/scripts/rvm
rvm use 1.9.1-p378
exec setuidgid app_user ruby main.rb
This script works but setuidgid has a major problem: the application will be run by user <x> and group <x> and only this group. If the user is in other groups the process will not have their rights.
So it led me to another approach:
#!/bin/sh
exec 2>&1
cd /app
exec sudo -u app_user rvm 1.9.1-p378 exec ruby main.rb
This one works fine except it is the RVM process which is spawned by daemontools and it does not react when it receives a SIGTERM which is not really nice. Basically it means the service cannot be restarted by hand, which is not good.
I found the answer but looking at the rvmsudo script installed with rvm, here is a working run script:
#!/bin/sh
# redirect stderr to stdout
exec 2>&1
cd /app
# load rvm
. /usr/local/rvm/scripts/rvm
# select ruby version for this application
rvm use 1.9.1
# # depending on your configuration you may need to provide the absolute path to rvm, like that:
# /usr/local/bin/rvm use 1.9.1
# build the exec command line preserving the rvm environment
command="exec sudo -u app_user /usr/bin/env PATH='$PATH'"
[[ -n "${GEM_HOME:-}" ]] && command="${command} GEM_HOME='$GEM_HOME' "
[[ -n "${GEM_PATH:-}" ]] && command="${command} GEM_PATH='$GEM_PATH' "
# this is where your real command line goes
command="${command} ruby main.rb"
# run the application
eval "${command}"

Resources