Drop database of Rails application from Ruby script, outside of application - ruby

I need to run 'rake db:drop' console command from ruby script (outside of Rails app). Generally it looks simple:
system("cd /my/path && rake db:drop")
But the problem is I'm under rvm hell, and when I'm doing cd /my/path it doesn't load correct gemset and environment, that's why as result I have
Could not find activesupport-3.1.12 in any of the sources
Run `bundle install` to install missing gems
Additionally, I have open connections problem, as Rails application is running.
How it could be done?
(my global task is to close existing connections to the database (we may have some, as I need to drop running rails app database), drop it, and then recreate again)
Thank you!

To kill active connections, I use this in a rake task. That should take care of one problem.
task :kill_postgres_connections => :environment do
db_name = "#{File.basename(Rails.root)}_#{Rails.env}"
sh = <<EOF
ps xa \
| grep postgres: \
| grep #{db_name} \
| grep -v grep \
| awk '{print $1}' \
| xargs kill
EOF
puts `#{sh}`
end
task "db:drop" => :kill_postgres_connections

To drop PostgreSQL db in Rails 4 you can also patch ActiveRecord drop method. Based on https://www.krautcomputing.com/blog/2014/01/10/how-to-drop-your-postgres-database-with-rails-4/
# config/initializers/postgresql_database_tasks.rb
module ActiveRecord
module Tasks
class PostgreSQLDatabaseTasks
def drop
establish_master_connection
connection.select_all "select pg_terminate_backend(pg_stat_activity.pid) from pg_stat_activity where datname='#{configuration['database']}' AND state='idle';"
connection.drop_database configuration['database']
end
end
end
end

Related

Sidekiq Broken Pipe Error

I am attempting to migrate from Heroku to AWS, but my Sidekiq jobs keep failing with the following error:
Errno::EPIPE: Broken pipe # io_write - <STDOUT>
I can successfully run jobs from the console using perform_now, and everything works just fine in Heroku, so I am presuming the issue lies somewhere with my AWS setup. I have seen references to improper daemonization around Stack Overflow and Github but not sure how to solve the problem.
Right now I am launching my processes with the following command:
foreman start -f Procfile -p 3000 -e $VAR_FILES &
and I have tried the command both with and without the & at the end.
My Procfile looks like this:
web: bundle exec puma -t 1:2 -p ${PORT:-3000} -e ${RACK_ENV:-production}
worker: bundle exec sidekiq -C config/sidekiq.yml
log: tail -f log/production.log
and I have also tried it like this, following the instructions here (https://github.com/mperham/sidekiq/wiki/Logging#syslog):
worker: bundle exec sidekiq -C config/sidekiq.yml 2>&1 | logger -t sidekiq
My sidekiq.yml has logfile set to ./log/sidekiq.log, which I believe is supposed to redirecting logs away from STDOUT anyway.
I have seen the discussion here (https://github.com/mperham/sidekiq/issues/3188) and can verify that the rails12factor gem is not in my Gemfile.
But still the error persists... Can anyone lend a hand?
UPDATE: I can finally get a stack trace and see it is coming from a puts statement inside of the Neo4j.rb gem:
2017-04-07T15:46:53.553Z 697 TID-12a6r4 WARN: Errno::EPIPE: Broken pipe # io_write - <STDOUT>
2017-04-07T15:46:53.553Z 697 TID-12a6r4 WARN: /var/lib/gems/2.3.0/bundler/gems/neo4j-c804cb33bef8/lib/neo4j/session_manager.rb:60:in `write'
/var/lib/gems/2.3.0/bundler/gems/neo4j-c804cb33bef8/lib/neo4j/session_manager.rb:60:in `puts'
/var/lib/gems/2.3.0/bundler/gems/neo4j-c804cb33bef8/lib/neo4j/session_manager.rb:60:in `puts'
But still not sure how I can mitigate the issue. I have tried with RAILS_LOG_TO_STDOUT=enabled both set and unset.
I spoke to the gem maintainers and they removed the puts statements in v 8.0.13. It fixed the problem for me!

Serverspec doesn't check package version correctly

I have a problem with serverspec. I'm trying to check installed package version on ubuntu.
I use this code:
describe 'java packages' do
it 'package openjdk-9-jre should be installed with the correct version' do
expect(package('openjdk-9-jre')).to be_installed.with_version('9~b114-0ubuntu1')
end
end
Serverspec run dpkg-query command to check package but escapes tilda character and it doesn't work.
serverspec runs:
dpkg-query -f '${Status} ${Version}' -W openjdk-9-jre | grep -E '^(install|hold) ok installed 9\\~b114-0ubuntu1$'
instead of
dpkg-query -f '${Status} ${Version}' -W openjdk-9-jre | grep -E '^(install|hold) ok installed 9~b114-0ubuntu1$'
How can I fix this problem?
The problem is here: https://github.com/mizzy/specinfra/blob/92ccc19714ead956589127c40e3cd65faf38cb8b/lib/specinfra/command/debian/base/package.rb#L6.
Specinfra is escaping the characters in the with_version chain as #{Regexp.escape(escape(version))} instead of #{Regexp.escape(version)). This would require a PR to Specinfra to fix due to the Specinfra/Serverspec contribution policy. I can put this on my list of things to do and notify you when finished, since I keep an up-to-date Specinfra fork around and am a contributor to both so I know the codebase.
In the meantime, you would have to do a command matcher workaround.
describe 'java packages' do
it 'package openjdk-9-jre should be installed with the correct version' do
describe command("dpkg-query -f '${Status} ${Version}' -W openjdk-9-jre") do
its(:stdout) { is_expected.to match('^(install|hold) ok installed 9\~b114\-0ubuntu1$') }
end
end
end
Specinfra PR: https://github.com/mizzy/specinfra/pull/608

Starting sidekiq with capistrano

I want to start sidekiq with capistrano. Below is code for that
namespace :sidekiq do
task :start do
run "cd #{current_path} && bundle exec sidekiq -c 10 -e production -L log/sidekiq.log &"
p capture("ps aux | grep sidekiq | awk '{print $2}' | sed -n 1p").strip!
end
end
It executes successfully but still sidekiq is not started on server.
output:
$ cap sidekiq:start
triggering load callbacks
* 2014-06-03 15:03:01 executing `sidekiq:start'
* executing "cd /home/project/current && bundle exec sidekiq -c 10 -e production -L log/sidekiq.log &"
servers: ["x.x.x.x"]
[x.x.x.x] executing command
command finished in 1229ms
* executing "ps aux | grep sidekiq | awk '{print $2}' | sed -n 1p"
servers: ["x.x.x.x"]
[x.x.x.x] executing command
command finished in 1229ms
"19291"
Your problem lies here:
cd /home/project/current && bundle exec sidekiq -c 10 -e production -L log/sidekiq.log &
When you add & at the end command is being executed in a separate process, but this process is still a child of a current process and is terminated when current process stops. Instead you need to run sidekiq as a deamon.
bundle exec sidekiq -c 10 -e production -L log/sidekiq.log -d
Note the extra -d option
Without making use of any gem, here is my solution working perfectly with Capistrano 3.4.0
namespace :sidekiq do
task :restart do
invoke 'sidekiq:stop'
invoke 'sidekiq:start'
end
before 'deploy:finished', 'sidekiq:restart'
task :stop do
on roles(:app) do
within current_path do
pid = p capture "ps aux | grep sidekiq | awk '{print $2}' | sed -n 1p"
execute("kill -9 #{pid}")
end
end
end
task :start do
on roles(:app) do
within current_path do
execute :bundle, "exec sidekiq -e #{fetch(:stage)} -C config/sidekiq.yml -d"
end
end
end
end
Just in case somehow trying to start/restart/stop environment with capistrano:
bundle exec cap production sidekiq:start
bundle exec cap production sidekiq:stop
bundle exec cap production sidekiq:restart
#staging
bundle exec cap staging sidekiq:start
bundle exec cap staging sidekiq:stop
bundle exec cap staging sidekiq:restart
#same with other dependencies
bundle exec cap production puma:restart
bundle exec cap staging puma:stop
Brief explanation
(in case you are hitting a repo online, like github remember to run your ssh agent to connect via ssh in the repo and pull latest version of the code/branch)
setup your own github ssh key locally
run ssh agent with the key eval $(ssh-agent) && ssh-add ~/.ssh/id_rsa
check agent with ssh -T git#github.com
After that i always use this to deploy
run capistrano targeting env bundle exec cap staging deploy
And these are really handy when you are already in prod and had issues but specially for staging, you could do individual exec depending on your Capfile (for instance most of the time i use puma as rack middleware server and sidekiq for scheculed-jobs)
Capfile
require "capistrano/setup"
# Include default deployment tasks
require "capistrano/deploy"
# Load the SCM plugin appropriate to your project:
#
# require "capistrano/scm/hg"
# install_plugin Capistrano::SCM::Hg
# or
# require "capistrano/scm/svn"
# install_plugin Capistrano::SCM::Svn
# or
require "capistrano/scm/git"
install_plugin Capistrano::SCM::Git
# Include tasks from other gems included in your Gemfile
#
# For documentation on these, see for example:
#
# https://github.com/capistrano/rvm
# https://github.com/capistrano/rbenv
# https://github.com/capistrano/chruby
# https://github.com/capistrano/bundler
# https://github.com/capistrano/rails
# https://github.com/capistrano/passenger
#
require "capistrano/rvm"
# require "capistrano/rbenv"
# require "capistrano/chruby"
require "capistrano/bundler"
require "capistrano/rails/assets"
require "capistrano/rails/migrations"
require "capistrano/yarn"
require "capistrano/puma"
install_plugin Capistrano::Puma # Default puma tasks
require 'capistrano/sidekiq'
require 'slackistrano/capistrano'
require_relative 'lib/capistrano/slack_deployment_message'
# Load custom tasks from `lib/capistrano/tasks` if you have any defined
Dir.glob("lib/capistrano/tasks/*.rake").each { |r| import r }
So in the end for executing start|stop|restart on these features enabled/installed/configured by capistrano
I could always restart puma with capistrano in production:
bundle exec cap production sidekiq:restart
bundle exec cap production puma:restart
As well as in staging:
bundle exec cap staging sidekiq:restart
bundle exec cap staging puma:restart
Hope this helps!
:D

Ruby RVM under another user from Bash script

What is the best way to launch ruby script under another user via su command and from another script?
I have a bash script with the following command for launching unicorn:
sudo -u unicornuser sh -l -c "bundle exec unicorn_rails -E production -D"
This script is an init.d script and it being execute when system starts and manually also.
The problem is the default ruby on system is 1.8, other rubyes (1.9) is working under RVM. I need to modify above script that it can execute ruby with RVM (non system wide)
Now I've migrated for this notation:
su -l -c "rvm use ruby-1.9.3-p125 && bundle exec unicorn_rails -E production -D" unicornuser
this worked for me, but I guess there must be a better way to do this.
You should use wrappers:
rvm wrapper ruby-1.9.3-p125 ext_1.9.3 bundle
This will create $rvm_path/bin/ext_1.9.3_bundle, so now you can use:
/full/path/to/rvm/bin/ext_1.9.3_bundle exec unicorn_rails -E production -D
replace /full/path/to/rvm with output from echo $rvm_path

Problem deploying Ruby+RVM and daemontools

I am using daemontools in production to keep the services alive and want to run a Ruby server, not Rails, and without RVM it works well but with RVM I have some issues.
My goal is to start a process by root, make it drop root rights to get another user rights and then spawn a Ruby process with RVM and a specified Ruby version.
Here is the run script I was using until now:
#!/bin/sh
exec 2>&1
cd /app/src
. /usr/local/rvm/scripts/rvm
rvm use 1.9.1-p378
exec setuidgid app_user ruby main.rb
This script works but setuidgid has a major problem: the application will be run by user <x> and group <x> and only this group. If the user is in other groups the process will not have their rights.
So it led me to another approach:
#!/bin/sh
exec 2>&1
cd /app
exec sudo -u app_user rvm 1.9.1-p378 exec ruby main.rb
This one works fine except it is the RVM process which is spawned by daemontools and it does not react when it receives a SIGTERM which is not really nice. Basically it means the service cannot be restarted by hand, which is not good.
I found the answer but looking at the rvmsudo script installed with rvm, here is a working run script:
#!/bin/sh
# redirect stderr to stdout
exec 2>&1
cd /app
# load rvm
. /usr/local/rvm/scripts/rvm
# select ruby version for this application
rvm use 1.9.1
# # depending on your configuration you may need to provide the absolute path to rvm, like that:
# /usr/local/bin/rvm use 1.9.1
# build the exec command line preserving the rvm environment
command="exec sudo -u app_user /usr/bin/env PATH='$PATH'"
[[ -n "${GEM_HOME:-}" ]] && command="${command} GEM_HOME='$GEM_HOME' "
[[ -n "${GEM_PATH:-}" ]] && command="${command} GEM_PATH='$GEM_PATH' "
# this is where your real command line goes
command="${command} ruby main.rb"
# run the application
eval "${command}"

Resources