Creating a variable per instance, rather than per request, with Sinatra modular style - ruby

I have a Sinatra app, written in modular style, running on Heroku. It uses Redis and I have a limited number (10) of Redis connections. I found that it would often throw errors complaining that it had run out of Redis connections. So I started using connection_pool in the hope that would fix things; a single pool of Redis connections and the app would choose one of those each time, rather than try to create a new connection on each request.
But I'm still getting the same issue. I can do loads of Redis queries on a single query without complaints. But if I reload a single test page, which just does some Redis queries, several times in fairly quick succession, I get the "Redis::CommandError - ERR max number of clients reached" error again.
So I'm assuming, maybe, it's creating a new instance of connection_pool on each request... I don't know. But it's not "pooling" as I would expect it to.
I have this kind of thing:
# myapp.rb
$LOAD_PATH.unshift(File.dirname(__FILE__))
$stdout.sync = true
require 'thin'
require 'myapp/frontend'
MyApp::Frontend.run!
And the Sinatra app:
# myapp/frontend.rb
require 'sinatra/base'
require 'redis'
require 'connection_pool'
require 'uuid'
module MyApp
class Frontend < Sinatra::Base
helpers do
def redis_pool
#redis_pool ||= ConnectionPool.new(:size => 8, :timeout => 5) do
redis_uri = URI.parse(ENV['REDISCLOUD_URL'])
client = ::Redis.new(:host => redis_uri.host,
:port => redis_uri.port,
:password => redis_uri.password)
end
end
end
get '/tester/'
redis_pool.with do |r|
id = UUID.generate
r.hset(:user, id, "Some data")
r.hget(:user, id)
r.hdel(:user, id)
end
p "DONE"
end
end
end
The Procfile looks like:
web: ruby myapp.rb
Any ideas? The current site is pretty low traffic, so this should be possible.

A new instance of #redis_pool is created every time a get request for /tester/ is processed because the helper method redis_pool is called every time.
You can use sinatra's settings helper to initialize a redis connection only once:
config do
redis_uri = URI.parse(ENV['REDISCLOUD_URL'])
set :redis, Redis.new(:host => redis_uri.host,
:port => redis_uri.port,
:password => redis_uri.password)
end
Now the each instance of the app has one redis connection that persists for all requests. Access the setting like so
get '/tester/'
id = UUID.generate
settings.redis.hset(:user, id, "some data")
settings.redis.hget(:user, id)
settings.redis.hdel(:user, id)
p "DONE"
end

Related

Is it better to create new controller instance for each HTTP request in Rack based app or to use the same instance?

I'm creating a very simple Rack based application as I want it to do a very specific task.
The server.rb looks something like this:
Path= File.expand_path("#{File.dirname __FILE__}/../../")
require "bundler/setup"
require "thin"
require "rack"
%w(parser auth controller).each do |file|
require "#{Path}/app/server/#{file}.rb"
end
builder = Rack::Builder.app do
use Auth
run Parser.new
end
Rack::Handler::Thin.run(builder, :Port => 8080, :threaded => true)
parser.rb looks like:
class Parser
def initialize
#controller = Controller.new
end
def call(env)
req = Rack::Request.new(env).params
res = Rack::Response.new
res['Content-Type'] = "text/plain"
command= req[:command]
if command =~ /\A(register|r|subscribe|s)\z/i
#controller.register
end
res.write command
res.finish
end
end
Now my question here, from design prospective, is it better to create one instance of Controller and use it with each request(like Idid with the code above), or to create new controller instance for each request(change #controller.register to Controller.new.register)? which is better to use and why?
Thanks in advance
The overhead of creating a new controller per request is likely not that large.
If you store state in the controller (instance variables etcetera) and you reuse it, you could run into concurrency issues such as race conditions or deadlock when under load.
If you take care to ensure that your Controller object stores no state, you can reuse it. If it does any sort of state storage per request, you will need to ensure that the shared resources are property synchronized.
My 2c - create a new controller per request, until you can confirm that you have a performance hit from creating a new controller per request. It's simpler, cleaner, and less prone to strange bugs.

Connection Pool returning an instance of Redis and not ConnectionPool

I am trying to make my Rails app use Resque for managing workers. But, I would like to continue using the ConnectionPool gem.
I have this in an initializer:
puts ENV["REDISTOGO_URL"]
uri = (not ENV["REDISTOGO_URL"].nil?) ? URI.parse(ENV["REDISTOGO_URL"]) : nil
# at this point, debugger confirms $redis is nil
$redis = ConnectionPool::Wrapper.new(:size => 5, :timeout => 3) {
if uri.nil?
Redis.connect
else
Redis.connect(:host => uri.host, :port => uri.port, :password => uri.password)
end
}
$redis # just put this in here for the debugger
# At this point, $redis is #<Redis:0x007fb1b0036bf0>
# when it should be an instance of ConnectionPool::Wrapper
Does anyone have an idea why $redis would not be returned as an instance ConnectionPool::Wrapper?
I've searched in all the gems source code, nowhere does it set the value of $redis. In ConnectionPool's source code, I did not find anything where it would return an instance of Redis instead of itself.
This only happens when I switched from DelayedJob to Resque. So, it would seem that is the problem. However, I'm at a loss.
I am using Unicorn. Here's that file in config.
worker_processes 2
timeout 30
preload_app true
before_fork do |server, worker|
# Replace with MongoDB or whatever
if defined?(ActiveRecord::Base)
ActiveRecord::Base.connection.disconnect!
Rails.logger.info('Disconnected from ActiveRecord')
end
# If you are using Redis but not Resque, change this
if defined?(Resque)
Resque.redis.quit
Rails.logger.info('Disconnected from Redis')
end
sleep 1
end
after_fork do |server, worker|
# Replace with MongoDB or whatever
if defined?(ActiveRecord::Base)
ActiveRecord::Base.establish_connection
Rails.logger.info('Connected to ActiveRecord')
end
# If you are using Redis but not Resque, change this
if defined?(Resque)
# Yes, commented the Resque out for debugging, still get the same problem.
#Resque.redis = ENV['REDISTOGO_URL']
Rails.logger.info('Connected to Redis')
end
end
And finally, the Procfile:
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
worker: env TERM_CHILD=1 QUEUE=* bundle exec rake resque:work
I'm using foreman in my develop environment.
Any help is greatly appreciated.
From the docs:
You can use ConnectionPool::Wrapper to wrap a single global connection.
I looks like the ConnectionPool::Wrapper is meant to wrap a single connection to Redis as a convenience for migrating large applications from using Redis directly to using ConnectionPools
If you call $redis.with, you get the #with defined by ConnectionPool
To get an actual connection pool, just change your
ConnectionPool::Wrapper.new(:size => 5, :timeout => 3) { #redis logic }
to
ConnectionPool.new(:size => 5, :timeout => 3) { #redis logic }
Internally ConnectionPool::Wrapper creates a normal ConnectionPool object, and uses method_missing to automatically checkout/checkin from that pool whenever any method is called on the wrapper.
This use of method_missing includes calls to inspect, or class, or any number of methods which are normally used to try to look at the object or figure out its type.
require 'connection_pool'
class MyClass
def foo
'bar'
end
end
obj = MyClass.new
obj.respond_to?(:foo) # true
obj.respond_to?(:with) # false
wrapper = ConnectionPool::Wrapper.new { MyClass.new }
wrapper.respond_to?(:foo) # true
wrapper.respond_to?(:with) # also true! 'with' is a method on ConnectionPool::Wrapper
You do have an instance of ConnectionPool::Wrapper, it just is a bit hard to tell.

Can't understand the ruby code

I can't understand what the below ruby code does. Can anyone give me some explanation. Thanks!
map '/healthz' do
run Healthz.new(logger)
end
The Healthz is:
class Healthz
def initialize(logger)
#logger = logger
end
def call(env)
#logger.debug "healthz access"
healthz = Component.updated_healthz
[200, { 'Content-Type' => 'application/json', 'Content-Length' => healthz.length.to_s }, healthz]
rescue => e
#logger.error "healthz error #{e.inspect} #{e.backtrace.join("\n")}"
raise e
end
end
And the lib used are:
require "eventmachine"
require 'thin'
require "yajl"
require "nats/client"
require "base64"
require 'set'
Since you're using eventmachine and thin, I'd guess that code is some kind of routing code for a simple web application.
That is, it maps the /healtz route of the application to the Healtz class, so that if you start up the app, and point your browser to localhost:<whatever_port_thin_uses>/healtz, it would start up a Healtz.new instance for you.
Since I don't know what Healtz actually does, I've no idea what will actually happen, but my guess is that it's some kind of rack application.
And, as I already stated, this is just my guess, from seeing the list of libs you're using.

using redis and ruby to implement a tiny short url app

I'm making a short URL app, using Ruby, Sinatra, and Redis. Currently it's under 15 lines:
require 'rubygems'
require 'sinatra'
require 'redis'
require 'uri'
configure do
REDISTOGO_URL = "redis://localhost:6379/"
uri = URI.parse(REDISTOGO_URL)
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
end
get '/' do
haml :index
end
post '/shorten' do
a = rand(9999)
REDIS.set(a.to_s, params[:long])
"<pre>http://199.19.118.186/get/#{a}</pre>"
#haml :shorten
end
get '/get/:url' do
redirect REDIS.get(params[:url])
end
Where index.haml is a form that POSTs long to /shorten. I've no problem with that.
Right now, however, when I try to use Redis (with the server running, yes), I get this error:
What am I doing wrong?
EDIT: Copy/paste from Emacs... facepalm
EDIT: When trying to access redis alone from ruby (code below), I get this:
/var/lib/gems/1.8/gems/redis-2.2.2/lib/redis/client.rb:47:in `call': ERR unknown command (RuntimeError)
from /var/lib/gems/1.8/gems/redis-2.2.2/lib/redis.rb:841:in `set'
from /usr/lib/ruby/1.8/monitor.rb:242:in `synchronize'
from /var/lib/gems/1.8/gems/redis-2.2.2/lib/redis.rb:840:in `set'
from test_redis.rb:9
With this code:
require 'rubygems'
require 'redis'
require 'uri'
REDISTOGO_URL = "redis://localhost:6379/"
uri = URI.parse(REDISTOGO_URL)
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
REDIS.set("test", "blah")
puts REDIS.get("test")
Ruby being case sensitive, I would try to replace REDIS.SET by REDIS.set and REDIS.GET by REDIS.get. You can find the documentation of the Redis client here:
https://github.com/ezmobius/redis-rb
I have tested your example with ruby 1.8.7. (default on my Linux box).
After installing sinatra, haml, redis and hiredis gems, I have modified the code as follows:
require 'rubygems'
require 'sinatra'
require 'redis'
require 'uri'
configure do
REDISTOGO_URL = "redis://localhost:6379/"
uri = URI.parse(REDISTOGO_URL)
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
end
get '/' do
"Hello"
haml :index
end
post '/shorten' do
a = rand(9999)
REDIS.set(a.to_s, "http://"+params[:long])
"<pre>http://localhost:4567/get/#{a}</pre>"
end
get '/get/:url' do
redirect REDIS.get(params[:url])
end
I have added the following template in views/index.haml.
!!!
%html
%head
%title My Sinatra Website
%body
%h1 Welcome
%p
Welcome to my website made with Sinatra and HAML
%form{ :action => "/shorten", :method=>"POST" }
%fieldset
%input{ :type =>"text", :name=>"long" }
%input{ :type =>"submit" }
Once Redis is started on port 6379 and sinatra on port 4567, it works like a charm.
I suggest you check your ruby installation and try to access Redis from ruby with a simple non sinatra script.
UPDATE:
The error message is peculiar because normally, when an unknown command is sent to the server, the faulty command is provided:
ERR unknown command 'dummy'
while you just have:
ERR unknown command
Actually, this specific fix was introduced in Redis server more than 2 years ago (in December 2009) - an eternity for Redis.
https://github.com/antirez/redis/commit/2c14807b2dd5c15f1471bec32a7c6dbb077720ee
In other words, you are trying to use a very old (i.e. pre 1-3) version of Redis server with the last version of the Redis client ruby gem, which probably does not support anymore the initial protocol. You may want to compile and install a recent version of Redis server (it is easy), it should work better.

How to perform simple web service client with Ruby and Savon

I'm trying to develop a simple example of a web service client in Ruby using Savon.
This is what I got so far:
class WebServiceController < ApplicationController
def index
puts "web_service: IN"
client = Savon::Client.new do
wsdl.document = "http://www.webservicex.com/CurrencyConvertor.asmx?wsdl"
end
response = client.request :conversion_rate do
soap.body = {
:from_currency => 'USD',
:to_currency => 'EUR'
}
end
puts response.to_hash;
render :text => response.to_hash.to_s
end
end
However, when I run that code I get:
uninitialized constant Savon::Client
I guess I have to add some reference to Savon? (I already installed the corresponding gem).
In addition: am I doing the right thing in that web service? Should it work?
Thank you for your time!
If this is a Rails 3 application, add this onto your Gemfile:
gem 'savon'
Then, run bundle install and restart your development server.
I suppose you've added
require 'savon'
somewhere in your file?

Resources