I have the following sinatra app:
require 'sinatra'
require 'redis'
require 'json'
class FeedStream < Sinatra::Application
helpers do
include SessionsHelper
def redis
#redis ||= Redis.connect
end
end
get '/feed', provides: 'text/event-stream' do
stream do |out|
redis.subscribe "feed" do |on|
on.message do |channel, message|
event_data = JSON.parse message
logger.info "received event = #{event_data}"
out << "event: #{event_data['event']}\n"
out << "data: #{{:data => event_data['data'],
:by => current_user}}.to_json\n\n"
end
end
end
end
end
basically, it receives events published by other users to a feed using redis pubsub, and then it sends those events with the sinatra streaming api.
The problem is that, when the browser reconnects to the feed, the redis client keeps connected, and it keeps receiving events, so the redis server gets full of useless connections.
how can i close all this connections once the broser closes the connection to the web server?
I know it's been a while.
Were you looking for quit?
After much research and experimentation, here's the code I'm using with sinatra + sinatra sse gem (which should be easily adapted to Rails 4):
class EventServer < Sinatra::Base
include Sinatra::SSE
set :connections, []
.
.
.
get '/channel/:channel' do
.
.
.
sse_stream do |out|
settings.connections << out
out.callback {
puts 'Client disconnected from sse';
settings.connections.delete(out);
}
redis.subscribe(channel) do |on|
on.subscribe do |channel, subscriptions|
puts "Subscribed to redis ##{channel}\n"
end
on.message do |channel, message|
puts "Message from redis ##{channel}: #{message}\n"
message = JSON.parse(message)
.
.
.
if settings.connections.include?(out)
out.push(message)
else
puts 'closing orphaned redis connection'
redis.unsubscribe
end
end
end
end
end
The redis connection blocks on.message and only accepts (p)subscribe/(p)unsubscribe commands. Once you unsubscribe, the redis connection is no longer blocked and can be released by the web server object which was instantiated by the initial sse request. It automatically clears when you receive a message on redis and sse connection to the browser no longer exists in the collection array.
Related
this is my ruby code:
require 'redis'
redis = Redis.new
Thread.new do
redis.subscribe('foo_bar') do |on|
on.message do |key, id|
puts id
end
end
end
redis.publish('foo_bar', 1337)
Can anyone tell me, why the "publish" instruction in last line is blocking my process?
Thanks
It appears that redis.subscribe blocks redis access in the entire process. So when you try to publish it locks.
Solution: trying using polling instead of subscribe
Solution 2: use several REDIS connections, i.e. create REDIS connection pool
To start with, the publish isnt blocking the process.
require 'redis'
redis = Redis.new
Thread.new do
redis.subscribe('foo_bar') do |on|
on.message do |key, id|
puts id
end
end
end
redis.publish('foo_bar', 1337)
Should output:
1337
And also,
require 'redis'
redis = Redis.new
Thread.new do
redis.subscribe('foo_bar') do |on|
on.message do |key, id|
puts id
end
end
end
redis.publish('foo_bar', 1337)
redis.publish('foo_bar', 1338)
Would output:
1337
1338
In the pub/sub system, the publisher pushes the message, and the subscriber is an always blocking synchronous process. Publisher is always asynchronous.
I'm building a Ruby app that runs both an EM::WebSocket server as well as a Sinatra server. Individually, I believe both of these are equipped to handle a SIGINT. However, when running both in the same app, the app continues when I press Ctrl+C. My assumption is that one of them is capturing the SIGINT, preventing the other from capturing it as well. I'm not sure how to go about fixing it, though.
Here's the code in a nutshell:
require 'thin'
require 'sinatra/base'
require 'em-websocket'
EventMachine.run do
class Web::Server < Sinatra::Base
get('/') { erb :index }
run!(port: 3000)
end
EM::WebSocket.start(port: 3001) do |ws|
# connect/disconnect handlers
end
end
I had the same issue. The key for me seemed to be to start Thin in the reactor loop with signals: false:
Thin::Server.start(
App, '0.0.0.0', 3000,
signals: false
)
This is complete code for a simple chat server:
require 'thin'
require 'sinatra/base'
require 'em-websocket'
class App < Sinatra::Base
# threaded - False: Will take requests on the reactor thread
# True: Will queue request for background thread
configure do
set :threaded, false
end
get '/' do
erb :index
end
end
EventMachine.run do
# hit Control + C to stop
Signal.trap("INT") {
puts "Shutting down"
EventMachine.stop
}
Signal.trap("TERM") {
puts "Shutting down"
EventMachine.stop
}
#clients = []
EM::WebSocket.start(:host => '0.0.0.0', :port => '3001') do |ws|
ws.onopen do |handshake|
#clients << ws
ws.send "Connected to #{handshake.path}."
end
ws.onclose do
ws.send "Closed."
#clients.delete ws
end
ws.onmessage do |msg|
puts "Received message: #{msg}"
#clients.each do |socket|
socket.send msg
end
end
end
Thin::Server.start(
App, '0.0.0.0', 3000,
signals: false
)
end
I downgrade thin to version 1.5.1 and it just works. Wired.
I have a very simple server for use in integration tests, built using eventmachine:
EM.run do
EM::start_server(server, port, HttpRecipient)
end
I can receive HTTP requests and parse them like so:
class HttpRecipient < EM::Connection
def initialize
##stored = ''
end
# Data is received in chunks, so here we wait until we've got a full web request before
# calling spool.
def receive_data(data)
##stored << data
begin
spool(##stored)
EM.stop
rescue WEBrick::HTTPStatus::BadRequest
#Not received a complete request yet
end
end
def spool(data)
#Parse the request
req = WEBrick::HTTPRequest.new(WEBrick::Config::HTTP)
req.parse(StringIO.new(##stored))
#Send a response, e.g. HTTP OK
end
end
The question is, how do I send a response? Eventmachine provides a send_data for sending responses, but that doesn't understand http. Similarly there is the em-http-request
module for sending requests, but it's not obvious that this is capable of generating responses.
I can generate HTTP messages manually and then send them using send_data, but I wonder if there is a clean way to use an existing http library, or the functionality built in to eventmachine?
If you want something easy then use Thin or Rainbows. It uses Eventmachine inside and provides Rack interface support.
# config.ru
http_server = proc do |env|
response = "Hello World!"
[200, {"Connection" => "close", "Content-Length" => response.bytesize.to_s}, [response]]
end
run http_server
And then
>> thin start -R config.ru
UPD.
If you need server to run in parallel you could run it in a Thread
require 'thin'
class ThreadedServer
def initialize(*args)
#server = Thin::Server.new(*args)
end
def start
#thread = Thread.start do
#server.start
end
end
def stop
#server.stop
if #thread
#thread.join
#thread = nil
end
end
end
http_server = proc do |env|
response = "Hello World!"
[200, {"Connection" => "close", "Content-Length" => response.bytesize.to_s}, [response]]
end
server = ThreadedServer.new http_server
server.start
# Some job with server
server.stop
# Server is down
I'm writing a ruby program that has 2 threads. One that listens on an incoming UDP connection and another that broadcasts on a websocket from which browsers on the client side read.I'm using the em-websocket gem. However, My UDP listener thread never gets called and it looks like the code stays within the websocket initialization code. I'm guessing because em-websocket is blocking, but I haven't been able to find any info online that suggests that. Is it an error on my side? I'm kinda new to ruby so I'm not able to figure out what I'm doing wrong.
require 'json'
require 'em-websocket'
require 'socket'
socket=nil
text="default"
$x=0
EventMachine.run do
EventMachine::WebSocket.start(:host => "0.0.0.0", :port => 8080) do |ws|
ws.onopen {
ws.send "Hello Client!"
socket=ws
$x=1
}
ws.onmessage { |msg| socket.send "Pong: #{msg}" }
ws.onclose { puts "WebSocket closed" }
end
end
def listen()
puts "listening..."
s = UDPSocket.new
s.bind(nil, 3000)
while 1<2 do
text, sender = s.recvfrom(1024)
puts text
if $x==1 then
socket.send text
end
end
end
t2=Thread.new{listen()}
t2.join
em-websocket is non-blocking, however UDPSocket#recv_from is. Might be better to just use EventMachine's open_datagram_socket instead.
Another thing to note: you should not expose socket as a "global" variable. Every time somebody connects the reference to the previously connected client will be lost. Maybe make some sort of repository for socket connections, or use an observer pattern to broadcast messages when something comes in. What I would do is have a dummy object act as an observer, and whenever a socket is connected/disconnect you register/unregister from the observer:
require 'observer'
class Dummy
include Observable
def receive_data data
changed true
notify_observers data
end
end
# ... later on ...
$broadcaster = Dummy.new
class UDPHandler < EventMachine::Connection
def receive_data data
$broadcaster.receive_data data
end
end
EventMachine.run do
EM.open_datagram_socket "0.0.0.0", 3000, UDPHandler
EM::WebSocket.start :host => "0.0.0.0", :port => 8080 do |ws|
ws.onopen do
$broadcaster.add_observer ws
end
ws.onclose do
$broadcaster.delete_observer ws
end
# ...
end
end
The whole point of EventMachine is to abstract away from the basic socket and threading structure, and handle all the asynchronous bits internally. It's best not to mix the classical libraries like UDPSocket or Thread with EventMachine stuff.
I'm trying to build a websocket server where each client establish its own redis connections used for publish and subscribe.
When the redis server is running I can see the two new connections being established when a client connects to the websocket server and I can also publish data to the client, but when the client drops the connection to the websocket server I also want to disconnect from Redis . How can I do this?
Maybe I'm doing it wrong, but this is my code.
#require 'redis'
require 'em-websocket'
require 'em-hiredis'
require 'json'
CLIENTS = Hash.new
class PubSub
def initialize(client)
#socket = client.ws
# These clients can only be used for pub sub commands
#publisher = EM::Hiredis.connect #Later I will like to disconnect this
#subscriber = EM::Hiredis.connect #Later I will like to disconnect this
client.connections << #publisher << #subscriber
end
def subscribe(channel)
#channel = channel
#subscriber.subscribe(channel)
#subscriber.on(:message) { |chan, message|
#socket.send message
}
end
def publish(channel,msg)
#publisher.publish(channel, msg).errback { |e|
puts [:publisherror, e]
}
end
def unsubscribe()
#subscriber.unsubscribe(#channel)
end
end
class Client
attr_accessor :connections, :ws
def initialize(ws)
#connections = []
#ws = ws
end
end
EventMachine.run do
# Creates a websocket listener
EventMachine::WebSocket.start(:host => '0.0.0.0', :port => 8081) do |ws|
ws.onopen do
# I instantiated above
puts 'CLient connected. Creating socket'
#client = Client.new(ws)
CLIENTS[ws] = #client
end
ws.onclose do
# Upon the close of the connection I remove it from my list of running sockets
puts 'Client disconnected. Closing socket'
#client.connections.each do |con|
#do something to disconnect from redis
end
CLIENTS.delete ws
end
ws.onmessage { |msg|
puts "Received message: #{msg}"
result = JSON.parse(msg)
if result.has_key? 'channel'
ps = PubSub.new(#client)
ps.subscribe(result['channel'])
elsif result.has_key? 'publish'
ps = PubSub.new(ws)
ps.publish(result['publish']['channel'],result['publish']['msg']);
end
}
end
end
This version of em-hiredis supports close connection: https://github.com/whatupdave/em-hiredis
Here is how I would (and did many times) this:
instead of always opening and closing connections for each client you can keep 1 connection open per Thread/Fiber dependeing on what you are basing your concurrency on, that way if you are using a poll of Thread/Fibers once each one of them have its connections they will keep it and reuse them.
I did not worked much with websocket until now (I was waiting for a standard implementation) but I am sure you can apply that thinking to it too.
You can also do what rails/activerecord: keeo a pool of redis connection, each time you need to use a connection you request one, use it and realease it, it could look like this:
def handle_request(request)
#redis_pool.get_connection do |c|
# [...]
end
end
before yielding the block a connection is taken from the available ones and after it the connection is marked as free.
This was added to em-hiredis: https://github.com/mloughran/em-hiredis/pull/6