I'm trying to build a notifications system with Redis and Sinatra streams. However I can't seem to catch when connection closes down, so the blocking Redis subscription block seems to never close down. What is the best way to achieve this?
get '/user/:id/next_notification' do
stream :keep_open do |out|
$redis.subscribe("notifications:#{params[:id]}") { |on|
on.message { |channel, msg|
$redis.unsubscribe
out << msg
}
}
out.callback {
puts "unsub"
# $redis.unsubscribe
}
out.errback {
puts "unsub"
# $redis.unsubscribe
}
end
end
Redis Subscription is a blocking call. So you need to execute it in a separate thread. Dunno how to do it in Ruby. But i'm sure there must be threading library in ruby.
Wrap the Blocking call in a try..catch and you will know when the connection has closed from the server side.
Related
Elm, phoenix, and elixir are quite new to me so I thought I would make channels test app simple example app to test the use of phoenix channels. The app has other stuff in it too because its made from old "parts" but bear with me on this.
The idea is that you have several genservers making http calls to a phoenix endpoint. Basically they are just updating a list held in an agent process.
That list is displayed in an Elm app through a phoenix channel. The goal was just to see what happens if the agent state is updated frequently with several processes.
So this is what I have so far. I have the phoenix site with the Elm app setup and a separate Elixir app with genservers making the updates. Everything works fine about 20 seconds but then the channel connection is cut and not reestablished unless I hit refresh on browser. I can see from logging that the backend is still working fine and there is no error on the browser console either. So whats the deal here? I thought that the channel connection should automatically reconnect if lost and why is it disconnecting anyway?
Im guessing that the problem is with the elm-phoenix-socket. Here is it is setup in the elm app:
socketServer : String
socketServer =
"ws://localhost:4000/socket/websocket"
initPhxSocket : Phoenix.Socket.Socket Msg
initPhxSocket =
Phoenix.Socket.init socketServer
|> Phoenix.Socket.withDebug
|> Phoenix.Socket.on "new:heartbeats" "heartbeats:lobby" ReceiveHeartbeats
Here is how the broadcast is done on the backend:
defmodule AbottiWeb.ApiController do
use AbottiWeb.Web, :controller
def index(conn, _params) do
beats = AbottiWeb.HeartbeatAgent.get()
json conn, beats
end
def heartbeat(conn, %{"agent" => agent} ) do
AbottiWeb.HeartbeatAgent.update(agent)
beats = AbottiWeb.HeartbeatAgent.get()
AbottiWeb.Endpoint.broadcast("heartbeats:lobby", "new:heartbeats", beats)
json conn, :ok
end
end
so in essence the genservers are constantly making calls to that heartbeat endpoint. I doubt the problem is here though. Another possibility where the problem lies is the channel setup which looks like this:
user_socket.ex:
defmodule AbottiWeb.UserSocket do
use Phoenix.Socket
channel "heartbeats:*", AbottiWeb.HeartbeatChannel
transport :websocket, Phoenix.Transports.WebSocket
def connect(_params, socket) do
{:ok, socket}
end
def id(_socket), do: nil
end
and heartbeat_channel.ex:
defmodule AbottiWeb.HeartbeatChannel do
use AbottiWeb.Web, :channel
require Logger
def join("heartbeats:lobby", payload, socket) do
Logger.debug "Hearbeats:lobby joined: #{inspect payload}"
if authorized?(payload) do
{:ok, socket}
else
{:error, %{reason: "unauthorized"}}
end
end
# Channels can be used in a request/response fashion
# by sending replies to requests from the client
def handle_in("ping", payload, socket) do
{:reply, {:ok, payload}, socket}
end
# It is also common to receive messages from the client and
# broadcast to everyone in the current topic (heartbeats:lobby).
def handle_in("shout", payload, socket) do
broadcast socket, "shout", payload
{:noreply, socket}
end
# This is invoked every time a notification is being broadcast
# to the client. The default implementation is just to push it
# downstream but one could filter or change the event.
def handle_out(event, payload, socket) do
Logger.debug "Broadcasting #{inspect event} #{inspect payload}"
push socket, event, payload
{:noreply, socket}
end
# Add authorization logic here as required.
defp authorized?(_payload) do
true
end
end
So any ideas what the problem is? Im guessing it is something really simple.
Ok, I know now that the socket transport times out. But why does it do that?
Well, I solved it with this:
transport :websocket, Phoenix.Transports.WebSocket,
timeout: :infinity
Don't know how harmful that is but this being a test app it doesn't really matter.
I want to use Ruby EventMachine and em-http-request to run parallel HTTP synchronous requests triggered from different threads.
The idea is to run a single reactor in its own thread and push HTTP requests to complete on its event queue thanks to EM.next_tick.
The pattern for every call could be
def complete_request(url, options)
Thread.new { EM.run } unless EM.reactor_running?
EM.run do
EM.next_tick do
con = EventMachine::HttpRequest.new(url, options[:connection_headers])
http = com.setup_request(verb, head: options[:headers], body: options[:body])
http.errback { }
http.callback { }
end
end
# wait for request completion (but how?)
...
end
reqs = []
responses = []
reqs << Thread.new { responses << complete_request('http://www.stackoverflow.com', verb: get) }
reqs << Thread.new { responses << complete_request('http://www.flickr.com', verb: get) }
reqs.each { |req| req.join }
To make the requests synchronous, I tried to use Fibers but unsuccessfully. Either the request fails to connect or it completes but never exits the event loop.
I don't want to call EM.stop in the callbacks because it would screw up other requests being executed in parallel I guess, and would also stop the reactor while I want it to run until I decide no more requests should be treated.
Does anyone already try to use EventMachine and em-http-request this way ? Can EventMachine support this use case ?
I'm using eventmachine to read from a HornetQ topic, push to a Channel which is subscribed to by EM websocket connections. I need to prevent the #topic.receive loop from blocking, so have created a proc and am calling EventMachine.defer with no callback. This will run indefinitely. This works fine. I could also have just used Thread.new.
My question is, is this the correct way to read from a stream/queue and pass the data to the channel and is there a better/any other way to do this?
require 'em-websocket'
require 'torquebox-messaging'
class WebsocketServer
def initialize
#channel = EM::Channel.new
#topic = TorqueBox::Messaging::Topic.new('/topics/mytopic')
end
def start
EventMachine.run do
topic_to_channel = proc do
while true
msg = #topic.receive
#channel.push msg
end
end
EventMachine.defer(topic_to_channel)
EventMachine::WebSocket.start(:host => "127.0.0.1", :port => 8081, :debug => false) do |connection|
connection.onopen do
sid = #channel.subscribe { |msg| connection.send msg }
connection.onclose do
#channel.unsubscribe(sid)
end
end
end
end
end
end
WebsocketServer.new.start
This is ok, but EM.defer will spawn 20 threads, so I would avoid it for your use case. In general I would avoid EM entirely, especially the Java reactor as we never finished it.
The Torquebox has a native stomp over websockets solution that would be a much better way to go in this context, and solves a bunch of other encapsulation challenges for you.
If you really want to stick with EM for this, then I'd use Thread.new instead of defer, so as to avoid having 19 idle threads taking up extra ram for no reason.
I'm writing a ruby program that has 2 threads. One that listens on an incoming UDP connection and another that broadcasts on a websocket from which browsers on the client side read.I'm using the em-websocket gem. However, My UDP listener thread never gets called and it looks like the code stays within the websocket initialization code. I'm guessing because em-websocket is blocking, but I haven't been able to find any info online that suggests that. Is it an error on my side? I'm kinda new to ruby so I'm not able to figure out what I'm doing wrong.
require 'json'
require 'em-websocket'
require 'socket'
socket=nil
text="default"
$x=0
EventMachine.run do
EventMachine::WebSocket.start(:host => "0.0.0.0", :port => 8080) do |ws|
ws.onopen {
ws.send "Hello Client!"
socket=ws
$x=1
}
ws.onmessage { |msg| socket.send "Pong: #{msg}" }
ws.onclose { puts "WebSocket closed" }
end
end
def listen()
puts "listening..."
s = UDPSocket.new
s.bind(nil, 3000)
while 1<2 do
text, sender = s.recvfrom(1024)
puts text
if $x==1 then
socket.send text
end
end
end
t2=Thread.new{listen()}
t2.join
em-websocket is non-blocking, however UDPSocket#recv_from is. Might be better to just use EventMachine's open_datagram_socket instead.
Another thing to note: you should not expose socket as a "global" variable. Every time somebody connects the reference to the previously connected client will be lost. Maybe make some sort of repository for socket connections, or use an observer pattern to broadcast messages when something comes in. What I would do is have a dummy object act as an observer, and whenever a socket is connected/disconnect you register/unregister from the observer:
require 'observer'
class Dummy
include Observable
def receive_data data
changed true
notify_observers data
end
end
# ... later on ...
$broadcaster = Dummy.new
class UDPHandler < EventMachine::Connection
def receive_data data
$broadcaster.receive_data data
end
end
EventMachine.run do
EM.open_datagram_socket "0.0.0.0", 3000, UDPHandler
EM::WebSocket.start :host => "0.0.0.0", :port => 8080 do |ws|
ws.onopen do
$broadcaster.add_observer ws
end
ws.onclose do
$broadcaster.delete_observer ws
end
# ...
end
end
The whole point of EventMachine is to abstract away from the basic socket and threading structure, and handle all the asynchronous bits internally. It's best not to mix the classical libraries like UDPSocket or Thread with EventMachine stuff.
I'm working on creating a background script that uses EventMachine to connect to a server with WebSockets. The script will be run using DelayedJob or Resque. I've been able to get it to talk to the WebSockets server and send messages, but whenever an error is raised within the EventMachine loop it doesn't crash the script - which is what should happen (and what I need to have happen). I don't have to use EventMachine as I'm only sending WebSocket messages and not receiving them - but I'd love any help on this :) thank you!
#!/usr/bin/env ruby
require 'rubygems'
require 'eventmachine'
require 'em-http'
class Job
include EventMachine::Deferrable
def self.perform
job = Job.new
EventMachine.run {
http = EventMachine::HttpRequest.new("ws://localhost:8080/").get :timeout => 0
http.errback { puts "oops" }
http.callback {
puts "WebSocket connected!"
http.send("Hello watcher")
}
http.stream { |msg| }
job.callback { puts "done" }
Thread.new {
job.execute(http)
http.close
EventMachine.stop
}
}
end
def execute(h)
sleep 1
puts "Job Runner!"
h.send("welcome!")
sleep 2
asdsadsa # here I am trying to simulate an error
sleep 1
h.send("we are all done!")
sleep 1
set_deferred_status :succeeded
end
end
Job.perform
Since you're causing an exception inside a thread, you should set Thread.abort_on_exception to true otherwise these errors will not be raised properly.
You don't need to use Thread.new here at all, in fact, it's not thread safe to do so (eventmachine itself is not thread safe, except for EM::Queue, EM::Channel and EM.schedule).
If you wanted to do synchronous things in execute, and you must have that thread, then, you'll want to call h.send via EM.schedule, for example:
EM.schedule { h.send("welcome!") }
If you must have that thread in this way, then, you want to catch exceptions from the thread you spawn yourself. You should then stop and shutdown on your own, or just raise back up in the main (eventmachine) thread:
EM.run do
thread = Thread.new do
raise 'boom'
end
EM.add_periodic_timer(0.1) { thread.join(0) }
end
The above pattern can easily just enumerate an array of threads in the periodic timer instead, if appropriate.
Finally, please note that exception bubbling (correct exception reporting) was only supported in EventMachine > 1.0, which is still in beta. To get usable backtraces when exceptions occur, either gem install eventmachine --pre, or better, use master from the Github repo.