is Ruby em-websocket blocking? - ruby

I'm writing a ruby program that has 2 threads. One that listens on an incoming UDP connection and another that broadcasts on a websocket from which browsers on the client side read.I'm using the em-websocket gem. However, My UDP listener thread never gets called and it looks like the code stays within the websocket initialization code. I'm guessing because em-websocket is blocking, but I haven't been able to find any info online that suggests that. Is it an error on my side? I'm kinda new to ruby so I'm not able to figure out what I'm doing wrong.
require 'json'
require 'em-websocket'
require 'socket'
socket=nil
text="default"
$x=0
EventMachine.run do
EventMachine::WebSocket.start(:host => "0.0.0.0", :port => 8080) do |ws|
ws.onopen {
ws.send "Hello Client!"
socket=ws
$x=1
}
ws.onmessage { |msg| socket.send "Pong: #{msg}" }
ws.onclose { puts "WebSocket closed" }
end
end
def listen()
puts "listening..."
s = UDPSocket.new
s.bind(nil, 3000)
while 1<2 do
text, sender = s.recvfrom(1024)
puts text
if $x==1 then
socket.send text
end
end
end
t2=Thread.new{listen()}
t2.join

em-websocket is non-blocking, however UDPSocket#recv_from is. Might be better to just use EventMachine's open_datagram_socket instead.
Another thing to note: you should not expose socket as a "global" variable. Every time somebody connects the reference to the previously connected client will be lost. Maybe make some sort of repository for socket connections, or use an observer pattern to broadcast messages when something comes in. What I would do is have a dummy object act as an observer, and whenever a socket is connected/disconnect you register/unregister from the observer:
require 'observer'
class Dummy
include Observable
def receive_data data
changed true
notify_observers data
end
end
# ... later on ...
$broadcaster = Dummy.new
class UDPHandler < EventMachine::Connection
def receive_data data
$broadcaster.receive_data data
end
end
EventMachine.run do
EM.open_datagram_socket "0.0.0.0", 3000, UDPHandler
EM::WebSocket.start :host => "0.0.0.0", :port => 8080 do |ws|
ws.onopen do
$broadcaster.add_observer ws
end
ws.onclose do
$broadcaster.delete_observer ws
end
# ...
end
end
The whole point of EventMachine is to abstract away from the basic socket and threading structure, and handle all the asynchronous bits internally. It's best not to mix the classical libraries like UDPSocket or Thread with EventMachine stuff.

Related

Ruby: Connect to remote WebSocket

I'm trying to connect to remote websocket using Celluloid and Websocket client based on celluloid (gem 'celluloid-websocket-client'). The main advantage of this client for me is that I can use callbacks in the form of class methods instead of blocks.
require 'celluloid/websocket/client'
class WSConnection
include Celluloid
def initialize(url)
#ws_client = Celluloid::WebSocket::Client.new url, Celluloid::Actor.current
end
# When WebSocket is opened, register callbacks
def on_open
puts "Websocket connection opened"
end
# When raw WebSocket message is received
def on_message(msg)
puts "Received message: #{msg}"
end
# When WebSocket is closed
def on_close(code, reason)
puts "WebSocket connection closed: #{code.inspect}, #{reason.inspect}"
end
end
m = WSConnection.new('wss://foo.bar')
while true; sleep; end
The expected output is
"Websocket connection opened"
However, I don't get any output at all. What could be the problem?
I am using
gem 'celluloid-websocket-client', '0.0.2'
rails 4.2.1
ruby 2.1.3
As you noticed in the comments, the gem had no SSL support. That is the trouble. To expound on the answer, here is a resolution, and also some next steps of what to expect for the future:
[ now ] Override methods in Celluloid::WebSocket::Client::Connection
This is an example injection to provide SSL support to the current gem. Mine is actually highly modified, but this shows you the basic solution:
def initialize(url, handler=nil)
#url = url
#handler = handler || Celluloid::Actor.current
#de If you want an auto-start:
start
end
def start
uri = URI.parse(#url)
port = uri.port || (uri.scheme == "ws" ? 80 : 443)
#socket.close rescue nil
#socket = Celluloid::IO::TCPSocket.new(uri.host, port)
#socket = Celluloid::IO::SSLSocket.new(#socket) if port == 443
#socket.connect
#client = ::WebSocket::Driver.client(self)
async.run
end
The above sends ripple effects through the other methods however, for example, #handler is used to hold the calling actor, which also has the emitter methods on it. Like I said, my version is very different from the stock gem because I got fed up with it and reworked mine. But then:
[ soon ] Use Reel::IO::Client and avoid near certain brain damage.
There are exciting things going on with WebSocket support, and a gem is coming to refactor both server and client implementations of websockets. No more monkeypatches required!
All websocket functionality is being extracted from Reel and being combined with a websocket-driver abstraction, as Reel::IO... in both ::Server and ::Client varieties.
Interestingly, this is prompted by Rails which is moving away from EventMachine to Celluloid::IO for websockets:
https://github.com/rails/actioncable/issues/16
https://github.com/celluloid/reel/issues/201
https://github.com/celluloid/reel-io/issues/2
A prealpha is online for preview: https://github.com/celluloid/reel-io

celluloid-io or eventmachine with mosquitto loops

I'm building a small ruby program to run a connection to a MQTT server and subscribe to a channel. I'm using the mosquitto gem which is just a bridge for libmosquitto C library.
I created a very simple implementation of a program that can run with ruby my_prog.rb:
# Dependencies
require File.expand_path(File.join('..', 'environment'), __FILE__)
# MQTT Application
module Pulsr
class MQTT
attr_reader :host, :port, :alive
def initialize(host = 'iot.eclipse.org', port = 1883, alive = 60)
#client ||= Mosquitto::Client.new SecureRandom.hex(8)
Signal.trap(Signal.list.has_key?('INT') ? 'SIGINT' : 'SIGTERM') do
#client.log 'Shutdown'
shutdown
end
#host = host
#port = port
#alive = alive
start
end
private
def on_connect
Proc.new { |return_code|
#client.log "Connected RC #{return_code}"
#client.subscribe(nil, '/pulsr', Mosquitto::EXACTLY_ONCE)
}
end
def on_disconnect
Proc.new { |return_code| #client.log "Disconnected RC #{return_code}" }
end
def on_subscribe
Proc.new { |message_id, granted_qos| #client.log "Subscribed MID #{message_id} QoS #{granted_qos}" }
end
def on_unsubscribe
Proc.new { |message_id| #client.log "Unsubscribed MID #{message_id}" }
end
def on_message
Proc.new { |message| Pulsr::Workers::TrackingEvent.perform_async message.to_s }
end
def configure
#client.logger = Logger.new(STDOUT)
#client.on_connect &on_connect
#client.on_disconnect &on_disconnect
#client.on_subscribe &on_subscribe
#client.on_unsubscribe &on_unsubscribe
#client.on_message &on_message
end
def connect
#client.connect_async(#host, #port, #alive)
end
def start
#client.loop_start
configure
connect
sleep
end
def shutdown
#client.loop_stop(true)
Process.exit
end
end
end
# MQTT Start
Pulsr::MQTT.new :host => 'iot.eclipse.org', :port => 1883, :alive => 60
I was wondering, if I wanted to use Celluloid or EventMachine to run the loops that the mosquitto gem provides, how would I do it?
The mosquitto gem provides a good documentation and presents a few loop methods that can be used, but I have no clue where to start or how to do it, neither I have ever used EM or Celluloid.
Could anyone help get started with this, I think it could bring some value to the community and it can end up as a open source project, a small addition to the mosquitto gem?
I think it is not that hard.
Mosquitto has a good library.
Yo need to connect these functions:
mosquitto_loop_misc() <-> EventMachine::PeriodicTimer.new
mosquitto_read() <-> EventMachine.watch
mosquitto_write() <-> EventMachine.watch
The em-mqtt gem provides an MQTT protocol implementation for eventmachine. This uses the pure ruby mqtt implementation to process the messages rather than libmosquitto.
If you really have to use the libmosquitto implementation for the parsing via the mosquitto gem then the above delineation would hold. The eventmachine component will be pretty much as is. All the calls to the protocol specific MQTT module would be replaced with the equivalent in libmosquitto. The main problem looks to be that the libmosquitto public API and subsequent Ruby API hides all of this away, down in libmosquitto's own network implementation, which is being replaced with eventmachine, so you would have a lot of hacking to expose the required methods to Ruby before you can get started.

EventMachine and em-websocket - reading from a queue and pushing to a channel

I'm using eventmachine to read from a HornetQ topic, push to a Channel which is subscribed to by EM websocket connections. I need to prevent the #topic.receive loop from blocking, so have created a proc and am calling EventMachine.defer with no callback. This will run indefinitely. This works fine. I could also have just used Thread.new.
My question is, is this the correct way to read from a stream/queue and pass the data to the channel and is there a better/any other way to do this?
require 'em-websocket'
require 'torquebox-messaging'
class WebsocketServer
def initialize
#channel = EM::Channel.new
#topic = TorqueBox::Messaging::Topic.new('/topics/mytopic')
end
def start
EventMachine.run do
topic_to_channel = proc do
while true
msg = #topic.receive
#channel.push msg
end
end
EventMachine.defer(topic_to_channel)
EventMachine::WebSocket.start(:host => "127.0.0.1", :port => 8081, :debug => false) do |connection|
connection.onopen do
sid = #channel.subscribe { |msg| connection.send msg }
connection.onclose do
#channel.unsubscribe(sid)
end
end
end
end
end
end
WebsocketServer.new.start
This is ok, but EM.defer will spawn 20 threads, so I would avoid it for your use case. In general I would avoid EM entirely, especially the Java reactor as we never finished it.
The Torquebox has a native stomp over websockets solution that would be a much better way to go in this context, and solves a bunch of other encapsulation challenges for you.
If you really want to stick with EM for this, then I'd use Thread.new instead of defer, so as to avoid having 19 idle threads taking up extra ram for no reason.

How to disconnect redis client in websocket eventmachine

I'm trying to build a websocket server where each client establish its own redis connections used for publish and subscribe.
When the redis server is running I can see the two new connections being established when a client connects to the websocket server and I can also publish data to the client, but when the client drops the connection to the websocket server I also want to disconnect from Redis . How can I do this?
Maybe I'm doing it wrong, but this is my code.
#require 'redis'
require 'em-websocket'
require 'em-hiredis'
require 'json'
CLIENTS = Hash.new
class PubSub
def initialize(client)
#socket = client.ws
# These clients can only be used for pub sub commands
#publisher = EM::Hiredis.connect #Later I will like to disconnect this
#subscriber = EM::Hiredis.connect #Later I will like to disconnect this
client.connections << #publisher << #subscriber
end
def subscribe(channel)
#channel = channel
#subscriber.subscribe(channel)
#subscriber.on(:message) { |chan, message|
#socket.send message
}
end
def publish(channel,msg)
#publisher.publish(channel, msg).errback { |e|
puts [:publisherror, e]
}
end
def unsubscribe()
#subscriber.unsubscribe(#channel)
end
end
class Client
attr_accessor :connections, :ws
def initialize(ws)
#connections = []
#ws = ws
end
end
EventMachine.run do
# Creates a websocket listener
EventMachine::WebSocket.start(:host => '0.0.0.0', :port => 8081) do |ws|
ws.onopen do
# I instantiated above
puts 'CLient connected. Creating socket'
#client = Client.new(ws)
CLIENTS[ws] = #client
end
ws.onclose do
# Upon the close of the connection I remove it from my list of running sockets
puts 'Client disconnected. Closing socket'
#client.connections.each do |con|
#do something to disconnect from redis
end
CLIENTS.delete ws
end
ws.onmessage { |msg|
puts "Received message: #{msg}"
result = JSON.parse(msg)
if result.has_key? 'channel'
ps = PubSub.new(#client)
ps.subscribe(result['channel'])
elsif result.has_key? 'publish'
ps = PubSub.new(ws)
ps.publish(result['publish']['channel'],result['publish']['msg']);
end
}
end
end
This version of em-hiredis supports close connection: https://github.com/whatupdave/em-hiredis
Here is how I would (and did many times) this:
instead of always opening and closing connections for each client you can keep 1 connection open per Thread/Fiber dependeing on what you are basing your concurrency on, that way if you are using a poll of Thread/Fibers once each one of them have its connections they will keep it and reuse them.
I did not worked much with websocket until now (I was waiting for a standard implementation) but I am sure you can apply that thinking to it too.
You can also do what rails/activerecord: keeo a pool of redis connection, each time you need to use a connection you request one, use it and realease it, it could look like this:
def handle_request(request)
#redis_pool.get_connection do |c|
# [...]
end
end
before yielding the block a connection is taken from the available ones and after it the connection is marked as free.
This was added to em-hiredis: https://github.com/mloughran/em-hiredis/pull/6

Ruby Tcp Server class with non-blocking or multithread functionality

Can't find any gem or class which can help to made a non-blocking/multithread server.
Where to find any?
The Ruby docs on sockets have some pretty good examples. Using information from that page, I cobbled together a simple client and server using non-blocking sockets. These are mostly copies of code from that page with a few changes.
The simple server code (with the accept_nonblock call that you may be interested in):
require 'socket'
include Socket::Constants
socket = Socket.new(AF_INET, SOCK_STREAM, 0)
sockaddr = Socket.sockaddr_in(6212, 'localhost')
socket.bind(sockaddr)
socket.listen(5)
begin
client_socket, client_sockaddr = socket.accept_nonblock
rescue Errno::EAGAIN, Errno::ECONNABORTED, Errno::EINTR, Errno::EWOULDBLOCK
IO.select([socket])
retry
end
puts client_socket.readline.chomp
client_socket.puts "hi from the server"
client_socket.close
socket.close
And a client that talks to it:
require 'socket'
include Socket::Constants
socket = Socket.new(AF_INET, SOCK_STREAM, 0)
sockaddr = Socket.sockaddr_in(6212, 'localhost')
begin
socket.connect_nonblock(sockaddr)
rescue Errno::EINPROGRESS
IO.select(nil, [socket])
begin
socket.connect_nonblock(sockaddr)
rescue Errno::EINVAL
retry
rescue Errno::EISCONN
end
end
socket.write("hi from the client\n")
results = socket.read
puts results
socket.close
Take a look at EventMachine. Here’s a quick example:
require "rubygems"
require "eventmachine"
module EchoServer
def receive_data (data)
send_data "You said: #{data}"
end
end
EventMachine::run do
EventMachine::start_server "0.0.0.0", 5000, EchoServer
end
Use Celluloid::IO
This is the primary purpose of Celluloid::IO and it is extremely good at what it does:
https://github.com/celluloid/celluloid-io
A few example servers...
https://github.com/celluloid/celluloid-dns
https://github.com/celluloid/reel

Resources