I'm building a small ruby program to run a connection to a MQTT server and subscribe to a channel. I'm using the mosquitto gem which is just a bridge for libmosquitto C library.
I created a very simple implementation of a program that can run with ruby my_prog.rb:
# Dependencies
require File.expand_path(File.join('..', 'environment'), __FILE__)
# MQTT Application
module Pulsr
class MQTT
attr_reader :host, :port, :alive
def initialize(host = 'iot.eclipse.org', port = 1883, alive = 60)
#client ||= Mosquitto::Client.new SecureRandom.hex(8)
Signal.trap(Signal.list.has_key?('INT') ? 'SIGINT' : 'SIGTERM') do
#client.log 'Shutdown'
shutdown
end
#host = host
#port = port
#alive = alive
start
end
private
def on_connect
Proc.new { |return_code|
#client.log "Connected RC #{return_code}"
#client.subscribe(nil, '/pulsr', Mosquitto::EXACTLY_ONCE)
}
end
def on_disconnect
Proc.new { |return_code| #client.log "Disconnected RC #{return_code}" }
end
def on_subscribe
Proc.new { |message_id, granted_qos| #client.log "Subscribed MID #{message_id} QoS #{granted_qos}" }
end
def on_unsubscribe
Proc.new { |message_id| #client.log "Unsubscribed MID #{message_id}" }
end
def on_message
Proc.new { |message| Pulsr::Workers::TrackingEvent.perform_async message.to_s }
end
def configure
#client.logger = Logger.new(STDOUT)
#client.on_connect &on_connect
#client.on_disconnect &on_disconnect
#client.on_subscribe &on_subscribe
#client.on_unsubscribe &on_unsubscribe
#client.on_message &on_message
end
def connect
#client.connect_async(#host, #port, #alive)
end
def start
#client.loop_start
configure
connect
sleep
end
def shutdown
#client.loop_stop(true)
Process.exit
end
end
end
# MQTT Start
Pulsr::MQTT.new :host => 'iot.eclipse.org', :port => 1883, :alive => 60
I was wondering, if I wanted to use Celluloid or EventMachine to run the loops that the mosquitto gem provides, how would I do it?
The mosquitto gem provides a good documentation and presents a few loop methods that can be used, but I have no clue where to start or how to do it, neither I have ever used EM or Celluloid.
Could anyone help get started with this, I think it could bring some value to the community and it can end up as a open source project, a small addition to the mosquitto gem?
I think it is not that hard.
Mosquitto has a good library.
Yo need to connect these functions:
mosquitto_loop_misc() <-> EventMachine::PeriodicTimer.new
mosquitto_read() <-> EventMachine.watch
mosquitto_write() <-> EventMachine.watch
The em-mqtt gem provides an MQTT protocol implementation for eventmachine. This uses the pure ruby mqtt implementation to process the messages rather than libmosquitto.
If you really have to use the libmosquitto implementation for the parsing via the mosquitto gem then the above delineation would hold. The eventmachine component will be pretty much as is. All the calls to the protocol specific MQTT module would be replaced with the equivalent in libmosquitto. The main problem looks to be that the libmosquitto public API and subsequent Ruby API hides all of this away, down in libmosquitto's own network implementation, which is being replaced with eventmachine, so you would have a lot of hacking to expose the required methods to Ruby before you can get started.
Related
im building a small game in ruby to practice programming, so far everything has went well but im trying to implement multiplayer support, i can connect to the server and i can send information but when I try to read form the server it just freezes and my screen goes completely black. and i cant find the cause, ive read the documentation for the gem im using for TCP and i dont know, maybe i missed something, but if any of you have some insight I would really appreciate it
heres the repo if this code isnt enough
https://github.com/jaypitti/ruby-2d-gosu-game
heres the client side code
class Client
include Celluloid::IO
def initialize(server, port)
begin
#socket = TCPSocket.new(server, port)
rescue
$error_message = "Cannot find game server."
end
end
def send_message(message)
#socket.write(message) if #socket
end
def read_message
#socket.readpartial(4096) if #socket
end
end
heres the gameserver
require 'celluloid/autostart'
require 'celluloid/io'
class Server
include Celluloid::IO
finalizer :shutdown
def initialize(host, port)
puts "Starting Server on #{host}:#{port}."
#server = TCPServer.new(host, port)
#objects = Hash.new
#players = Hash.new
async.run
end
def shutdown
#server.close if #server
end
def run
loop { async.handle_connection #server.accept }
end
def handle_connection(socket)
_, port, host = socket.peeraddr
user = "#{host}:#{port}"
puts "#{user} has joined the arena."
loop do
data = socket.readpartial(4096)
data_array = data.split("\n")
if data_array and !data_array.empty?
begin
data_array.each do |row|
message = row.split("|")
if message.size == 10
case message[0]
when 'obj'
#players[user] = message[1..9] unless #players[user]
#objects[message[1]] = message[1..9]
when 'del'
#objects.delete message[1]
end
end
response = String.new
#objects.each_value do |obj|
(response << obj.join("|") << "\n") if obj
end
socket.write response
end
rescue Exception => exception
puts exception.backtrace
end
end # end data
end # end loop
rescue EOFError => err
player = #players[user]
puts "#{player[3]} has left"
#objects.delete player[0]
#players.delete user
socket.close
end
end
server, port = ARGV[0] || "0.0.0.0", ARGV[1] || 1234
supervisor = Server.supervise(server, port.to_i)
trap("INT") do
supervisor.terminate
exit
end
sleep
it just freezes and my screen goes completely black. and i cant find the cause
A good trick you can look at is attaching to your process with either rbspy or rbtrace to see that is going on when it is stuck.
You can also try first reducing dependencies here a bit and doing this with a simple threadpool prior to going full async with celluloid or event machine.
First of all you should not be rescuing Exception all over the place. Wrapping long begin rescue blocks around nested iterators is begging for trouble.
It sounds like a threading issues, memory and/or CPU but that's just a guess. Try to monitor your resources or use some performance checking gems. But for the love of Satoshi Nakamoto, please write some test coverage and see your methods fail miserably, then fix them!
Some of these may help:
group :development do
gem 'bullet', require: false
gem 'flamegraph', require: false
gem 'memory_profiler', require: false
gem 'rack-mini-profiler', require: false
gem 'seed_dump'
gem 'stackprof', require: false
gem 'traceroute', require: false
end
I'm trying to connect to remote websocket using Celluloid and Websocket client based on celluloid (gem 'celluloid-websocket-client'). The main advantage of this client for me is that I can use callbacks in the form of class methods instead of blocks.
require 'celluloid/websocket/client'
class WSConnection
include Celluloid
def initialize(url)
#ws_client = Celluloid::WebSocket::Client.new url, Celluloid::Actor.current
end
# When WebSocket is opened, register callbacks
def on_open
puts "Websocket connection opened"
end
# When raw WebSocket message is received
def on_message(msg)
puts "Received message: #{msg}"
end
# When WebSocket is closed
def on_close(code, reason)
puts "WebSocket connection closed: #{code.inspect}, #{reason.inspect}"
end
end
m = WSConnection.new('wss://foo.bar')
while true; sleep; end
The expected output is
"Websocket connection opened"
However, I don't get any output at all. What could be the problem?
I am using
gem 'celluloid-websocket-client', '0.0.2'
rails 4.2.1
ruby 2.1.3
As you noticed in the comments, the gem had no SSL support. That is the trouble. To expound on the answer, here is a resolution, and also some next steps of what to expect for the future:
[ now ] Override methods in Celluloid::WebSocket::Client::Connection
This is an example injection to provide SSL support to the current gem. Mine is actually highly modified, but this shows you the basic solution:
def initialize(url, handler=nil)
#url = url
#handler = handler || Celluloid::Actor.current
#de If you want an auto-start:
start
end
def start
uri = URI.parse(#url)
port = uri.port || (uri.scheme == "ws" ? 80 : 443)
#socket.close rescue nil
#socket = Celluloid::IO::TCPSocket.new(uri.host, port)
#socket = Celluloid::IO::SSLSocket.new(#socket) if port == 443
#socket.connect
#client = ::WebSocket::Driver.client(self)
async.run
end
The above sends ripple effects through the other methods however, for example, #handler is used to hold the calling actor, which also has the emitter methods on it. Like I said, my version is very different from the stock gem because I got fed up with it and reworked mine. But then:
[ soon ] Use Reel::IO::Client and avoid near certain brain damage.
There are exciting things going on with WebSocket support, and a gem is coming to refactor both server and client implementations of websockets. No more monkeypatches required!
All websocket functionality is being extracted from Reel and being combined with a websocket-driver abstraction, as Reel::IO... in both ::Server and ::Client varieties.
Interestingly, this is prompted by Rails which is moving away from EventMachine to Celluloid::IO for websockets:
https://github.com/rails/actioncable/issues/16
https://github.com/celluloid/reel/issues/201
https://github.com/celluloid/reel-io/issues/2
A prealpha is online for preview: https://github.com/celluloid/reel-io
I'm using eventmachine to read from a HornetQ topic, push to a Channel which is subscribed to by EM websocket connections. I need to prevent the #topic.receive loop from blocking, so have created a proc and am calling EventMachine.defer with no callback. This will run indefinitely. This works fine. I could also have just used Thread.new.
My question is, is this the correct way to read from a stream/queue and pass the data to the channel and is there a better/any other way to do this?
require 'em-websocket'
require 'torquebox-messaging'
class WebsocketServer
def initialize
#channel = EM::Channel.new
#topic = TorqueBox::Messaging::Topic.new('/topics/mytopic')
end
def start
EventMachine.run do
topic_to_channel = proc do
while true
msg = #topic.receive
#channel.push msg
end
end
EventMachine.defer(topic_to_channel)
EventMachine::WebSocket.start(:host => "127.0.0.1", :port => 8081, :debug => false) do |connection|
connection.onopen do
sid = #channel.subscribe { |msg| connection.send msg }
connection.onclose do
#channel.unsubscribe(sid)
end
end
end
end
end
end
WebsocketServer.new.start
This is ok, but EM.defer will spawn 20 threads, so I would avoid it for your use case. In general I would avoid EM entirely, especially the Java reactor as we never finished it.
The Torquebox has a native stomp over websockets solution that would be a much better way to go in this context, and solves a bunch of other encapsulation challenges for you.
If you really want to stick with EM for this, then I'd use Thread.new instead of defer, so as to avoid having 19 idle threads taking up extra ram for no reason.
I'm writing a ruby program that has 2 threads. One that listens on an incoming UDP connection and another that broadcasts on a websocket from which browsers on the client side read.I'm using the em-websocket gem. However, My UDP listener thread never gets called and it looks like the code stays within the websocket initialization code. I'm guessing because em-websocket is blocking, but I haven't been able to find any info online that suggests that. Is it an error on my side? I'm kinda new to ruby so I'm not able to figure out what I'm doing wrong.
require 'json'
require 'em-websocket'
require 'socket'
socket=nil
text="default"
$x=0
EventMachine.run do
EventMachine::WebSocket.start(:host => "0.0.0.0", :port => 8080) do |ws|
ws.onopen {
ws.send "Hello Client!"
socket=ws
$x=1
}
ws.onmessage { |msg| socket.send "Pong: #{msg}" }
ws.onclose { puts "WebSocket closed" }
end
end
def listen()
puts "listening..."
s = UDPSocket.new
s.bind(nil, 3000)
while 1<2 do
text, sender = s.recvfrom(1024)
puts text
if $x==1 then
socket.send text
end
end
end
t2=Thread.new{listen()}
t2.join
em-websocket is non-blocking, however UDPSocket#recv_from is. Might be better to just use EventMachine's open_datagram_socket instead.
Another thing to note: you should not expose socket as a "global" variable. Every time somebody connects the reference to the previously connected client will be lost. Maybe make some sort of repository for socket connections, or use an observer pattern to broadcast messages when something comes in. What I would do is have a dummy object act as an observer, and whenever a socket is connected/disconnect you register/unregister from the observer:
require 'observer'
class Dummy
include Observable
def receive_data data
changed true
notify_observers data
end
end
# ... later on ...
$broadcaster = Dummy.new
class UDPHandler < EventMachine::Connection
def receive_data data
$broadcaster.receive_data data
end
end
EventMachine.run do
EM.open_datagram_socket "0.0.0.0", 3000, UDPHandler
EM::WebSocket.start :host => "0.0.0.0", :port => 8080 do |ws|
ws.onopen do
$broadcaster.add_observer ws
end
ws.onclose do
$broadcaster.delete_observer ws
end
# ...
end
end
The whole point of EventMachine is to abstract away from the basic socket and threading structure, and handle all the asynchronous bits internally. It's best not to mix the classical libraries like UDPSocket or Thread with EventMachine stuff.
I am trying to implement a request-response pattern using the em-zeromq gem, but I can't get the response socket to send a message back to the request socket in its handler. I have written some very simple code to test it:
em_req.rb
require 'em-zeromq'
client_id = ARGV[0] ? ARGV[0].to_i : 1
message = ARGV[1] || "Foo"
Thread.abort_on_exception = true
class ReqHandler
attr_reader :received
def on_readable(socket, messages)
messages.each do |m|
puts "Received message from server: #{m.copy_out_string}"
end
end
end
trap('INT') do
EM.stop
end
ctx = EM::ZeroMQ::Context.new(1)
EM.run do
conn = ctx.connect(ZMQ::REQ, 'tcp://127.0.0.1:9000', ReqHandler.new, identity: "client#{client_id}")
conn.socket.send_string(message)
end
em_rep.rb
require 'em-zeromq'
Thread.abort_on_exception = true
class ResponseHandler
attr_reader :received
def on_readable(socket, messages)
message = messages.first.copy_out_string
puts "Received message from client: #{message}"
socket.send_msg("re: #{message}")
end
end
trap('INT') do
EM.stop
end
ctx = EM::ZeroMQ::Context.new(1)
EM.run do
socket = ctx.bind(ZMQ::REP, 'tcp://127.0.0.1:9000', ResponseHandler.new)
end
I have written similar code using the push-pull pattern and got that to work, but for request-response all I get is the response code printing "Received message from client1: Foo" but the reply never reaches the request code. I suspect it has to do with writing to the socket in the response code's handler, because the same thing happens when I use a request-router pattern. The only time it works is when I send a message from the server without sending a message from the client first (using push-pull).
Any ideas about what might be causing this? The author of the gem isn't maintaining it anymore, but I thought I would post this issue anyway in the hopes of other developers with similar experiences seeing this.
I am using em-zeromq 0.2.2 on Ruby 1.9.2p290.
I commmited a fix in the master branch which should solve your problem, can you give it a try ?
You can use bundler to easily test it:
Create a file called Gemfile in your application folder:
source :rubygems
gem 'em-zeromq', :git => "git://github.com/andrewvc/em-zeromq.git"
And add this on top of your ruby files:
require 'rubygems'
require 'bundler/setup'
And last run this in the application folder ($ is your prompt):
$ bundle
Now you can execute your ruby files they will use the latest code from github
Edit: I am the new maintainer for the em-zeromq gem.