I looked over this documentation:
http://redis.io/topics/pubsub
It states:
When you subscribe to a channel, you will get a message that is represented as a multi-bulk reply with three elements. The first element of a message is the kind of message (e.g. SUBSCRIBE or UNSUBSCRIBE). The second element of the message is the name of the given channel you are subscribing or unsubscribing to. The third element of the message is the number of channels you are currently subscribed to:
> SUBSCRIBE first second
*3 #three elements in this message: “subscribe”, “first”, and 1
$9 #number of bytes in the element
subscribe #kind of message
$5 #number of bytes in the element
first #name of channel
:1 #number of channels we are subscribed to
That's cool you can see the number of channels you are subscribed to as part of a bulk reply from subscribing to a channel. Now I try to get this reply back when using ruby:
require 'rubygems'
require 'redis'
require 'json'
redis = Redis.new(:timeout => 0)
redis.subscribe('chatroom') do |on|
on.message do |channel, msg, total_channels|
data = JSON.parse(msg)
puts "##{channel} - [#{data['user']}]: #{data['msg']} - channels subscribed to: #{total_channels}"
end
end
However, I do not get that kind of reply at all. What it gives me is the name of channel, the data published to that channel, and then total_channels is nil, because there is no third parameter sent back.
So where is this "multi-bulk reply" that redis speaks of?
Actually, the protocol is to send a subscribe reply message as the first message just after a subscribe operation. You do not get the number of subscribed channels in all messages you receive (just as a reply to subscribe/unsubscribe).
With the current version of redis-rb, you need a separate handler to process subscribe/unsubscribe reply messages:
require 'rubygems'
require 'redis'
require 'json'
redis = Redis.new(:timeout => 0)
redis.subscribe('chatroom') do |on|
on.subscribe do |channel, subscriptions|
puts "Subscribed to ##{channel} (#{subscriptions} subscriptions)"
end
on.message do |channel, msg|
data = JSON.parse(msg)
puts "##{channel} - [#{data['user']}]: #{data['msg']}"
end
end
Please note that in your example, the number of subscriptions will be always 1.
Related
I am working on a project that returns SMS messages to a user who has just sent a SMS message to the server.
The process is:
The user sends a SMS message to the server.
The server will send two SMS messages back to this user. Note that these are two separate short messages and will be sent pretty much at the same time.
I've got the sending part working, but just for sending one SMS message, not two. When I add more code to send another message only the second message part works, which means only the second message has been sent out, the first message has been ignored.
The code looks pretty much like:
else
sms = SMS.create(:body => params['Body'], :from => params['From'], :to => params['To'], :created_at => Time.now)
#return a random saved sms
return_secret = SMS.first(:offset => rand(SMS.count))
twiml = Twilio::TwiML::Response.new do |r|
r.Sms return_secret.body
#send another message to remind user for rating
ask_rating = remind_rating
if ask_rating
twiml = Twilio::TwiML::Response.new do |r|
r.Sms ask_rating
end
twiml.text
end
Does anyone know how to send two messages in Twilio?
You've got some variable shadowing going on with twiml. As you wrote it, the second message's code is inside of the first message's block. Yet, you refer to a variable with the same name as one outside of the block. I would try flattening your code so you aren't nesting like that.
I think the issue here is you're instantiating a second TwiML::Response object when you already have one, so you can just references the previous one which you assigned to r in the first block. You also called it r in the second block so you just remove the block that encloses it:
sms = SMS.create(:body => params['Body'], :from => params['From'], :to => params['To'], :created_at => Time.now)
#return a random saved sms
return_secret = SMS.first(:offset => rand(SMS.count))
twiml = Twilio::TwiML::Response.new do |r|
r.Sms return_secret.body
#send another message to remind user for rating
ask_rating = remind_rating
if ask_rating
r.Sms ask_rating
end
end
Also the blocks weren't balanced in the initial code snippet so I stripped out the else to make it syntactically accurate.
Thank you all, really appreciate your replies.
After consulting with twilio team, they gave me an example like this:
require 'rubygems'
require 'twilio-ruby'
require 'sinatra'
get '/sms-quickstart' do
twiml = Twilio::TwiML::Response.new do |r|
r.Message "Hey Monkey. Thanks for the message!"
r.Message "this is the 2nd message"
end
twiml.text
end
I just deleted
if ask_rating
twiml = Twilio::TwiML::Response.new do |r|
everything works...
I have multiple clients connecting on UDP. Their addresses are stored for future communications. If one dies and message is sent to it then recvfrom(2) gives me econnreset.
Is there a way to know which peer died at this point?
the scenario looks like this
def broadcast message
# one of the clients is dead
clients.each { |x| #socket.send message, 0, x.ip, x.port }
end
def fill_inbox
#inbox << #socket.recvfrom(1000) while IO.select([#socket],nil,nil,0)
rescue Errno::ECONNRESET => e
# who died?
end
I'm using Pubnub to publish live messages from a Server to a client (browser page). When using Pubnub, one must abide by their message size constraints, sometimes resulting in the need to chunk the message, send it in pieces, and reconstruct on the client side. Following Pubnub's advice, one can ensure delivery of each chunk of a message if the Pubnub.publish() function is not called too quickly (i.e. if the message pieces are simply being pumped through a for loop).
The Pubnub Ruby API specifies 3 required arguments in a Pubnub.publish(), a channel, a message, and a callback function. The callback function shown below is from Pubnub's Ruby examples:
#my_callback = lambda { |message| puts(message) }
pn.publish(:channel => :hello_world,
:message => "hi",
:callback => #my_callback)
The message in the callback (not the "hi" message in the publish), contains the status information of the publish() call with values like "sent" and "message to large", both of which would be accompanied by a unique identifier.
So somewhere under the Pubnub hood, this callback is getting a .call() - I'm wondering if there is a way for me to get inbetween this process. More detailed, say I have a message that needs to be broken up into three chunks, I would like for chunk 0 to be sent, and upon receipt of a "sent" status in the callback, I would like to then send chunk 1, etc...
I'm not very familiar with lambda functions and their scopes, and this was my first attempt:
#my_callback = lambda { |message|
puts(message)
Rails.logger.debug("Setting pubnub_do_send to true from callback")
pubnub_do_send = true
}
pubnub_do_send = true
while !pubnub_message.nil?
if pubnub_do_send
#Send pubnub message
#Cut off first chunk of message (this is why I'm testing for nil)
#Set pubnub_do_send to false
Rails.logger.debug("Message #{message_id} chunk #{chunk_id} sent")
pubnub_do_send = false
end
end
This resulted in an utter failure - getting the server completely locked into an infinite while loop because (if I had to guess) pubnub_do_send was never set to true again. Looking at the debug log, I see the first message print ("Message 1 chunk 0 sent") but never the output from the callback function. (Probably because of the infinite while loop it's found itself in)
There must be a clean way to do this, and I'm ok with refactoring code to some extent, chunking up the messages and storing into an array first and then simply looping through the array to send, but I feel like the solution can't be far off, I'm just not too handy with lambda functions and callbacks.
I feel like the solution should look like:
#my_callback = lambda { |message|
puts(message)
send_pubnub_message(message_id, chunk_id, chunk)
}
def send_pubnub_message(message_id, chunk_id, chunk)
#Send pubnub message
#Send message_id, next chunk_id, and next chunk to my_callback
end
But the problem is my_callback is called by Pubnub when the Pubnub.publish() gets some status back about the message rather than me directly calling it! (Is there a way for me to insert message_id, chunk_id, and chunk into the callback while still letting pubnub attach its message to the mix? That sounds way wrong, but maybe with Ruby...)
Thanks in advance for all help.
You shouldn't be trying to handle message chunk state in your callback. It has only a single responsibility, notifying about the status of the publish. However, you can insert things into the message. I might create a message wrapper that knows it's current state, and send that in the lambda, so you don't have to keep track of it. I haven't tested the following code, but here's an example of what I'm talking about:
class Message
attr_accessor :id, :message, :chunked_message, :chunk_id
def initialize(id, message)
#id, #message = id, message
chunk_message
end
def current_chunk
#chunked_message[#chunk_id]
end
def next_chunk
#chunk_id += 1
self
end
def more?
#chunked_message.length > chunk_id
end
private
def chunk_message
implement splitting message here
end
end
def send_pubnub_message(message)
pn.publish(:channel => :hello_world,
:message => message.current_chunk
:callback => lambda { |status|
puts(status)
case status[0] // 1 = success, 0 = fail
when 1
send_pubnub_message(message.next_chunk) if message.more?
when 0
handle_pubnub_error(status[1], message)
end
}
end
I have an application that reacts to messages sent by clients. One message is reload_credentials, that the application receives any time a new client registers. This message will then connect to a PostgreSQL database, do a query for all the credentials, and then store them in a regular Ruby hash ( client_id => client_token ).
Some other messages that the application may receive are start,stop,pause which are used to keep track of some session times. My point is that I envision the application functioning in the following way:
client sends a message
message gets queued
queue is being processed
However, for example, I don't want to block the reactor. Furthermore, let's imagine I have a reload_credentials message that's next in queue. I don't want any other message from the queue to be processed until the credentials are reloaded from the DB. Also, while I am processing a certain message ( like waiting for the credentials query to finish) , I want to allow other messages to be enqueued .
Could you please guide me towards solving such a problem? I'm thinking I may have to use em-synchrony, but I am not sure.
Use one of the Postgresql EM drivers, or EM.defer so that you won't block the reactor.
When you receive the 'reload_credentials' message just flip a flag that causes all subsequent messages to be enqueued. Once the 'reload_credentials' has finished, process all messages from the queue. After the queue is empty flip the flag that causes messages to be processed as they are received.
EM drivers for Postgresql are listed here: https://github.com/eventmachine/eventmachine/wiki/Protocol-Implementations
module Server
def post_init
#queue = []
#loading_credentials = false
end
def recieve_message(type, data)
return #queue << [type, data] if #loading_credentials || !#queue.empty?
return process_msg(type, data) unless :reload_credentials == type
#loading_credentials = true
reload_credentials do
#loading_credentials = false
process_queue
end
end
def reload_credentials(&when_done)
EM.defer( proc { query_and_load_credentials }, when_done )
end
def process_queue
while (type, data = #queue.shift)
process_msg(type, data)
end
end
# lots of other methods
end
EM.start_server(HOST, PORT, Server)
If you want all connections to queue messages whenever any connection receives a 'reload_connections' message you'll have to coordinate via the eigenclass.
The following is I presume, something like your current implementation:
class Worker
def initialize queue
#queue = queue
dequeue
end
def dequeue
#queue.pop do |item|
begin
work_on item
ensure
dequeue
end
end
end
def work_on item
case item.type
when :reload_credentials
# magic happens here
else
# more magic happens here
end
end
end
q = EM::Queue.new
workers = Array.new(10) { Worker.new q }
The problem above, if I understand you correctly, is that you don't want workers working on new jobs (jobs that have arrived earlier in the producer timeline), than any reload_credentials jobs. The following should service this (additional words of caution at the end).
class Worker
def initialize queue
#queue = queue
dequeue
end
def dequeue
#queue.pop do |item|
begin
work_on item
ensure
dequeue
end
end
end
def work_on item
case item.type
when :reload_credentials
# magic happens here
else
# more magic happens here
end
end
end
class LockingDispatcher
def initialize channel, queue
#channel = channel
#queue = queue
#backlog = []
#channel.subscribe method(:dispatch_with_locking)
#locked = false
end
def dispatch_with_locking item
if locked?
#backlog << item
else
# You probably want to move the specialization here out into a method or
# block that's passed into the constructor, to make the lockingdispatcher
# more of a generic processor
case item.type
when :reload_credentials
lock
deferrable = CredentialReloader.new(item).start
deferrable.callback { unlock }
deferrable.errback { unlock }
else
dispatch_without_locking item
end
end
end
def dispatch_without_locking item
#queue << item
end
def locked?
#locked
end
def lock
#locked = true
end
def unlock
#locked = false
bl = #backlog.dup
#backlog.clear
bl.each { |item| dispatch_with_locking item }
end
end
channel = EM::Channel.new
queue = EM::Queue.new
dispatcher = LockingDispatcher.new channel, queue
workers = Array.new(10) { Worker.new queue }
So, input to the first system comes in on q, but in this new system it comes in on channel. The queue is still used for work distribution among workers, but the queue is not populated while a refresh credentials operation is going on. Unfortunately, as I didn't take more time, I have not generalized the LockingDispatcher such that it isn't coupled with the item type and code for dispatching CredentialsReloader. I'll leave that to you.
You should note here that whilst this services what I understand of your original request, it is generally better to relax this kind of requirement. There are several outstanding problems that essentially cannot be eradicated without alterations in that requirement:
The system does not wait for executing jobs to complete before starting credentials jobs
The system will handle bursts of credentials jobs very badly - other items that might be processable, won't be.
In the case of a bug in the credentials code, the backlog could fill up ram and cause failure. A simple timeout might be enough to avoid catastrophic effects, iff the code is abortable, and subsequent messages are sufficiently processable to avoid further deadlocks.
It actually sounds like you have some notion of a userid in the system. If you think through your requirements, it's likely possible that you only need to backlog items that pertain to a userid who's credentials are in a refresh state. This is a different problem, that involves a different kind of dispatching. Try a hash of locked backlogs for those users, with a callback on credential completion to drain those backlogs into the workers, or some similar arrangement.
Good luck!
I'm trying to write a simple zmq system to replace an http client/server. My client times out when the server is off/unavailable, but does not retry or stop. What am I missing?
zmq_client.rb (modified version of Han Holl's lazy pirate client from zeromq guide)
require 'rubygems'
require 'zmq'
context = ZMQ::Context.new
socket = context.socket(ZMQ::REQ)
socket.connect('tcp://localhost:5559')
retries = 2
timeout = 10
retries.times do |tries|
message = "Hello #{tries}"
raise("Send: #{message} failed") unless socket.send(message)
puts "Sending string [#{message}]"
if ZMQ.select( [socket], nil, nil, timeout)
message = socket.recv
puts "Received reply [#{message}]"
break
else
puts "timeout"
end
end
socket.close
zmq_broker.rb (modified version of Oleg Sidorov's code found on zeromq guide)
require 'rubygems'
require 'ffi-rzmq'
context = ZMQ::Context.new
frontend = context.socket(ZMQ::ROUTER)
frontend.bind('tcp://*:5559')
poller = ZMQ::Poller.new
poller.register(frontend, ZMQ::POLLIN)
loop do
poller.poll(:blocking)
poller.readables.each do |socket|
if socket === frontend
loop do
socket.recv_string(message = '')
more = socket.more_parts?
puts "#{message}#{more}"
socket.send_string(message, more ? ZMQ::SNDMORE : 0)
break unless more
end
end
end
end
You should get an error Send: #{message} failed as soon as you try to send again after the first timeout, because your 2nd send will happen directly after the 1st send, and the REQ socket enforces that each send must go after (successful, not timeout-ed) recv.
In the lazy pirate pattern, you may need to send several requests before getting a reply. Solution suggested in the 0MQ Guide is to close and reopen the REQ socket after an error. Your client doesn't close/reopen the REQ socket.
You may find helpful the "Lazy Pirate client in Ruby" example from the Guide.