What is wrong with my Celluloid actors - ruby

I'm playing with celluloid gem. The example works well, but when I press Ctrl-C I get the unexpected message:
^CD, [2015-10-07T09:53:19.784411 #16326] DEBUG -- : Terminating 8 actors...
and after few seconds, I get the error:
E, [2015-10-07T09:53:29.785162 #16326] ERROR -- : Couldn't cleanly terminate all actors in 10 seconds!
/usr/local/rvm/gems/ruby-2.0.0-p353/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run_machine': Interrupt
from /usr/local/rvm/gems/ruby-2.0.0-p353/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run'
Strange that I create only 4 actors, not 8, and my TERM, INT signals handler isn't be called.
#!/usr/bin/env ruby
require './config/environment'
opts = CommandlineOptions.new.to_h
iface = opts[:iface] || '0.0.0.0'
port = opts[:port] || 3000
App.logger.info('Starting communication server')
connections = Connections.new
local_inbox = LocalQueue.new
auth_server = AuthServer.new(connections, local_inbox)
inbox_service = InboxService.new('inbox', iface, port)
inbox_service.async.process_inbox(local_inbox) # <--------
remote_outbox_name = "outbox_#{iface}:#{port}"
outbox_service = OutboxService.new(connections)
outbox_service.async.subscribe(remote_outbox_name) # <--------
conn_server_opts = { host: iface, port: port }
conn_server_opts.merge!(auth_server.callbacks)
conn_server = ConnServer.new(conn_server_opts)
%W(INT TERM).each do |signal|
trap(signal) do
info("Shutting down...")
conn_server.stop
end
end
conn_server.start
Here InboxService is an actor which creates another actor - there are 2 actors, then OutboxService also creates one actor, so I got created 4 actors.
require 'redis'
require 'celluloid/current'
class InboxServiceActor
include Celluloid
def initialize(remote_inbox_name)
#remote_inbox_name = remote_inbox_name
create_redis_connection
end
def publish(full_msg)
#redis.publish(#remote_inbox_name, full_msg)
end
private
def create_redis_connection
#redis = Redis.new
end
end
require 'json'
require 'redis'
require 'celluloid/current'
class OutboxServiceActor
include Celluloid
include HasLoggerMethods
def initialize
create_redis_connection
end
def subscribe(remote_outbox_name, &block)
#redis.subscribe(remote_outbox_name) do |on|
on.message do |_channel, full_msg|
debug("Outbox message received: '#{full_msg}'")
hash = parse_msg(full_msg)
block.call(hash['signature'], hash['msg']) if message_valid?(hash)
end
end
end
private
def create_redis_connection
#redis = Redis.new
end
def parse_msg(full_msg)
JSON.parse(full_msg)
rescue JSON::ParserError
error('Outbox message JSON parse error')
nil
end
def message_valid?(msg)
msg.is_a?(Hash) && msg.key?('signature') && msg.key?('msg') ||
error('Invalid outbox message. Should '\
'contain "signature" and "msg" keys') && false
end
end

Related

How to test WebSockets For Hanami?

Using the following:
Hanami cookbook websockets
IoT Saga - Part 3 - Websockets! Connecting LiteCable to Hanami
I've been able to add WebSockets to Hanami, however as this is for production code I want to add specs; but I can't find information on how to test WebSockets and Hanami using Rspec.
I've been able to find this for RoR but nothing non-Rails specific or Hanami Specific, I have asked on the Hanami Gitter but not gotten a response yet.
Is the TCR gem the only way? I would prefer something simpler but If I must how would I set it up for anycable-go via litecable.
How can I test WebSockets for Hanami using Rspec?
To get this working requires several moving parts, the first is the Socket simulator which simulates the receiving socket on the webserver:
Note: url_path should be customized to what works for your web socket specific endpoint
# frozen_string_literal: true
require 'puma'
require 'lite_cable/server'
require_relative 'sync_client'
class SocketSimulator
def initialize(x_site_id_header: nil)
#server_logs = []
#x_site_id_header = x_site_id_header
end
attr_accessor :server_logs
def client
return #client if #client
url_path = "/ws?connection_token=#{connection_token}"
#client = SyncClient.new("ws://127.0.0.1:3099#{url_path}", headers: headers, cookies: '')
end
def connection_token
#connection_token ||= SecureRandom.hex
end
def user
return #user if #user
email = "#{SecureRandom.hex}#mailinator.com"
password = SecureRandom.hex
#user = Fabricate.create :user, email: email, site_id: site_id, password: password
end
def start
#server = Puma::Server.new(
LiteCable::Server::Middleware.new(nil, connection_class: Api::Sockets::Connection),
Puma::Events.strings
).tap do |server|
server.add_tcp_listener '127.0.0.1', 3099
server.min_threads = 1
server.max_threads = 4
end
#server_thread = Thread.new { #server.run.join }
end
def teardown
#server&.stop(true)
#server_thread&.join
#server_logs.clear
end
def headers
{
'AUTHORIZATION' => "Bearer #{jwt}",
'X_HANAMI_DIRECT_BOOKINGS_SITE_ID' => #x_site_id_header || site_id
}
end
def site_id
#site_id ||= SecureRandom.hex
end
def jwt
#jwt ||= Interactors::Users::GenerateJwt.new(user, site_id).call.jwt
end
end
The next thing is the SyncClient which is a fake client you can use to actually connect to the simulated socket:
# frozen_string_literal: true
# Synchronous websocket client
# Copied and modified from https://github.com/palkan/litecable/blob/master/spec/support/sync_client.rb
class SyncClient
require 'websocket-client-simple'
require 'concurrent'
require 'socket'
WAIT_WHEN_EXPECTING_EVENT = 5
WAIT_WHEN_NOT_EXPECTING_EVENT = 0.5
attr_reader :pings
def initialize(url, headers: {}, cookies: '')
#messages = Queue.new
#closed = Concurrent::Event.new
#has_messages = Concurrent::Semaphore.new(0)
#pings = Concurrent::AtomicFixnum.new(0)
#open = Concurrent::Promise.new
#ws = set_up_web_socket(url, headers.merge('COOKIE' => cookies))
#open.wait!(WAIT_WHEN_EXPECTING_EVENT)
end
def ip
Socket.ip_address_list.detect(&:ipv4_private?).try(:ip_address)
end
def set_up_web_socket(url, headers)
WebSocket::Client::Simple.connect(
url,
headers: headers
) do |ws|
ws.on(:error, &method(:on_error))
ws.on(:open, &method(:on_open))
ws.on(:message, &method(:on_message))
ws.on(:close, &method(:on_close))
end
end
def on_error(event)
event = RuntimeError.new(event.message) unless event.is_a?(Exception)
if #open.pending?
#open.fail(event)
else
#messages << event
#has_messages.release
end
end
def on_open(_event = nil)
#open.set(true)
end
def on_message(event)
if event.type == :close
#closed.set
else
message = JSON.parse(event.data)
if message['type'] == 'ping'
#pings.increment
else
#messages << message
#has_messages.release
end
end
end
def on_close(_event = nil)
#closed.set
end
def read_message
#has_messages.try_acquire(1, WAIT_WHEN_EXPECTING_EVENT)
msg = #messages.pop(true)
raise msg if msg.is_a?(Exception)
msg
end
def read_messages(expected_size = 0)
list = []
loop do
list_is_smaller = list.size < expected_size ? WAIT_WHEN_EXPECTING_EVENT : WAIT_WHEN_NOT_EXPECTING_EVENT
break unless #has_messages.try_acquire(1, list_is_smaller)
msg = #messages.pop(true)
raise msg if msg.is_a?(Exception)
list << msg
end
list
end
def send_message(message)
#ws.send(JSON.generate(message))
end
def close
sleep WAIT_WHEN_NOT_EXPECTING_EVENT
raise "#{#messages.size} messages unprocessed" unless #messages.empty?
#ws.close
wait_for_close
end
def wait_for_close
#closed.wait(WAIT_WHEN_EXPECTING_EVENT)
end
def closed?
#closed.set?
end
end
The last part is a fake channel to test against:
# frozen_string_literal: true
class FakeChannel < Api::Sockets::ApplicationChannel
identifier :fake
def subscribed
logger.info "Can Reject? #{can_reject?}"
reject if can_reject?
logger.debug "Streaming from #{stream_location}"
stream_from stream_location
end
def unsubscribed
transmit message: 'Goodbye channel!'
end
def can_reject?
logger.info "PARAMS: #{params}"
params.fetch('value_to_check', 0) > 5
end
def foo
transmit('bar')
end
end
To use in specs:
# frozen_string_literal: true
require_relative '../../../websockets-test-utils/fake_channel'
require_relative '../../../websockets-test-utils/socket_simulator'
RSpec.describe Interactors::Channels::Broadcast, db_truncation: true do
subject(:interactor) { described_class.new(token: connection_token, loc: 'fake', message: message) }
let(:identifier) { { channel: 'fake' }.to_json }
let(:socket_simulator) { SocketSimulator.new }
let(:client) { socket_simulator.client }
let(:user) { socket_simulator.user }
let(:connection_token) { socket_simulator.connection_token }
let(:channel) { 'fake' }
let(:message) { 'woooooo' }
before do
socket_simulator.start
end
after do
socket_simulator.teardown
end
describe 'call' do
before do
client.send_message command: 'subscribe',
identifier: identifier
end
it 'broadcasts a message to the correct channel' do
expect(client.read_message).to eq('type' => 'welcome')
expect(client.read_message).to eq(
'identifier' => identifier,
'type' => 'confirm_subscription'
)
interactor.call
expect(client.read_message).to eq(
'identifier' => identifier,
'message' => message
)
end
context 'with other connection' do
let(:user2) { Fabricate.create :user }
let(:jwt) { Interactors::Users::GenerateJwt.new(user2, site_id).call.jwt }
let(:site_id) { socket_simulator.site_id }
let(:url_path) { "/ws?connection_token=#{SecureRandom.hex}" }
let(:client2) { SyncClient.new("ws://127.0.0.1:3099#{url_path}", headers: {}, cookies: '') }
before do
client2.send_message command: 'subscribe',
identifier: identifier
end
it "doesn't broadcast to connections that shouldn't get it" do
aggregate_failures 'broadcast!' do
expect(client2.read_message).to eq('type' => 'welcome')
expect(client2.read_message).to eq(
'identifier' => identifier,
'type' => 'confirm_subscription'
)
expect(client.read_message).to eq('type' => 'welcome')
expect(client.read_message).to eq(
'identifier' => identifier,
'type' => 'confirm_subscription'
)
interactor.call
sleep 1
expect(client.read_message).to eq(
'identifier' => identifier,
'message' => message
)
expect { client2.close }.not_to raise_exception
end
end
end
end
end

API integration error HTTParty

I'm learning how to work with HTTParty and API and I'm having an issue with my code.
Users/admin/.rbenv/versions/2.0.0-p481/lib/ruby/2.0.0/uri/generic.rb:214:in `initialize': the scheme http does not accept registry part: :80 (or bad hostname?)
I've tried using debug_output STDOUT both as an argument to my method and after including HTTParty to have a clue but with no success. Nothing gets displayed:
require 'httparty'
class LolObserver
include HTTParty
default_timeout(1) #timeout after 1 second
attr_reader :api_key, :playerid
attr_accessor :region
def initialize(region,playerid,apikey)
#region = region_server(region)
#playerid = playerid
#api_key = apikey
end
def region_server(region)
case region
when "euw"
self.class.base_uri "https://euw.api.pvp.net"
self.region = "EUW1"
when "na"
self.class.base_uri "https://na.api.pvp.net"
self.region = "NA1"
end
end
def handle_timeouts
begin
yield
#Timeout::Error, is raised if a chunk of the response cannot be read within the read_timeout.
#Timeout::Error, is raised if a connection cannot be created within the open_timeout.
rescue Net::OpenTimeout, Net::ReadTimeout
#todo
end
end
def base_path
"/observer-mode/rest/consumer/getSpectatorGameInfo"
end
def current_game_info
handle_timeouts do
url = "#{ base_path }/#{region}/#{playerid}?api_key=#{api_key}"
puts '------------------------------'
puts url
HTTParty.get(url,:debug_output => $stdout)
end
end
end
I verified my URL which is fine so I'm lost as to where the problem is coming from.
I tested with a static base_uri and it doesn't change anything.
The odd thing is when I do:
HTTParty.get("https://euw.api.pvp.net/observer-mode/rest/consumer/getSpectatorGameInfo/EUW1/randomid?api_key=myapikey")
Everything is working fine and I'm getting a response.
HTTParty doesn't seem to like the way you set your base_uri.
Unless you need it to be like that just add another attr_reader called domain and it will work.
require 'httparty'
class LolObserver
include HTTParty
default_timeout(1) #timeout after 1 second
attr_reader :api_key, :playerid, :domain
attr_accessor :region
def initialize(region,playerid,apikey)
#region = region_server(region)
#playerid = playerid
#api_key = apikey
end
def region_server(region)
case region
when "euw"
#domain = "https://euw.api.pvp.net"
self.region = "EUW1"
when "na"
#domain = "https://na.api.pvp.net"
self.region = "NA1"
end
end
def handle_timeouts
begin
yield
#Timeout::Error, is raised if a chunk of the response cannot be read within the read_timeout.
#Timeout::Error, is raised if a connection cannot be created within the open_timeout.
rescue Net::OpenTimeout, Net::ReadTimeout
#todo
end
end
def base_path
"/observer-mode/rest/consumer/getSpectatorGameInfo"
end
def current_game_info
handle_timeouts do
url = "#{domain}/#{ base_path }/#{region}/#{playerid}?api_key=#{api_key}"
puts '------------------------------'
puts url
HTTParty.get(url,:debug_output => $stdout)
end
end
end

I must be misunderstanding Celluloid

I currently have a script written in Ruby that scans a range of IP addresses and tries to connect to them. It's extremely slow at the moment. It takes up to 300 seconds to scan 254 hosts on the network, and that's obviously not very practical. What I'm trying to do is give the script some concurrency in hopes of speeding up the script. So far this is what I have:
require 'socket'
require 'celluloid'
$res_arr = []
class Ranger
include Celluloid
def initialize(host)
#host = host
#timeout = 1
end
def ip_range(host)
host =~ /(?:\d{1,3}\.){3}[xX*]{1,3}/
end
def ctrl(host)
begin
if ip_range(host)
strIP = host.gsub(/[xX*]/, '')
(1..254).each do |oct|
$res_arr << strIP+oct.to_s
end
else
puts "Invalid host!"
end
rescue
puts "onnection terminated."
end
end
def connect
addr = Socket.getaddrinfo(#host, nil)
sock = Socket.new(Socket.const_get(addr[0][0]), Socket::SOCK_STREAM, 0)
begin
sock.connect_nonblock(Socket.pack_sockaddr_in(22, addr[0][3]))
rescue Errno::EINPROGRESS
resp = IO.select(nil, [sock], nil, #timeout.to_i)
if resp.nil?
$res_arr << "#{#host} Firewalled!"
end
begin
if sock.connect_nonblock(Socket.pack_sockaddr_in(22, addr[0][3]))
$res_arr << "#{#host}Connected!"
end
rescue Errno::ECONNREFUSED
$res_arr << "#{#host} Refused!"
rescue
false
end
end
sock
end
def output(contents)
puts contents.value
end
end # Ranger
main = Ranger.new(ARGV[0])
main.ctrl(ARGV[0])
$res_arr.each do |ip|
scan = Ranger.new(ip)
scnftr = scan.future :connect
scan.output(scnftr)
end
The script works, but it takes just as long as before I included Celluloid at all. Am I misunderstanding how Celluloid works and what it's supposed to do?
Your problem is that each iteration of your loop starts a future, then immediately waits for it to return a value. What you want instead is start all futures, then wait for all futures to finish in two separate steps:
futures = $res_arr.map do |ip|
scan = Ranger.new(ip)
scan.future :connect
end
# now that all futures are running, we can start
# waiting for the first one to finish
futures.each do |future|
puts future.value
end
Here's another example from the celluloid source: https://github.com/celluloid/celluloid/blob/master/examples/simple_pmap.rb

WARN TCPServer Error: Address already in use - bind(2) in linux EC2 and Heroku servers

[2013-01-29 09:17:50] INFO WEBrick 1.3.1
[2013-01-29 09:17:50] INFO ruby 1.8.7 (2012-10-12) [i386-linux]
[2013-01-29 09:17:50] WARN TCPServer Error: Address already in use - bind(2)
[2013-01-29 09:17:50] INFO WEBrick::HTTPServer#start: pid=4107 port=8080
When I run the file attached below in linux I get the error described. I tried all possible command and strategies online to listen to processes (including rogue) and kill them. I did this in lots of ports. No luck.
As soon as I run the script in Mac OS and it works. Nevertheless I have to mount it on a server and clients have to communicate with it. It happens on every instance of amazon ec2 and on heroku. I have seen this error one too many times and spend many hours trying to fix it. I configured the security group of ec2 instances and still did not work. I am beyond desperate. At this point I have to think that the problem must be WEBrick itself or something in my code.
require 'webrick'
require 'uri'
require 'net/http'
$own_address = 8080
class AuctionInfo
# The representation is a hash mapping item names to [highest_bidder, highest_bid, end_time]
def initialize
#data = {}
end
def new_item(item, endTime)
#data[item] = ["UNKNOWN", 0, endTime]
end
def bid(item, bid, client)
if #data.has_key?(item)
endTime = #data[item][2]
if #data[item][1].to_i < bid.to_i and Time.new.to_i < endTime.to_i
#data[item] = [client, bid, endTime]
end
end
end
def get_status(item)
if #data.has_key?(item)
return #data[item][0]
end
end
def winner(item)
if #data.has_key?(item)
if #data[item][2].to_i + 1 <= Time.new.to_i
return #data[item][0]
else return "UNKNOWN"
end
end
end
def reset
#data = {}
end
def has_item(item)
return #data.has_key?(item)
end
def get_data
return {}.replace(#data)
end
end
class StartAuctionServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_POST(request, response)
if request.query['name'] and request.query['end_time']
#data.new_item(request.query['name'], request.query['end_time'].to_i)
end
response.status = 200
end
alias_method :do_GET, :do_POST
end
class BidServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_POST(request, response)
if request.query['name'] and request.query['client'] and request.query['bid']
#data.bid(request.query['name'], request.query['bid'].to_i, request.query['client'])
end
response.status = 200
end
alias_method :do_GET, :do_POST
end
class StatusServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_GET(request, response)
if request.query['name']
response.body = #data.get_status(request.query['name'])
end
response.status = 200
end
alias_method :do_POST, :do_GET
end
class WinnerServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_GET(request, response)
if request.query['name']
response.body = #data.winner(request.query['name'])
end
response.status = 200
end
alias_method :do_POST, :do_GET
end
class ResetServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_POST(request, response)
#data.reset
response.status = 200
end
alias_method :do_GET, :do_POST
end
class RandomServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_GET(request, response)
response.status = 200
response.body = #data.get_data.to_s
end
alias_method :do_POST, :do_GET
end
data = AuctionInfo.new
server = WEBrick::HTTPServer.new(:Port => $own_address)
server.mount '/start_auction', StartAuctionServlet, data
server.mount '/bid', BidServlet, data
server.mount '/status', StatusServlet, data
server.mount '/winner', WinnerServlet, data
server.mount '/rst', ResetServlet, data
server.mount '/', RandomServlet, data
trap("INT") { server.shutdown }
server.start
Have you checked whether the linux server is running apache, tomcat, trinidad or any other web server? Odds are one of them is already running on port 8080 on the server.
lsof is a useful command. Try lsof | grep 8080 and see whether anything shows up

Ruby and Redis, threads not processing commands

I'm working with redis and ruby and attempting to issue a blpop within a thread, so that I can wait for an incoming item on a list.
The problem is that the code within the block for blpop never seems to get called. Here's the sample code that I'm running (ruby 1.9.3):
require 'rubygems'
require 'redis'
def start_thread
#thread = Thread.new do
r = Redis.new
r.blpop("test", 0) do |key, message|
process_message(key, message)
end
end
redis = Redis.new
redis.rpush "test", "hello world"
end
def process_message(key, message)
#message = "#{key} was sent #{message}"
end
start_thread
#thread.join
p #message
Any help is greatly appreciated!
require 'rubygems'
require 'redis'
def start_thread
#thread = Thread.new do
r = Redis.new
key, message = r.blpop(:test, 0)
process_message(key, message)
end
redis = Redis.new
redis.rpush :test, "hello world"
end
def process_message(key, message)
#message = "#{key} was sent #{message}"
end
start_thread
#thread.join
p #message

Resources