I'm trying to run a mock webserver within a thread within a class. I've tried passing the class' #server property to the thread block but as soon as I try to do server.accept the thread stops. Is there some way to make this work? I want to basically be able to run a webserver off of this script while still taking user input via stdin.gets. Is this possible?
class Server
def initialize()
#server = TCPServer.new(8080)
end
def run()
#thread = Thread.new(#server) { |server|
while true
newsock = server.accept
puts "some stuff after accept!"
next if !newsock
# some other stuff
end
}
end
end
def processCommand()
# some user commands here
end
test = Server.new
while true do
processCommand(STDIN.gets)
end
In the above sample, the thread dies on server.accept
In the code you posted, you're not calling Server#run. That's probably just an oversight in making the post. Server.accept is supposed to block a thread, returning only when someone has connected.
Anyone who goes into writing an HTTP server with bright eyes soon learns that it's more fun to let someone else do that work. For quick and dirty HTTP servers, I've got good results enlisting the aid of WEBrick. It's a part of the Ruby library. Here's a WEBrick server that will serve up "Boo!" When you connect your browser to localhost:8080/:
#!/usr/bin/ruby1.8
require 'webrick'
class MiniServer
def initialize
Thread.new do
Thread::abort_on_exception = true
server = WEBrick::HTTPServer.new(:BindAddress=>'127.0.0.1',
:Port=>8080,
:Logger=>WEBrick::Log.new('/dev/stdout'))
server.mount('/', Servlet, self)
server.start
end
end
private
class Servlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(webrick_server, mini_server)
end
def do_GET(req, resp)
resp.body = "<html><head></head><body>Boo!</body></html>"
end
alias :do_POST :do_GET
end
end
server = MiniServer.new
gets
I don't know ruby, but it looks like server.accept is blocking until you get a tcp connection... your thread will continue as soon as a connection is accepted.
You should start the server in your main thread and then spawn a new thread for each connection that you accept, that way your server will immediately go to accept another connection and your thread will service the one that was just accepted.
Related
I am encountering an interesting issue with Ruby TCPServer, where once a client connects, it continually uses more and more CPU processing power until it hits 100% and then the entire system starts to bog down and can't process incoming data.
The processing class that is having an issue is designed to be a TCP Client that receives data from an embedded system, processes it, then returns the processed data to be further used (either by other similar data processors, or output to a user).
In this particular case, there is an external piece of code that would like this processed data, but cannot access it from the main parent code (the thing that the original process class is returning it's data to). This external piece may or may not be connected at any point while it is running.
To solve this, I set up a Thread with a TCPServer, and the processing class continually adds to a queue, and the Thread pulls from the queue and sends it to the client.
It works great, except for the performance issues. I am curious if I have something funky going on in my code, or if it's just the nature of this methodology and it will never be performant enough to work.
Thanks in advance for any insight/suggestions with this problem!
Here is my code/setup, with some test helpers:
process_data.rb
require 'socket'
class ProcessData
def initialize
super
#queue = Queue.new
#client_active = false
Thread.new do
# Waiting for connection
#server = TCPServer.open('localhost', 5000)
loop do
Thread.start(#server.accept) do |client|
puts 'Client connected'
# Connection established
#client_active = true
begin
# Continually attempt to send data to client
loop do
unless #queue.empty?
# If data exists, send it to client
begin
until #queue.empty?
client.puts(#queue.pop)
end
rescue Errno::EPIPE => error
# Client disconnected
client.close
end
end
sleep(1)
end
rescue IOError => error
# Client disconnected
#client_active = false
end
end # Thread.start(#server.accept)
end # loop do
end # Thread.new do
end
def read(data)
# Data comes in from embedded system on this method
# Do some processing
processed_data = data.to_i + 5678
# Ready to send data to external client
if #client_active
#queue << processed_data
end
return processed_data
end
end
test_embedded_system.rb (source of the original data)
require 'socket'
#data = '1234'*100000 # Simulate lots of data coming ing
embedded_system = TCPServer.open('localhost', 5555)
client_connection = embedded_system.accept
loop do
client_connection.puts(#data)
sleep(0.1)
end
parent.rb (this is what will create/call the ProcessData class)
require_relative 'process_data'
processor = ProcessData.new
loop do
begin
s = TCPSocket.new('localhost', 5555)
while data = s.gets
processor.read(data)
end
rescue => e
sleep(1)
end
end
random_client.rb (wants data from ProcessData)
require 'socket'
loop do
begin
s = TCPSocket.new('localhost', 5000)
while processed_data = s.gets
puts processed_data
end
rescue => e
sleep(1)
end
end
To run the test in linux, open 3 terminal windows:
Window 1: ./test_embedded_system.rb
Window 2: ./parent.rb
\CPU usage is stable
Window 3: ./random_client.rb
\CPU usage continually grows
I ended up figuring out what the issue was, and unfortunately I lead folks astray with my example.
It turns out my example didn't quite have the issue I was having, and the main difference was the sleep(1) was not in my version of process_data.rb.
That sleep is actually incredibly important, because it is inside of a loop do, and without the sleep, the Thread won't yield the GVL, and will continually eat up CPU resources.
Essentially, it was unrelated to TCP stuff, and more related to Threads and loops.
If you stumble on this question later on, you can put a sleep(0) in your loops if you don't want it to wait, but you want it to yield the GVL.
Check out these answers as well for more info:
Ruby infinite loop causes 100% cpu load
sleep 0 has special meaning?
I'm developing API, based on EventMachine echo server. It listens requests on specific port and returns html-page with qr-code on it, which is generated depending on params from request query. The problem is that evaluating method, which packs string into qr-code takes from 8 to 11 seconds, it is unacceptable. I have no ideas why it happens, except it could be associated with event-machine.
P.S. In irb the same code RQRCode::QRCode.new(my_string, :size => 10,:level => :l) takes less than 1 second.
I have tried two different gems: rqrcode and barby+rqrcode. Both show the same results.
Code example:
require 'eventmachine'
require 'rqrcode'
class Handler < EventMachine::Connection
def receive_data(data)
puts Time.now
qrcode = RQRCode::QRCode.new('some_string', :size => 10,:level => :l)
puts Time.now
return qrcode
end
end
EventMachine::run {
EventMachine::start_server("0.0.0.0", 8081, Handler)
puts "Listening..."
}
Output:
2015-05-12 18:03:38 +0300
2015-05-12 18:03:48 +0300
It looks like you need to call close_connection. Here's EventMachine's echo server example
require 'eventmachine'
module EchoServer
def post_init
puts "-- someone connected to the echo server!"
end
def receive_data data
send_data ">>>you sent: #{data}"
close_connection if data =~ /quit/i # <---- This is what you're missing -----
end
def unbind
puts "-- someone disconnected from the echo server!"
end
end
# Note that this will block current thread.
EventMachine.run {
EventMachine.start_server "127.0.0.1", 8081, EchoServer
}
If your timestamps are always (nearly) exactly 10 seconds apart, I'd wager the request is ending because of a timeout rather than because of a response. After you respond with the QR data, call close_connection to let the browser know you're finished properly.
Edit It also looks like you're not calling send_data (and referencing Tume instead of Time). It looks like you need to use send_data, not return. Is your code snippet what's actually running on your server?
Update Running in debug mode was significantly slowing down QR generation.
The problem was that I've run server in debug-mode from RubyMine. Starting server from console solves the problem, qr-code generation takes about 1 sec.
I'm trying to build a websocket server where each client establish its own redis connections used for publish and subscribe.
When the redis server is running I can see the two new connections being established when a client connects to the websocket server and I can also publish data to the client, but when the client drops the connection to the websocket server I also want to disconnect from Redis . How can I do this?
Maybe I'm doing it wrong, but this is my code.
#require 'redis'
require 'em-websocket'
require 'em-hiredis'
require 'json'
CLIENTS = Hash.new
class PubSub
def initialize(client)
#socket = client.ws
# These clients can only be used for pub sub commands
#publisher = EM::Hiredis.connect #Later I will like to disconnect this
#subscriber = EM::Hiredis.connect #Later I will like to disconnect this
client.connections << #publisher << #subscriber
end
def subscribe(channel)
#channel = channel
#subscriber.subscribe(channel)
#subscriber.on(:message) { |chan, message|
#socket.send message
}
end
def publish(channel,msg)
#publisher.publish(channel, msg).errback { |e|
puts [:publisherror, e]
}
end
def unsubscribe()
#subscriber.unsubscribe(#channel)
end
end
class Client
attr_accessor :connections, :ws
def initialize(ws)
#connections = []
#ws = ws
end
end
EventMachine.run do
# Creates a websocket listener
EventMachine::WebSocket.start(:host => '0.0.0.0', :port => 8081) do |ws|
ws.onopen do
# I instantiated above
puts 'CLient connected. Creating socket'
#client = Client.new(ws)
CLIENTS[ws] = #client
end
ws.onclose do
# Upon the close of the connection I remove it from my list of running sockets
puts 'Client disconnected. Closing socket'
#client.connections.each do |con|
#do something to disconnect from redis
end
CLIENTS.delete ws
end
ws.onmessage { |msg|
puts "Received message: #{msg}"
result = JSON.parse(msg)
if result.has_key? 'channel'
ps = PubSub.new(#client)
ps.subscribe(result['channel'])
elsif result.has_key? 'publish'
ps = PubSub.new(ws)
ps.publish(result['publish']['channel'],result['publish']['msg']);
end
}
end
end
This version of em-hiredis supports close connection: https://github.com/whatupdave/em-hiredis
Here is how I would (and did many times) this:
instead of always opening and closing connections for each client you can keep 1 connection open per Thread/Fiber dependeing on what you are basing your concurrency on, that way if you are using a poll of Thread/Fibers once each one of them have its connections they will keep it and reuse them.
I did not worked much with websocket until now (I was waiting for a standard implementation) but I am sure you can apply that thinking to it too.
You can also do what rails/activerecord: keeo a pool of redis connection, each time you need to use a connection you request one, use it and realease it, it could look like this:
def handle_request(request)
#redis_pool.get_connection do |c|
# [...]
end
end
before yielding the block a connection is taken from the available ones and after it the connection is marked as free.
This was added to em-hiredis: https://github.com/mloughran/em-hiredis/pull/6
I have an eventmachine app where one script is reading from a file, sending data to another script line by line, and the "server" script is acting upon that data. Unfortunately, the "server" script fails to execute receive_data as it should. I know that a connection is being made because it eecutes post_init, and I know the sender script is sending data. Here is some of my code along with how I start the server.
module BT_Server
def post_init
puts "-- someone connected to the echo server!"
end
def receive_data(data)
puts "hi"
int, time, *int_macs = data.split("-")
# more stuff that isn't needed here
end
def bt_left(dev)
dev.save
if t = Device.macs.index(dev.mac)
Device.all[t].add_int(dev.int, dev.t_0, dev.t_l)
else
Device.new(dev.mac, dev.int, dev.t_0, dev.t_l)
end
return false
end
def unbind
puts "disconnection"
end
end
EventMachine::run {
EventMachine::start_server 'localhost', 8081, BT_Server
puts t_0 = Time.new
puts 'listening...'
}
Note: I have the Module definition in a separate file, along with my classes, which I require into the server script, if that makes any difference.
i tested your code and it outputs 'hi' every time i send something via telnet.
from my point of view, the code is correct.
are you sure the sending script is working? try with a manual telnet on port 8081.
regards.
I am writing a ruby script to be used as Postfix SMTP access policy delegation. The script needs to access a Tokyo Tyrant database. I am using EventMachine to take care of network connections. EventMachine needs a EventMachine::Connection class that is instantiated by EventMachine‘s processing loop whenever a new connection is created. so for each connection a class is instantiated and destroyed.
I am creating a connection to Tokyo Tyrant from the post_init of the EventMachine::Connection (ie right after connection is setup) and tearing it down after connection is terminated.
My question is if this is the proper way to connect to db? ie making a connection every yime I need it and tearing it down after I am finished? Wouldn't be better to connect to DB once (when program is started) tear it down during program shutdown? If that is so how should I code that ?
My code is:
require 'rubygems'
require 'eventmachine'
require 'rufus/tokyo/tyrant'
class LineCounter < EM::Connection
ActionAllow = "action=dunno\n\n"
def post_init
puts "Received a new connection"
#tokyo = Rufus::Tokyo::Tyrant.new('server', 1978)
#data_received = ""
end
def receive_data data
#data_received << data
#data_received.lines do |line|
key = line.split('=')[0]
value = line.split('=')[1]
#reverse_client_name = value.strip() if key == 'reverse_client_name'
#client_address = value.strip() if key == 'client_address'
#tokyo[#client_address] = #reverse_client_name
end
puts #client_address, #reverse_client_name
send_data ActionAllow
end
def unbind
#tokyo.close
end
end
EventMachine::run {
host,port = "127.0.0.1", 9997
EventMachine::start_server host, port, LineCounter
puts "Now accepting connections on address #{host}, port #{port}..."
EventMachine::add_periodic_timer( 10 ) { $stderr.write "*" }
}
with regards,
raj
Surprising there's no answers to this question.
What you probably need is a connection pool where you can fetch, use, and return connections as they are required.
class ConnectionPool
def initialize(&block)
#pool = [ ]
#generator = block
end
def fetch
#pool.shift or #generator and #generator.call
end
def release(handle)
#pool.push(handle)
end
def use
if (block_given?)
handle = fetch
yield(handle)
release(handle)
end
end
end
# Declare a pool with an appropriate connection generator
tokyo_pool = ConnectionPool.new do
Rufus::Tokyo::Tyrant.new('server', 1978)
end
# Fetch/Release cycle
tokyo = tokyo_pool.fetch
tokyo[#client_address] = #reverse_client_name
tokyo_pool.release(tokyo)
# Simple block-method for use
tokyo_pool.use do |tokyo|
tokyo[#client_address] = #reverse_client_name
end