I am new to ruby and thought it would be a great idea to rebuild a simple chat program I made in C#.
I am using Ruby 2.0.0 MRI (Matz’s Ruby Implementation).
The problem is I want to have I/O for simple server commands while the server is running.
This is the server that was taken from the sample. I added the commands method that uses gets() to get input. I want this method to run as a thread in the background, but the thread is blocking the other thread.
require 'socket' # Get sockets from stdlib
server = TCPServer.open(2000) # Socket to listen on port 2000
def commands
x = 1
while x == 1
exitProgram = gets.chomp
if exitProgram == "exit" || exitProgram == "Exit"
x = 2
abort("Exiting the program.")
end
end
end
def main
Thread.start(commands)
Thread.start(server.accept)
loop { # Servers run forever
Thread.start(server.accept) do |client|
client.puts(Time.now.ctime) # Send the time to the client
client.puts "Closing the connection. Bye!"
client.close # Disconnect from the client
end
}
end
main
This is the client so far.
require 'socket' # Sockets are in standard library
hostname = 'localhost'
port = 2000
s = TCPSocket.open(hostname, port)
while line = s.gets # Read lines from the socket
puts line.chop # And print with platform line terminator
end
s.close # Close the socket when done
gets.chomp
Read the documentation for Thread.new (which is the same as Thread.start here)
Thread.start(commands) runs the commands method and passes its return value to a thread (which then does nothing). It's blocking because you aren't starting any threads when gets is called. You want
Thread.start { commands }
Here's a similar demo script that works just like you would expect
def commands
while gets.strip !~ /^exit$/i
puts "Invalid command"
end
abort "Exiting the program"
end
Thread.start { commands }
loop do
puts "Type exit:"
sleep 2
end
Related
I am writing a headless Ruby application using EventMachine that communicates over sockets. I want to write some unit tests for this app. This means that my Ruby test script needs to launch that app in the background, perform socket communication with it, and then close that process.
This code fails. The socket connection is refused.
require 'socket'
PORT = 7331
CMD = File.expand_path("../../bin/rb3jay",__FILE__)
#thread = Thread.new{ `#{CMD} -D --port #{PORT}` }
#socket = TCPSocket.open('localhost', PORT)
#=> Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 7331
If I inject a 2 second delay before attempting the socket connection, it works as desired:
#thread = Thread.new{ `#{CMD} -D --port #{PORT}` }
sleep 2
#socket = TCPSocket.open('localhost', PORT)
This seems a gross hack. Maybe 2 seconds is long enough for my machine, but too short somewhere else.
How should I correctly launch my EventMachine application in the background, and create a socket connection to it as soon as it is ready?
Not sure if there's a better way, but I solved this by using retry:
#thread = Thread.new{ `#{CMD} -D --port #{PORT}` }
begin
#socket = TCPSocket.open('localhost', PORT)
rescue Errno::ECONNREFUSED
sleep 0.1
retry
end
This will indefinitely keep trying to establish the connection 10 times a second until it works. A more robust solution might be to use a counter or timer to eventually give up, just in case something is seriously awry.
The full test code looks like this:
require 'socket'
require 'minitest/autorun'
PORT = 7331
CMD = File.expand_path("../../bin/rb3jay",__FILE__)
class TestServer < MiniTest::Unit::TestCase
def setup
#pid = Process.spawn "#{CMD} -D --port #{PORT}"
begin
#socket = TCPSocket.open('localhost', PORT)
rescue Errno::ECONNREFUSED
sleep 0.1
retry
end
end
def teardown
if #socket && !#socket.closed?
#socket.puts("quit") # try for a nice shutdown
#socket.close
end
Process.kill("HUP",#pid)
end
def test_aaa
# my test code
end
def test_bbb
# more test code
end
end
The problem is that from the main thread you can't know when Thread.new code block has actually been executed. By using sleep, you just give it enough time so it is executed.
In such cases, I prefer to use a Queue, where the Thread.new block does a push (usually a nil) into it after it has finished doing what it's supposed to do, while the thread that used to sleep, does a pop from it. pop waits until there are data available in the Queue.
require 'socket'
PORT = 7331
CMD = File.expand_path("../../bin/rb3jay",__FILE__)
q = Queue.new
#thread = Thread.new do
Process.spawn "#{CMD} -D --port #{PORT}"
q.push(nil)
end
q.pop
#socket = TCPSocket.open('localhost', PORT)
However, you might face problems, since spawning your command doesn't mean that the server is actually ready (listening for new connections). So, I'd try an approach were I could have more control over the life-cycle of the server.
require 'socket'
PORT = 7331
q = Queue.new
#thread = Thread.new do
server = TCPServer.new PORT
q.push(nil)
loop do
client = server.accept
client.puts "Hello !"
client.puts "Time is #{Time.now}"
client.close
end
end
q.pop
#socket = TCPSocket.open('localhost', PORT)
I have been searching the whole day for socket accept non blocking. I found recv non blocking but that wouldn't benefit me in anyway. My script first starts a new socket class. It binds to the client with ip 127.0.0.1 and port 6112. Then it starts multi threading. Multi threading takes #sock.accept. << That is blocking. I have then used accept_nonblock. Though, that would throw me the following error:
IO::EWOULDBLOCKWaitReadable : A non-blocking socket operation could not be completed immediately. - accept(2) would block
I am using Ruby 2.2.
NOTE: I do not intend to use Rails to solve my problem, or give me a shortcut. I am sticking with pure Ruby 2.2.
Here is my script:
require 'socket'
include Socket::Constants
#sock = Socket.new(AF_INET, SOCK_STREAM, 0)
#sockaddr = Socket.sockaddr_in(6112, '127.0.0.1')
#sock.bind(#sockaddr)
#sock.listen(5)
Thread.new(#sock.accept_nonblock) do |connection|
#client = Client.new(ip, connection, self)
#clients.push(#client)
begin
while connection
packet = connection.recv(55555)
if packet == nil
DeleteClient(connection)
else
#toput = "[RECV]: #{packet}"
puts #toput
end
end
rescue Exception => e
if e.class != IOError
line1 = e.backtrace[0].split(".rb").last
line = line1.split(":")[1]
#Log.Error(e, e.class, e.backtrace[0].split(".rb").first + ".rb",line)
puts "#{ e } (#{ e.class })"
end
end
def DeleteClient(connection)
#clients.delete(#client)
connection.close
end
http://docs.ruby-lang.org/en/2.2.0/Socket.html#method-i-accept_nonblock
accept_nonblock raises an exception when it can't immediately accept a connection. You are expected to rescue this exception and then IO.select the socket.
begin # emulate blocking accept
client_socket, client_addrinfo = socket.accept_nonblock
rescue IO::WaitReadable, Errno::EINTR
IO.select([socket])
retry
end
A patch has recently been accepted which will add an exception: false option to accept_nonblock, which will allow you to use it without using exceptions for flow control. I don't know that it's shipped yet, though.
I'm going on a limb here, and posting a large chunk of code.
I hope it will answer both your question and the any related questions others reading this answer might raise.
I'm sorry if I went overboard, I just thought it was almost all relevant.
Issues like looping through an event stack, using IO.select to push events in a non-block manner and other performance issues are all related (in my opinion) to the nonblocking concept of socket programming.
So i'm posting a ruby module which acts as a server with a reactor, using a limited number of threads, rather than thousands of threads, each per connection (12 threads will give you better performance than a hundred). The reactor utilizes the IO.select method with a timeout once all it's active events are handled.
The module can set up multiple listening sockets which use #accept_nonblock, and they all currently act as an echo server.
It's basically the same code I used for the Plezi framework's core... with some stripped down functionality.
The following is a thread-pool with 12 working threads + the main thread (which will sleep and wait for the "TERM" signal)...
...And it's an example of an accept_nonblock with exception handling and a thread pool.
It's a simple socket echo server, test it as a remote client using telnet:
> telnet localhost 3000
Hi!
# => Hi!
bye
#=> will disconnect
here's the code - Good Luck!!!
require 'socket'
module SmallServer
module_function
####
# Replace this method with your actual server logic.
#
# this code will be called when a socket recieves data.
#
# For now, we will just echo.
def got_data io, io_params
begin
got = io.recv_nonblock( 1048576 ) # with maximum number of bytes to read at a time...
puts "echoing: #{got}"
if got.match /^(exit|bye|q)\R/
puts 'closing connection.'
io.puts "bye bye!"
remove_connection io
else
io.puts "echoing: #{got}"
end
rescue => e
# should also log error
remove_connection io
end
end
#########
# main loop and activation code
#
# This will create a thread pool and set them running.
def start
# prepare threads
exit_flag = false
max_threads = 12
threads = []
thread_cycle = Proc.new do
io_review rescue false
true while fire_event
end
(max_threads).times { Thread.new { thread_cycle.call until exit_flag } }
# set signal tarps
trap('INT'){ exit_flag = true; raise "close!" }
trap('TERM'){ exit_flag = true; raise "close!" }
puts "Services running. Press ^C to stop"
# sleep until trap raises exception (cycling might cause the main thread to loose signals that might be caught inside rescue clauses)
(sleep unless SERVICES.empty?) rescue true
# start shutdown.
exit_flag = true
# set fallback tarps
trap('INT'){ puts 'Forced exit.'; Kernel.exit }
trap('TERM'){ puts 'Forced exit.'; Kernel.exit }
puts 'Started shutdown process. Press ^C to force quit.'
# shut down listening sockets
stop_services
# disconnect active connections
stop_connections
# cycle down threads
puts "Waiting for workers to cycle down"
threads.each {|t| t.join if t.alive?}
# rundown any active events
thread_cycle.call
end
#######################
## Events (Callbacks) / Multi-tasking Platform
EVENTS = []
E_LOCKER = Mutex.new
# returns true if there are any unhandled events
def events?
E_LOCKER.synchronize {!EVENTS.empty?}
end
# pushes an event to the event's stack
# if a block is passed along, it will be used as a callback: the block will be called with the values returned by the handler's `call` method.
def push_event handler, *args, &block
if block
E_LOCKER.synchronize {EVENTS << [(Proc.new {|a| push_event block, handler.call(*a)} ), args]}
else
E_LOCKER.synchronize {EVENTS << [handler, args]}
end
end
# Runs the block asynchronously by pushing it as an event to the event's stack
#
def run_async *args, &block
E_LOCKER.synchronize {EVENTS << [ block, args ]} if block
!block.nil?
end
# creates an asynchronous call to a method, with an optional callback (shortcut)
def callback object, method, *args, &block
push_event object.method(method), *args, &block
end
# event handling FIFO
def fire_event
event = E_LOCKER.synchronize {EVENTS.shift}
return false unless event
begin
event[0].call(*event[1])
rescue OpenSSL::SSL::SSLError => e
puts "SSL Bump - SSL Certificate refused?"
rescue Exception => e
raise if e.is_a?(SignalException) || e.is_a?(SystemExit)
error e
end
true
end
#####
# Reactor
#
# IO review code will review the connections and sockets
# it will accept new connections and react to socket input
IO_LOCKER = Mutex.new
def io_review
IO_LOCKER.synchronize do
return false unless EVENTS.empty?
united = SERVICES.keys + IO_CONNECTION_DIC.keys
return false if united.empty?
io_r = (IO.select(united, nil, united, 0.1) )
if io_r
io_r[0].each do |io|
if SERVICES[io]
begin
callback self, :add_connection, io.accept_nonblock, SERVICES[io]
rescue Errno::EWOULDBLOCK => e
rescue => e
# log
end
elsif IO_CONNECTION_DIC[io]
callback(self, :got_data, io, IO_CONNECTION_DIC[io] )
else
puts "what?!"
remove_connection(io)
SERVICES.delete(io)
end
end
io_r[2].each { |io| (remove_connection(io) || SERVICES.delete(io)).close rescue true }
end
end
callback self, :clear_connections
true
end
#######################
# IO - listening sockets (services)
SERVICES = {}
S_LOCKER = Mutex.new
def add_service port = 3000, parameters = {}
parameters[:port] ||= port
parameters.update port if port.is_a?(Hash)
service = TCPServer.new(parameters[:port])
S_LOCKER.synchronize {SERVICES[service] = parameters}
callback Kernel, :puts, "Started listening on port #{port}."
true
end
def stop_services
puts 'Stopping services'
S_LOCKER.synchronize {SERVICES.each {|s, p| (s.close rescue true); puts "Stoped listening on port #{p[:port]}"}; SERVICES.clear }
end
#####################
# IO - Active connections handling
IO_CONNECTION_DIC = {}
C_LOCKER = Mutex.new
def stop_connections
C_LOCKER.synchronize {IO_CONNECTION_DIC.each {|io, params| io.close rescue true} ; IO_CONNECTION_DIC.clear}
end
def add_connection io, more_data
C_LOCKER.synchronize {IO_CONNECTION_DIC[io] = more_data} if io
end
def remove_connection io
C_LOCKER.synchronize { IO_CONNECTION_DIC.delete io; io.close rescue true }
end
# clears closed connections from the stack
def clear_connections
C_LOCKER.synchronize { IO_CONNECTION_DIC.delete_if {|c| c.closed? } }
end
end
start the echo server in irb with:
SmallServer.add_service(3000) ; SmallServer.start
I wrote a TCP server using below code. This is to receive GPS location data via GSM network from a remote GPS sensor.
require 'socket'
server = TCPServer.open(2000) # Listen on port 2000
sockets = [server] # An array of sockets we'll monitor
log = STDOUT # Send log messages to standard out
while true
ready = select(sockets)
readable = ready[0]
readable.each do |socket|
if socket == server
client = server.accept
sockets << client
log.puts "Accepted connection from #{client.peeraddr[2]}"
while msg = client.gets
puts msg
end
else
input = socket.gets
if !input
log.puts "Client on #{socket.peeraddr[2]} disconnected"
sockets.delete(socket)
socket.close
next
end
input.chop!
if (input == "quit")
socket.puts("Bye");
log.puts "Closing connnection to #{socket.peeraddr[2]}"
sockets.delete(socket)
socket.close
else
socket.puts(input.reverse)
end
end
end
end
and then I wrote one using Eventmachine. Code as below:
require 'eventmachine'
module EchoServer
def post_init
puts "-- someone connected to the echo server!"
end
def receive_data data
puts data
end
def unbind
puts "-- someone disconnected from the echo server!"
end
end
EventMachine::run {
EventMachine::start_server "127.0.0.1", 2000, EchoServer
}
However, this eventmachine code will not receive nor display the data. Any part of the Eventmachine code that is wrong?
Thanks
I think your problem is that you are listening on localhost only, try this:
EM::run do
EM.start_server "0.0.0.0", 2000, EchoServer
end
I have an eventmachine app where one script is reading from a file, sending data to another script line by line, and the "server" script is acting upon that data. Unfortunately, the "server" script fails to execute receive_data as it should. I know that a connection is being made because it eecutes post_init, and I know the sender script is sending data. Here is some of my code along with how I start the server.
module BT_Server
def post_init
puts "-- someone connected to the echo server!"
end
def receive_data(data)
puts "hi"
int, time, *int_macs = data.split("-")
# more stuff that isn't needed here
end
def bt_left(dev)
dev.save
if t = Device.macs.index(dev.mac)
Device.all[t].add_int(dev.int, dev.t_0, dev.t_l)
else
Device.new(dev.mac, dev.int, dev.t_0, dev.t_l)
end
return false
end
def unbind
puts "disconnection"
end
end
EventMachine::run {
EventMachine::start_server 'localhost', 8081, BT_Server
puts t_0 = Time.new
puts 'listening...'
}
Note: I have the Module definition in a separate file, along with my classes, which I require into the server script, if that makes any difference.
i tested your code and it outputs 'hi' every time i send something via telnet.
from my point of view, the code is correct.
are you sure the sending script is working? try with a manual telnet on port 8081.
regards.
This is related to a question I asked here:
Thread Locking in Ruby (use of soap4r and QT)
However it is particular to one part of that question and is supported by a simpler example. The test code is:
require 'rubygems'
require 'thread'
require 'soap/rpc/standaloneserver'
class SOAPServer < SOAP::RPC::StandaloneServer
def initialize(* args)
super
# Exposed methods
add_method(self, 'test', 'x', 'y')
end
def test(x, y)
return x + y
end
end
myServer = SOAPServer.new('monitorservice', 'urn:ruby:MonitorService', 'localhost', 4004)
Thread.new do
puts 'Starting web services'
myServer.start
puts 'Ending web services'
end
sleep(4)
#Thread.new do
testnum = 0
while testnum < 4000 do
testnum += 1
puts myServer.test(0,testnum)
sleep(2)
end
#end
puts myServer.test(0,4001)
puts myServer.test(0,4002)
puts myServer.test(0,4003)
puts myServer.test(0,4004)
gets
When I run this with the thread commented out everything runs along fine. However, once the thread is put in the process hangs. I poked into Webrick and found that the stop occurs here (the puts are, of course, mine):
while #status == :Running
begin
puts "1.1"
if svrs = IO.select(#listeners, nil, nil, 2.0)
svrs[0].each{|svr|
puts "-+-"
#tokens.pop # blocks while no token is there.
if sock = accept_client(svr)
th = start_thread(sock, &block)
th[:WEBrickThread] = true
thgroup.add(th)
else
#tokens.push(nil)
end
}
end
puts ".+."
When run with the thread NOT commented out I get something like this:
Starting web services
1.1
.+.
1.1
4001
4002
4003
4004
1
.+.
1.1
If the problem is caused by the gets() call and the purpose of the gets() call in your code is to prevent the Ruby interpreter from exiting, you can replace it with Thread.join() calls for each thread that you create. Join() will block until that thread has finished executing and therefore it'll prevent the Ruby interpreter from exiting.
E.g.:
t1 = Thread.new do
puts 'Starting web services'
myServer.start
puts 'Ending web services'
end
t2 = ...
...
t1.join
t2.join
Alternatively, if you can join() only one of the threads if there is a single thread that controls the execution of the application, and the other threads will be killed on exit.
The trailing gets blocks Ruby's IO. I'm not sure why. If it is replaced with pretty much anything the program works. I used a sleeping loop:
loop do
sleep 1
end
ADDED:
I should note that I also get strange behavior with sleep based on the sleep increment. In the end I abandoned Ruby since the threading behavior was too wonky.