I'm reading lines of input on a TCP socket, similar to this:
class Bla
def getcmd
#sock.gets unless #sock.closed?
end
def start
srv = TCPServer.new(5000)
#sock = srv.accept
while ! #sock.closed?
ans = getcmd
end
end
end
If the endpoint terminates the connection while getline() is running then gets() hangs.
How can I work around this? Is it necessary to do non-blocking or timed I/O?
You can use select to see whether you can safely gets from the socket, see following implementation of a TCPServer using this technique.
require 'socket'
host, port = 'localhost', 7000
TCPServer.open(host, port) do |server|
while client = server.accept
readfds = true
got = nil
begin
readfds, writefds, exceptfds = select([client], nil, nil, 0.1)
p :r => readfds, :w => writefds, :e => exceptfds
if readfds
got = client.gets
p got
end
end while got
end
end
And here a client that tries to break the server:
require 'socket'
host, port = 'localhost', 7000
TCPSocket.open(host, port) do |socket|
socket.puts "Hey there"
socket.write 'he'
socket.flush
socket.close
end
The IO#closed? returns true when both reader and writer are closed.
In your case, the #sock.gets returns nil, and then you call the getcmd again, and this runs in a never ending loop. You can either use select, or close the socket when gets returns nil.
I recommend using readpartial to read from your socket and also catching peer resets:
while true
sockets_ready = select(#sockets, nil, nil, nil)
if sockets_ready != nil
sockets_ready[0].each do |socket|
begin
if (socket == #server_socket)
# puts "Connection accepted!"
#sockets << #server_socket.accept
else
# Received something on a client socket
if socket.eof?
# puts "Disconnect!"
socket.close
#sockets.delete(socket)
else
data = ""
recv_length = 256
while (tmp = socket.readpartial(recv_length))
data += tmp
break if (!socket.ready?)
end
listen socket, data
end
end
rescue Exception => exception
case exception
when Errno::ECONNRESET,Errno::ECONNABORTED,Errno::ETIMEDOUT
# puts "Socket: #{exception.class}"
#sockets.delete(socket)
else
raise exception
end
end
end
end
end
This code borrows heavily from some nice IBM code by M. Tim Jones. Note that #server_socket is initialized by:
#server_socket = TCPServer.open(port)
#sockets is just an array of sockets.
I simply pgrep "ruby" to find the pid, and kill -9 the pid and restart.
If you believe the rdoc for ruby sockets, they don't implement gets. This leads me to believe gets is being provided by a higher level of abstraction (maybe the IO libraries?) and probably isn't aware of socket-specific things like 'connection closed.'
Try using recvfrom instead of gets
Related
I am working on a project where I have implemented a TCP client server for a device communication. In order to send a command from the server to the client, I am building a command that the device understands and sending to it but the response is not what should be returned
while 1
Thread.start(#otd.accept) do |client|
loop do
command_to_send ="<R-2,3,4>"
client.puts command_to_send
puts "Command #{command_to_send}sent"
#sleep 2
response = **client.gets** # here it halts and never puts the the next statement.
puts "Reponse #{response}"
end # end of nested loop
client.close
end #END OF THREAD.
end #end of while loop
Can someone tell me what I am missing?
Do not use gets as it expects '\n' to be a delimiter of the message.
Instead use: recv here is a method that could help you:
def read(timeout = 2, buffer = 1024)
message = ''
begin
Timeout::timeout(timeout) do
buffer = client.recv(buffer)
message += buffer
end
rescue Timeout::Error
puts "Received nothing from client: #{client.__id__}"
message = ''
rescue Exception => e
raise "Client failed to read for reason - #{e.message}"
end
message
end
You do not have to use sleep anymore as recv like gets is blocking. But the timeout makes sure you are not stuck waiting for a response not existing.
I have been trying to get port forwarding to work correctly with Net::SSH. From what I understand I need to fork out the Net::SSH session if I want to be able to use it from the same Ruby program so that the event handling loop can actually process packets being sent through the connection. However, this results in the ugliness you can see in the following:
#!/usr/bin/env ruby -w
require 'net/ssh'
require 'httparty'
require 'socket'
include Process
log = Logger.new(STDOUT)
log.level = Logger::DEBUG
local_port = 2006
child_socket, parent_socket = Socket.pair(:UNIX, :DGRAM, 0)
maxlen = 1000
hostname = "www.example.com"
pid = fork do
parent_socket.close
Net::SSH.start("hostname", "username") do |session|
session.logger = log
session.logger.sev_threshold=Logger::Severity::DEBUG
session.forward.local(local_port, hostname, 80)
child_socket.send("ready", 0)
pidi = fork do
msg = child_socket.recv(maxlen)
puts "Message from parent was: #{msg}"
exit
end
session.loop do
status = waitpid(pidi, Process::WNOHANG)
puts "Status: #{status.inspect}"
status.nil?
end
end
end
child_socket.close
puts "Message from child: #{parent_socket.recv(maxlen)}"
resp = HTTParty.post("http://localhost:#{local_port}/", :headers => { "Host" => hostname } )
# the write cannot be the last statement, otherwise the child pid could end up
# not receiving it
parent_socket.write("done")
puts resp.inspect
Can anybody show me a more elegant/better working solution to this?
I spend a lot of time trying to figure out how to correctly implement port forwarding, then I took inspiration from net/ssh/gateway library. I needed a robust solution that works after various possible connection errors. This is what I'm using now, hope it helps:
require 'net/ssh'
ssh_options = ['host', 'login', :password => 'password']
tunnel_port = 2222
begin
run_tunnel_thread = true
tunnel_mutex = Mutex.new
ssh = Net::SSH.start *ssh_options
tunnel_thread = Thread.new do
begin
while run_tunnel_thread do
tunnel_mutex.synchronize { ssh.process 0.01 }
Thread.pass
end
rescue => exc
puts "tunnel thread error: #{exc.message}"
end
end
tunnel_mutex.synchronize do
ssh.forward.local tunnel_port, 'tunnel_host', 22
end
begin
ssh_tunnel = Net::SSH.start 'localhost', 'tunnel_login', :password => 'tunnel_password', :port => tunnel_port
puts ssh_tunnel.exec! 'date'
rescue => exc
puts "tunnel connection error: #{exc.message}"
ensure
ssh_tunnel.close if ssh_tunnel
end
tunnel_mutex.synchronize do
ssh.forward.cancel_local tunnel_port
end
rescue => exc
puts "tunnel error: #{exc.message}"
ensure
run_tunnel_thread = false
tunnel_thread.join if tunnel_thread
ssh.close if ssh
end
That's just how SSH in general is. If you're offended by how ugly it looks, you should probably wrap up that functionality into a port forwarding class of some sort so that the exposed part is a lot more succinct. An interface like this, perhaps:
forwarder = PortForwarder.new(8080, 'remote.host', 80)
So I have found a slightly better implementation. It only requires a single fork but still uses a socket for the communication. It uses IO#read_nonblock for checking if a message is ready. If there isn't one, the method throws an exception, in which case the block continues to return true and the SSH session keeps serving requests. Once the parent is done with the connection it sends a message, which causes child_socket.read_nonblock(maxlen).nil? to return false, making the loop exit and therefore shutting down the SSH connection.
I feel a little better about this, so between that and #tadman's suggestion to wrap it in a port forwarding class I think it's about as good as it'll get. However, any further suggestions for improving this are most welcome.
#!/usr/bin/env ruby -w
require 'net/ssh'
require 'httparty'
require 'socket'
log = Logger.new(STDOUT)
log.level = Logger::DEBUG
local_port = 2006
child_socket, parent_socket = Socket.pair(:UNIX, :DGRAM, 0)
maxlen = 1000
hostname = "www.example.com"
pid = fork do
parent_socket.close
Net::SSH.start("ssh-tunnel-hostname", "username") do |session|
session.logger = log
session.logger.sev_threshold=Logger::Severity::DEBUG
session.forward.local(local_port, hostname, 80)
child_socket.send("ready", 0)
session.loop { child_socket.read_nonblock(maxlen).nil? rescue true }
end
end
child_socket.close
puts "Message from child: #{parent_socket.recv(maxlen)}"
resp = HTTParty.post("http://localhost:#{local_port}/", :headers => { "Host" => hostname } )
# the write cannot be the last statement, otherwise the child pid could end up
# not receiving it
parent_socket.write("done")
puts resp.inspect
I have a 2 part question on reading from sockets and how is it managed on Ruby servers like Unicorn or Mongrel
I've learnt that to read from a socket is different from reading a file and that there are no distinct EOF message sent and the data is an endless stream. So how do you know when to stop reading? My TCPServer for example in this case when I hit my server by accessing http://localhost:9799 from a browser, it hangs after there is no more data to read and it won't throw the EOFError either.
require 'socket'
READ_CHUNK = 1024
socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM)
addr = Socket.pack_sockaddr_in(9799, '127.0.0.1')
socket.bind(addr)
socket.listen(Socket::SOMAXCONN)
socket.setsockopt(:SOCKET, :REUSEADDR, true)
puts "Server is listening on port = 9799"
loop do
connection, addr_info = socket.accept
data_buffer = ""
loop do
begin
connection.read_nonblock(READ_CHUNK, data_buffer)
puts "Buffer = #{data_buffer}"
rescue Errno::EAGAIN => e
IO.select([connection])
retry
rescue EOFError
break
end
end
connection.write("HTTP/1.1 200 \r\n")
connection.write("Content-Type: text/html\r\n")
connection.write("Status 200 \r\n")
connection.write("Connection: close \r\n")
connection.write("Hello World \r\n")
connection.close
end
I'd like to know whats the best practice/standard approach used by Ruby servers. I see the Unicorn uses read_nonblock from kgio library and mongrel uses readpartial (I'm not sure about these but going through the code this is what I feel is the approach adopted.) Even with checks for \r\n how does the server know the input is complete.
Could explain how this should be done (and I think gets is not the approach - its with read, readpartial, read_nonblock).
2). I would really appreciate a few lines on how this is achieved in servers like unicorn or passenger
Thank you.
It's done in unicorn here
https://github.com/defunkt/unicorn/blob/master/lib/unicorn/http_request.rb#L69-L71
There is add_parse method(read the comments above methods)
https://github.com/defunkt/unicorn/blob/master/ext/unicorn_http/unicorn_http.rl#L760-L778
Also take a look at some explanations here http://www.ruby-forum.com/topic/2267632#1014288
Here is your working code using http_parser.rb https://gist.github.com/4136962
gem install http_parser.rb
require 'socket'
require "http/parser"
READ_CHUNK = 1024 * 4
socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM)
addr = Socket.pack_sockaddr_in(9799, '127.0.0.1')
socket.bind(addr)
socket.listen(Socket::SOMAXCONN)
socket.setsockopt(:SOCKET, :REUSEADDR, true)
puts "Server is listening on port = 9799"
loop do
connection, addr_info = socket.accept
parser = Http::Parser.new
begin
data = connection.readpartial(READ_CHUNK)
puts "Buffer = #{data}"
parser << data
end until parser.headers
connection.write("HTTP/1.1 200 \r\n")
connection.write("Content-Type: text/html\r\n")
connection.write("Status 200 \r\n")
connection.write("Connection: close \r\n")
connection.write("\r\n\r\n")
connection.write("Hello World \r\n")
connection.close
end
How can I detect that a socket is half-open? The case I'm dealing with is when the other side of a socket has sent a FIN and the Ruby app has ACKed that FIN. Is there a way for me to tell that the socket is in this condition?
Take, for example:
require 'socket'
s = TCPServer.new('0.0.0.0', 5010)
loop do
c = s.accept
until c.closed?
p c.recv(1024)
end
end
In this case, when I telnet into port 5010, I'll see all my input until I close the telnet session. At that point, it will print empty strings over and over as fast as it can.
You are using the blocking call recv, which will return nil when the other end closes. The socket won't be closed until you close it. Change
until c.closed?
p c.recv(1024)
end
to
while (s = c.recv(1024)) && s > 0
p s
end
c.close
You could combine IO#read and IO#eof? to check this.
require 'socket'
server = TCPServer.new('0.0.0.0', 5010)
loop do
client = server.accept
client.read(1024) until client.eof?
puts 'client closed connection'
client.close
end
How do you set the timeout for blocking operations on a Ruby socket?
The solution I found which appears to work is to use Timeout::timeout:
require 'timeout'
...
begin
timeout(5) do
message, client_address = some_socket.recvfrom(1024)
end
rescue Timeout::Error
puts "Timed out!"
end
The timeout object is a good solution.
This is an example of asynchronous I/O (non-blocking in nature and occurs asynchronously to
the flow of the application.)
IO.select(read_array
[, write_array
[, error_array
[, timeout]]] ) => array or nil
Can be used to get the same effect.
require 'socket'
strmSock1 = TCPSocket::new( "www.dn.se", 80 )
strmSock2 = TCPSocket::new( "www.svd.se", 80 )
# Block until one or more events are received
#result = select( [strmSock1, strmSock2, STDIN], nil, nil )
timeout=5
timeout=100
result = select( [strmSock1, strmSock2], nil, nil,timeout )
puts result.inspect
if result
for inp in result[0]
if inp == strmSock1 then
# data avail on strmSock1
puts "data avail on strmSock1"
elsif inp == strmSock2 then
# data avail on strmSock2
puts "data avail on strmSock2"
elsif inp == STDIN
# data avail on STDIN
puts "data avail on STDIN"
end
end
end
I think the non blocking approach is the way to go.
I tried the mentioned above article and could still get it to hang.
this article non blocking networking and the jonke's approach above got me on the right path. My server was blocking on the initial connect so I needed it to be a little lower level.
the socket rdoc can give more details into the connect_nonblock
def self.open(host, port, timeout=10)
addr = Socket.getaddrinfo(host, nil)
sock = Socket.new(Socket.const_get(addr[0][0]), Socket::SOCK_STREAM, 0)
begin
sock.connect_nonblock(Socket.pack_sockaddr_in(port, addr[0][3]))
rescue Errno::EINPROGRESS
resp = IO.select([sock],nil, nil, timeout.to_i)
if resp.nil?
raise Errno::ECONNREFUSED
end
begin
sock.connect_nonblock(Socket.pack_sockaddr_in(port, addr[0][3]))
rescue Errno::EISCONN
end
end
sock
end
to get a good test. startup a simple socket server and then do a ctrl-z to background it
the IO.select is expecting data to come in on the input stream within 10 seconds. this may not work if that is not the case.
It should be a good replacement for the TCPSocket's open method.