I've this ruby code that connects to a TCP server (namely, netcat). It loops 20 times, and sends "ABCD ". If I kill netcat, it takes TWO iterations of the loop for an exception to be triggered. On the first loop after netcat is killed, no exception is triggered, and "send" reports that 5 bytes have been correctly written... Which in the end is not true, since of course the server never received them.
Is there a way to work around this issue ? Right now I'm losing data : since I think it's been correctly transfered, I'm not replaying it.
#!/usr/bin/env ruby
require 'rubygems'
require 'socket'
sock = TCPSocket.new('192.168.0.10', 5443)
sock.sync = true
20.times do
sleep 2
begin
count = sock.write("ABCD ")
puts "Wrote #{count} bytes"
rescue Exception => myException
puts "Exception rescued : #{myException}"
end
end
When you're sending data your blocking call will return when the data is written to the TCP output buffer. It would only block if the buffer was full, waiting for the server to acknowledge receipt of previous data that was sent.
Once this data is in the buffer, the network drivers try to send the data. If the connection is lost, on the second attempt to write, your application discovers the broken state of the connection.
Also, how does the connection close? Is the server actively closing the connection? In which case client socket would be notified at its next socket call. Or has it crashed? Or perhaps there's a network fault which means you can no longer communicate.
Discovering a broken connection only occurs when you try to send or receive data over the socket. This is different from having the connection actively closed. You simply can't determine if the connection is still alive without doing something with it.
So try doing sock.recv(0) after the write - if the socket has failed this would raise "Errno::ECONNRESET: Connection reset by peer - recvfrom(2)". You could also try sock.sendmsg "", 0 (not sock.write, or sock.send), and this would report a "Errno::EPIPE: Broken pipe - sendmsg(2)".
Even if you got your hands on the TCP packets and get acknowledgement that the data had been received at the other end, there's still no guarantee that the server will have processed this data - it might in its input buffer but not yet processed.
All of this might help identify a broken connection earlier, but it still won't guarantee that the data was received and processed by the server. The only sure way to know that the application has processed your message is with an application level response.
I tried without the sleep function (just to make sure it wasn't putting on hold anything) and still no luck:
#!/usr/bin/env ruby
require 'rubygems'
require 'socket'
require 'activesupport' # Fixnum.seconds
sock = TCPSocket.new('127.0.0.1', 5443)
sock.sync = true
will_restart_at = Time.now + 2.seconds
should_continue = true
while should_continue
if will_restart_at <= Time.now
will_restart_at = Time.now + 2.seconds
begin
count = sock.write("ABCD ")
puts "Wrote #{count} bytes"
rescue Exception => myException
puts "Exception rescued : #{myException}"
should_continue = false
end
end
end
I analyzed with Wireshark and the two solutions are exactly behaving identically.
I think (and can't be sure) that until you actually call your_socket.write (which will not fail as the socket is still opened because you weren't probing for its possible destruction), the socket won't raise any error.
I tried to simulate this with nginx and manual TCP sockets. And look at that:
irb> sock = TCPSocket.new('127.0.0.1', 80)
=> #<TCPSocket:0xb743b824>
irb> sock.write("salut")
=> 5
irb> sock.read
=> "<html>\r\n<head><title>400 Bad Request</title></head>\r\n<body>\r\n</body>\r\n</html>\r\n"
# Here, I kill nginx
irb> sock.write("salut")
=> 5
irb> sock.read
=> ""
irb> sock.write("salut")
Errno::EPIPE: Broken pipe
So what's the conclusion from here? Unless you're actually expecting some data from the server, you're screwed to detect that you've lost the connection :)
To detect a gracefully close, you'll have to read from the socket - read returning 0 indicates the socket has closed.
If you do need know if data got sent successfully though, there's no way other than implementing ACKs of the data at the application level.
Related
I'm putting together a TCPServer in Ruby 3.0.2 and I'm finding that I can't seem to read the entire packet without blocking (until the socket is closed).
Edit: There was some confusion on what I was trying to do - my bad - so just to help clarify: I wanted to read everything that had been sent over the TCP connection so far. (end edit)
My first try was:
#!/snap/bin/ruby
require 'socket'
server = TCPServer.new('localhost', 4200)
loop {
Thread.start(server.accept) do |connection|
puts connection.gets # The important line
end
}
But that hangs until the client closes the connection. Okay, so I take a look at connection.methods, and the ruby docs and try a bunch of options that seem promising. Basically, there is two types of read methods: blocking and nonblocking.
The blocking methods that I tried are .read, .gets, .readlines, .readline, .recv, and .recvmsg. Now .read, .readlines, and .gets all hang (until the socket is closed) - so that's not helpful. The other ones (eg. .readline, the recv methods) don't read the entire message. Now, I could read each line until I see an empty line and parse the HTTP header from there. But there's got to be a better way; I don't want to have to worry about getting a corrupted message and hanging because I didn't read an empty line at the end of the header.
So I went looking at the non-blocking options. Specifically .recv_nonblock and .recvmsg_nonblock. Both of these throw errors (Resource temporarily unavailable - recvfrom(2) would block and Resource temporarily unavailable - recvmsg(2) respectively).
Any ideas on what could be going on? I think it has something to with me using Ruby 3, because trying out the code on Ruby 2.5, client.gets returns a line (doesn't hang), although .readlines does hang - so not sure what's going on.
Ideally, I could just call something along the lines of client.get_message and I would get the entire message that has been sent, but I'd also be okay with working at the TCP level and getting the packet size, reading that size, and reconstructing the message from there.
TCP just transmits the bytes that you write to the socket, and guarantees that the are received in the order they were sent. If you have the concept of a 'message' then you'll need to add that into your server and client.
.gets specifically will block until it reads a new 'line', or whatever you define as the separator for the string - see the docs IO#gets. This means that until your server receives that byte from the client, it will block.
In your client have a look at how you're writing your data - if you're using ruby then puts would work, as it will terminate the string with a new line. If you're using write then it will only write the string without a new line
Ie.
# client.rb
c = TCPSocket.new 'localhost', 5000
c.puts "foo"
c.write "bar"
c.write "baz\n"
# server.rb
s = TCPServer.new 5000
loop do
client = s.accept
puts client.gets
puts client.gets
end
will output
foo
barbaz
Thanks to everyone who commented/answered, but I found the solution that I think was intended by the creators of the Socket class!
The recv_nonblock method takes some optional arguments - one of which is a buffer that the Socket will store what it has read to. So a call like client.recv_nonblock(1000, 0, buffer) stores up to 1000 characters from the Socket into buffer and then exits instead of blocking.
Just to make life easy, I put together a monkey patch to the TCPSocket class:
class TCPSocket
def eat_buffer
contents = ''
buffer = ''
begin
loop {
recv_nonblock(256, 0, buffer)
contents += buffer
}
rescue IO::EAGAINWaitReadable
contents
end
end
end
The point that Steffen makes in the comments is well taken - TCP isn't designed to be used this way. This is a hacky (in the bad sense) method, and should be avoided.
I've read some things suggesting that because of the design of TCP this might not be possible (such as: Java socket API: How to tell if a connection has been closed?), but I'm trying to find explicit confirmation. I have a basic TCP server that accepts connections, and a client that initiates a connection, sends a message, and then closes the connection. Is there a way for the server to know that the client closed the connection?
I found some suggestions to look into checking the file descriptors for the sockets (source: How to check if a given file descriptor stored in a variable is still valid?), using the kernel select command (source: https://bytes.com/topic/c/answers/866296-detecting-if-file-descriptor-closed) as well as using recv to check if the client returns 0 (source: http://man7.org/linux/man-pages/man2/recv.2.html#RETURN_VALUE), but these do not seem to work, at least not when called by Ruby. To test this, I wrote a basic server and client:
test_server.rb
require 'socket'
require 'fcntl'
TIMEOUT = 5
server = TCPServer.new('localhost', 8080)
puts "Starting server"
loop do
client = server.accept
puts "New client: #{client}"
puts "** before closed #{Time.now.to_i} closed=#{client.closed?}"
result = IO.select([client], nil, nil, TIMEOUT)
puts "select result=#{result}"
fd = client.fcntl(Fcntl::F_GETFD, 0)
puts "client fd=#{fd}"
stuff = client.recv(30)
puts "received '#{stuff}'"
begin
r = client.recv(1)
rescue => e
end
puts "received #{r} nil?=#{r.nil?}"
sleep 3
puts "** after closed #{Time.now.to_i} closed=#{client.closed?}"
result = IO.select([client], nil, nil, TIMEOUT)
puts "select result=#{result}"
fd = client.fcntl(Fcntl::F_GETFD, 0)
puts "client fd=#{fd}"
begin
r = client.recv(1)
rescue => e
end
puts "received #{r} nil?=#{r.nil?}"
puts "done!"
end
test_client.rb
require 'socket'
class Client
def initialize
#socket = tcp_socket
end
def tcp_socket
Thread.current[:socket] = TCPSocket.new("localhost", 8080)
end
def send(s, args={})
puts "sending str '#{s}'"
nbytes = #socket.send(s, 0)
puts "received #{nbytes} bytes"
sleep 1
#socket.close
puts "done at #{Time.now.to_i}: #{#socket.closed?}"
end
end
msg = 'hello world this is my message'
server = Client.new
server.send(msg)
The client sends a 30-byte message, waits 1s, then closes the connection.
The server accepts the connection, calls select and fcntl on it to check its status, receives the message, tries to read 1 more byte, sleeps for 3 seconds, then calls select and fcntl and again tries to read 1 byte. The intent here is to check if anything changes that the server can see before and after the client closed the connection (hence the 3-second sleep). The result I get from running the server and then the client code is:
Starting server
New client: #<TCPSocket:0x00007fa0930f0880>
** before closed 1578005539 closed=false
select result=[[#<TCPSocket:fd 10>], [], []]
client fd=1
received 'hello world this is my message'
received nil?=false
** after closed 1578005543 closed=false
select result=[[#<TCPSocket:fd 10>], [], []]
client fd=1
received nil?=false
done!
Before and after the client closed the connection, select still sees the socket as readable, the underlying file descriptor does not change, and recv returns empty string (It's possible the kernel call is returning 0 as specified in the man-page but Ruby is capturing that, and if so I don't know how to see it.). Thus none of these seem to be a reliable indicator of whether the connection was closed from the other side. Is there something I'm missing?
I have seen some other suggestions to incorporate a regular heartbeat back to the client, but I'm wondering if there's a way to avoid that. Reason is that I'm trying to accommodate a case where the client may be sending a message in several pieces separated by a delay (e.g. 100 bytes at 1 second each byte). If the server sends a heartbeat message in the middle of that operation and listens for an OK, I presume the client has to be listening for the heartbeat as well and send its OK back, separate from the ongoing message send, and in my test case, I can't change the client to do that.
I have seen some other suggestions to incorporate a regular heartbeat back to the client, but I'm wondering if there's a way to avoid that.
A heartbeat (ping) is the only viable solution.
There is no way to reliably know if the connection is live except by trying to send data over the wire.
Since TCP/IP doesn't require any traffic when data isn't being sent (or received), there's no way for the TCP stack (not even in the OS kernel) to know if the connection is "live" without attempting to exchange data over the wire.
Some connections will close gracefully, allowing the TCP stack to recognize that the connection was closed - but this isn't always true (you can read more about "half-open" or "half-closed" connections).
For this reason, all servers implement a timeout / ping mechanism to test for lost connectivity.
I'm trying to accommodate a case where the client may be sending a message in several pieces separated by a delay (e.g. 100 bytes at 1 second each byte)...
Remember that TCP/IP is a stream based protocol, not a message based protocol.
This means that your 100 bytes might arrive fragmented or they might be combined with a previous message.
If you're sending messages (rather than streaming data), you need - by design - to mark message boundaries.
Since these message boundaries must be marked, it becomes relatively easy to add a message type marker (to mark ping/pong messages).
You can observer the WebSocket protocol message format to learn more about message based protocol design using a TCP/IP (streamed) connection.
In a Ruby script I'm having a problem with socket connections.
What I am doing is the following:
I have two threads and each one creates a connection to a different web server
Any time thread 1 receives data from server 1, I want thread 1 to post this data to server 2
Any time thread 2 receives data from server 2, I want thread 2 to post this data to server 1
Basically I am kind of acting as a bridge between the 2 servers.
Code looks like this:
require 'uri'
require 'net/http'
require 'json'
#connection1 = Net::HTTP.start 'server1.com'
#connection2 = Net::HTTP.start 'server2.com'
# reads data from server 1 as it comes and sends it to server 2
Thread.new{
while JSON.parse(#connection1.post('/receive').body) !nil
#connection2.post '/send', JSON.parse(#connection1.post('/receive').body)
end
}
# reads data from server 2 as it comes and sends it to server 2
while JSON.parse(#connection2.post('/receive').body) !nil
#connection1.post '/send', JSON.parse(#connection2.post('/receive').body)
end
# Thread.join
# not actually needed because the two connections are supposed to continuously stream data
However as soon as one of the two connections receives data and tries sending it to the other connection I'm receiving the following error:
Socket operation on non-socket - Errno::ENOTSOCK
More in deep stack trace:
C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:176:in
wait_readable': socket operation on non-socket. (Errno::ENOTSOCK)
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:176:in 'rbuf_fill'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:154:in 'readuntil'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/protocol.rb:164:in 'readline'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:40:in
'read_status_line'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/http/response.rb:29:in 'read_new'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1446:in block in 'transport_request'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1443:in 'catch'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1443:in 'transport_request'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1416:in 'request'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1430:in 'send_entity'
from C:/Dev/Ruby24-x64/lib/ruby/2.4.0/net/http.rb:1218:in 'post'
So what do you think I am doing wrong?
I should add that for reasons beyond my control the two remote servers are configured to serve data when contacted with a POST rather than with a GET.
Core problem
You lack any sort of synchronization between both threads and Net::HTTP is not thread-safe.
What's possibly happening here is that you call #connection1.post /receive in one thread, that said thread gets paused and the second thread tries to use #connection1.post /send while connection1 is still being used.
Another problem is that your code in inefficient, you issue two /receive requests per thread to get information.
while JSON.parse(#connection1.post('/receive').body) !nil
#connection2.post '/send', JSON.parse(#connection1.post('/receive').body)
end
This makes three requests total
Could be
while True
result = JSON.parse(#connection1.post('/receive').body)
break if result.nil?
#connection2.post '/send', result)
end
This makes two requests total
Suggested Solution
Use a Mutex to make sure that while connection1 is sending/receiving a request, no other thread touches it.
require 'uri'
require 'net/http'
require 'json'
#connection1 = Net::HTTP.start 'server1.com'
#connection2 = Net::HTTP.start 'server2.com'
connection_1_lock = Mutex.new
connection_2_lock = Mutex.new
# reads data from server 1 as it comes and sends it to server 2
Thread.new do
while True
receive_result = nil
connection_1_lock.synchronize do
receive_result = JSON.parse(#connection1.post('/receive').body)
end
connection_2_lock.synchronize do
#connection2.post '/send', receive_result
end
end
end
Thread.new do
while True
receive_result = nil
connection_2_lock.synchronize do
receive_result = JSON.parse(#connection2.post('/receive').body)
end
connection_1_lock.synchronize do
#connection1.post '/send', receive_result
end
end
end
I believe the code above should fix your problem, although I cannot guarantee it. Concurrent programming is hard.
Further reading:
I suggest you read up on concurrent/multithreaded programming and its pitfalls. There are numerous Ruby resources online.
Since Ruby's documentation on Mutex is notoriously bad, I'll shamelessly plug my own article here and suggest you read it:
https://dev.to/enether/working-with-multithreaded-ruby-part-i-cj3 (The 'How To Protect Yourself' paragraph introduces mutexes)
I'm connecting to a TCP server using Ruby's TCPSocket class.
I send some data about an address and I must wait for the server to do some processing to give me the geocoding of said address. Since the process in the server takes some time, I cannot read the response immediately.
When I used socket.readpartial() I got a response of two white spaces.
I temporarily solved this using sleep(5) but I don't like this at all, because it is hackish and clumsy, and I risk that even after 5 seconds the response is not ready and I still get an empty response.
I know that the responses will always be 285 characters long.
Is there a more correct and elegant way of having my TCP socket wait for the full response?
Here's my code:
def matchgeocode(rua, nro, cidade, uf)
count = 0
begin
socket = TCPSocket.new(GEOCODER_URL, GEOCODER_PORT)
# Needed for authentication
socket.write("TICKET #{GEOCODER_TICKET}")
socket.read(2)
# Here's the message I send to the server
socket.write("MATCHGEOCODE -Rua:\"#{rua}\" -Nro:#{nro} -Cidade:\"#{cidade}\" -Uf:\"#{uf}\"")
# My hackish sleep
sleep(5)
# Reading the fixed size response
response = socket.readpartial(285)
socket.write('QUIT')
socket.close
rescue Exception => e
count += 1
puts e.message
if count <= 5 && response.eql?('')
retry
end
end
response
end
Since you know the length of the response you should use read, not readpartial.
readpartial returns immediately if ANY data is available, even one byte is enough. That's why you need the sleep call so that the response has time to return to you before readpartial tries to peek at what data is present.
read on the other hand blocks completely until ALL requested data is available. Since you know the length of the result then read is the natural solution here.
So, I'm trying to simulate some basic HTTP persistent connections using sockets and Ruby - for a college class.
The point is to build a server - able to handle multiple clients - that receives a file path and gives back the file content - just like an HTTP GET.
The current server implementation loops listening for clients, fires a new thread when there's an incoming connection and reads the file paths from this socket. It's very dumb, but it works fine when working with non-presistent connections - one request per connection.
But they should be persistent.
Which means the client shouldn't worry about closing the connection. In the non-persistent version the servers echoes the response and close the connection - goodbye client, farewell.
But being persistent means the server thread should loop and wait for more incoming requests until... well until there's no more requests. How does the server knows that? It doesn't! Some sort of timeout is needed. I tried to do that with Ruby's Timeout, but it didn't work.
Googling for some solutions - besides being thoroughly advised to avoid using Timeout module - I've seen a lot of posts about the IO.select method, that should handle this socket waiting issue way better than using threads and stuff (which really sounds cool, considering how Ruby threads (don't) work). I'm trying to understand here how IO.select works, but still wasn't able to make it work in the current scenario.
So I aske basically two things:
how can I efficiently work this timeout issue on the server-side, either using some thread based solution, low-level socket options or some IO.select magic?
how can the client side know that the server has closed its side of the connection?
Here's the current code for the server:
require 'date'
module Sockettp
class Server
def initialize(dir, port = Sockettp::DEFAULT_PORT)
#dir = dir
#port = port
end
def start
puts "Starting Sockettp server..."
puts "Serving #{#dir.yellow} on port #{#port.to_s.green}"
Socket.tcp_server_loop(#port) do |socket, client_addrinfo|
handle socket, client_addrinfo
end
end
private
def handle(socket, addrinfo)
Thread.new(socket) do |client|
log "New client connected"
begin
loop do
if client.eof?
puts "#{'-' * 100} end connection"
break
end
input = client.gets.chomp
body = content_for(input)
response = {}
if body
response.merge!({
status: 200,
body: body
})
else
response.merge!({
status: 404,
body: Sockettp::STATUSES[404]
})
end
log "#{addrinfo.ip_address} #{input} -- #{response[:status]} #{Sockettp::STATUSES[response[:status]]}".send(response[:status] == 200 ? :green : :red)
client.puts(response.to_json)
end
ensure
socket.close
end
end
end
def content_for(path)
path = File.join(#dir, path)
return File.read(path) if File.file?(path)
return Dir["#{path}/*"] if File.directory?(path)
end
def log(msg)
puts "#{Thread.current} -- #{DateTime.now.to_s} -- #{msg}"
end
end
end
Update
I was able to simulate the timeout behaviour using the IO.select method, but the implementation doesn't feel good when combining with a couple of threads for accepting new connections and another couple for handling requests. The concurrency makes the situation mad and unstable, and I'm probably not sticking with it unless I can figure out a better way of using this solution.
Update 2
Seems like Timeout is still the best way to handle this. I'm sticking with it till find a better option.
I still don't know how to deal with zombie client connections.
Solution
I endend up using IO.select (got inspired when looking at the webrick code). You cha check the final version here (lib/http/server/client_handler.rb)
You should implement something like heartbeat packets.Client side should send special packets to after few secs/mins to ensure that server doesn't time out the connection on the client end.You just avoid doing anything in this call.