Try to start 5 TCP sessions in a non-blocking manner. However, using wireshark, saw the TCP SYN's were sent out after 25ms from each other. Had expected to see 5 connection attempts at the same. Why?
Here is the code
require 'socket'
i = 0;
while i < 5
socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM, 0)
sockaddr = Socket.pack_sockaddr_in(80, '<some_homename>')
begin
socket.connect_nonblock(sockaddr)
rescue Errno::EINPROGRESS
#nothing to do
end
i += 1
end
Related
require 'socket'
socket = TCPSocket.open('stream-api.betfair.com', '443')
while line = socket.gets
puts line.chop
end
socket.close
I should receive something like {"op":"connection","connectionId":"002-230915140112-174"}
but I receive Connection reset by peer which
means the remote end would have sent a reset packet (RST) to kill the connection without an orderly shutdown (close). In that case you know it was the peer(client).
betfair included a nodejs example and also csharp/java examples
Any help is much appreciated. Thanks!
First thing, you should replace the string '443' by an integer :
TCPSocket.open('stream-api.betfair.com', 443)
Anyway, it seems to be related with the SSL negociation : the following Stackoverflow post gives a quick idea about what would work : How to establish a SSL enabled TCP/IP Connection in Ruby. Using this method, I works.
require 'socket'
require 'openssl'
host = 'stream-api.betfair.com'
port = 443
socket = TCPSocket.open(host,port)
ssl_context = OpenSSL::SSL::SSLContext.new()
ssl_context.ssl_version = :SSLv23
ssl_socket = OpenSSL::SSL::SSLSocket.new(socket, ssl_context)
ssl_socket.sync_close = true
ssl_socket.connect
while line = ssl_socket.gets
p line
end
ssl_socket.close
Result :
"{\"op\":\"connection\",\"connectionId\":\"001-151118094105-259478\"}\r\n"
Dealing with SSL/TLS protected connection is sometime quite verbose with Ruby. In the example you gave, in NodeJS, the hint is the first line :
var tls = require('tls');
I want to proxy SSL data, HTTPS in this case.
Here is my Code:
begin
server = TCPServer.open(on_demand_port)
rescue Exception => e
sleep 5
retry
end
sslContext = OpenSSL::SSL::SSLContext.new
sslContext.verify_mode = OpenSSL::SSL::VERIFY_NONE
begin
sslContext.cert = OpenSSL::X509::Certificate.new(File.open("#{Dir.pwd}/Cert/cert.pem"))
sslContext.key = OpenSSL::PKey::RSA.new(File.open("#{Dir.pwd}/Cert/key.pem"), "1234")
rescue Exception => e
sleep 5
retry
end
begin
sslServer = OpenSSL::SSL::SSLServer.new(server, sslContext)
rescue Exception => e
sleep 5
retry
end
while true
begin
threads << Thread.new(sslServer.accept) do |client| # Putting new connections into the thread pool
tcp_proxy(client, db_name, db_user, db_password, remote_host, remote_port, patterns)
end
rescue Exception => e
end
threads = threads.select { |t| t.alive? ? true : (t.join; false) }
while threads.size >= on_demand_max_threads
sleep 1
threads = threads.select { |t| t.alive? ? true : (t.join; false) }
end
end
And this is the "tcp_proxy" which is the actual SSL Proxy
begin
begin
ssl_context = OpenSSL::SSL::SSLContext.new
ssl_context.verify_mode = OpenSSL::SSL::VERIFY_NONE
cert_store = OpenSSL::X509::Store.new
cert_store.set_default_paths
ssl_context.cert_store = cert_store
tcp_socket = TCPSocket.new(remote_host, remote_port)
server_socket = OpenSSL::SSL::SSLSocket.new tcp_socket, ssl_context
server_socket.sync_close = true
server_socket.connect
rescue Exception => e
client.close
end
while true
# Wait for data to be available on either socket.
(ready_sockets, dummy, dummy) = IO.select([client, server_socket])
begin
ready_sockets.each do |socket|
data = socket.readpartial(4096)
if socket == client
# Read from client, write to server.
server_socket.write data
server_socket.flush
else
# Read from server, write to client.
client.write data
client.flush
end
end
rescue Exception => e
end
end
rescue StandardError => e
end
begin
client.close
server_socket.close
rescue Exception => e
end
Now, this is working great in normal TCP and HTTP, but, when I use it in SSL\HTTPS when upgrading the socket it starts getting really really slow and sometimes will just timeout.
Any idea why ?
You have to be careful with read and select, because read is done at the SSL level while select is at the TCP level.
SSL puts the data into frames, where each frame can contain at most 16384 bytes. It needs to read the full frame from the underlying TCP socket before the read on the SSL socket can return any data from the frame. This means if you have a frame with 4097 bytes payload it will need to read the full frame from the TCP socket before you can read anything from the SSL socket. If you then only read 4096 bytes from the SSL socket it will return the first 4096 bytes and leave the rest (1 byte) in the SSL buffer. If you then check with select for new data at the TCP level it might block, because there are no unread data at the TCP level, even if there is still the single byte inside the SSL buffer.
There are two ways to work around this problem:
Check with pending if there are still data in the SSL buffer. If there are, read them instead of doing the select.
Or try to read at least 16384 bytes with each read, that is the maximum size of a SSL frame. I'm not sure about the implementation in ruby, but in Perl this read will just call the underlying SSL_read and this only reads the data from a single frame. Thus with a read size of 16384 bytes there can be no pending data and you can just call select like you do now.
I have a 2 part question on reading from sockets and how is it managed on Ruby servers like Unicorn or Mongrel
I've learnt that to read from a socket is different from reading a file and that there are no distinct EOF message sent and the data is an endless stream. So how do you know when to stop reading? My TCPServer for example in this case when I hit my server by accessing http://localhost:9799 from a browser, it hangs after there is no more data to read and it won't throw the EOFError either.
require 'socket'
READ_CHUNK = 1024
socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM)
addr = Socket.pack_sockaddr_in(9799, '127.0.0.1')
socket.bind(addr)
socket.listen(Socket::SOMAXCONN)
socket.setsockopt(:SOCKET, :REUSEADDR, true)
puts "Server is listening on port = 9799"
loop do
connection, addr_info = socket.accept
data_buffer = ""
loop do
begin
connection.read_nonblock(READ_CHUNK, data_buffer)
puts "Buffer = #{data_buffer}"
rescue Errno::EAGAIN => e
IO.select([connection])
retry
rescue EOFError
break
end
end
connection.write("HTTP/1.1 200 \r\n")
connection.write("Content-Type: text/html\r\n")
connection.write("Status 200 \r\n")
connection.write("Connection: close \r\n")
connection.write("Hello World \r\n")
connection.close
end
I'd like to know whats the best practice/standard approach used by Ruby servers. I see the Unicorn uses read_nonblock from kgio library and mongrel uses readpartial (I'm not sure about these but going through the code this is what I feel is the approach adopted.) Even with checks for \r\n how does the server know the input is complete.
Could explain how this should be done (and I think gets is not the approach - its with read, readpartial, read_nonblock).
2). I would really appreciate a few lines on how this is achieved in servers like unicorn or passenger
Thank you.
It's done in unicorn here
https://github.com/defunkt/unicorn/blob/master/lib/unicorn/http_request.rb#L69-L71
There is add_parse method(read the comments above methods)
https://github.com/defunkt/unicorn/blob/master/ext/unicorn_http/unicorn_http.rl#L760-L778
Also take a look at some explanations here http://www.ruby-forum.com/topic/2267632#1014288
Here is your working code using http_parser.rb https://gist.github.com/4136962
gem install http_parser.rb
require 'socket'
require "http/parser"
READ_CHUNK = 1024 * 4
socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM)
addr = Socket.pack_sockaddr_in(9799, '127.0.0.1')
socket.bind(addr)
socket.listen(Socket::SOMAXCONN)
socket.setsockopt(:SOCKET, :REUSEADDR, true)
puts "Server is listening on port = 9799"
loop do
connection, addr_info = socket.accept
parser = Http::Parser.new
begin
data = connection.readpartial(READ_CHUNK)
puts "Buffer = #{data}"
parser << data
end until parser.headers
connection.write("HTTP/1.1 200 \r\n")
connection.write("Content-Type: text/html\r\n")
connection.write("Status 200 \r\n")
connection.write("Connection: close \r\n")
connection.write("\r\n\r\n")
connection.write("Hello World \r\n")
connection.close
end
I use the following code to check the server status of a certain game server to see if the game server is online.
begin
sock = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM, 0)
sockaddr = Socket.sockaddr_in(game_server.gameserver_port, game_server.gameserver_hostname)
sock.connect(sockaddr)
server_status.status = 1
rescue
server_status.status = 0
end
However it seems that the code blindly hangs up on the line without proceeding anywhere sock.connect(sockaddr) and does not throw an error when there's no services listening on that port. Is there a better way to do this in Ruby?
Could timeout be a good solution?
require 'timeout'
begin
timeout(5) do
# socket stuff...
end
rescue Timeout::Error
puts "Timed out!"
end
How can I detect that a socket is half-open? The case I'm dealing with is when the other side of a socket has sent a FIN and the Ruby app has ACKed that FIN. Is there a way for me to tell that the socket is in this condition?
Take, for example:
require 'socket'
s = TCPServer.new('0.0.0.0', 5010)
loop do
c = s.accept
until c.closed?
p c.recv(1024)
end
end
In this case, when I telnet into port 5010, I'll see all my input until I close the telnet session. At that point, it will print empty strings over and over as fast as it can.
You are using the blocking call recv, which will return nil when the other end closes. The socket won't be closed until you close it. Change
until c.closed?
p c.recv(1024)
end
to
while (s = c.recv(1024)) && s > 0
p s
end
c.close
You could combine IO#read and IO#eof? to check this.
require 'socket'
server = TCPServer.new('0.0.0.0', 5010)
loop do
client = server.accept
client.read(1024) until client.eof?
puts 'client closed connection'
client.close
end