Ruby HTTP2 GET request - ruby

I'm trying to use the Ruby gem http-2 to send a GET request to Google.
I've lifted the code directly from the example and simplified it slightly:
require 'http/2'
require 'socket'
require 'openssl'
require 'uri'
uri = URI.parse('http://www.google.com/')
tcp = TCPSocket.new(uri.host, uri.port)
sock = tcp
conn = HTTP2::Client.new
conn.on(:frame) do |bytes|
# puts "Sending bytes: #{bytes.unpack("H*").first}"
sock.print bytes
sock.flush
end
conn.on(:frame_sent) do |frame|
puts "Sent frame: #{frame.inspect}"
end
conn.on(:frame_received) do |frame|
puts "Received frame: #{frame.inspect}"
end
stream = conn.new_stream
stream.on(:close) do
puts 'stream closed'
sock.close
end
stream.on(:half_close) do
puts 'closing client-end of the stream'
end
stream.on(:headers) do |h|
puts "response headers: #{h}"
end
stream.on(:data) do |d|
puts "response data chunk: <<#{d}>>"
end
head = {
':scheme' => uri.scheme,
':method' => 'GET',
':path' => uri.path
}
puts 'Sending HTTP 2.0 request'
stream.headers(head, end_stream: true)
while !sock.closed? && !sock.eof?
data = sock.read_nonblock(1024)
# puts "Received bytes: #{data.unpack("H*").first}"
begin
conn << data
rescue => e
puts "#{e.class} exception: #{e.message} - closing socket."
e.backtrace.each { |l| puts "\t" + l }
sock.close
end
end
The output is:
Sending HTTP 2.0 request
Sent frame: {:type=>:settings, :stream=>0, :payload=>[[:settings_max_concurrent_streams, 100]]}
Sent frame: {:type=>:headers, :flags=>[:end_headers, :end_stream], :payload=>[[":scheme", "http"], [":method", "GET"], [":path", "/"]], :stream=>1}
closing client-end of the stream
(Note: you get pretty much the same output as above by running the actual example file, i.e., ruby client.rb http://www.google.com/)
Why is no response data being displayed?

Public servers like google.com do not support HTTP/2 in clear text.
You are trying to connect to http://google.com, while you should really connect to https://google.com (note the https scheme).
In order to do that, you may need to wrap the TCP socket using TLS (see for example here), if http-2 does not do it for you.
Note also that HTTP/2 requires strong TLS ciphers and ALPN, so make sure that you have an updated version of OpenSSL (at least 1.0.2).
Given that the author of http-2 is a strong HTTP/2 supporter, I am guessing that your only problem is the fact that you tried clear-text http rather than https, and I expect that TLS cipher strength and ALPN are taken care of by the http-2 library.

Related

how to use the ruby analytical receive binary stream from the TCP

I'm going to the receiving device over the data, but these data are binary stream, I put these data storage, then read to display them correctly, is there a better way?
require 'socket'
server = TCPServer.open(2000)
loop {
Thread.start(server.accept) do |client|
File.open("tmp","w") { |file| file.write(client.gets)}
File.open("tmp").each do |f|
puts f.unpack('H*')
end
client.puts(Time.now.ctime) # Send the time to the client
client.puts "Closing the connection. Bye!"
client.close # Disconnect from the client
end
}
the received data like this: xx^Q^A^Hb0# <90>26 2^B^#<83>ev
I want like this: 787811010862304020903236202032020001c26c0d0a
sorry about my poor English!
Using a temporary file with a name will cause a problem if there are multiple clients sending data; the temporary file will be overwritten.
You don't need to use a temporary file.
require 'socket'
server = TCPServer.open(2000)
loop {
Thread.start(server.accept) do |client|
puts client.gets.unpack('H*')
client.puts(Time.now.ctime) # Send the time to the client
client.puts "Closing the connection. Bye!"
client.close
end
}

How do I read a in incoming POST Multipart request using ruby and tcpserver

I have created a very simple server:
#!/bin/ruby
require 'socket'
server = TCPServer.open 2000
puts "Listening on port 2000"
loop {
client = server.accept
client.puts "HTTP/1.1 200/OK\r\nContent-type:text/html\r\n\r\n"
response = "My super slim ruby http server"
client.puts response
received = client.recv(1024)
puts received
puts "\n"
client.close
}
So far, it serves my purpose, which is to print out the requests that might come from a given client. However, if I use, for example, the following curl command to create a request:
curl -F "data=someData" http://localhost:2000
My ruby server only prints out the HTTP headers, but not the body of the request.
Is there a way to do this?
Looks like you have to call recv again to get the body:
#!/bin/ruby
require 'socket'
server = TCPServer.open 2000
puts "Listening on port 2000"
loop {
client = server.accept
client.puts "HTTP/1.1 200/OK\r\nContent-type:text/html\r\n\r\n"
response = "My super slim ruby http server"
client.puts response
headers = client.recv(1024)
headers =~ /Content-Length: (\d+)/ # get content length
body = $1 ? client.recv($1.to_i) : '' # <- call again to get body if there is one
puts headers + body
client.close
}
Thanks bundacia, I ended up mixing what you sent and some other findings and this is the result:
#!/bin/ruby
require 'socket'
server = TCPServer.open 2000
puts "Listening on port 2000"
loop {
client = server.accept
client.puts "HTTP/1.1 200/OK\r\nContent-type:text/xml\r\n\r\n"
response = "My super slim ruby http server"
client.puts response
all_data = []
i = 1024
firstrun = "yes"
while i > 0
partial_data = client.recv(i)
if (firstrun == "no")
i = 0
end
if (firstrun == "yes")
partial_data =~ /Content-Length: (\d+)/ # get content length
length = $1
if (nil != length && !length.empty?)
i = length.to_i
firstrun = "no"
end
end
all_data << partial_data
end
puts all_data.join()
client.close
}

How to capture POST data from a simple Ruby server

I have a basic Ruby server that I'd like to listen to a specific port, read incoming POST data and do blah...
I have this:
require 'socket' # Get sockets from stdlib
server = TCPServer.open(2000) # Socket to listen on port 2000
loop { # Servers run forever
client = server.accept # Wait for a client to connect
client.puts(Time.now.ctime) # Send the time to the client
client.puts "Closing the connection. Bye!"
client.close # Disconnect from the client
}
How would I go about capturing the POST data?
Thanks for any help.
It's possible to do this without adding much to your server:
require 'socket' # Get sockets from stdlib
server = TCPServer.open(2000) # Socket to listen on port 2000
loop { # Servers run forever
client = server.accept # Wait for a client to connect
method, path = client.gets.split # In this case, method = "POST" and path = "/"
headers = {}
while line = client.gets.split(' ', 2) # Collect HTTP headers
break if line[0] == "" # Blank line means no more headers
headers[line[0].chop] = line[1].strip # Hash headers by type
end
data = client.read(headers["Content-Length"].to_i) # Read the POST data as specified in the header
puts data # Do what you want with the POST data
client.puts(Time.now.ctime) # Send the time to the client
client.puts "Closing the connection. Bye!"
client.close # Disconnect from the client
}
For really simple apps you probably want to write something using Sinatra which is about as basic as you can get.
post('/') do
# Do stuff with post data stored in params
puts params[:example]
end
Then you can stick this in a Rack script, config.ru, and host it easily using any Rack-compliant server.
client.read(length) # length is length of request header content

How to read from a TCPServer socket in ruby using read, readpartial and read_nonblock

I have a 2 part question on reading from sockets and how is it managed on Ruby servers like Unicorn or Mongrel
I've learnt that to read from a socket is different from reading a file and that there are no distinct EOF message sent and the data is an endless stream. So how do you know when to stop reading? My TCPServer for example in this case when I hit my server by accessing http://localhost:9799 from a browser, it hangs after there is no more data to read and it won't throw the EOFError either.
require 'socket'
READ_CHUNK = 1024
socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM)
addr = Socket.pack_sockaddr_in(9799, '127.0.0.1')
socket.bind(addr)
socket.listen(Socket::SOMAXCONN)
socket.setsockopt(:SOCKET, :REUSEADDR, true)
puts "Server is listening on port = 9799"
loop do
connection, addr_info = socket.accept
data_buffer = ""
loop do
begin
connection.read_nonblock(READ_CHUNK, data_buffer)
puts "Buffer = #{data_buffer}"
rescue Errno::EAGAIN => e
IO.select([connection])
retry
rescue EOFError
break
end
end
connection.write("HTTP/1.1 200 \r\n")
connection.write("Content-Type: text/html\r\n")
connection.write("Status 200 \r\n")
connection.write("Connection: close \r\n")
connection.write("Hello World \r\n")
connection.close
end
I'd like to know whats the best practice/standard approach used by Ruby servers. I see the Unicorn uses read_nonblock from kgio library and mongrel uses readpartial (I'm not sure about these but going through the code this is what I feel is the approach adopted.) Even with checks for \r\n how does the server know the input is complete.
Could explain how this should be done (and I think gets is not the approach - its with read, readpartial, read_nonblock).
2). I would really appreciate a few lines on how this is achieved in servers like unicorn or passenger
Thank you.
It's done in unicorn here
https://github.com/defunkt/unicorn/blob/master/lib/unicorn/http_request.rb#L69-L71
There is add_parse method(read the comments above methods)
https://github.com/defunkt/unicorn/blob/master/ext/unicorn_http/unicorn_http.rl#L760-L778
Also take a look at some explanations here http://www.ruby-forum.com/topic/2267632#1014288
Here is your working code using http_parser.rb https://gist.github.com/4136962
gem install http_parser.rb
require 'socket'
require "http/parser"
READ_CHUNK = 1024 * 4
socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM)
addr = Socket.pack_sockaddr_in(9799, '127.0.0.1')
socket.bind(addr)
socket.listen(Socket::SOMAXCONN)
socket.setsockopt(:SOCKET, :REUSEADDR, true)
puts "Server is listening on port = 9799"
loop do
connection, addr_info = socket.accept
parser = Http::Parser.new
begin
data = connection.readpartial(READ_CHUNK)
puts "Buffer = #{data}"
parser << data
end until parser.headers
connection.write("HTTP/1.1 200 \r\n")
connection.write("Content-Type: text/html\r\n")
connection.write("Status 200 \r\n")
connection.write("Connection: close \r\n")
connection.write("\r\n\r\n")
connection.write("Hello World \r\n")
connection.close
end

ruby - zeromq detects client timeout, but client hangs

I'm trying to write a simple zmq system to replace an http client/server. My client times out when the server is off/unavailable, but does not retry or stop. What am I missing?
zmq_client.rb (modified version of Han Holl's lazy pirate client from zeromq guide)
require 'rubygems'
require 'zmq'
context = ZMQ::Context.new
socket = context.socket(ZMQ::REQ)
socket.connect('tcp://localhost:5559')
retries = 2
timeout = 10
retries.times do |tries|
message = "Hello #{tries}"
raise("Send: #{message} failed") unless socket.send(message)
puts "Sending string [#{message}]"
if ZMQ.select( [socket], nil, nil, timeout)
message = socket.recv
puts "Received reply [#{message}]"
break
else
puts "timeout"
end
end
socket.close
zmq_broker.rb (modified version of Oleg Sidorov's code found on zeromq guide)
require 'rubygems'
require 'ffi-rzmq'
context = ZMQ::Context.new
frontend = context.socket(ZMQ::ROUTER)
frontend.bind('tcp://*:5559')
poller = ZMQ::Poller.new
poller.register(frontend, ZMQ::POLLIN)
loop do
poller.poll(:blocking)
poller.readables.each do |socket|
if socket === frontend
loop do
socket.recv_string(message = '')
more = socket.more_parts?
puts "#{message}#{more}"
socket.send_string(message, more ? ZMQ::SNDMORE : 0)
break unless more
end
end
end
end
You should get an error Send: #{message} failed as soon as you try to send again after the first timeout, because your 2nd send will happen directly after the 1st send, and the REQ socket enforces that each send must go after (successful, not timeout-ed) recv.
In the lazy pirate pattern, you may need to send several requests before getting a reply. Solution suggested in the 0MQ Guide is to close and reopen the REQ socket after an error. Your client doesn't close/reopen the REQ socket.
You may find helpful the "Lazy Pirate client in Ruby" example from the Guide.

Resources