I'm trying to make an HTTP Head request using Net::HTTP.
require 'net/http'
uri = URI("https://github.com/rails/rails")
http = Net::HTTP.new(uri.host, uri.port)
request = http.head(uri)
puts request
fails.
AFAICT, this is because Net::HTTP is waiting on a response body which will never come. How do I ask Net::HTTP to make a request and not wait on the response body?
If you follow the documentation properly, it works just fine. The library implementation probably has some assumptions on the usage when it determines whether to read the payload.
response = nil
Net::HTTP.start('github.com', :use_ssl => true) do |http|
response = http.head('/rails/rails')
end
response.each { |k, v| [k, v] }
Related
Maybe I'm just blind but many post about passing headers in Net::HTTP follows the lines of
require 'net/http'
uri = URI("http://www.ruby-lang.org")
req = Net::HTTP::Get.new(uri)
req['some_header'] = "some_val"
res = Net::HTTP.start(uri.hostname, uri.port) {|http|
http.request(req)
}
puts res.body
(From Ruby - Send GET request with headers metaphori's answer)
And from the Net::HTTP docs (https://docs.ruby-lang.org/en/2.0.0/Net/HTTP.html)
uri = URI('http://example.com/cached_response')
file = File.stat 'cached_response'
req = Net::HTTP::Get.new(uri)
req['If-Modified-Since'] = file.mtime.rfc2822
res = Net::HTTP.start(uri.hostname, uri.port) {|http|
http.request(req)
}
open 'cached_response', 'w' do |io|
io.write res.body
end if res.is_a?(Net::HTTPSuccess)
But what is the advantage of doing the above when you can pass the headers via the following way?
options = {
'headers' => {
'Content-Type' => 'application/json'
}
}
request = Net::HTTP::Get.new('http://www.stackoverflow.com/', options['headers'])
This allows you to parameterize the headers and can allow for multiple headers very easily.
My main question is, what is the advantage of passing the headers in the creation of Net::HTTP::Get vs passing them after the creation of Net::HTTP::Get
Net::HTTPHeader already goes ahead and assigns the headers in the function
def initialize_http_header(initheader)
#header = {}
return unless initheader
initheader.each do |key, value|
warn "net/http: duplicated HTTP header: #{key}", uplevel: 1 if key?(key) and $VERBOSE
if value.nil?
warn "net/http: nil HTTP header: #{key}", uplevel: 1 if $VERBOSE
else
value = value.strip # raise error for invalid byte sequences
if value.count("\r\n") > 0
raise ArgumentError, 'header field value cannot include CR/LF'
end
#header[key.downcase] = [value]
end
end
end
So doing
request['some_header'] = "some_val" almost seems like code duplication.
There is no advantage for setting headers one way or another, at least not that I can think of. It comes down to your own preference. In fact, if you take a look at what happens when you supply headers while initializing a new Net::Http::Get, you will find that internally, Ruby simply sets the headers onto a #headers variable:
https://github.com/ruby/ruby/blob/c5eb24349a4535948514fe765c3ddb0628d81004/lib/net/http/header.rb#L25
And if you set the headers using request[name] = value, you can see that Net::Http does the exact same thing, but in a different method:
https://github.com/ruby/ruby/blob/c5eb24349a4535948514fe765c3ddb0628d81004/lib/net/http/header.rb#L46
So the resulting object has the same configuration no matter which way you decide to pass the request headers.
I have the following method in our Ruby 1.8.7 project:
def self.ping_server
request_data = get_request_data
uri = 'another_server.our_company.com'
https = Net::HTTP.new(uri)
https.use_ssl = true
path = "/our_service"
data = request_data.to_json
response = https.post(path, data, {'Content-Type' => 'application/json'})
return response
end
Whenever I run this method, I get the following time out error:
Completed 500 Internal Server Error in 128936ms
Errno::ETIMEDOUT (Connection timed out - connect(2)):
lib/my_class.rb:51:in `ping_our_server'
I checked with a coworker who has access to the logs for another_server.our_company.com. My request isn't arriving at the other server.
What should I do to get my request to work?
EDIT: On further inspection, this is what I think is happening (but I'm not completely sure): Our other server will only accept HTTPS requests, but it looks like my request is being sent over HTTP for some reason. Is there something I need to add to make sure my request is sent over HTTPS?
According to this website, this is how you send an HTTPS request:
require "net/https"
require "uri"
uri = URI.parse("https://secure.com/")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Get.new(uri.request_uri)
response = http.request(request)
response.body
response.status
response["header-here"] # All headers are lowercase
According to this website (which is linked from my first link), you should also do this to close up a vulnerability in the net/https library:
To get going, you need a local CA certificates bundle, the official
curl site maintains an up to date cacert.pem / ca-bundle.crt file
containing all of the major certificates if you need one.
Next, after a gem install always_verify_ssl_certificates, you can be
up and running with a test as simply as:
require 'always_verify_ssl_certificates'
AlwaysVerifySSLCertificates.ca_file = "/path/path/path/cacert.pem"
http= Net::HTTP.new('https://some.ssl.site', 443)
http.use_ssl = true
req = Net::HTTP::Get.new('/')
response = http.request(req)
If the site
has a bad certificate an error will be raised at this point. If not, a
legitimate HTTP response object will be returned.
In C# it was fairly simple and didn't take more than a couple minutes to google:
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(#"http://www.example.com?q=someValue");
request.Headers.Add("Authorization: OAuth realm=\"example.com\" oauth_consumer_key=\"BCqrstoO\" ... so on and so forth");
string resultString = "";
using (StreamReader read = new StreamReader(request.GetResponse().GetResponseStream(), true))
{
resultString = read.ReadToEnd();
}
Trying to do it in Ruby hasn't quite been as straight forward (or is just something stupid that I'm missing).
I have been looking and the closest things I've come to finding my answer are How to make an HTTP GET with modified headers? and Send Custom Headers in Ruby.
So my problem, I suppose, boils down to
How do I set the headers as just a just a straight forward string?
Why do these two examples show headers formatted the way they are?
Is what I'm asking for even good convention and if not, how do I format what I'm trying to do in the convention these Ruby methods are asking for?
So far I tried the two examples and here's my most recent non-working attempt:
headers = "Authorization: OAuth realm=\"example.com\" oauth_consumer_key=\"BCqrstoO\" ... so on and so forth"
uri = URI("www.example.com")
http = Net::HTTP.new(uri.host, uri.port)
http.get(uri.path, headers) do |chunk|
puts chunk
end
Use open-uri. Example:
require 'open-uri'
open("http://www.ruby-lang.org/en/",
"User-Agent" => "Ruby/#{RUBY_VERSION}",
"From" => "foo#bar.invalid",
"Referer" => "http://www.ruby-lang.org/") {|f|
# ...
}
Just in case you check this at this point on time, the Net:HTTPRequest object allows you to add headers easily.
Net::HTTP.start(uri.host, uri.port) do |http|
request = Net::HTTP::Get.new uri
request['my-header'] = '1'
http.request request do |response|
puts response
end
end
To do a Net::HTTP https request without the block form you can do this:
...
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
...
But is there a way to tell Net::HTTP to use https when doing the block form?
u = URI.parse(url)
Net::HTTP.start(u.host, u.port) do |http|
# if I put http.use_ssl = true here, ruby complains that this can't
# be done becuase the sesion has already started
resp = http.get(u.request_uri)
end
I'm on ruby 1.8.7
See the documentation for Net::HTTP.start which takes an optional hash. From the documentation:
opt sets following values by its accessor. The keys are ca_file, ca_path, cert, cert_store, ciphers, close_on_empty_response, key, open_timeout, read_timeout, ssl_timeout, ssl_version, use_ssl, verify_callback, verify_depth and verify_mode. If you set :use_ssl as true, you can use https and default value of verify_mode is set as OpenSSL::SSL::VERIFY_PEER.
Net::HTTP.start(url.host, url.port, :use_ssl => true)
I have the following bit of code:
uri = URI.parse("https://rs.xxx-travel.com/wbsapi/RequestListenerServlet")
https = Net::HTTP.new(uri.host,uri.port)
https.use_ssl = true
req = Net::HTTP::Post.new(uri.path)
req.body = searchxml
req["Accept-Encoding"] ='gzip'
res = https.request(req)
This normally works fine but the server at the other side is complaining about something in my XML and the techies there need the xml message AND the headers that are being sent.
I've got the xml message, but I can't work out how to get at the Headers that are being sent with the above.
To access headers use the each_header method:
# Header being sent (the request object):
req.each_header do |header_name, header_value|
puts "#{header_name} : #{header_value}"
end
# Works with the response object as well:
res.each_header do |header_name, header_value|
puts "#{header_name} : #{header_value}"
end
you can add:
https.set_debug_output $stderr
before the request and you will see in console the real http request sent to the server.
very useful to debug this kind of scenarios.
Take a look at the docs for Net::HTTP's post method. It takes the path of the uri value, the data (XML) you want to post, then the headers you want to set. It returns the response and the body as a two-element array.
I can't test this because you've obscured the host, and odds are good it takes a registered account, but the code looks correct from what I remember when using Net::HTTP.
require 'net/http'
require 'uri'
uri = URI.parse("https://rs.xxx-travel.com/wbsapi/RequestListenerServlet")
https = Net::HTTP.new(uri.host, uri.port)
https.use_ssl = true
req, body = https.post(uri.path, '<xml><blah></blah></xml>', {"Accept-Encoding" => 'gzip'})
puts "#{body.size} bytes received."
req.each{ |h,v| puts "#{h}: #{v}" }
Look at Typhoeus as an alternate, and, in my opinion, easier to use gem, especially the "Making Quick Requests" section.