How can I send HTTP GET request with parameters via ruby?
I have tried a lot of examples but all of those failed.
I know this post is old but for the sake of those brought here by google, there is an easier way to encode your parameters in a URL safe manner. I'm not sure why I haven't seen this elsewhere as the method is documented on the Net::HTTP page. I have seen the method described by Arsen7 as the accepted answer on several other questions also.
Mentioned in the Net::HTTP documentation is URI.encode_www_form(params):
# Lets say we have a path and params that look like this:
path = "/search"
params = {q: => "answer"}
# Example 1: Replacing the #path_with_params method from Arsen7
def path_with_params(path, params)
encoded_params = URI.encode_www_form(params)
[path, encoded_params].join("?")
end
# Example 2: A shortcut for the entire example by Arsen7
uri = URI.parse("http://localhost.com" + path)
uri.query = URI.encode_www_form(params)
response = Net::HTTP.get_response(uri)
Which example you choose is very much dependent on your use case. In my current project I am using a method similar to the one recommended by Arsen7 along with the simpler #path_with_params method and without the block format.
# Simplified example implementation without response
# decoding or error handling.
require "net/http"
require "uri"
class Connection
VERB_MAP = {
:get => Net::HTTP::Get,
:post => Net::HTTP::Post,
:put => Net::HTTP::Put,
:delete => Net::HTTP::Delete
}
API_ENDPOINT = "http://dev.random.com"
attr_reader :http
def initialize(endpoint = API_ENDPOINT)
uri = URI.parse(endpoint)
#http = Net::HTTP.new(uri.host, uri.port)
end
def request(method, path, params)
case method
when :get
full_path = path_with_params(path, params)
request = VERB_MAP[method].new(full_path)
else
request = VERB_MAP[method].new(path)
request.set_form_data(params)
end
http.request(request)
end
private
def path_with_params(path, params)
encoded_params = URI.encode_www_form(params)
[path, encoded_params].join("?")
end
end
con = Connection.new
con.request(:post, "/account", {:email => "test#test.com"})
=> #<Net::HTTPCreated 201 Created readbody=true>
I assume that you understand the examples on the Net::HTTP documentation page but you do not know how to pass parameters to the GET request.
You just append the parameters to the requested address, in exactly the same way you type such address in the browser:
require 'net/http'
res = Net::HTTP.start('localhost', 3000) do |http|
http.get('/users?id=1')
end
puts res.body
If you need some generic way to build the parameters string from a hash, you may create a helper like this:
require 'cgi'
def path_with_params(page, params)
return page if params.empty?
page + "?" + params.map {|k,v| CGI.escape(k.to_s)+'='+CGI.escape(v.to_s) }.join("&")
end
path_with_params("/users", :id => 1, :name => "John&Sons")
# => "/users?name=John%26Sons&id=1"
Related
Maybe I'm just blind but many post about passing headers in Net::HTTP follows the lines of
require 'net/http'
uri = URI("http://www.ruby-lang.org")
req = Net::HTTP::Get.new(uri)
req['some_header'] = "some_val"
res = Net::HTTP.start(uri.hostname, uri.port) {|http|
http.request(req)
}
puts res.body
(From Ruby - Send GET request with headers metaphori's answer)
And from the Net::HTTP docs (https://docs.ruby-lang.org/en/2.0.0/Net/HTTP.html)
uri = URI('http://example.com/cached_response')
file = File.stat 'cached_response'
req = Net::HTTP::Get.new(uri)
req['If-Modified-Since'] = file.mtime.rfc2822
res = Net::HTTP.start(uri.hostname, uri.port) {|http|
http.request(req)
}
open 'cached_response', 'w' do |io|
io.write res.body
end if res.is_a?(Net::HTTPSuccess)
But what is the advantage of doing the above when you can pass the headers via the following way?
options = {
'headers' => {
'Content-Type' => 'application/json'
}
}
request = Net::HTTP::Get.new('http://www.stackoverflow.com/', options['headers'])
This allows you to parameterize the headers and can allow for multiple headers very easily.
My main question is, what is the advantage of passing the headers in the creation of Net::HTTP::Get vs passing them after the creation of Net::HTTP::Get
Net::HTTPHeader already goes ahead and assigns the headers in the function
def initialize_http_header(initheader)
#header = {}
return unless initheader
initheader.each do |key, value|
warn "net/http: duplicated HTTP header: #{key}", uplevel: 1 if key?(key) and $VERBOSE
if value.nil?
warn "net/http: nil HTTP header: #{key}", uplevel: 1 if $VERBOSE
else
value = value.strip # raise error for invalid byte sequences
if value.count("\r\n") > 0
raise ArgumentError, 'header field value cannot include CR/LF'
end
#header[key.downcase] = [value]
end
end
end
So doing
request['some_header'] = "some_val" almost seems like code duplication.
There is no advantage for setting headers one way or another, at least not that I can think of. It comes down to your own preference. In fact, if you take a look at what happens when you supply headers while initializing a new Net::Http::Get, you will find that internally, Ruby simply sets the headers onto a #headers variable:
https://github.com/ruby/ruby/blob/c5eb24349a4535948514fe765c3ddb0628d81004/lib/net/http/header.rb#L25
And if you set the headers using request[name] = value, you can see that Net::Http does the exact same thing, but in a different method:
https://github.com/ruby/ruby/blob/c5eb24349a4535948514fe765c3ddb0628d81004/lib/net/http/header.rb#L46
So the resulting object has the same configuration no matter which way you decide to pass the request headers.
I have the following methods in a Ruby script:
def parse_endpoint(endpoint)
return URI.parse(endpoint)
end
def verify_url(endpoint, fname)
url = “#{endpoint}#{fname}”
req = Net::HTTP.new(url.host, url.port)
res = req.request_head(url.path)
if res.code == “200”
true
else
puts “#{fname} is an invalid file”
false
end
end
Testing the url manually like so works fine (returns true since the url is indeed valid):
endpoint = parse_endpoint('http://mywebsite.com/mySubdirectory/')
verify_url(endpoint, “myFile.json”)
However, when I try to do the following in rspec
describe 'my functionality'
let (:endpoint) { parse_endpoint(“http://mywebsite.com/mySubdirectory/”) }
it 'should verify valid url' do
expect(verify_url(endpoint, “myFile.json”).to eq(true))
end
end
it gives me this error
“NoMethodError:
undefined method `host' for "http://mysebsite.com/mySubdirectory/myFile.json":String”
What am I doing wrong?
url is a String object, and you are trying to access a method called host which does not exist in String:
url = “#{endpoint}#{fname}”
req = Net::HTTP.new(url.host, url.port)
EDIT you probably need an URI object. I think this is what you want:
2.2.1 :004 > require 'uri'
=> true
2.2.1 :001 > url = 'http://mywebsite.com/mySubdirectory/'
=> "http://mywebsite.com/mySubdirectory/"
2.2.1 :005 > parsed_url = URI.parse url
=> #<URI::HTTP http://mywebsite.com/mySubdirectory/>
2.2.1 :006 > parsed_url.host
=> "mywebsite.com"
So just add url = URI.parse url before using url.host.
Testing the url manually like so works fine (returns true since the url is indeed valid):
endpoint = parse_endpoint('http://mywebsite.com/mySubdirectory/')
verify_url(endpoint, “myFile.json”)
It seems you missed something when you tested code above (maybe you tested old version) because it can't work as it is now.
Look at these lines of code:
url = "#{endpoint}#{fname}"
req = Net::HTTP.new(url.host, url.port)
You're creating a string variable url from other two variables endpoint and fname. So far, so good.
But then you're trying to access method host on url variable, which doesn't exist (but it exists on the endpoint variable), that's why you get this error.
You may want to use this code instead:
def verify_url(endpoint, fname)
url = endpoint.merge(fname)
res = Net::HTTP.start(url.host, url.port) do |http|
http.head(url.path)
end
# it's actually a bad idea to puts some text in a query method
# let's just return value instead
res.code == "200"
end
I have to add a new param to an indeterminate URL, let's say param=value.
In case the actual URL has already params like this
http://url.com?p1=v1&p2=v2
I should transform the URL to this other:
http://url.com?p1=v1&p2=v2¶m=value
But if the URL has not any param yet like this:
http://url.com
I should transform the URL to this other:
http://url.com?param=value
I feel worry to solve this with Regex because I'm not sure that looking for the presence of & could be enough. I'm thinking that maybe I should transform the URL to an URI object, and then add the param and transform it to String again.
Looking for any suggestion from someone who has been already in this situation.
Update
To help with the participation I'm sharing a basic test suite:
require "minitest"
require "minitest/autorun"
def add_param(url, param_name, param_value)
# the code here
"not implemented"
end
class AddParamTest < Minitest::Test
def test_add_param
assert_equal("http://url.com?param=value", add_param("http://url.com", "param", "value"))
assert_equal("http://url.com?p1=v1&p2=v2¶m=value", add_param("http://url.com?p1=v1&p2=v2", "param", "value"))
assert_equal("http://url.com?param=value#&tro&lo&lo", add_param("http://url.com#&tro&lo&lo", "param", "value"))
assert_equal("http://url.com?p1=v1&p2=v2¶m=value#&tro&lo&lo", add_param("http://url.com?p1=v1&p2=v2#&tro&lo&lo", "param", "value"))
end
end
require 'uri'
uri = URI("http://url.com?p1=v1&p2=2")
ar = URI.decode_www_form(uri.query) << ["param","value"]
uri.query = URI.encode_www_form(ar)
p uri #=> #<URI::HTTP:0xa0c44c8 URL:http://url.com?p1=v1&p2=2¶m=value>
uri = URI("http://url.com")
uri.query = "param=value" if uri.query.nil?
p uri #=> #<URI::HTTP:0xa0eaee8 URL:http://url.com?param=value>
EDIT:(by fguillen, to merge all the good propositions and also to make it compatible with his question test suite.)
require 'uri'
def add_param(url, param_name, param_value)
uri = URI(url)
params = URI.decode_www_form(uri.query || "") << [param_name, param_value]
uri.query = URI.encode_www_form(params)
uri.to_s
end
More elegant solution:
url = 'http://example.com?exiting=0'
params = {new_param: 1}
uri = URI.parse url
uri.query = URI.encode_www_form URI.decode_www_form(uri.query || '').concat(params.to_a)
uri.to_s #=> http://example.com?exiting=0&new_param=1
Well, you may also not know if this parameter already exists in url. If you want to replace it with new value in this case, you can do this:
url = 'http://example.com?exists=0&other=3'
params = {'exists' => 1, "not_exists" => 2}
uri = URI.parse url
uri.query = URI.encode_www_form(URI.decode_www_form(uri.query || '').to_h.merge(params))
uri.to_s
You can try to use my gem iri:
Iri.new('http://url.com?p1=v1&p2=v2').add(:param, 'value').to_s
I'm having trouble getting parameters passed in an HTTP Put call, using ruby. Take a look at the "put_data" variable.
When I leave it as a hash, ruby says:
undefined method `bytesize' for #<Hash:0x007fbf41a109e8>
if I convert to a string, I get:
can't convert Net::HTTPUnauthorized into String
I've also tried doing just - '?token=wBsB16NSrfVDpZPoEpM'
def process_activation
uri = URI("http://localhost:3000/api/v1/activation/" + self.member_card_num)
Net::HTTP.start(uri.host, uri.port) do |http|
headers = {'Content-Type' => 'text/plain; charset=utf-8'}
put_data = {:token => "wBsB16NSrfVDpZPoEpM"}
response = http.send_request('PUT', uri.request_uri, put_data, headers)
result = JSON.parse(response)
end
if result['card']['state']['state'] == "active"
return true
else
return false
end
end
I've searched all around, including rubydocs, but can't find an example of how to encode parameters. Any help would be appreciated.
Don't waste your time with NET::HTTP. I used 'rest-client' and had this thing done in minutes...
def process_activation
response = RestClient.put 'http://localhost:3000/api/v1/card_activation/'+ self.member_card_num, :token => "wBsB1pjJNNfiK6NSrfVDpZPoEpM"
result = JSON.parse(response)
return result['card']['state']['state'] == "active"
end
Although the HTTP spec says that headers are case insensitive; Paypal, with their new adaptive payments API require their headers to be case-sensitive.
Using the paypal adaptive payments extension for ActiveMerchant (http://github.com/lamp/paypal_adaptive_gateway) it seems that although the headers are set in all caps, they are sent in mixed case.
Here is the code that sends the HTTP request:
headers = {
"X-PAYPAL-REQUEST-DATA-FORMAT" => "XML",
"X-PAYPAL-RESPONSE-DATA-FORMAT" => "JSON",
"X-PAYPAL-SECURITY-USERID" => #config[:login],
"X-PAYPAL-SECURITY-PASSWORD" => #config[:password],
"X-PAYPAL-SECURITY-SIGNATURE" => #config[:signature],
"X-PAYPAL-APPLICATION-ID" => #config[:appid]
}
build_url action
request = Net::HTTP::Post.new(#url.path)
request.body = #xml
headers.each_pair { |k,v| request[k] = v }
request.content_type = 'text/xml'
proxy = Net::HTTP::Proxy("127.0.0.1", "60723")
server = proxy.new(#url.host, 443)
server.use_ssl = true
server.start { |http| http.request(request) }.body
(i added the proxy line so i could see what was going on with Charles - http://www.charlesproxy.com/)
When I look at the request headers in charles, this is what i see:
X-Paypal-Application-Id ...
X-Paypal-Security-Password...
X-Paypal-Security-Signature ...
X-Paypal-Security-Userid ...
X-Paypal-Request-Data-Format XML
X-Paypal-Response-Data-Format JSON
Accept */*
Content-Type text/xml
Content-Length 522
Host svcs.sandbox.paypal.com
I verified that it is not Charles doing the case conversion by running a similar request using curl. In that test the case was preserved.
The RFC does specify that header keys are case-insensitive, so unfortunately you seem to have hit an annoying requirement with the PayPal API.
Net::HTTP is what is changing the case, although I'm surprised they're not all getting downcased:
# File net/http.rb, line 1160
def []=(key, val)
unless val
#header.delete key.downcase
return val
end
#header[key.downcase] = [val]
end
"Sets the header field corresponding to the case-insensitive key."
As the above is a simple class it could be monkey-patched. I will think further for a nicer solution.
Use following code to force case sensitive headers.
class CaseSensitivePost < Net::HTTP::Post
def initialize_http_header(headers)
#header = {}
headers.each{|k,v| #header[k.to_s] = [v] }
end
def [](name)
#header[name.to_s]
end
def []=(name, val)
if val
#header[name.to_s] = [val]
else
#header.delete(name.to_s)
end
end
def capitalize(name)
name
end
end
Usage example:
post = CaseSensitivePost.new(url, {myCasedHeader: '1'})
post.body = body
http = Net::HTTP.new(host, port)
http.request(post)
If you are still looking for an answer that works. Newer versions have introduced some changes to underlying capitalize method by using to_s. Fix is to make the to_s and to_str return the self so that the returned object is an instance of ImmutableKey instead of the base string class.
class ImmutableKey < String
def capitalize
self
end
def to_s
self
end
alias_method :to_str, :to_s
end
Ref: https://jatindhankhar.in/blog/custom-http-header-and-ruby-standard-library/
I got several issues with the code proposed by #kaplan-ilya because the Net::HTTP library tries to detect the post content-type, and the I ended up with 2 content-type and other fields repeated with different cases.
So the code below should ensure than once a particular case has been choosen, it will stick to the same.
class Post < Net::HTTP::Post
def initialize_http_header(headers)
#header = {}
headers.each { |k, v| #header[k.to_s] = [v] }
end
def [](name)
_k, val = header_insensitive_match name
val
end
def []=(name, val)
key, _val = header_insensitive_match name
key = name if key.nil?
if val
#header[key] = [val]
else
#header.delete(key)
end
end
def capitalize(name)
name
end
def header_insensitive_match(name)
#header.find { |key, _value| key.match Regexp.new(name.to_s, Regexp::IGNORECASE) }
end
end