net/http PUT with body causes problems - ruby

I want to use the Cloudera API from ruby. Therefore I want to
update the configuration of the cloudera manager, which is done
with an HTTP PUT request containing some json data.
The URL is http://localhost:7180/api/v11/cm/config, my first approach was the following code:
require 'net/http'
require 'base64'
port = 7180
host = 'localhost'
req = Net::HTTP::Put.new('/api/v11/cm/config')
req.body = '{"items":[{"name":"TSQUERY_STREAMS_LIMIT","value":1000},{"name":"parcel_proxy_server","value":"proxy"},{"name":"parcel_proxy_port","value":"8080"},{"name":"parcel_update_freq","value":"1"}]}'
req['Content-Type'] = 'application/json'
req['Authorization'] = "Basic #{Base64.encode64('admin:admin')}"
client = Net::HTTP.new(host, port)
resp = client.request(req)
puts resp
puts resp.to_hash
puts resp.body
This variant returns a 400 Bad Request response with the message
"message" : "No content to map due to end-of-input\n at [Source: org.apache.cxf.transport.http.AbstractHTTPDestination$1#4561cbc9; line: 1, column: 1]"
If I want to trace this in wireshark with the setting tcp.port == 7180, this request does somehow not show up.
Then I switched to using Net::HTTP.start with the following source code:
require 'net/http'
require 'base64'
port = 7180
host = 'localhost'
req = Net::HTTP::Put.new('/api/v11/cm/config')
req.body = '{"items":[{"name":"TSQUERY_STREAMS_LIMIT","value":1000},{"name":"parcel_proxy_server","value":"proxy"},{"name":"parcel_proxy_port","value":"8080"},{"name":"parcel_update_freq","value":"1"}]}'
req['Content-Type'] = 'application/json'
req['Authorization'] = "Basic #{Base64.encode64('admin:admin')}"
resp = Net::HTTP.start(host, port) { |client| client.request(req) }
puts resp
puts resp.to_hash
puts resp.body
This thing also returned a bad request, but without any body or else. In contrast to the first approach, this one showed up in wireshark but the body it posted was depicted as with Hypertext-Transfer-Protocol. If I do the same request with cURL, the request body is correctly shown as JavaScript Object Notation.
Does anybody know what the problem of my requests is?

Ok so I tried around for several hours but this behavior is rather strange:
To resolve this I first used Net::HTTP.start instead of Net::HTTP.new. Second and even more important:
I used to do basic authentication by setting the header value manually as you could see in
req['Authorization'] = "Basic #{Base64.encode64('admin:admin')}"
But Net::HTTP::Put already provides a basic_auth method. If this one is not used, strange errors as the ones that I got will occur.
So the working version looks like this:
req = Net::HTTP::Put.new('/api/v11/cm/config')
req.body = '{"items":[{"name":"TSQUERY_STREAMS_LIMIT","value":1000},{"name":"parcel_proxy_server","value":"proxy"},{"name":"parcel_proxy_port","value":"8080"},{"name":"parcel_update_freq","value":"1"}]}'
req['Content-Type'] = 'application/json'
req.basic_auth 'admin', 'admin'
resp = Net::HTTP.start(host, port) { |client| client.request(req) }

Related

Ruby putting token key in requests

I dont know how to put my key into my requests so they are sent back as
{"status"=>"400", "message"=>"Token parameter is required."}
This is the code I have been using
require 'net/http'
require 'json'
token = 'YiwwVvywLngtPT***************'
url = 'https://www.ncdc.noaa.gov/cdo-web/api/v2/stations?locationid=FIPS:23&limit=5&sortfield=mindate'
uri = URI(url)
response = Net::HTTP.get(uri)
response.authorization = token
puts JSON.parse(response)
I have tried a couple different things I've found on the internet but all of them just give errors for
undefined method `methodname' for #<String:0x00007fd97519abd0>
According to the API documentation (based on the URL you referenced), you need to provide the token in a header named token.
So, you should probably try some variation of the below (code untested):
token = 'YiwwVvywLngtPT***************'
url = 'https://www.ncdc.noaa.gov/cdo-web/api/v2/stations?locationid=FIPS:23&limit=5&sortfield=mindate'
uri = URI(url)
request = Net::HTTP::Get.new(uri)
request['token'] = token
response = Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(request)
end
More information about Net:HTTP headers can be found in this StackOverflow answer.
As a side note, if you are not locked on using Net::HTTP, consider switching to a friendlier HTTP client, perhaps HTTParty. Then, the complete code looks like this:
require 'httparty'
token = 'YiwwVvywLngtPT***************'
url = 'https://www.ncdc.noaa.gov/cdo-web/api/v2/stations?locationid=FIPS:23&limit=5&sortfield=mindate'
response = HTTParty.get url, headers: { token: token }
puts response.body

Ruby GET NET HTTP request does not work with AUTHORIZATION and ACCEPT when passed in a header

I've been using the code below to call a third party API . This code works fine (i've changed the url and the credentials but the structure of the code is the same) :
require 'base64'
require 'httparty'
require 'json'
######################################################################
# Get the token first
######################################################################
consumer_key = "my_key"
consumer_secret = "my_secret"
credentials = Base64.encode64("#{consumer_key}:#{consumer_secret}").gsub("\n", '')
url = "https://mysite/token"
body = "grant_type=client_credentials"
headers = {
"Authorization" => "Basic #{credentials}",
"Content-Type" => "application/x-www-form-urlencoded;charset=UTF-8"
}
r = HTTParty.post(url, body: body, headers: headers)
bearer_token = JSON.parse(r.body)['access_token']
######################################################################
# Use the token in a call as authorisation header
######################################################################
api_url = "https://apisite/the_value_i_am_looking_for_in_the_api"
url = URI.parse(api_url)
req = Net::HTTP.new(url.host, url.port)
req.use_ssl = true
# If we are just passing a key that doesn't need to be in the token format
headers = {
'Authorization' => "Bearer #{bearer_token}"
}
# Get the response back (he data is in the response body: resp.body )
resp = req.get(url, headers)
My issue is that the API providers have changed their API so you now need to pass an "accept" into the call via the header. I used POSTMAN to make the call, added the accept to the header and was able to get it working without issue. So far so good.
I then changed my ruby code to extend the headers section to include the accept, using the code below:
headers = {
'Authorization' => "Bearer #{bearer_token}",
'Accept' => 'application/vnd.bluebadge-api.v1+json'
}
I've not added an accept to a header before so I may have gotten the syntax wrong.
However, this returns an unauthorised 401 response code:
#<Net::HTTPUnauthorized 401 Unauthorized readbody=true>
I thought I might have the credentials wrong so remove the accept, try again and this changes to a 406 response code:
#<Net::HTTPNotAcceptable 406 Not Acceptable readbody=true>
If I examine the response I get the message I would expect that the accept header is not the supported version. So I know the credentials are correct (and the fact they match the postman credentials which works):
"{\"apiVersion\": \"1\",\"context\": null,\"id\": null,\"method\": null,\"error\": {\"code\": null,\"message\": null,\"reason\": \"Accept header version is not a currently supported version.\",\"errors\": null}}\n"
So I know all my credentials are correct because I've copied them into the postman request which works with no errors. The value for the accept header is correct because I copied that from a working postman request too.
I am at a loss for why this wouldn't work.
I've looked through the NET HTTP library and cant find anything to help me there. I've seen a couple of posts elsewhere which I've tried and they haven't worked either.
I appreciate any help in trying to solve this.
Found the problem. I was using the credentials from the production environment to get the token then trying to query the test environment API. In my defence they look very similar (only 3 characters different). I think I had a case of the code blindness.
The code I posted does work when I put the correct URL for the environments.
I also found that I could use this:
uri = URI.parse("https://myapi/some_text")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Get.new(uri.request_uri)
request["Authorization"] = "Bearer #{bearer_token}"
request["Accept"] = "application/vnd.bluebadge-api.v1+json"
response = http.request(request)
Or using HTTParty like this:
response = HTTParty.get('https://myapi/some_text', {
headers: {"Authorization" => "Bearer #{bearer_token}", "Accept" => "application/vnd.bluebadge-api.v1+json" }
})
I would prefer the format of my orginal code or the HTTparty code because it is easy to see from the code that you're passing headers. Hopefully this will help others to double check their authorization credentials...

How to add "Content-type" header in HTTP POST

I keep getting a
400 "Bad Request" (Net::HTTPServerException)
error whenever I try to add a content-type header from various methods.
I've seen several different examples and I can't get any to work. My goal is to add a content type of JSON to my request. Without the header, my request doesn't error:
def post_data(notice)
uri = URI('my uri is here')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
req = Net::HTTP::Post.new("#{uri.path}?#{uri.query}")
text = notice
req.add_field('Content-Type', 'application/json')
req.body = "{\"sensu_payload\" = #{payload(text).to_json}}"
response = http.request(req)
verify_response(response)
end
I've also tried this method of adding a header:
request = Net::HTTP::Post.new(#path, initheader = {'Content-Type' =>'application/json'})
Use uri.path instead of #path:
request = Net::HTTP::Post.new(uri.path, initheader = {'Content-Type' =>'application/json'})
Instead of add_field, I think you should use the hash [] form:
req['Content-Type'] = 'application/json'
See the "Setting Headers" example in the documentation and []=.
Also, using:
"#{uri.path}?#{uri.query}"
is never a good idea. String concatenation can't manage the complexities of correctly encoding illegal values in a query, which can break a request. Consider doing something like this instead:
require 'uri'
require 'uri'
foo = URI('http://www.example.com') # => #<URI::HTTP http://www.example.com>
foo.query = URI::encode_www_form({'bar' => 'path/to/file', 'baz' => 'this & that'})
foo.to_s # => "http://www.example.com?bar=path%2Fto%2Ffile&baz=this+%26+that"
Beyond that, I'd recommend using any of the other HTTP-client gems available. They make it much easier to deal with unexpected situations, like redirects and retries than Net::HTTP. It's more like the building block for features that aren't available other ways.

POST request with Ruby and Calabash

I use Calabsh to test iOS app. Duringtest I need to create POST request to change some values and then verify that changes are reflected in UI.
Request looks like:
wwww.testserver.com/userAddMoney?user_id=1&amount=999
To authorize on server I need to pass special parameters to Header of request:
Headers: X-Testing-Auth-Secret: kI7wGju76kjhJHGklk76
require 'net/http'
uri = URI.parse('http://www.testserver.com/userAddMoney?user_id=1&amount=999')
http = Net::HTTP.new(uri.host,uri.port)
## https.use_ssl = true # use https, need require net/https
req = Net::HTTP::Post.new(uri.path)
req['X-Testing-Auth-Secret'] = 'kI7wGju76kjhJHGklk76'
res = http.request(req)
Docs here: Net::HTTP::Post Net::HTTPSession

Ruby send JSON request

How do I send a JSON request in ruby? I have a JSON object but I dont think I can just do .send. Do I have to have javascript send the form?
Or can I use the net/http class in ruby?
With header - content type = json and body the json object?
uri = URI('https://myapp.com/api/v1/resource')
body = { param1: 'some value', param2: 'some other value' }
headers = { 'Content-Type': 'application/json' }
response = Net::HTTP.post(uri, body.to_json, headers)
require 'net/http'
require 'json'
def create_agent
uri = URI('http://api.nsa.gov:1337/agent')
http = Net::HTTP.new(uri.host, uri.port)
req = Net::HTTP::Post.new(uri.path, 'Content-Type' => 'application/json')
req.body = {name: 'John Doe', role: 'agent'}.to_json
res = http.request(req)
puts "response #{res.body}"
rescue => e
puts "failed #{e}"
end
HTTParty makes this a bit easier I think (and works with nested json etc, which didn't seem to work in other examples I've seen.
require 'httparty'
HTTParty.post("http://localhost:3000/api/v1/users", body: {user: {email: 'user1#example.com', password: 'secret'}}).body
This works on ruby 2.4 HTTPS Post with JSON object and the response body written out.
require 'net/http' #net/https does not have to be required anymore
require 'json'
require 'uri'
uri = URI('https://your.secure-url.com')
Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
request = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
request.body = {parameter: 'value'}.to_json
response = http.request request # Net::HTTPResponse object
puts "response #{response.body}"
end
real life example, notify Airbrake API about new deployment via NetHttps
require 'uri'
require 'net/https'
require 'json'
class MakeHttpsRequest
def call(url, hash_json)
uri = URI.parse(url)
req = Net::HTTP::Post.new(uri.to_s)
req.body = hash_json.to_json
req['Content-Type'] = 'application/json'
# ... set more request headers
response = https(uri).request(req)
response.body
end
private
def https(uri)
Net::HTTP.new(uri.host, uri.port).tap do |http|
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
end
end
end
project_id = 'yyyyyy'
project_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
url = "https://airbrake.io/api/v4/projects/#{project_id}/deploys?key=#{project_key}"
body_hash = {
"environment":"production",
"username":"tomas",
"repository":"https://github.com/equivalent/scrapbook2",
"revision":"live-20160905_0001",
"version":"v2.0"
}
puts MakeHttpsRequest.new.call(url, body_hash)
Notes:
in case you doing authentication via Authorisation header set header req['Authorization'] = "Token xxxxxxxxxxxx" or http://api.rubyonrails.org/classes/ActionController/HttpAuthentication/Token.html
A simple json POST request example for those that need it even simpler than what Tom is linking to:
require 'net/http'
uri = URI.parse("http://www.example.com/search.json")
response = Net::HTTP.post_form(uri, {"search" => "Berlin"})
I like this light weight http request client called `unirest'
gem install unirest
usage:
response = Unirest.post "http://httpbin.org/post",
headers:{ "Accept" => "application/json" },
parameters:{ :age => 23, :foo => "bar" }
response.code # Status code
response.headers # Response headers
response.body # Parsed body
response.raw_body # Unparsed body
It's 2020 - nobody should be using Net::HTTP any more and all answers seem to be saying so, use a more high level gem such as Faraday - Github
That said, what I like to do is a wrapper around the HTTP api call,something that's called like
rv = Transporter::FaradayHttp[url, options]
because this allows me to fake HTTP calls without additional dependencies, ie:
if InfoSig.env?(:test) && !(url.to_s =~ /localhost/)
response_body = FakerForTests[url: url, options: options]
else
conn = Faraday::Connection.new url, connection_options
Where the faker looks something like this
I know there are HTTP mocking/stubbing frameworks, but at least when I researched last time they didn't allow me to validate requests efficiently and they were just for HTTP, not for example for raw TCP exchanges, this system allows me to have a unified framework for all API communication.
Assuming you just want to quick&dirty convert a hash to json, send the json to a remote host to test an API and parse response to ruby this is probably fastest way without involving additional gems:
JSON.load `curl -H 'Content-Type:application/json' -H 'Accept:application/json' -X POST localhost:3000/simple_api -d '#{message.to_json}'`
Hopefully this goes without saying, but don't use this in production.
The net/http api can be tough to use.
require "net/http"
uri = URI.parse(uri)
Net::HTTP.new(uri.host, uri.port).start do |client|
request = Net::HTTP::Post.new(uri.path)
request.body = "{}"
request["Content-Type"] = "application/json"
client.request(request)
end
data = {a: {b: [1, 2]}}.to_json
uri = URI 'https://myapp.com/api/v1/resource'
https = Net::HTTP.new uri.host, uri.port
https.use_ssl = true
https.post2 uri.path, data, 'Content-Type' => 'application/json'
Using my favourite http request library in ruby:
resp = HTTP.timeout(connect: 15, read: 30).accept(:json).get('https://units.d8u.us/money/1/USD/GBP/', json: {iAmOne: 'Hash'}).parse
resp.class
=> Hash

Resources