I'm writing an API Wrapper and running into an issue that I'm not exactly sure how to solve.
Here's the Faraday setup:
#connection = Faraday.new(url: #api_url, params: params,
headers: default_headers,
ssl: { verify: true } ) do |faraday|
faraday.use FaradayMiddleware::Mashify
faraday.use FaradayMiddleware::ParseJson, content_type: /\bjson$/
faraday.use FaradayMiddleware::FollowRedirects
faraday.adapter Faraday.default_adapter
end
This works for 95% of the API calls I'm making - it parses JSON just like I need it to and Mashify's it. Great.
The issue is when the API returns a JSON value NOT wrapped in a JSON object. The API call is this:
https://hacker-news.firebaseio.com/v0/maxitem.json
I can see this isn't a valid JSON object in jsonlint (screenshot below).
Do I need to reconfigure a new Faraday instance for this call alone? That seems somewhat redundant but it's clearly blowing up on the FaradayMiddleware::ParseJson class.
My RSpec test returns this:
Failure/Error: latest = client.max_item
Faraday::ParsingError:
757: unexpected token at '8438316'
Any help is appreciated.
According to the JSON grammar a bare number isn't JSON, it's just a number. So yeah it looks like their API is a little inconsistent and you need a different configuration for that method.
Related
When using Octokit to do post request to the graphql endpoint, how do you set custom Accept: headers in order to turn on a "preview" API?
I need to do a GraphQL query that is part of an API preview, so it requires me to set the Accept header to application/vnd.github.starfox-preview+json (see GitHub API documentation).
I am using Octokit (4.15.0) in a Rails project, and am successfully doing normal queries (not preview mode) using both the v3 and v4 (GraphQL) APIs.
For normal queries (not API preview), my code looks like this, and works flawlessly. Here, the method graphql_query_string forms the correct query string, and github_machine_user is an instance of an Octokit Client object.
Code that works (when custom Accept: header not needed)
def perform_graphql_query(repo_name, org_name, after="")
graphql_query_string = graphql_query(repo_name, org_name, after)
options = { query: graphql_query_string }.to_json
github_machine_user.post '/graphql', options
end
This results in the default Accept header being used, which is, apparently "application/vnd.github.v3+json" (more on how I know this below.)
I tried several ways of adding a custom Accept: header, but none of them work. Here's what I tried:
Unsuccessful Attempt #1:
def perform_graphql_query(repo_name, org_name, after="")
graphql_query_string = graphql_query(repo_name, org_name, after)
options = {:query => graphql_query_string,
:headers => {:accept => "application/vnd.github.starfox-preview+json"}
}.to_json
github_machine_user.post('/graphql', options)
end
Unsuccessful Attempt #2:
def perform_graphql_query(repo_name, org_name, after="")
graphql_query_string = graphql_query(repo_name, org_name, after)
options = {
:query => graphql_query_string,
:accept => "application/vnd.github.starfox-preview+json"
}.to_json
github_machine_user.post('/graphql', options)
end
Neither of theses sets the proper header. From the source code for the post method and the underlying request method, I would have expected this to work. Here's what those look like.
# Make a HTTP POST request
#
# #param url [String] The path, relative to {#api_endpoint}
# #param options [Hash] Body and header params for request
# #return [Sawyer::Resource]
def post(url, options = {})
request :post, url, options
end
def request(method, path, data, options = {})
if data.is_a?(Hash)
options[:query] = data.delete(:query) || {}
options[:headers] = data.delete(:headers) || {}
if accept = data.delete(:accept)
options[:headers][:accept] = accept
end
end
Is it me, or a bug in Octokit?
I can tell the header isn't being set not just because I get results indicating Field 'project' doesn't exist on type 'AddedToProjectEvent', because I also followed the advice in the Octokit documentation about turning on debugging info and I can see the headers on my request.
Here's how they appear (other than the redacted credential)
I, [2020-07-24T12:26:37.989030 #64350] INFO -- request: POST https://api.github.com/graphql
I, [2020-07-24T12:26:37.989109 #64350] INFO -- request: Accept: "application/vnd.github.v3+json"
User-Agent: "Octokit Ruby Gem 4.15.0"
Content-Type: "application/json"
Authorization: "token REDACTED_FOR_STACK_OVERFLOW_POST"
So my requests to set the accept header are not being respected.
I did verify that using curl, I was able to get the query to work properly by passing the correct Accept header. So the graphQl query is properly formed, and when the Accept header is properly put into the query, everything is fine. But I just can't seem to figure out how to get Octokit to respect my wishes in setting the Accept header. I've even looked into the source, and it seems like what I'm doing should work.
Can anyone help?
Update: Posted as an issue on Octokit's GitHub Repo too.
Update #2: I tried removing the to_json. Unfortunately, that gives a stack trace as shown below.
If I remove the to_json and then only pass the {:accept => "application/vnd.github.starfox-preview+json"} then it does set the correct header. But if try including :query in that hash, it results in the error below, unless I have the to_json on the hash. I just can't seem to win.
NoMethodError undefined method `each' for #<String:0x00007fb05be10b00>
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/utils/params_hash.rb:28:in `update'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/sawyer-0.8.2/lib/sawyer/agent.rb:99:in `block in call'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:489:in `block in run_request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:506:in `block in build_request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/request.rb:55:in `block in create'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/request.rb:54:in `tap'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/request.rb:54:in `create'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:502:in `build_request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:484:in `run_request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:279:in `post'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/sawyer-0.8.2/lib/sawyer/agent.rb:94:in `call'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/octokit-4.18.0/lib/octokit/connection.rb:156:in `request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/octokit-4.18.0/lib/octokit/connection.rb:28:in `post'
/Users/pconrad/github/project-anacapa/anacapa-github-linker/app/jobs/course/course_github_repo_get_sdlc_events.rb:105:in `perform_graphql_query'
Update #3: Added my own answer below but it seems hacky. I had to call a private method, which one is not supposed to do, and definitely not supposed to "have to do". So, still hoping for a better solution.
This is what finally worked. I had to get "dirty" and invoke a private method using .send which is not ideal.
graphql_query_string = graphql_query(repo_name, org_name, after).gsub("\n","")
data = {
:query => graphql_query_string,
}.to_json
options = {
:headers => {
:accept => Octokit::Preview::PREVIEW_TYPES[:project_card_events]
}
}
github_machine_user.send :request, :post, '/graphql', data,options
If there is a better way, I'd welcome the suggestion. I agree with the other posters that said it doesn't make any sense to pass to_json on the data part, but without it, I get the stack trace about each not being defined on a string.
Remove calling #to_json on options. You're passing a String into #post as options. options gets put into #request as data. #request will do its thing on data if it was a Hash but since it's a String, it won't populate the new options that is passed to the client in #request.
I'm new to Ruby and API, so my apologies if this is super simple...
I need to have script that will first POST to initiate the creation of an export file, and then have a GET call to retrieve the file. The GET call needs to use part of the POST json response.
I'm using the httparty gem.
I think I need to create a variable that equals the parsed json, and then make that variable part of the GET call, but I'm not clear on how to do that.
Help is appreciated.
require 'httparty'
url = 'https://api.somewhere.org'
response = HTTParty.post(url)
puts response.parse_response
json response:
export_files"=>
{"id"=> #####,
"export_id"=> #####,
"status"=>"Queued"}}
In my GET call I need to use the export_id number in the url.
HTTParty.get('https://api.somewhere.org/export_id/####')
As described in the comments but a bit more verbose and skeleton for error:
require 'httparty'
require 'json'
url = 'https://api.somewhere.org'
response = HTTParty.post(url)
if hash = JSON.parse(response.body)
if export_id = hash[:export_files][:export_id]
post = HTTParty.post("https://api.somewhere.org/export_id/#{export_id}")
end
else
# handle error
end
I am trying to view the JSON request which is sent from POSTMAN through a POST request to add a security group info to a table, and my request looks like one below
POST /securitygroup HTTP/1.1
Host: localhost:9292
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: c4bef1db-d544-c923-3b0b-e7004e2dd093
{
"securitygroup":{
"secgrp_id": 124,
"secgrp_nm": "SECURITY ADMIN",
"secgrp_profile_nme": "ADMIN"
}
}
Roda code looks like below
# cat config.ru
require "roda"
require "sequel"
require "oci8"
require "json"
DB = Sequel.oracle(host: 'xyz.dev.com', port: '1525', database: 'devbox1', user: 'abc', password: 'pass')
class App < Roda
plugin :json, classes: [Array, Hash, Sequel::Model, Sequel::Dataset]
route do |r|
# secgroup = DB[:security_groups]
# secgroup.insert(r.params["securitygroup"])
# secgroup
# above insert threw the following error
# OCIError: ORA-00947: not enough values,
# because SQL generated as below
# INSERT INTO "SECURITYGROUPS" VALUES (NULL)
# so I am trying to access the request object 'r', I feel that I am doing
# something which is not correct
{"response": r.params.["securitygroup"]["secgrp_id"]}
# throws undefined method `[]' for nil:NilClass
end
end
Can you please take a look at the request and point me, where am I going wrong, Request format not correct or is there a different way to handle the request on the ruby code.
I need help to parse the request that comes in as JSON similar to code presented in https://twin.github.io/introduction-to-roda/
r.post "albums" do
album = Album.create(r.params["album"])
r.redirect album_path(album) # /albums/1
end
You need just a little tweak: Add to the App the plugin :json_parser.
class App < Roda
# use this plugin to convert response to json
plugin :json
# use this plugin to convert request from json
plugin :json_parser
...
end
See in the Roda documentation "Plugins that Ship with Roda" in the group "others" there is the json_parser plugin.
I am pretty new to working with Ruby, especially with APIs but I've been trying to get the Darksky API to work, but I'm afraid I'm missing something obvious with how I'm using it.
Here is what I have
require 'darksky'
darksky = Darksky::API.new('my api key')
forecast = darksky.forecast('34.0500', '118.2500')
forecast
When I run this from the command line nothing happens. What am I doing wrong here?
Simply using forecast isn't going to do anything. You need to use puts at a minimum:
puts forecast
Or, see if Ruby's object pretty-printer can return something more interesting:
require 'pp'
pp forecast
Digging in further, I think their API doesn't work. Based on their examples, using a valid key and their location samples, plus the locations from their source site Forecast.io, also returns nil.
Using the REST interface directly from Forecast.io's site does return JSON. JSON is very easy to work with in Ruby, so it's a good way to go.
Here's some code to test the API, and Forecast.io's REST interface:
API_KEY = 'xxxxxxxxxxxxxxxxxxx'
LOCATION = %w[37.8267 -122.423]
require 'darksky'
darksky = Darksky::API.new(API_KEY)
forecast = darksky.forecast(*LOCATION)
forecast # => nil
brief_forecast = darksky.brief_forecast(*LOCATION)
brief_forecast # => nil
require 'json'
require 'httparty'
URL = "https://api.forecast.io/forecast/#{ API_KEY }/37.8267,-122.423"
puts URL
# >> https://api.forecast.io/forecast/xxxxxxxxxxxxxxxxxxx/37.8267,-122.423
puts HTTParty.get(URL).body[0, 80]
# >> {"latitude":37.8267,"longitude":-122.423,"timezone":"America/Los_Angeles","offse
Notice that LOCATION is 37.8267,-122.423 in both cases, which is Alcatraz according to the Forecast.io site. Also notice that the body output displayed is a JSON string.
Pass the returned JSON to the Ruby's JSON class like:
JSON[returned_json]
to get it parsed back into a Ruby Hash. Using OpenURI (because it comes with Ruby) instead of HTTParty, and passing it to JSON for parsing looks like:
body = open(URL).read
puts JSON[body]
I'm brand new to ruby (first day working with ruby) so please forgive any novice questions and lack of understanding.
I'm trying to validate the responses to http callouts.
For example, let's say the endpoint is the following:
https://applicationname-api-sbox02.herokuapp.com
And, I'm trying to authenticate a user by sending a get request like this:
get_response = RestClient.get( "https://applicationname-api-sbox02.herokuapp.com/api/v1/users",
{
"Content-Type" => "application/json",
"Authorization" => "token 4d012314b7e46008f215cdb7d120cdd7",
"Manufacturer-Token" => "8d0693ccfe65104600e2555d5af34213"
}
)
Now, I want to validate the response and do the following:
- parse the response to ensure that it is valid JSON
- do some validation and verify the JSON has the correct data (verify that id == 4 for example)
- if an error is encountered, raise an exception using the 'raise' method.
In my first feeble attempt I tried the following:
puts get_response.body
if get_response.code == 200
puts "********* Get current user successful"
else
puts "Get current user failed!!"
end
Now, this returned that getting the current user was successful, but how do I actually parse the json, verify the correct id, and raise an exception if an error occurred?
Instead of raising an exception, write a test.
A straightforward approach, using the json parser and unit test framework from the std lib:
require 'minitest/autorun'
require 'rest_client'
require 'json'
class APITest < MiniTest::Unit::TestCase
def setup
response = RestClient.get("https://applicationname-api-sbox02.herokuapp.com/api/v1/users",
{
"Content-Type" => "application/json",
"Authorization" => "token 4d012314b7e46008f215cdb7d120cdd7",
"Manufacturer-Token" => "8d0693ccfe65104600e2555d5af34213"
}
)
#data = JSON.parse response.body
end
def test_id_correct
assert_equal 4, #data['id']
end
end
Execute with ruby $filename
JSON.parse parses a JSON string into a ruby hash
Getting started with minitest
If you are using ruby 1.8, you'll need to install the json gem and either install the minitest gem, or switch to the older testunit API. If you choose the latter, then you'll need to change require 'minitest/autorun' -> require 'test/unit' and MiniTest::Unit::TestCase -> Test::Unit::TestCase
I'm a little late to the party, but I recently co-created an rspec driven framework called Airborne for just this purpose. Check it out: https://github.com/brooklynDev/airborne
here is an example from our specs so you can see how we test json api:
it 'returns charge' do
get "/charges/#{charge.id}", '', headers
expect(response.status).to eq(200)
expect(response).to match_response_schema(:charge)
expect(response).to match_json(<<-JSON)
{
"id":"{id}",
"email": "{email}",
"ip": "127.0.0.1",
"amount": 10500,
"state": "captured",
"captured_amount": 10500,
}
JSON
end
Lets look at it closely
match_response_schema(:charge)
This matcher checks that json we get in response is in general valid. We use json-schema (json schema validator) for it. Guys from Thoughtbot have a detailed guide how to use json schema validator and create own matcher in this blog post.
Understanding JSON Schema is where I got a lot of useful information on how to create schemas for JSON documents.
match_json
This is our own matcher and we have released match_json gem recently. Using it you can test structure and values of your json. Here are two great features of this matcher:
if you don't know exact values, you can use patterns like {id}, {uuid} {date_time}, etc. we have predefined patterns but you can add your own too.
you get clear failure message what is wrong with your json e.g. "5" was not found in " > array":[1,2,3]
Parsing json can be done with the json gem: http://flori.github.com/json/
Parsed json is accessed through key/value just like in javascript. You can easily verify the values and conditionally raise errors.
Raising errors is done like so:
raise "the ID was #{id} instead of 4"
And writing unit tests can be done with Test::Unit - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/test/unit/rdoc/Test/Unit.html