ORM JSON request handling for dataset insert - ruby

I am trying to view the JSON request which is sent from POSTMAN through a POST request to add a security group info to a table, and my request looks like one below
POST /securitygroup HTTP/1.1
Host: localhost:9292
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: c4bef1db-d544-c923-3b0b-e7004e2dd093
{
"securitygroup":{
"secgrp_id": 124,
"secgrp_nm": "SECURITY ADMIN",
"secgrp_profile_nme": "ADMIN"
}
}
Roda code looks like below
# cat config.ru
require "roda"
require "sequel"
require "oci8"
require "json"
DB = Sequel.oracle(host: 'xyz.dev.com', port: '1525', database: 'devbox1', user: 'abc', password: 'pass')
class App < Roda
plugin :json, classes: [Array, Hash, Sequel::Model, Sequel::Dataset]
route do |r|
# secgroup = DB[:security_groups]
# secgroup.insert(r.params["securitygroup"])
# secgroup
# above insert threw the following error
# OCIError: ORA-00947: not enough values,
# because SQL generated as below
# INSERT INTO "SECURITYGROUPS" VALUES (NULL)
# so I am trying to access the request object 'r', I feel that I am doing
# something which is not correct
{"response": r.params.["securitygroup"]["secgrp_id"]}
# throws undefined method `[]' for nil:NilClass
end
end
Can you please take a look at the request and point me, where am I going wrong, Request format not correct or is there a different way to handle the request on the ruby code.
I need help to parse the request that comes in as JSON similar to code presented in https://twin.github.io/introduction-to-roda/
r.post "albums" do
album = Album.create(r.params["album"])
r.redirect album_path(album) # /albums/1
end

You need just a little tweak: Add to the App the plugin :json_parser.
class App < Roda
# use this plugin to convert response to json
plugin :json
# use this plugin to convert request from json
plugin :json_parser
...
end
See in the Roda documentation "Plugins that Ship with Roda" in the group "others" there is the json_parser plugin.

Related

With octokit.rb (Ruby library for GitHub API) how do you set Accept: header in post request (for API Previews)?

When using Octokit to do post request to the graphql endpoint, how do you set custom Accept: headers in order to turn on a "preview" API?
I need to do a GraphQL query that is part of an API preview, so it requires me to set the Accept header to application/vnd.github.starfox-preview+json (see GitHub API documentation).
I am using Octokit (4.15.0) in a Rails project, and am successfully doing normal queries (not preview mode) using both the v3 and v4 (GraphQL) APIs.
For normal queries (not API preview), my code looks like this, and works flawlessly. Here, the method graphql_query_string forms the correct query string, and github_machine_user is an instance of an Octokit Client object.
Code that works (when custom Accept: header not needed)
def perform_graphql_query(repo_name, org_name, after="")
graphql_query_string = graphql_query(repo_name, org_name, after)
options = { query: graphql_query_string }.to_json
github_machine_user.post '/graphql', options
end
This results in the default Accept header being used, which is, apparently "application/vnd.github.v3+json" (more on how I know this below.)
I tried several ways of adding a custom Accept: header, but none of them work. Here's what I tried:
Unsuccessful Attempt #1:
def perform_graphql_query(repo_name, org_name, after="")
graphql_query_string = graphql_query(repo_name, org_name, after)
options = {:query => graphql_query_string,
:headers => {:accept => "application/vnd.github.starfox-preview+json"}
}.to_json
github_machine_user.post('/graphql', options)
end
Unsuccessful Attempt #2:
def perform_graphql_query(repo_name, org_name, after="")
graphql_query_string = graphql_query(repo_name, org_name, after)
options = {
:query => graphql_query_string,
:accept => "application/vnd.github.starfox-preview+json"
}.to_json
github_machine_user.post('/graphql', options)
end
Neither of theses sets the proper header. From the source code for the post method and the underlying request method, I would have expected this to work. Here's what those look like.
# Make a HTTP POST request
#
# #param url [String] The path, relative to {#api_endpoint}
# #param options [Hash] Body and header params for request
# #return [Sawyer::Resource]
def post(url, options = {})
request :post, url, options
end
def request(method, path, data, options = {})
if data.is_a?(Hash)
options[:query] = data.delete(:query) || {}
options[:headers] = data.delete(:headers) || {}
if accept = data.delete(:accept)
options[:headers][:accept] = accept
end
end
Is it me, or a bug in Octokit?
I can tell the header isn't being set not just because I get results indicating Field 'project' doesn't exist on type 'AddedToProjectEvent', because I also followed the advice in the Octokit documentation about turning on debugging info and I can see the headers on my request.
Here's how they appear (other than the redacted credential)
I, [2020-07-24T12:26:37.989030 #64350] INFO -- request: POST https://api.github.com/graphql
I, [2020-07-24T12:26:37.989109 #64350] INFO -- request: Accept: "application/vnd.github.v3+json"
User-Agent: "Octokit Ruby Gem 4.15.0"
Content-Type: "application/json"
Authorization: "token REDACTED_FOR_STACK_OVERFLOW_POST"
So my requests to set the accept header are not being respected.
I did verify that using curl, I was able to get the query to work properly by passing the correct Accept header. So the graphQl query is properly formed, and when the Accept header is properly put into the query, everything is fine. But I just can't seem to figure out how to get Octokit to respect my wishes in setting the Accept header. I've even looked into the source, and it seems like what I'm doing should work.
Can anyone help?
Update: Posted as an issue on Octokit's GitHub Repo too.
Update #2: I tried removing the to_json. Unfortunately, that gives a stack trace as shown below.
If I remove the to_json and then only pass the {:accept => "application/vnd.github.starfox-preview+json"} then it does set the correct header. But if try including :query in that hash, it results in the error below, unless I have the to_json on the hash. I just can't seem to win.
NoMethodError undefined method `each' for #<String:0x00007fb05be10b00>
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/utils/params_hash.rb:28:in `update'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/sawyer-0.8.2/lib/sawyer/agent.rb:99:in `block in call'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:489:in `block in run_request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:506:in `block in build_request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/request.rb:55:in `block in create'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/request.rb:54:in `tap'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/request.rb:54:in `create'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:502:in `build_request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:484:in `run_request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/faraday-1.0.0/lib/faraday/connection.rb:279:in `post'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/sawyer-0.8.2/lib/sawyer/agent.rb:94:in `call'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/octokit-4.18.0/lib/octokit/connection.rb:156:in `request'
/Users/pconrad/.rvm/gems/ruby-2.6.5/gems/octokit-4.18.0/lib/octokit/connection.rb:28:in `post'
/Users/pconrad/github/project-anacapa/anacapa-github-linker/app/jobs/course/course_github_repo_get_sdlc_events.rb:105:in `perform_graphql_query'
Update #3: Added my own answer below but it seems hacky. I had to call a private method, which one is not supposed to do, and definitely not supposed to "have to do". So, still hoping for a better solution.
This is what finally worked. I had to get "dirty" and invoke a private method using .send which is not ideal.
graphql_query_string = graphql_query(repo_name, org_name, after).gsub("\n","")
data = {
:query => graphql_query_string,
}.to_json
options = {
:headers => {
:accept => Octokit::Preview::PREVIEW_TYPES[:project_card_events]
}
}
github_machine_user.send :request, :post, '/graphql', data,options
If there is a better way, I'd welcome the suggestion. I agree with the other posters that said it doesn't make any sense to pass to_json on the data part, but without it, I get the stack trace about each not being defined on a string.
Remove calling #to_json on options. You're passing a String into #post as options. options gets put into #request as data. #request will do its thing on data if it was a Hash but since it's a String, it won't populate the new options that is passed to the client in #request.

how do I dynamically generate multiple HTTP mock responses with VCR for Ruby/RSpec?

I'm hitting an API and for each request, there was a recorded request/response in the cassette YAML file. However, the only difference between the requests was the id in the query parameters.
How do I shrink my YAML file so that URLs are dynamically generated for each request?
You can use Dynamic ERB Cassettes with VCR, you just have to pass in the :erb option which can have a value of true or a hash containing the template variables to be passed to the cassette:
ids = [0, 1, 2, 3]
VCR.use_cassette('dynamic_generated_requests', :erb => { :ids => ids }) do
# Make HTTP Requests
end
YAML File with ERB
And your YAML file would look like this:
---
http_interactions:
<% ids.each do |id| %>
- request:
method: post
uri: https://api.example.com/path/to/rest_api/<%= id %>/method
body:
encoding: UTF-8
headers:
content-type:
- application/json
response:
status:
code: 200
message: OK
headers:
cache-control:
- no-cache, no-store
content-type:
- application/json
connection:
- Close
body:
encoding: UTF-8
string: '{"status_code": <%= id %>}'
http_version: '1.1'
recorded_at: Tue, 15 Jan 2019 16:14:14 GMT
<% end %>
recorded_with: VCR 3.0.0
Note: the .yml file extension is still used because VCR handles the ERB processing through the :erb option.
Debugging: raw_cassette_bytes
If you want to debug this and make sure the YAML file looks good, you can print out the rendered YAML file with the raw_cassette_bytes method:
puts VCR.current_cassette.send(:raw_cassette_bytes)
Use that within the VCR.use_cassette block:
VCR.use_cassette('dynamic_generated_requests', :erb => { :ids => ids }) do
puts VCR.current_cassette.send(:raw_cassette_bytes)
# Make HTTP Requests
end

Faraday::ParsingError - 757: unexpected token

I'm writing an API Wrapper and running into an issue that I'm not exactly sure how to solve.
Here's the Faraday setup:
#connection = Faraday.new(url: #api_url, params: params,
headers: default_headers,
ssl: { verify: true } ) do |faraday|
faraday.use FaradayMiddleware::Mashify
faraday.use FaradayMiddleware::ParseJson, content_type: /\bjson$/
faraday.use FaradayMiddleware::FollowRedirects
faraday.adapter Faraday.default_adapter
end
This works for 95% of the API calls I'm making - it parses JSON just like I need it to and Mashify's it. Great.
The issue is when the API returns a JSON value NOT wrapped in a JSON object. The API call is this:
https://hacker-news.firebaseio.com/v0/maxitem.json
I can see this isn't a valid JSON object in jsonlint (screenshot below).
Do I need to reconfigure a new Faraday instance for this call alone? That seems somewhat redundant but it's clearly blowing up on the FaradayMiddleware::ParseJson class.
My RSpec test returns this:
Failure/Error: latest = client.max_item
Faraday::ParsingError:
757: unexpected token at '8438316'
Any help is appreciated.
According to the JSON grammar a bare number isn't JSON, it's just a number. So yeah it looks like their API is a little inconsistent and you need a different configuration for that method.

Why can't I post variables using HTTParty?

I'm trying to use HTTParty to manage requests to an API. These are the instructions from the documentation:
url: https://www.havelockinvestments.com/r/orderbook
Required Post Variables symbol: "VTX"
Return Data
status: Contains 'ok' or 'error'
message: Contains error message if applicable
bids: Contains Bids array, as price=>amount pairs
asks: Contains Asks array, as price=>amount pairs
This is what I'm putting in my Ruby script:
require 'httparty'
response = HTTParty.post(
'https://www.havelockinvestments.com/r/orderbook',
:query => { :symbol => "VTX" }
)
But I'm getting an error response:
{"status":"error","message":"post:symbol is required"}
What am I doing wrong here when posting the symbol variable?
The original documentation is at: https://www.havelockinvestments.com/apidoc.php
Documentation seems a bit sparse on the HTTParty Github page, but from the examples it looks like you specify the parameters in a hash as a value to the :body key in the options for HTTParty#post
Like so:
response = HTTParty.post('https://www.havelockinvestments.com/r/orderbook', {body: {symbol: "VTX"}})

How to test a JSON REST API

I'm brand new to ruby (first day working with ruby) so please forgive any novice questions and lack of understanding.
I'm trying to validate the responses to http callouts.
For example, let's say the endpoint is the following:
https://applicationname-api-sbox02.herokuapp.com
And, I'm trying to authenticate a user by sending a get request like this:
get_response = RestClient.get( "https://applicationname-api-sbox02.herokuapp.com/api/v1/users",
{
"Content-Type" => "application/json",
"Authorization" => "token 4d012314b7e46008f215cdb7d120cdd7",
"Manufacturer-Token" => "8d0693ccfe65104600e2555d5af34213"
}
)
Now, I want to validate the response and do the following:
- parse the response to ensure that it is valid JSON
- do some validation and verify the JSON has the correct data (verify that id == 4 for example)
- if an error is encountered, raise an exception using the 'raise' method.
In my first feeble attempt I tried the following:
puts get_response.body
if get_response.code == 200
puts "********* Get current user successful"
else
puts "Get current user failed!!"
end
Now, this returned that getting the current user was successful, but how do I actually parse the json, verify the correct id, and raise an exception if an error occurred?
Instead of raising an exception, write a test.
A straightforward approach, using the json parser and unit test framework from the std lib:
require 'minitest/autorun'
require 'rest_client'
require 'json'
class APITest < MiniTest::Unit::TestCase
def setup
response = RestClient.get("https://applicationname-api-sbox02.herokuapp.com/api/v1/users",
{
"Content-Type" => "application/json",
"Authorization" => "token 4d012314b7e46008f215cdb7d120cdd7",
"Manufacturer-Token" => "8d0693ccfe65104600e2555d5af34213"
}
)
#data = JSON.parse response.body
end
def test_id_correct
assert_equal 4, #data['id']
end
end
Execute with ruby $filename
JSON.parse parses a JSON string into a ruby hash
Getting started with minitest
If you are using ruby 1.8, you'll need to install the json gem and either install the minitest gem, or switch to the older testunit API. If you choose the latter, then you'll need to change require 'minitest/autorun' -> require 'test/unit' and MiniTest::Unit::TestCase -> Test::Unit::TestCase
I'm a little late to the party, but I recently co-created an rspec driven framework called Airborne for just this purpose. Check it out: https://github.com/brooklynDev/airborne
here is an example from our specs so you can see how we test json api:
it 'returns charge' do
get "/charges/#{charge.id}", '', headers
expect(response.status).to eq(200)
expect(response).to match_response_schema(:charge)
expect(response).to match_json(<<-JSON)
{
"id":"{id}",
"email": "{email}",
"ip": "127.0.0.1",
"amount": 10500,
"state": "captured",
"captured_amount": 10500,
}
JSON
end
Lets look at it closely
match_response_schema(:charge)
This matcher checks that json we get in response is in general valid. We use json-schema (json schema validator) for it. Guys from Thoughtbot have a detailed guide how to use json schema validator and create own matcher in this blog post.
Understanding JSON Schema is where I got a lot of useful information on how to create schemas for JSON documents.
match_json
This is our own matcher and we have released match_json gem recently. Using it you can test structure and values of your json. Here are two great features of this matcher:
if you don't know exact values, you can use patterns like {id}, {uuid} {date_time}, etc. we have predefined patterns but you can add your own too.
you get clear failure message what is wrong with your json e.g. "5" was not found in " > array":[1,2,3]
Parsing json can be done with the json gem: http://flori.github.com/json/
Parsed json is accessed through key/value just like in javascript. You can easily verify the values and conditionally raise errors.
Raising errors is done like so:
raise "the ID was #{id} instead of 4"
And writing unit tests can be done with Test::Unit - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/test/unit/rdoc/Test/Unit.html

Resources