Testing External Api in ruby using rspec - ruby

I am working on a project where I have to test an external api which is making call to github.
Let's take the code below as example, the class below does not have intializer
Class CodeReviewSignedOff
def pullrequestreview(github)
#the following line return a json response, it require two parameter a string and a number
github.client.pull_request_reviews("string",number).each do |github|
#block of code which perform some operation
end
return count #return integer response
end
end
As you can see my class CodeReviewSignedOff relies on the github object which is defined in my existing project and using the object to make external API call.
But the thing is, I do not want to actually call the API.
I want stub request github.client.pull_request_reviews
allow(github).to receive_message_chain(:client,:pull_request_review_requests).and_return json
My question here is,
How should I return the json response because if I include it in the double quote it will be interpreted as string.
My second question
How do I test my pullrequestreview function.
github=double #I am creating a fake github
object=CodeReviewSignedOff.new
expect(pullrequestreview).with(double).and_return(3)
Is the above logic correct?

Good question!
Disclaimer: I'm still learning about RoR, but I can try to answer this question as best as I can. If anyone has suggestions about how I can improve my response, please let me know!
Testing a class method (using stubs)
So, in order to test your method called "pullrequestreview", you'll need to stub a method within the Github::Client class. I personally have not worked with GitHub's API, so let's assume that it is a static class with a few class methods (including pull_request_reviews).
(Note: I renamed "pullrequestreview" to pull_request_review, because it is more idiomatic in Ruby)
In the example below, I stubbed Github::Client.pull_request_reviews, so that it will return res_data whenever it is called:
Github::Client.stubs(:pull_request_reviews).returns(res_data)
Next, I use a "factory" to generate a github_client argument to pass into code_review_sign_off.pull_request_review. Since the GitHub method is stubbed, it doesn't really matter what I pass into it. In more complex test scenarios, it might be a good idea to make use of a factory bot gem to build out different GitHub related objects (with different trait options). For example, using a factory, you could define a new github_client with a "pull_request" trait like so:
github_client = create(:github_client, :with_pull_request)
In your post, code_review_sign_off.pull_request_review is expecting an iterable object from GitHub. Let's assume that code_review_sign_off.pull_request_review returns the number of elements within the array. Since I have control over GitHub's response, I know that only one element will be returned, so the result should be 1, which I assert using:
assert_equal pull_request_review_response, 1
So, in general when testing, you want to issolate the function that you're interested in and use stubs to keep everything else constant.
require 'test_helper'
class CodeReviewSignedOffTest < ActionDispatch::IntegrationTest
test 'with CodeReviewSignedOff method' do
res_data = [{
:id => 1,
:data => "some_data"
}]
Github::Client.stubs(:pull_request_reviews).returns(res_data)
github_client = create(:github_client, :with_pull_request)
code_review_sign_off = CodeReviewSignedOff.new
pull_request_review_response = code_review_sign_off.pull_request_review(github_client)
...
assert_equal pull_request_review_response, 1
end
end
(Note: I'm using the mocha/minitest in the examples)
Bonus: Mocking an API endpoint
So, first, I am going to assume you have your test environment for RSpec properly configured (you ran rails generate rspec:install command) and you have the appropriate /spec directory.
The first step to mocking an API endpoint is to stub the calls to the actual endpoints by using a gem like WebMock. So, install the gem and add require 'webmock/rspec' to /spec/spec_helper.rb
Within /spec/spec_helper.rb you'll need to define a few things,
First, add the following to ensure that you're not continuing to make real calls to the API:
WebMock.disable_net_connect!(allow_localhost: true)
Next, you'll need a stub for each API endpoint that you're mocking. For example, in your case, you'd need to define something like:
RSpec.configure do |config|
config.before(:each) do
pull_request_review_response = [
{
"id": 80,
"node_id": "MDE3OlB1bGxSZXF1ZXN0UmV2aWV3ODA=",
"user": {
"login": "octocat",
"id": 1,
"node_id": "MDQ6VXNlcjE=",
"avatar_url": "https://github.com/images/error/octocat_happy.gif",
"gravatar_id": "",
"url": "https://api.github.com/users/octocat",
"html_url": "https://github.com/octocat",
"followers_url": "https://api.github.com/users/octocat/followers",
"following_url": "https://api.github.com/users/octocat/following{/other_user}",
"gists_url": "https://api.github.com/users/octocat/gists{/gist_id}",
"starred_url": "https://api.github.com/users/octocat/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/octocat/subscriptions",
"organizations_url": "https://api.github.com/users/octocat/orgs",
"repos_url": "https://api.github.com/users/octocat/repos",
"events_url": "https://api.github.com/users/octocat/events{/privacy}",
"received_events_url": "https://api.github.com/users/octocat/received_events",
"type": "User",
"site_admin": false
},
"body": "Here is the body for the review.",
"state": "APPROVED",
"html_url": "https://github.com/octocat/Hello-World/pull/12#pullrequestreview-80",
"pull_request_url": "https://api.github.com/repos/octocat/Hello-World/pulls/12",
"_links": {
"html": {
"href": "https://github.com/octocat/Hello-World/pull/12#pullrequestreview-80"
},
"pull_request": {
"href": "https://api.github.com/repos/octocat/Hello-World/pulls/12"
}
},
"submitted_at": "2019-11-17T17:43:43Z",
"commit_id": "ecdd80bb57125d7ba9641ffaa4d7d2c19d3f3091",
"author_association": "COLLABORATOR"
}
]
owner = 'octocat',
repo = 'hello-world',
pull_number = 42
stub_request(:get, "https://api.github.com/repos/#{owner}/#{repo}/pulls/#{pull_number}/reviews").
to_return(status: 200, body: pull_request_review_response.to_json,
headers: {'Accept'=>'*/*', 'User-Agent'=>'Ruby'})
end
# ....
end
The important part of this code is stub_request which takes an HTTP request method, a URL path, and it returns the arguments passed into to_return (as an HTTP response). The response object will include a status, body, and a header.
I used the examples from the WebMock README and the API doc from GitHub to produce the example above (Note: It is untested).
Now, you can define a new test that calls the Mock endpoint:
require 'spec_helper'
class CodeReviewSignedOffTest < ActionDispatch::IntegrationTest
test 'GET pull_request_review_response endpoint from GitHub mock' do
uri = URI('https://api.github.com/repos/octocat/hello-world/pulls/42/reviews')
response = Net::HTTP.get(uri)
data = JSON.parse(response&.body)
...
end
end
Hopefully you'll see the data you defined within your mock API come through. I haven't tested this code, so it might require some debugging.
Let me know if you have any questions or run into trouble writing your tests! Good luck.

Related

Every error in the book from google-api-ruby-client but no data

I have been attempting to work on a request from my boss this week that requires using the google admin directory api.
At this point I am questioning if what I am trying to do is even possible.
Can I retrieve data from the scope "https://www.googleapis.com/auth/admin.directory.device.mobile.readonly" with a service account? Is it even possible?
The errors I have seen in the past hour are below...
Many of them sound the same and I have no idea what is going on or why this is such a difficult journey for such basic information.
PERMISSION_DENIED: Request had insufficient authentication scopes. (Google::Apis::ClientError)
`check_status': Unauthorized (Google::Apis::AuthorizationError)
Authorization failed. Server message: (Signet::AuthorizationError)
{
"error": "unauthorized_client",
"error_description": "Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested."
}
`check_status': permission_denied: request had insufficient authentication scopes
`check_status': badRequest: Bad Request
My current test script is below...
require "google/apis/admin_directory_v1"
require "googleauth"
require "googleauth/stores/file_token_store"
require "fileutils"
APPLICATION_NAME = "Directory API Ruby Quickstart".freeze
CREDENTIALS_PATH = "credentials.json".freeze
CUSTOMER_ID = "thasgunnabeanopefrommedawg".freeze
SCOPE = ["https://www.googleapis.com/auth/admin.directory.device.mobile.readonly"].freeze
authorizer = Google::Auth::ServiceAccountCredentials.make_creds(
json_key_io:
File.open('credentials.json'),
scope: SCOPE)
authorizer.update!(sub: "fullbl00m#citadelny.com")
authorizer.fetch_access_token!
# puts authorize
# Initialize the API
service = Google::Apis::AdminDirectoryV1::DirectoryService.new
service.client_options.application_name = APPLICATION_NAME
service.authorization = Google::Auth.get_application_default(SCOPE)
response = service.list_mobile_devices(customer_id: CUSTOMER_ID)
puts response.to_json
EDITS BELOW *** [27th, MAY, 2022]
I have been trying with ruby, python, and postman for two weeks at this point :/
Last night I took the ruby snippet that was posted by user:Daimto below.
I was able to return a token with the following modified version of the ruby snippet provided in the answer below.
require 'googleauth'
require 'google/apis/admin_directory_v1'
creds = {
"type": "service_account",
"project_id": "MYPROJECTNAME",
"private_key_id": "MYPRIVATEKEYID",
"private_key": "-----BEGIN PRIVATE KEY-----\n-MY PRIVATE KEY
WILL BE HERE BUT REMOVED FOR SECURITY-----END PRIVATE KEY-----\n",
"client_email": "emailfromserviceaccount-compute#developer.gserviceaccount.com",
"client_id": "MYCLIENTIDISACTUALLYHERE",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/compute%40developer.gserviceaccount.com"
}
creds_json = creds.to_json
creds_json_io = StringIO.new(creds_json)
auth = Google::Auth::ServiceAccountCredentials.make_creds(
json_key_io: creds_json_io,
scope["https://www.googleapis.com/auth/admin.directory.device.mobile.readonly","https://www.googleapis.com/auth/admin.directory.device.chromeos.readonly","https://www.googleapis.com/auth/admin.directory.device.mobile"]
)
auth.sub = "emailfrommyserviceaccount-
compute#developer.gserviceaccount.com"
puts auth.fetch_access_token
Please excuse the formatting.
I took the service account out of the env variable for now to make sure I can get it to work without adding extra layers of abstraction at this time.
When trying to add the additional code from the Directory Api Quickstart to the above snip I STILL RETURN THE ERROR
/var/lib/gems/2.7.0/gems/google-apis-core-0.5.0/lib/google/apis/core/http_command.rb:224:in `check_status': Unauthorized (Google::Apis::AuthorizationError)
The additional code added is below...
The last line of the previous snip gets changed to the first line of the snip that comes after this. This is to properly pass the token to the example after modifying user:Daimto's response.
authorize = auth.fetch_access_token
# Initialize the API
service = Google::Apis::AdminDirectoryV1::DirectoryService.new
service.client_options.application_name = "my-application-name"
service.authorization = authorize
# List the first 10 users in the domain
response = service.list_users(customer: "my_customer",
max_results: 10,
order_by: "email")
puts "Users:"
puts "No users found" if response.users.empty?
response.users.each { |user| puts "- #{user.primary_email} (#{user.name.full_name})" }
The method Method: mobiledevices.list requires one of the following scopes.
So to answer your first question yes you can use the https://www.googleapis.com/auth/admin.directory.device.mobile.readonly scope.
Error number 1
PERMISSION_DENIED: Request had insufficient authentication scopes.
You were probably getting this error when you had supplied a different scope.
Error 3;
Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested.
There are three types of clients you can create on google cloud console.
web client
native client
service account
The json file you get from creating these clients is all different. The code that uses them is also different. The error is telling you that you have a client.json file that you are using which does not match the type of code you are using.
How to create service account credetinals
The code for a service account would be like this Not tested you may need to fix the scope. Remember that the service account needs to be configured properly on your workspace domain for the sub to work.
require 'googleauth'
require 'google/apis/admin_v1'
creds = ENV['GOOGLE_SERVICE_ACCOUNT'] # JSON downloaded from cloud console
# is saved in this ENV variable
creds_json = JSON.parse(creds)
creds_json_io = StringIO.new(creds_json.to_json)
auth = Google::Auth::ServiceAccountCredentials.make_creds(
json_key_io: creds_json_io,
scope: [Google::Apis::ADMINV1::ADMIN_DIRECTORY_MOBILE_READONLY]
)
auth.sub = 'admin#yourdomain.com'
auth.fetch_access_token
Tip: You have a lot of errors there, I feel that you have been struggling with this for a while. Advice step back, have a look at the sample on the readme for the Google-api-ruby-client. Start over. Just get your auth to work. Once you get the code right and the client right all the pieces will fit into place.

Grails 3 - Spring Rest Docs using Rest assured giving SnippetException when using JSON views

I am trying to integrate Spring REST docs with rest assured with Grails 3.1.4 application. I am using JSON Views.
Complete code is at https://github.com/rohitpal99/rest-docs
In NoteController when I use
List<Note> noteList = Note.findAll()
Map response = [totalCount: noteList.size(), type: "note"]
render response as grails.converters.JSON
Document generation works well.
But I want to use JSON views like
respond Note.findAll()
where I have _notes.gson and index.gson files in /views directory. I get a SnippetException. A usual /notes GET request response is correct.
rest.docs.ApiDocumentationSpec > test and document get request for /index FAILED
org.springframework.restdocs.snippet.SnippetException at ApiDocumentationSpec.groovy:54
with no message. Unable to track why it occurs.
Please suggest.
Full stacktrace
org.springframework.restdocs.snippet.SnippetException: The following parts of the payload were not documented:
{
"instanceList" : [ {
"title" : "Hello, World!",
"body" : "Integration Test from Hello"
}, {
"title" : "Hello, Grails",
"body" : "Integration Test from Grails"
} ]
}
at org.springframework.restdocs.payload.AbstractFieldsSnippet.validateFieldDocumentation(AbstractFieldsSnippet.java:134)
at org.springframework.restdocs.payload.AbstractFieldsSnippet.createModel(AbstractFieldsSnippet.java:74)
at org.springframework.restdocs.snippet.TemplatedSnippet.document(TemplatedSnippet.java:64)
at org.springframework.restdocs.generate.RestDocumentationGenerator.handle(RestDocumentationGenerator.java:192)
at org.springframework.restdocs.restassured.RestDocumentationFilter.filter(RestDocumentationFilter.java:63)
at com.jayway.restassured.internal.filter.FilterContextImpl.next(FilterContextImpl.groovy:73)
at org.springframework.restdocs.restassured.RestAssuredRestDocumentationConfigurer.filter(RestAssuredRestDocumentationConfigurer.java:65)
at com.jayway.restassured.internal.filter.FilterContextImpl.next(FilterContextImpl.groovy:73)
at com.jayway.restassured.internal.RequestSpecificationImpl.applyPathParamsAndSendRequest(RequestSpecificationImpl.groovy:1574)
at com.jayway.restassured.internal.RequestSpecificationImpl.get(RequestSpecificationImpl.groovy:159)
at rest.docs.ApiDocumentationSpec.$tt__$spock_feature_0_0(ApiDocumentationSpec.groovy:54)
at rest.docs.ApiDocumentationSpec.test and document get request for /index_closure2(ApiDocumentationSpec.groovy)
at groovy.lang.Closure.call(Closure.java:426)
at groovy.lang.Closure.call(Closure.java:442)
at grails.transaction.GrailsTransactionTemplate$1.doInTransaction(GrailsTransactionTemplate.groovy:70)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at grails.transaction.GrailsTransactionTemplate.executeAndRollback(GrailsTransactionTemplate.groovy:67)
at rest.docs.ApiDocumentationSpec.test and document get request for /index(ApiDocumentationSpec.groovy)
REST Docs will fail a test if you try to document something that isn't there or fail to document something that is there. You've documented two fields in your test:
responseFields(
fieldWithPath('totalCount').description('Total count'),
fieldWithPath('type').description("Type of result")
)))
REST Docs has failed the test as some parts of the response haven't been documented. Specifically an instanceList array that contains maps with two keys: title and body. You can document those and the other two fields with something like this:
responseFields(
fieldWithPath('totalCount').description('Total count'),
fieldWithPath('type').description("Type of result"),
fieldWithPath('instanceList[].title').description('Foo'),
fieldWithPath('instanceList[].body').description('Bar')
)))
If you don't care about potentially missing fields, you can use relaxedResponseFields instead of responseFields:
relaxedResponseFields(
fieldWithPath('totalCount').description('Total count'),
fieldWithPath('type').description("Type of result")
))
This won't fail the test if some fields are not mentioned.

savon-multipart - How to obtain attachments

I am using savon-multipart https://github.com/savonrb/savon-multipart to request a SOAP multipart response with an attachment (PDF). So far, this is my code:
require "savon-multipart"
client = Savon.client(
wsdl: "http://something.de?wsdl",
wsse_auth: [username: "uu", password: "??"]
)
reponse = client.call(:get_report, message: {
pdfId: 1
})
response.attachments
Authentication works fine. I can also fetch the XML-reponse. What I can't do is extract the attachment. There does not seem to exist a method for it.
According to savon-multipart's documentation
response.attachments
should contain the attachment(s). Unfortunately ruby tells me that this method is not defined.
I could't find an example implementation of savon-multipart so I'm coming to you guys :) Hope you can help me.
We had this same problem in some code. I hope this saves someone else some time in finding the solution.
When using savon-multipart, we had to add multipart: true to the parameters in call. When that parameter was added the response returned was of type Savon::Multipart::Response which has the attachments and parts methods.
reponse = client.call(:get_report, message: {
pdfId: 1
}, multipart: true)
Without that parameter, or with it set to false, the returned response is a Savon::Response object which does not have those methods.

Goliath + Redis: Testing sequence of requests

I wrote an endpoint that has two operations: PUT and GET. GET retrieves what was PUT, so a good way to ensure both work would be to test them in sequence.
My cURL testing tells me the service works. However, I can't get my test that does the same thing to pass!
it "accepts authenticated PUT data" do
attributes1 = {
user: {
lat: '12.34',
lng: '56.78'
}
}
with_api(Location, { log_stdout: true, verbose: true }) do
put_request({ query: { auth_token: 'abc' }, body: attributes1.to_json }) do |client|
client.response_header.status.must_equal 201
get_request({ query: { auth_token: 'abc' } }) do |client|
client.response_header.status.must_equal 200
client.response.must_match_json_expression([attributes1[:user]])
end
end
end
end
Note that PUT accepts JSON in the body, and GET returns JSON. The service uses Redis as the datastore, and I use mock_redis to mock it.
test_helper.rb:
$redis = MockRedis.new
class Goliath::Server
def load_config(file = nil)
config['redis'] = $redis
end
end
top of spec:
before do
$redis.flushdb
end
The GET should retrieve what was just PUT, but instead it retrieves JSON "null".
Am I doing anything obviously wrong? Perhaps with how I am passing along body data to the request?
It would definitely help to get some better logging when my tests are running. I tried the { log_stdout: true, verbose: true } options, but this still only seems to output basic INFO logging from the Goliath server, which doesn't tell me anything about data and params.
Any help would be appreciated!

Rails pagination in API using Her, Faraday

I've been trying to figure this out all day, and it's driving me crazy.
I have two rails apps, ServerApp and ClientApp. ClientApp gets data from ServerApp through an API, using the Her gem. Everything was great until I needed pagination information.
This is the method I am using to get the orders (this uses kamainari for pagination and ransack for search):
# ServerApp
def search
#search = Order.includes(:documents, :client).order('id desc').search(params[:q])
#orders = #search.result(distinct: true).page(params[:page]).per(params[:per])
respond_with #orders.as_json(include: :documents)
end
It returns an array of hashes in json, which Her uses as a collection of orders. That works fine.
# Response
[
{
"client_id": 239,
"created_at": "2013-05-15T15:37:03-07:00",
"id": 2422,
"ordered_at": "2013-05-15T15:37:03-07:00",
"origin": "online",
"updated_at": "2013-05-15T15:37:03-07:00",
"documents": [
{ ... }
]
},
...
]
But I needed pagination information. It looked like I needed to send it as metadata with my json. So I change my response to this:
respond_to do |format|
format.json do
render json: { orders: #orders.as_json(include: :documents), metadata: 'sent' }
end
end
This does indeed send over metadata, so in my ClientApp I can write #orders.metadata and get 'sent'. But now my orders are nested in an array inside of 'orders', so I need to use #orders.orders, and then it treats it like an array instead of a Her collection.
After doing some reading, it seemed sending pagination info through headers was the way a lot of other people did this (I was able to get the headers set up in an after_filter using this guide). But I am even more lost on how to get those response headers in my ClientApp - I believe I need a Faraday Middleware but I just am having no luck getting this to work.
If anyone knows how I can just get this done, I would be very grateful. I can't take another day of banging my head against the wall on this, but I feel like I am just one vital piece of info away from solving this!
I encountered the same issue and solved it by adding my own middleware and rewriting the "parse" and "on_complete" methods without that much hassle and avoiding the use of global variables.
Here's the code:
class CustomParserMiddleware < Her::Middleware::DefaultParseJSON
def parse(env)
json = parse_json(env[:body])
pagination = parse_json(env[:response_headers][:pagination_key]) || {}
errors = json.delete(:errors) || {}
metadata = json.delete(:metadata) || {}
{
:data => json,
:errors => errors,
:metadata => {
:pagination => pagination,
:additional_metadata => metadata
},
end
def on_complete(env)
env[:body] = case env[:status]
when 204
parse('{}')
else
parse(env)
end
end
end
then, you can access the pagination as follows:
model = Model.all
model.metadata[:pagination]
I finally got this working. The trick was to use a global variable in the faraday on_complete - I tried to find a better solution but this was the best I could do. Once again, I got the header code from here. Here's the full guide to how to get pagination working with Her:
First, on my server side, I have the Kaminari gem, and I pass page and per as params to the server from the client. (This is also using ransack for searching)
def search
#search = Order.order('id desc').search(params[:q])
#orders = #search.result(distinct: true).page(params[:page]).per(params[:per])
respond_with #orders.as_json(include: :items)
end
My client makes the request like so:
#orders = Order.search(q: { client_id_eq: #current_user.id }, page: params[:page], per: 3)`
Back on the server, I have this in my ApiController (app controller for api):
protected
def self.set_pagination_headers(name, options = {})
after_filter(options) do |controller|
results = instance_variable_get("##{name}")
headers["X-Pagination"] = {
total_count: results.total_count,
offset_value: results.offset_value
}.to_json
end
end
In the server orders_controller.rb, I set the pagination headers for the search method:
class OrdersController < ApiController
set_pagination_headers :orders, only: [:search]
...
end
Now to receive the headers we need a Faraday middleware in Her on the client.
# config/initializers/her.rb
Her::API.setup url: Constants.api.url do |c|
c.use TokenAuthentication
c.use HeaderParser # <= This is my middleware for headers
c.use Faraday::Request::UrlEncoded
c.use Her::Middleware::DefaultParseJSON
c.use Faraday::Adapter::NetHttp
c.use Faraday::Response::RaiseError
end
# lib/header_parser.rb
# don't forget to load this file in application.rb with something like:
# config.autoload_paths += Dir[File.join(Rails.root, "lib", "*.rb")].each { |l| require l }
class HeaderParser < Faraday::Response::Middleware
def on_complete(env)
unless env[:response_headers]['x-pagination'].nil?
# Set the global var for pagination
$pagination = JSON.parse(env[:response_headers]['x-pagination'], symbolize_names: true)
end
end
end
Now back in your client controller, you have a global variable of hash called $pagination; mine looks like this:
$pagintation = { total_count: 0, offset_value: 0 }`
Finally, I added Kaminari gem to my client app to paginate the array and get those easy pagination links:
#orders = Kaminari.paginate_array(#orders, total_count: $pagination[:total_count]).page(params[:page]).per(params[:per_page])`
I hope this can help someone else, and if anyone knows a better way to do this, let me know!
You can pass header options to Faraday when setting up the connection, see the docs at http://rubydoc.info/gems/faraday/0.8.7/Faraday/Connection:initialize
Sometimes it helps to do a curl request first, esp. use -vv option for verbose output where you will see all headers. (Maybe you can attach some log outputs from the Server too)
You can use e.g. clogger (http://clogger.rubyforge.org/) do monitor header information on the Rails server side

Resources