I am trying to simulate unexpected behaviour from a web api, such as not finding the server and timeouts, using webmock.
What would be the best way to do this? All I can think of is to do something like this:
stubbed_request = stub_request(:get, "#{host}/api/something.json").
with(:headers => {'Accept'=>'*/*', 'Content-Type'=>'application/json', 'User-Agent'=>'Ruby'}).
to_return(:status => [500, "Internal Server Error"])
That should work for things like 404 etc., but how can I test timeouts, server not found/offline server, and no internet connection?
After some digging I found some solutions to this.
Apparently you can change the to_return(...) to to_timeout, which will throw a timeout error. You can also have to_raise(StandardError). For full reference, see https://github.com/bblimke/webmock#raising-timeout-errors.
Timeout, or server not found, example:
stubbed_request = stub_request(:get, "#{host}/api/something.json").
with(:headers => {'Accept'=>'*/*', 'Content-Type'=>'application/json', 'User-Agent'=>'Ruby'}).
to_timeout
Raise StandardError, or no internet/other exception, example:
stubbed_request = stub_request(:get, "#{host}/api/something.json").
with(:headers => {'Accept'=>'*/*', 'Content-Type'=>'application/json', 'User-Agent'=>'Ruby'}).
to_raise(StandardError)
#Error example 2:
stubbed_request = stub_request(:get, "#{host}/api/something.json").
with(:headers => {'Accept'=>'*/*', 'Content-Type'=>'application/json', 'User-Agent'=>'Ruby'}).
to_raise("My special error")
There you go, not too hard after all.
I have no idea how I didn't find this the first time. Anyway, hope this helps someone out one day.
Came across this question and decided to add supportive materials. According to a discussion in WebMock issue (https://github.com/bblimke/webmock/issues/16), the timeout can be simulated in two ways.
The first way is two use .to_raise(e):
stubbed_request = stub_request(:get, "#{host}/api/something.json").
with(:headers => {'Accept'=>'*/*', 'Content-Type'=>'application/json', 'User-
Agent'=>'Ruby'}).to_raise(e)
Where e is a library specific timeout exception.
quotation: "The whole point of WebMock is to be HTTP client library independent so to_timeout should work with every library.
The problem is that different libraries return different errors, ie Net::HTTP return Ruby Timeout::Error while HTTPClient raises HTTPClient:: TimeoutError.
This behavior can be replicated in WebMock but the error handling code will have to be different any time you change the library."
The second way is to use the following sample:
stub_request(:any, 'www.example.net').to_timeout
RestClient.post('www.example.net', 'abc') # ===> RestClient::RequestTimeout
Here is the original source: https://github.com/bblimke/webmock/issues/16
Related
How to enforce Faraday adapter typhoeus to use HTTP/2 for requests to servers which supported HTTP/2?
I have tested this over service https://http2.pro/doc/api and result was like this:
body="{\"http2\":1,\"protocol\":\"HTTP\\/2.0\",\"push\":0,\"user_agent\":\"Faraday v0.12.2\"}",
\"http2\":1, what means that HTTP/2 not used for request!
There are two things at play here. The first is that the remote API is lying to you in the response body. Their documentation says:
http2: Possible values are 0 (HTTP/2 was used) and 1 (HTTP/2 was not used).
Even though the response body shows 'http2': 1 indicating that HTTP2 was not used, it is being used. You can most easily confirm this using Chrome's dev tools:
So once we know that the API is lying in the response body, how can we independently confirm that Typhoeus is using HTTP2?
(this answer assumes you are using pry as your REPL, not IRB)
First let's confirm that Typhoeus alone will use HTTP2:
require 'typhoeus'
response = Typhoeus.get("https://http2.pro/api/v1", http_version: :httpv2_0)
response.class
=> Typhoeus::Response < Object
response.body
=> "{\"http2\":1,\"protocol\":\"HTTP\\/2.0\",\"push\":0,\"user_agent\":\"Typhoeus - https:\\/\\/github.com\\/typhoeus\\/typhoeus\"}" # this is the lying API response
response.http_version
=> "2" # this is what Typhoeus tells us was actually used
Now let's test it in Faraday:
require 'faraday'
require 'typhoeus'
require 'typhoeus/adapters/faraday'
conn = Faraday.new do |faraday|
faraday.adapter :typhoeus, http_version: :httpv2_0
end
response = conn.get("https://http2.pro/api/v1")
response.body
=> "{\"http2\":1,\"protocol\":\"HTTP\\/2.0\",\"push\":0,\"user_agent\":\"Faraday v0.17.0\"}" # again we get the lying API response
But how can we confirm it was HTTP2? This doesn't work:
response.http_version
NoMethodError: undefined method `http_version' for #<Faraday::Response:0x00007f99935519a8>
Because response isn't a Typhoeus::Response object, it's a Faraday object:
response.class
=> Faraday::Response < Object
So we need to get into the gem itself to figure out where it's creating the Typhoeus::Response object so we can call .http_version on it manually and confirm it's using the protocol we expect. As it turns out, that's right here.
Let's take the easy route and stick binding.pry into our local copy of the gem (you'll need to restart pry to pick up the changes to the gem):
def typhoeus_request(env)
opts = {
:method => env[:method],
:body => env[:body],
:headers => env[:request_headers]
}.merge(#adapter_options)
binding.pry
::Typhoeus::Request.new(env[:url].to_s, opts)
end
Then re-run the request:
require 'faraday'
require 'typhoeus'
require 'typhoeus/adapters/faraday'
conn = Faraday.new do |faraday|
faraday.adapter :typhoeus, http_version: :httpv2_0
end
response = conn.get("https://http2.pro/api/v1")
And you'll see:
Frame number: 0/3
From: /Users/foo/.rvm/gems/ruby-2.6.3/gems/typhoeus-1.3.1/lib/typhoeus/adapters/faraday.rb # line 127 Faraday::Adapter::Typhoeus#typhoeus_request:
120: def typhoeus_request(env)
121: opts = {
122: :method => env[:method],
123: :body => env[:body],
124: :headers => env[:request_headers]
125: }.merge(#adapter_options)
126: binding.pry
=> 127: ::Typhoeus::Request.new(env[:url].to_s, opts)
128: end
Now enter:
response = ::Typhoeus::Request.new(env[:url].to_s, opts).run
And confirm it's a Typhoeus::Response object:
response.class
=> Typhoeus::Response < Object
And confirm it's using HTTP2:
response.http_version
=> "2"
And confirm the API response body is a dirty liar:
response.body
=> "{\"http2\":1,\"protocol\":\"HTTP\\/2.0\",\"push\":0,\"user_agent\":\"Faraday v0.17.0\"}"
And that's how you use Typhoeus as a Faraday adapter to make an HTTP2 request.
I have some code that looks like this:
while response.droplet.status != env["user_droplet_desired_state"] do
sleep 2
response = ocean.droplet.show env["droplet_id"]
say ".", nil, false
end
The idea being you can set the app to wait until the server is in a certain state (eg. restart it, then watch it until it's active again)
However, I'm using webmock in the tests, and I can't figure out a way to give a different response the second time.
For example, code like this:
stub_request(:get, "https://api.digitalocean.com/v2/droplets/6918990?per_page=200").
with(:headers => {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Authorization'=>'Bearer foo', 'Content-Type'=>'application/json', 'User-Agent'=>'Faraday v0.9.2'}).
to_return(:status => 200, :body => fixture('show_droplet_inactive'), :headers => {})
stub_request(:get, "https://api.digitalocean.com/v2/droplets/6918990?per_page=200").
with(:headers => {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Authorization'=>'Bearer foo', 'Content-Type'=>'application/json', 'User-Agent'=>'Faraday v0.9.2'}).
to_return(:status => 200, :body => fixture('show_droplet'), :headers => {})
With the idea being "First time mark as in-active so the loop goes through one-time, then mark as active afterwards"
The documentation says that stubs are just done as "Last one found will work":
Always the last declared stub matching the request will be applied
i.e:
stub_request(:get, "www.example.com").to_return(:body => "abc")
stub_request(:get, "www.example.com").to_return(:body => "def")
Net::HTTP.get('www.example.com', '/') # ====> "def"
Is it possible to model multiple calls to the same endpoint with different results in webmock?
If you pass multiple arguments to #to_return, it will respond each time with the next response, and then just keep returning the last one over and over. For example:
require 'webmock/rspec'
require 'uri'
describe "something" do
it "happens" do
stub_request(:get, 'example.com/blah').
to_return({status: 200, body: 'ohai'}, {status: 200, body: 'there'})
puts Net::HTTP.get(URI('http://example.com/blah'))
puts Net::HTTP.get(URI('http://example.com/blah'))
puts Net::HTTP.get(URI('http://example.com/blah'))
puts Net::HTTP.get(URI('http://example.com/blah'))
end
end
When run as rspec <file>, this will print:
ohai
there
there
there
I'm trying to use webmock with rspec to stub out requests to Aws but I can't seem to get the regex to work for SQS polling. If I run rspec, webmock generates a 'correct' stub for me to use in a before(:each) block, in my spec_helper.rb like this:
You can stub this request with the following snippet:
stub_request(:post, "https://sqs.us-west-2.amazonaws.com/123456789012/backlog").
with(:body => "Action=ReceiveMessage&AttributeName.1=All&MaxNumberOfMessages=1&MessageAttributeName.1=All&QueueUrl=https%3A%2F%2Fsqs.us-west-2.amazonaws.com%2F123456789012%2Fbacklog&Version=2012-11-05&VisibilityTimeout=0&WaitTimeSeconds=20",
:headers => {'Accept'=>'*/*', 'Accept-Encoding'=>'', 'Authorization'=>'AWS4-HMAC-SHA256 Credential=MY_ACCESS_KEY/20150726/us-west-2/sqs/aws4_request, SignedHeaders=content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=large_alpha-numeric-signature', 'Content-Length'=>'224', 'Content-Type'=>'application/x-www-form-urlencoded; charset=utf-8', 'Host'=>'sqs.us-west-2.amazonaws.com', 'User-Agent'=>'aws-sdk-ruby2/2.1.7 ruby/2.2.2 x86_64-darwin14', 'X-Amz-Content-Sha256'=>'69336339ae76cf370477d4dsaf667as0b5dd8d25762c7c78sad8a', 'X-Amz-Date'=>'20150726T143009Z'}).
to_return(:status => 200, :body => "", :headers => {})
So in my spec_helper.rb I have
RSpec.configure do |config|
config.before(:each) do
stub_request(:post, "https://sqs.us-west-2.amazonaws.com/123456789012/backlog").
with(:body => "Action=ReceiveMessage&AttributeName.1=All&MaxNumberOfMessages=1&MessageAttributeName.1=All&QueueUrl=https%3A%2F%2Fsqs.us-west-2.amazonaws.com%2F123456789012%2Fbacklog&Version=2012-11-05&VisibilityTimeout=0&WaitTimeSeconds=20",
:headers => {'Accept'=>'*/*',
'Accept-Encoding'=>'',
'Authorization'=>"AWS4-HMAC-SHA256 Credential=MY_ACCESS_KEY/20150726/us-west-2/sqs/aws4_request, SignedHeaders=content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=" + /"^[a-zA-Z0-9]*$"/,
'Content-Length'=>'224',
'Content-Type'=>'application/x-www-form-urlencoded; charset=utf-8',
'Host'=>'sqs.us-west-2.amazonaws.com',
'User-Agent'=>'aws-sdk-ruby2/2.1.7 ruby/2.2.2 x86_64-darwin14',
'X-Amz-Content-Sha256'=>'694236339ae76cf370477d4dsaf667as0b5dd8d25762c7c78sad8a',
'X-Amz-Date'=>""+ /"^[a-zA-Z0-9]*$"/}).
to_return(:status => 200, :body => "", :headers => {})
end
The areas I'm trying to use regex against are the Signature and the X-Amz-Date because they're the only two that seem to change between different attempts to run the rspec.
The problem is the regex seems to not be working because even though I've added it into the spec_helper.rb, every time I run the suite, I get back the recommended stub from webmock instead of a passing or failing test. It should be passing at this point, from what I understand from the webmock docs and several tutorials.
How should I change this to get webmock to work for my test suite against Aws SQS polling?
I've been bashing my head against my desk for a few days now so any help is much appreciated.
Signature is likely generated using a byproduct of Time.now and I'm guessing you don't actually want to test for that. Instead simply do:
stub_request(:post, "https://sqs.us-west-2.amazonaws.com/123456789012/backlog").and_return(:status => 200, :body => "", :headers => {})
If you want to be even less specific on the URL (like ommiting that ID) you can even use a regex match:
stub_request(:post, /amazonaws.com/).and_return(:status => 200, :body => "", :headers => {})
I have the same problem and have the solution only for 'X-Amz-Date'.
Since it is date in special format, use Timecop.freeze block around your mock and method call.
Timecop.freeze do
stub_request(:get, "https://s3.eu-central-1.amazonaws.com/test_bucket/test").
with(headers: {'Accept'=>'*/*', 'Accept-Encoding'=>'', 'Authorization'=>'AWS4-HMAC-SHA256 Credential=access_key/20190814/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=9e6a6a209a0cb05346a058c12ef706ebff185ae3b72f3e542e1becbc97e8ea7a', 'Content-Length'=>'0', 'Content-Type'=>'', 'Host'=>'s3.eu-central-1.amazonaws.com', 'User-Agent'=>'aws-sdk-ruby2/2.9.44 ruby/2.6.3 x86_64-darwin18 resources', 'X-Amz-Content-Sha256'=>'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'X-Amz-Date'=>"#{time.utc.iso8601(0).gsub(/[^\p{Alnum}]/, '')}"}).
to_return(status: 200, body: "", headers: {})
AwsService.bucket.object("test").get
end
The regexp leaves only alphanumerical characters and Timecop takes care of the time.
I have been trying for a while to stub multipart requests using webmock and have not found a satisfying solution.
Ideally, I would like to stub the request as follow:
stub_request(:post, 'http://test.api.com').with(:body => { :file1 => File.new('filepath1'), file2 => File.new('filepath2') })
However, this does not seem to work and RSpec complains that the request has not been stubbed. The non-stubbed request is printed:
stub_request(:post, "http://test.api.com").
with(:body => "--785340\r\nContent-Disposition: form-data; name=\"file1\"; filename=\"filepath1\"\r\nContent-Type: text/plain\r\n\r\nhello\r\n--785340\r\nContent-Disposition: form-data; name=\"file2\"; filename=\"filepath2\"\r\nContent-Type: text/plain\r\n\r\nhello2\r\n--785340\r\n",
:headers => {'Accept'=>'*/*; q=0.5, application/xml', 'Accept-Encoding'=>'gzip, deflate', 'Content-Length'=>'664', 'Content-Type'=>'multipart/form-data; boundary=785340', 'User-Agent'=>'Ruby'}).
to_return(:status => 200, :body => "", :headers => {})
Of course, I can't really follow this suggestion because the boundaries are generated dynamically. Any idea how I could properly stub these requests?
Thanks!
Bruno
Kind of late but I'll leave an answer for future overflowers and googlers.
I had the same problem and used Rack::Multipart::Parser in conjunction with Webmock as a work around. Quick and dirty code should look something like this (warning: uses activesupport extensions):
stub_request(:post, 'sample.com').with do |req|
env = req.headers.transform_keys { |key| key.underscore.upcase }
.merge('rack.input' => StringIO.new(req.body))
parsed_request = Rack::Multipart::Parser.new(env).parse
# Expectations:
assert_equal parsed_request["file1"][:tempfile].read, "hello world"
end
WebMock doesn't support multipart requests at the moment. Check author's comment here for more info: https://github.com/vcr/vcr/issues/295#issuecomment-20181472
I suggest you consider one of the following routes:
stubbing without matching the post multipart body
wrapping the request in a method with file path arguments and setting more fine-grained expectation on this method
using VCR for mocking external requests in integration tests
Here's a workaround using WebMock with regex to match multipart/form-data requests, especially handy for testing uploading of images:
stub_request(:post, 'sample.com').with do |req|
# Test the headers.
assert_equal req.headers['Accept'], 'application/json'
assert_equal req.headers['Accept-Encoding'], 'gzip, deflate'
assert_equal req.headers['Content-Length'], 796588
assert_match %r{\Amultipart/form-data}, req.headers['Content-Type']
assert_equal req.headers['User-Agent'], 'Ruby'
# Test the body. This will exclude the image blob data which follow these lines in the body
req.body.lines[1].match('Content-Disposition: form-data; name="FormParameterNameForImage"; filename="image_filename.jpeg"').size >= 1
req.body.lines[2].match('Content-Type: img/jpeg').size >= 1
end
Testing only the headers could also have been done using the normal WebMock way of using stub_request(:post, 'sample.com').with(headers: {'Accept' => 'application/json}), and simply not include any body specification in the with clause.
I need to send a POST request as an XML string but I get odd results. The code:
require 'rest_client'
response = RestClient.post "http://127.0.0.1:2000", "<tag1>text</tag1>", :content_type => "text/xml"
I expect to receive "<tag1>text</tag1>" as the parameter on the request server. Instead, I get "tag1"=>"text". It converts the XML to a hash. Why is that? Any way around this?
Try this:
response = RestClient.post "http://127.0.0.1:2000",
"<tag1>text</tag1>",
{:accept => :xml, :content_type => :xml}
I think you just needed to specify the ":accept" to let it know you wanted to receive it in the XML format. Assuming it's your own server, you can debug on the server and see the request format used is probably html.
Hope that helps.
Instead of using RestClient, use Ruby's built-in Open::URI for GET requests or something like Net::HTTP or the incredibly powerful Typhoeus:
uri = URI('http://www.example.com/search.cgi')
res = Net::HTTP.post_form(uri, 'q' => 'ruby', 'max' => '50')
In Typhoeus, you'd use:
res = Typhoeus::Request.post(
'http://localhost:3000/posts',
:params => {
:title => 'test post',
:content => 'this is my test'
}
)
Your resulting page, if it's in XML will be easy to parse using Nokogiri:
doc = Nokogiri::XML(res.body)
At that point you'll have a fully parsed DOM, ready to be searched, using Nokogiri's search methods, such as search and at, or any of their related methods.