Stub multipart requests with webmock/rspec - ruby

I have been trying for a while to stub multipart requests using webmock and have not found a satisfying solution.
Ideally, I would like to stub the request as follow:
stub_request(:post, 'http://test.api.com').with(:body => { :file1 => File.new('filepath1'), file2 => File.new('filepath2') })
However, this does not seem to work and RSpec complains that the request has not been stubbed. The non-stubbed request is printed:
stub_request(:post, "http://test.api.com").
with(:body => "--785340\r\nContent-Disposition: form-data; name=\"file1\"; filename=\"filepath1\"\r\nContent-Type: text/plain\r\n\r\nhello\r\n--785340\r\nContent-Disposition: form-data; name=\"file2\"; filename=\"filepath2\"\r\nContent-Type: text/plain\r\n\r\nhello2\r\n--785340\r\n",
:headers => {'Accept'=>'*/*; q=0.5, application/xml', 'Accept-Encoding'=>'gzip, deflate', 'Content-Length'=>'664', 'Content-Type'=>'multipart/form-data; boundary=785340', 'User-Agent'=>'Ruby'}).
to_return(:status => 200, :body => "", :headers => {})
Of course, I can't really follow this suggestion because the boundaries are generated dynamically. Any idea how I could properly stub these requests?
Thanks!
Bruno

Kind of late but I'll leave an answer for future overflowers and googlers.
I had the same problem and used Rack::Multipart::Parser in conjunction with Webmock as a work around. Quick and dirty code should look something like this (warning: uses activesupport extensions):
stub_request(:post, 'sample.com').with do |req|
env = req.headers.transform_keys { |key| key.underscore.upcase }
.merge('rack.input' => StringIO.new(req.body))
parsed_request = Rack::Multipart::Parser.new(env).parse
# Expectations:
assert_equal parsed_request["file1"][:tempfile].read, "hello world"
end

WebMock doesn't support multipart requests at the moment. Check author's comment here for more info: https://github.com/vcr/vcr/issues/295#issuecomment-20181472
I suggest you consider one of the following routes:
stubbing without matching the post multipart body
wrapping the request in a method with file path arguments and setting more fine-grained expectation on this method
using VCR for mocking external requests in integration tests

Here's a workaround using WebMock with regex to match multipart/form-data requests, especially handy for testing uploading of images:
stub_request(:post, 'sample.com').with do |req|
# Test the headers.
assert_equal req.headers['Accept'], 'application/json'
assert_equal req.headers['Accept-Encoding'], 'gzip, deflate'
assert_equal req.headers['Content-Length'], 796588
assert_match %r{\Amultipart/form-data}, req.headers['Content-Type']
assert_equal req.headers['User-Agent'], 'Ruby'
# Test the body. This will exclude the image blob data which follow these lines in the body
req.body.lines[1].match('Content-Disposition: form-data; name="FormParameterNameForImage"; filename="image_filename.jpeg"').size >= 1
req.body.lines[2].match('Content-Type: img/jpeg').size >= 1
end
Testing only the headers could also have been done using the normal WebMock way of using stub_request(:post, 'sample.com').with(headers: {'Accept' => 'application/json}), and simply not include any body specification in the with clause.

Related

Regex with webmock in aws-sdk not working

I'm trying to use webmock with rspec to stub out requests to Aws but I can't seem to get the regex to work for SQS polling. If I run rspec, webmock generates a 'correct' stub for me to use in a before(:each) block, in my spec_helper.rb like this:
You can stub this request with the following snippet:
stub_request(:post, "https://sqs.us-west-2.amazonaws.com/123456789012/backlog").
with(:body => "Action=ReceiveMessage&AttributeName.1=All&MaxNumberOfMessages=1&MessageAttributeName.1=All&QueueUrl=https%3A%2F%2Fsqs.us-west-2.amazonaws.com%2F123456789012%2Fbacklog&Version=2012-11-05&VisibilityTimeout=0&WaitTimeSeconds=20",
:headers => {'Accept'=>'*/*', 'Accept-Encoding'=>'', 'Authorization'=>'AWS4-HMAC-SHA256 Credential=MY_ACCESS_KEY/20150726/us-west-2/sqs/aws4_request, SignedHeaders=content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=large_alpha-numeric-signature', 'Content-Length'=>'224', 'Content-Type'=>'application/x-www-form-urlencoded; charset=utf-8', 'Host'=>'sqs.us-west-2.amazonaws.com', 'User-Agent'=>'aws-sdk-ruby2/2.1.7 ruby/2.2.2 x86_64-darwin14', 'X-Amz-Content-Sha256'=>'69336339ae76cf370477d4dsaf667as0b5dd8d25762c7c78sad8a', 'X-Amz-Date'=>'20150726T143009Z'}).
to_return(:status => 200, :body => "", :headers => {})
So in my spec_helper.rb I have
RSpec.configure do |config|
config.before(:each) do
stub_request(:post, "https://sqs.us-west-2.amazonaws.com/123456789012/backlog").
with(:body => "Action=ReceiveMessage&AttributeName.1=All&MaxNumberOfMessages=1&MessageAttributeName.1=All&QueueUrl=https%3A%2F%2Fsqs.us-west-2.amazonaws.com%2F123456789012%2Fbacklog&Version=2012-11-05&VisibilityTimeout=0&WaitTimeSeconds=20",
:headers => {'Accept'=>'*/*',
'Accept-Encoding'=>'',
'Authorization'=>"AWS4-HMAC-SHA256 Credential=MY_ACCESS_KEY/20150726/us-west-2/sqs/aws4_request, SignedHeaders=content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=" + /"^[a-zA-Z0-9]*$"/,
'Content-Length'=>'224',
'Content-Type'=>'application/x-www-form-urlencoded; charset=utf-8',
'Host'=>'sqs.us-west-2.amazonaws.com',
'User-Agent'=>'aws-sdk-ruby2/2.1.7 ruby/2.2.2 x86_64-darwin14',
'X-Amz-Content-Sha256'=>'694236339ae76cf370477d4dsaf667as0b5dd8d25762c7c78sad8a',
'X-Amz-Date'=>""+ /"^[a-zA-Z0-9]*$"/}).
to_return(:status => 200, :body => "", :headers => {})
end
The areas I'm trying to use regex against are the Signature and the X-Amz-Date because they're the only two that seem to change between different attempts to run the rspec.
The problem is the regex seems to not be working because even though I've added it into the spec_helper.rb, every time I run the suite, I get back the recommended stub from webmock instead of a passing or failing test. It should be passing at this point, from what I understand from the webmock docs and several tutorials.
How should I change this to get webmock to work for my test suite against Aws SQS polling?
I've been bashing my head against my desk for a few days now so any help is much appreciated.
Signature is likely generated using a byproduct of Time.now and I'm guessing you don't actually want to test for that. Instead simply do:
stub_request(:post, "https://sqs.us-west-2.amazonaws.com/123456789012/backlog").and_return(:status => 200, :body => "", :headers => {})
If you want to be even less specific on the URL (like ommiting that ID) you can even use a regex match:
stub_request(:post, /amazonaws.com/).and_return(:status => 200, :body => "", :headers => {})
I have the same problem and have the solution only for 'X-Amz-Date'.
Since it is date in special format, use Timecop.freeze block around your mock and method call.
Timecop.freeze do
stub_request(:get, "https://s3.eu-central-1.amazonaws.com/test_bucket/test").
with(headers: {'Accept'=>'*/*', 'Accept-Encoding'=>'', 'Authorization'=>'AWS4-HMAC-SHA256 Credential=access_key/20190814/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=9e6a6a209a0cb05346a058c12ef706ebff185ae3b72f3e542e1becbc97e8ea7a', 'Content-Length'=>'0', 'Content-Type'=>'', 'Host'=>'s3.eu-central-1.amazonaws.com', 'User-Agent'=>'aws-sdk-ruby2/2.9.44 ruby/2.6.3 x86_64-darwin18 resources', 'X-Amz-Content-Sha256'=>'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'X-Amz-Date'=>"#{time.utc.iso8601(0).gsub(/[^\p{Alnum}]/, '')}"}).
to_return(status: 200, body: "", headers: {})
AwsService.bucket.object("test").get
end
The regexp leaves only alphanumerical characters and Timecop takes care of the time.

DELETE method with a payload using Ruby Rest:Client?

While I don't think it is very restful to have to include a payload in a DELETE request. I ran into an instance where I am testing a service that requires a payload for DELETE. Might there be a way using Ruby's Rest Client to accomplish this? Unfortunately, I am having a hard time with this one.
#json_request = '{"user_id": 5, "meta_data": "foo"}'
resource = RestClient::Resource.new "http://www.foo.com/some/process"
#response_update = resource.delete(#json_request, :content_type => :json, :accept => :json)
Output:
ArgumentError:
wrong number of arguments (2 for 0..1)
Try this
RestClient::Request.execute(:method => 'delete', :url => "http://www.foo.com", :payload => json_data)
Currently it's not possible with that gem. You can see a PL addressing that. Maybe you could fork it and pull those changes to your own fork of the rest-client gem.
The pull request https://github.com/rest-client/rest-client/pull/98
As a very modern update, from the ReadMe
RestClient::Request.execute(method: :delete, url: 'http://example.com/resource',
payload: 'foo', headers: {myheader: 'bar'})

Ruby RestClient converts XML to Hash

I need to send a POST request as an XML string but I get odd results. The code:
require 'rest_client'
response = RestClient.post "http://127.0.0.1:2000", "<tag1>text</tag1>", :content_type => "text/xml"
I expect to receive "<tag1>text</tag1>" as the parameter on the request server. Instead, I get "tag1"=>"text". It converts the XML to a hash. Why is that? Any way around this?
Try this:
response = RestClient.post "http://127.0.0.1:2000",
"<tag1>text</tag1>",
{:accept => :xml, :content_type => :xml}
I think you just needed to specify the ":accept" to let it know you wanted to receive it in the XML format. Assuming it's your own server, you can debug on the server and see the request format used is probably html.
Hope that helps.
Instead of using RestClient, use Ruby's built-in Open::URI for GET requests or something like Net::HTTP or the incredibly powerful Typhoeus:
uri = URI('http://www.example.com/search.cgi')
res = Net::HTTP.post_form(uri, 'q' => 'ruby', 'max' => '50')
In Typhoeus, you'd use:
res = Typhoeus::Request.post(
'http://localhost:3000/posts',
:params => {
:title => 'test post',
:content => 'this is my test'
}
)
Your resulting page, if it's in XML will be easy to parse using Nokogiri:
doc = Nokogiri::XML(res.body)
At that point you'll have a fully parsed DOM, ready to be searched, using Nokogiri's search methods, such as search and at, or any of their related methods.

How to Send a Multidimensional Array using Rack::Test post

I am sending a multidimensional array using the post method given by the Rack::Test like
post "#{url}.json",:data => [["Company","Website"],["this is the dummy text, with,comma","www.technology.com"],["some company","www.url.com"]]
But in my controller when check my params params[:data] i am receiving it as a single dimensioned array ["Company", "Website", "this is the dummy text, with comma", "www.technology.com", "some company", "www.url.com"]
But i want it to retain it's property as a multi-dimensional array. I need help to do this.
The Rack::Test methods all have the same signature... and the second param is a hash of params
i.e.
post '/path', params={}, rack_env={}
This is because they're just URL params - which are typical key/value structures (i.e. a hash)
Why do you need it to be a multi-dimensional array?
EDIT: oh, I get it - you have a single hash with one key (:data)
If it's still causing you grief you could explicitly call to_param in there
ruby-1.9.2-p180 :003 > h = {:data => [["Company","Website"],["this is the dummy text, with,comma","www.technology.com"],["some company","www.url.com"]]}
=> {:data=>[["Company", "Website"], ["this is the dummy text, with,comma", "www.technology.com"], ["some company", "www.url.com"]]}
ruby-1.9.2-p180 :004 > h.to_param
=> "data[][]=Company&data[][]=Website&data[][]=this+is+the+dummy+text%2C+with%2Ccomma&data[][]=www.technology.com&data[][]=some+company&data[][]=www.url.com"
A workaround if you really need nested arrays is to change the request content type to JSON:
post url, JSON.dump([[1, 2], [3, 4]]), { "CONTENT_TYPE" => "application/json" }
This will correctly send a nested array through to the rack app.
Neither of the above worked too well for me but this did (see my original answer here).
The problem is body is sent as application/x-www-form-urlencoded by default, which doesn't handle multi-dimensional arrays too well. You can send it as application/json, but Sinatra probably won't merge the data into the request params. I use a middleware from rack-contrib which parses a json body from a POST request and merges it for you:
# Gemfile
`gem 'rack-contrib'`
then config.ru:
require 'rack/contrib'
require './app'
use Rack::PostBodyContentTypeParser
run Sinatra::Application
This won't be used in testing by default, but you can specify it:
# spec_helper.rb
OUTER_APP = Rack::Builder.parse_file("config.ru").first
module RSpecMixin
include Rack::Test::Methods
def app
OUTER_APP # typically this might just be Sinatra::Application
end
end
RSpec.configure do |config|
config.include RSpecMixin
end
And example usage:
it 'is ok' do
post '/', { key: 'value' }.to_json, { 'CONTENT_TYPE' => 'application/json' }
expect(last_response).to be_ok
end

Rails 3 - Testing controller's GET method - trying to use JSON and getting 406 errors

I have a simple controller that specifies:
respond_to :json
When I try to build a functional test that calls it like this:
test "GET" do
get 'index', :format => :json
end
Everything works just fine. If, however, I try to pass a query parameter in like this:
test "GET" do
get 'index', {:my_param = '1234'}, :format => :json
end
I get a 406 error returned from the controller. If I dump the response via response.inspect, I can see that the #status = 406 and the #header has a content-type of text/html. If I dump the response via response.inspect for the simple case that doesn't pass a query parameter, I see that the #header has a content-type of application/json as I request in the code.
Does anyone have any ideas what I'm doing wrong here? I'm suspecting that I'm screwing up ruby syntax with hashes or something but I'm bumping my head against the wall and not getting anywhere.
I'm on a Mac running Ruby 1.9 and Rails 3.0.5.
Thanks in advance!
This is the function you are calling:
get(action, parameters = nil, session = nil, flash = nil)
The :format part is still a parameter, certainly not a session or flash. Try:
get 'index', {:my_param => '1234', :format => :json}
Oh, important note, use '=>', not '=' ... that's a hash, not an assignment.

Resources