Testing rack-timeout in sinatra and ruby - ruby

This is something that I thought would be straightforward but I'm having issues around testing the rack-timeout gem. I have a sinatra base class with an endpoint which does some logic.
module MyModule
class MySinatra < Sinatra::Base
use Rack::Timeout
Rack::Timeout.timeout = 10
get '/dosomething' do
#do the normal logic.
end
end
end
More information on the rack-timeout gem is here. I'm trying to setup a test where I can send a request which I know will take more than a few seconds in order for it to fail.
Here is the test so far
require "test/unit"
require "mocha/setup"
require 'rack/timeout'
def test_rack_timeout_should_throw_timed_out_exception_test
Rack::Timeout.stubs(:timeout).returns(0.0001)
assert_raises TimeoutError do
get "/dosomething"
end
Rack::Timeout.unstub
end
There are a number of ways this could be done but I am not sure how they would be implemented
Override the '/dosomething' method as part of the test to {sleep 3}
Do the same as above but with a stubbing or mocking library
instead of using get "/dosomething" in the test, create a net::http response which will keep the request open.
Any thoughts on this would be very much appreciated.

First of all your test will not actually pass, because the error is not handed through to the test. It is only raised on the server side. Luckily, rack-test provides the last_response.errors method to check whether there were errors. Therefore i would write the above test as follows:
def test_rack_timeout_should_throw_timed_out_exception
Rack::Timeout.stubs(:timeout).returns(0.0001)
get '/dosomething'
assert last_response.server_error?, 'There was no server error'
assert last_response.errors.include?('Timeout::Error'), 'No Timeout::Error raised'
Rack::Timeout.unstub
end
Now the only thing left to do is to simulate a slow response by overriding the route. It seemed simple at first but then i realized it is not so simple at all when i got my hands on it. I fiddled around a lot and came up with this here:
class Sinatra::Base
def self.with_fake_route method, route, body
old_routes = routes.dup
routes.clear
self.send(method.to_sym, route.to_s, &body.to_proc)
yield
routes.merge! old_routes
end
end
It will allow you to temporarily use only a route, within the block you pass to the method. For example now you can simulate a slow response with:
MyModule::MySinatra.with_fake_route(:get, '/dosomething', ->{ sleep 0.0002 }) do
get '/dosomething'
end
Note that the get '/dosomething' inside the block is not the definition of the temporary route, but a method of rack-test firing a mock request. The actual override route is specified in form of arguments to with_route.
This is the best solution i could come up with but i would love to see a more elegant way to solve this.
Complete working example (ran on Ruby 1.9.3.p385):
require 'sinatra/base'
require 'rack/timeout'
module MyModule
class MySinatra < Sinatra::Base
use Rack::Timeout
Rack::Timeout.timeout = 10
get '/dosomething' do
'foo'
end
end
end
require 'test/unit'
require 'rack/test'
require 'mocha/setup'
class Sinatra::Base
def self.with_fake_route method, route, body
old_routes = routes.dup
routes.clear
self.send(method.to_sym, route, &body)
yield
routes.merge! old_routes
end
end
class Tests < Test::Unit::TestCase
include Rack::Test::Methods
def app
MyModule::MySinatra
end
def test_rack_timeout_should_throw_timed_out_exception
Rack::Timeout.stubs(:timeout).returns(0.0001)
MyModule::MySinatra.with_fake_route(:get, '/dosomething', ->{ sleep 0.0002 }) do
get '/dosomething'
end
assert last_response.server_error?, 'There was no server error'
assert last_response.errors.include?('Timeout::Error'), 'No Timeout::Error raised'
Rack::Timeout.unstub
end
end
produces:
1 tests, 2 assertions, 0 failures, 0 errors, 0 skips

Related

Raising 500 errors deliberately in Sinatra in order to test how they are handled

I want to write an RSpec test which verifies that, should a 500 error occur in my Sinatra-powered API, the error will be caught by a Sinatra error definition and returned to the client in a JSON format. That is, rather than returning some HTML error page, it returns JSON like this to conform with the rest of the API:
{
success: "false",
response: "Internal server error"
}
However, I'm unsure how to actually trigger a 500 error in my Sinatra app in order to test this behaviour with RSpec. I can't find a way to mock Sinatra routes, so currently my best idea is this route which deliberately causes a 500. This feels like a pretty dreadful solution:
get '/api/v1/testing/internal-server-error' do
1 / 0
end
Is there a way to mock Sinatra routes so that I can have, say, /'s route handler block raise an exception, therefore triggering a 500? If not, is there some other way to deliberately cause a 500 error in my app?
When facing a situation like this, what I usually do is separate concerns, and move logic outside of the Sinatra get ... block. Then, it is easy to stub it and make it raise an error.
For example, given this server code:
# server.rb
require 'sinatra'
class SomeModel
def self.some_action
"do what you need to do"
end
end
get '/' do
SomeModel.some_action
end
You can then use this code to have the model, or any other class/function you are using to actually generate the response, raise an error, using this spec:
# spec
describe '/' do
context 'on error' do
before do
allow(SomeModel).to receive(:some_action) { raise ArgumentError }
end
it 'errors gracefully' do
get '/'
expect(last_response.status).to eq 500
end
end
end
For completeness, here is a self contained file that can be tested to demonstrate this approach by running rspec thisfile.rb:
# thisfile.rb
require 'rack/test'
require 'rspec'
require 'sinatra'
# server
class SomeModel
def self.some_action
"do what you need to do"
end
end
get '/' do
SomeModel.some_action
end
# spec_helper
ENV['APP_ENV'] = 'test'
module RSpecMixin
include Rack::Test::Methods
def app() Sinatra::Application end
end
RSpec.configure do |c|
c.include RSpecMixin
end
# spec
describe '/' do
context 'on error' do
before do
allow(SomeModel).to receive(:some_action) { raise ArgumentError }
end
it 'errors gracefully' do
get '/'
expect(last_response.status).to eq 500
end
end
end
Use the halt method:
require 'sinatra'
get '/' do
halt 500, {
success: 'false',
response: 'Internal server error'
}.to_json
end

How do I get the Sinatra app instance that's being tested by rack-test?

I want to get hold of the app instance being tested by rack-test so that I can mock some of its methods. I thought I could simply save the app instance in the app method, but for some strange reason that doesn't work. It seems like rack-test simply uses the instance to get the class, then creates its own instance.
I've made a test to demonstrate my issue (it requires the gems "sinatra", "rack-test" and "rr" to run):
require "sinatra"
require "minitest/spec"
require "minitest/autorun"
require "rack/test"
require "rr"
describe "instantiated app" do
include Rack::Test::Methods
def app
cls = Class.new(Sinatra::Base) do
get "/foo" do
$instance_id = self.object_id
generate_response
end
def generate_response
[200, {"Content-Type" => "text/plain"}, "I am a response"]
end
end
# Instantiate the actual class, and not a wrapped class made by Sinatra
#app = cls.new!
return #app
end
it "should have the same object id inside response handlers" do
get "/foo"
assert_equal $instance_id, #app.object_id,
"Expected object IDs to be the same"
end
it "should trigger mocked instance methods" do
mock(#app).generate_response {
[200, {"Content-Type" => "text/plain"}, "I am MOCKED"]
}
get "/foo"
assert_equal "I am MOCKED", last_response.body
end
end
How come rack-test isn't using the instance I provided? How do I get hold of the instance that rack-test is using, so that I can mock the generate_response method?
Update
I have made no progress. It turns out rack-test creates the tested instance on the fly when the first request is made (i.e. get("/foo")), so it's not possible to mock the app instance before then.
I have used rr's stub.proxy(...) to intercept .new, .new! and .allocate; and added a puts statement with the instance's class name and object_id. I have also added such statements in the tested class' constructor, as well as a request handler.
Here's the output:
From constructor: <TestSubject 47378836917780>
Proxy intercepted new! instance: <TestSubject 47378836917780>
Proxy intercepted new instance: <Sinatra::Wrapper 47378838065200>
From request handler: <TestSubject 47378838063980>
Notice the object ids. The tested instance (printed from the request handler) never went through .new and was never initialized.
So, confusingly, the instance being tested is never created, but somehow exists none the less. My guess was that allocate was being used, but the proxy intercept shows that it doesn't. I ran TestSubject.allocate myself to verify that the intercept works, and it does.
I also added the inherited, included, extended and prepended hooks to the tested class and added print statements, but they were never called. This leaves me completely and utterly stumped as to what kind of horrible black magic rack-test is up to under the hood.
So to summarize: the tested instance is created on the fly when the first request is sent. The tested instance is created by fel magic and dodges all attempts to catch it with a hook, so I can find no way to mock it. It almost feels like the author of rake-test has gone to extraordinary lengths to make sure the app instance can't be touched during testing.
I am still fumbling around for a solution.
Ok, I finally got it.
The problem, all along, turned out to be Sinatra::Base.call. Internally, it does dup.call!(env). In other words, every time you run call, Sinatra will duplicate your app instance and send the request to the duplicate, circumventing all mocks and stubs. That explains why none of the life cycle hooks were triggered, since presumably dup uses some low level C magic to clone the instance (citation needed.)
rack-test doesn't do anything convoluted at all, all it does it call app() to retrieve the app, and then call .call(env) on the app. All I need to do then, is stub out the .call method on my class and make sure Sinatra's magic isn't being inserted anywhere. I can use .new! on my app to stop Sinatra from inserting a wrapper and a stack, and I can use .call! to call my app without Sinatra duplicating my app instance.
Note: I can no longer just create an anonymous class inside the app function, since that would create a new class each time app() is called and leave me unable to mock it.
Here's the test from the question, updated to work:
require "sinatra"
require "minitest/spec"
require "minitest/autorun"
require "rack/test"
require "rr"
describe "sinatra app" do
include Rack::Test::Methods
class TestSubject < Sinatra::Base
get "/foo" do
generate_response
end
def generate_response
[200, {"Content-Type" => "text/plain"}, "I am a response"]
end
end
def app
return TestSubject
end
it "should trigger mocked instance methods" do
stub(TestSubject).call { |env|
instance = TestSubject.new!
mock(instance).generate_response {
[200, {"Content-Type" => "text/plain"}, "I am MOCKED"]
}
instance.call! env
}
get "/foo"
assert_equal "I am MOCKED", last_response.body
end
end
Yes, rack test instantiates new app per each request (possibly to avoid collisions and start with a fresh state.) The option here would be to mock the Sinatra::Base derived class itself, inside app:
require "sinatra"
require "minitest/spec"
require "minitest/autorun"
require "rack/test"
require "rr"
describe "instantiated app" do
include Rack::Test::Methods
def app
Class.new(Sinatra::Base) do
get "/foo" do
generate_response
end
def generate_response
[200, {"Content-Type" => "text/plain"}, "I am a response"]
end
end.prepend(Module.new do # ⇐ HERE
def generate_response
[200, {"Content-Type" => "text/plain"}, "I am MOCKED"]
end
end).new!
end
it "should trigger mocked instance methods" do
get "/foo"
assert_equal "I am MOCKED", last_response.body
end
end
or, to mock app method in a whole.

No error when running unit test with an expectation on method calls (Ruby/Test::Unit/Mocha)

I have a question regaring Mocha expectations running with the Ruby Test::Unit framework.
My purpose is to verify the methods called when I call a wrapping method.
This test always seem to pass, I don't understand why...
I am expecting an error stating that the method clean has only be called once (with can be verified in the Expectation object).
Why is the following Unit Test not raising an error when I run the following test:
require 'test/unit'
require 'mocha'
require 'mocha/test_unit'
class DoMe
def stop_and_clean
clean
stop
end
def clean
true
end
def stop
true
end
end
class DoMeTest < Test::Unit::TestCase
def test_stop_and_clean
d = DoMe.new
d.expects(:clean).times(5)
assert(d.stop_and_clean)
end
end
It seems to me that expects(:clean).times(5) will not be satisfied, but running the test shows no error. Do I need to make an assertion on the Expectation object ?
exp = d.expects(:clean).times(5)
[...]
assert(that exp is invoked only once)

How to test if some specific rack middleware is being used?

To be more particular, I'm talking about sentry-raven and sinatra here. I saw examples testing sinatra applications, or middlewares. But I didn't see ones testing if some particular middleware is present. Or should I be testing behavior, not configuration (or how should I call it)?
The important thing (I'd say) is the behaviour, but if you wish to check for middleware there are 2 ways I'd suggest after taking a delve into the Sinatra source (there are possibly much easier/better ways):
The env
In the Sinatra source there's a method that uses the env to check if a middleware is already present:
# Behaves exactly like Rack::CommonLogger with the notable exception that it does nothing,
# if another CommonLogger is already in the middleware chain.
class CommonLogger < Rack::CommonLogger
def call(env)
env['sinatra.commonlogger'] ? #app.call(env) : super
end
You could do the same thing in a route, e.g.
get "/env-keys" do
env.keys.inspect
end
It'll only show you the middleware if it's inserted something in env hash, e.g.
class MyBad
def initialize app, options={}
#app = app
#options = options
end
def call env
#app.call env.merge("mybad" => "I'm sorry!")
end
end
output:
["SERVER_SOFTWARE", "SERVER_NAME", "rack.input", "rack.version", "rack.errors", "rack.multithread", "rack.multiprocess", "rack.run_once", "REQUEST_METHOD", "REQUEST_PATH", "PATH_INFO", "REQUEST_URI", "HTTP_VERSION", "HTTP_HOST", "HTTP_CONNECTION", "HTTP_CACHE_CONTROL", "HTTP_ACCEPT", "HTTP_USER_AGENT", "HTTP_DNT", "HTTP_ACCEPT_ENCODING", "HTTP_ACCEPT_LANGUAGE", "GATEWAY_INTERFACE", "SERVER_PORT", "QUERY_STRING", "SERVER_PROTOCOL", "rack.url_scheme", "SCRIPT_NAME", "REMOTE_ADDR", "async.callback", "async.close", "rack.logger", "mybad", "rack.request.query_string", "rack.request.query_hash", "sinatra.route"]
It's near the end of that list.
The middleware method
There's also a method called middleware in Sinatra::Base:
# Middleware used in this class and all superclasses.
def middleware
if superclass.respond_to?(:middleware)
superclass.middleware + #middleware
else
#middleware
end
end
Call it in the class definition of a modular app and you can get the middlewares in an array:
require 'sinatra/base'
class AnExample < Sinatra::Base
use MyBad
warn "self.middleware = #{self.middleware}"
output:
self.middleware = [[MyBad, [], nil]]
There may be a way to get it from Sinatra::Application, but I haven't looked.
With help from ruby-raven guys, we've got this:
ENV['RACK_ENV'] = 'test'
# the app: start
require 'sinatra'
require 'sentry-raven'
Raven.configure(true) do |config|
config.dsn = '...'
end
use Raven::Rack
get '/' do
'Hello, world!'
end
# the app: end
require 'rspec'
require 'rack/test'
Raven.configure do |config|
logger = ::Logger.new(STDOUT)
logger.level = ::Logger::WARN
config.logger = logger
end
describe 'app' do
include Rack::Test::Methods
def app
#app || Sinatra::Application
end
class TestRavenError < StandardError; end
it 'sends errors to sentry' do
#app = Class.new Sinatra::Application do
get '/' do
raise TestRavenError
end
end
allow(Raven.client).to receive(:send).and_return(true)
begin
get '/'
rescue TestRavenError
end
expect(Raven.client).to have_received(:send)
end
end
Or if raven sending requests is in the way (when tests fail because of raven sending requests, and not because of the underlying error), one can disable them:
Raven.configure(true) do |config|
config.should_send = Proc.new { false }
end
And mock Raven.send_or_skip instead:
...
allow(Raven).to receive(:send_or_skip).and_return(true)
begin
get '/'
rescue TestRavenError
end
expect(Raven).to have_received(:send_or_skip)
...

How can I test an action that handles exceptions using rack/test on Sinatra?

I want to test this route I made on Sinatra:
get '/party' do
begin
party_source.parties
rescue Exceptions::SourceNotFoundError
status 404
rescue Exceptions::SourceInternalError
status 503
end
end
And I wrote this test (assume the party_source is accessible by the test, in the actual code it is):
require 'rack/test'
def test_correct_status_code_when_get_error_404
source_404 = mock()
source_404.expects(:parties).with(nil).raises(Exceptions::SourceNotFoundError)
MyApp.party_source = source_404
get '/party'
assert_equal 404, last_response.status
end
When I run this test it fails because instead of getting 404 (my code) I get a status 500. No matter what exception I raise I always get and status 500, which I think is being generated by Sinatra or Rack.
How can I test this case?
Update
As I can understand it, the exceptions isn't getting caught by my rescues blocks. Rack or Sinatra is getting it and handling the HTTP Status 500 response message.
I can't understand how my rescue code block is being ignored.
Here's a short example, showing that you can test such an action:
hello_sinatra.rb:
require 'sinatra/base'
class Hello < Sinatra::Base
get '/party' do
begin
raise StandardError
rescue StandardError
status 404
end
end
end
Hello.run! if __FILE__ == $0
sinatra_test.rb:
$:.push('.')
require 'hello_sinatra'
require 'test/unit'
require 'rack/test'
ENV['RACK_ENV'] = 'test'
class HelloTest < Test::Unit::TestCase
include Rack::Test::Methods
def app
Hello
end
def test_correct_status_code_when_get_error_404
get '/party'
assert_equal 404, last_response.status
end
end
However, something looks strange in your code. Can you try to replace MyApp.party_source = source_404 with app.party_source = source_404
Update
You're only catching Exceptions::SourceNotFoundError and Exceptions::SourceInternalError, something else is likely going wrong in your mock, which gives the 500 error.
Add a catchall at the end of your begin/rescue block using rescue Exception and you will quickly see where the problem is.

Resources