I have some code in my tests which works with some external service. This service is not very stable, so sometimes it crash for no reason. But in 80% of runs it works well.
So I want a method to automatically rerun all failed rspecs several time (for example 2 or 3 time).
Is there any way to do it?
Many people would say that your test actually should never hit external services, and that's one of the reasons to do it. Your tests should not fail because some external service is down.
TL;DR use mocks and stubs or to replace those external service calls
Instead of re-running failed specs, couldn't you just run the method accessing the service a set number of times and run the expectation on the logical OR of the results?
So instead of:
it "returns expected value for some args" do
unstable_external_service(<some args>).should == <expected return value>
end
just do something like this:
def run_x_times(times, args)
return nil if times == 0
unstable_external_service(args) || run_x_times(times-1)
end
it "returns expected value for some args" do
run_x_times(10, <some args>).should == <expected return value>
end
You can use the same wrapper method throughout your tests anytime you access the service. I'm assuming here that your service returns nil on a failure, but if not you could change this to fit your particular case -- you get the general idea.
Related
Perhaps it is my lack of familiarity with rspec but i do not understand what is going on with my test.
I have 2 classes one called Scrape, the other Result (creative) Scrape is a web scraping class that searches a site and scrapes the results from the page, creating a new Result instance from each.
Result instances are stored in a class variable array accessible via Result.all
this works in practice in the actual program, however when I tried to write a test for this behavior it fails.
describe "#scrape_results" do
it "accepts a url scrapes the page and creates a Result for each" do
s = Scrape.new
s.scrape_results(#url)
expect(Result.all.count).not_to eq(0)
end
end
every time i run the test Result.all.count is 0
if i use pry and manually run #scrape_results the test passes.
I appreciate your time, patience, and help
thanks
I notice that you are passing #url to #scrape_results in your test. Unless you are defining that variable inside of the describe block or the test block it will be nil in your test. It is possible that since #url might be something other than nil from wherever you are pry-ing which is causing the Result to be created and the test to pass.
I'm working on some unit tests. One of them use a specific configuration variable as set in my application MyBigApp::Env which looks like:
{:country=>'uk', :another_hosts=>["192.168.99.105"]}
So I access it with MyBigApp::Env.country
However in my unit test I want that country for the test become something.
Using rspec I've seen stub but can't get it to work - any ideas where I'm going wrong:
MyBigApp::Env.stub(:[]).with('country').and_return('gr')
Also tried this (as above shows deprecated):
allow(MyBigApp::Env).to receive('country').and_return('gr')
Infact as a test, I also tried:
my_hash = {:uri=>nil}
allow(my_hash).to receive(:[]).with(:uri).and_return('Over written!')
p my_hash
and that didnt update either - it just returned {:uri=>nil}
As a workaround, at the moment I'm having to save the env var in a temp var in the before(each) block then return it back to the original in the after(each). This feels really risky to me. I'm thinking imagine the service running and someone runs unit tests it could effect the end user in that small instance the test is running.
Any help would be appreciated.
Thanks
Yes it possible, but keep in mind that stub only works when you trigger/call the method that you stubbed/mocked
my_hash = {:uri=>nil}
allow(my_hash).to receive(:[]).with(:uri).and_return('Over written!')
p my_hash[:url] # it will be 'Over written!'
This works for me:
my_hash = {:uri=>nil}
allow(my_hash).to receive(:[]).with(:uri).and_return('Over written!')
expect(my_hash[:uri]).to eq "Over written!"
In your sample test case, you are just calling p my_hash which doesn't actually call the [] method.
In terms of why this isn't working with MyBigApp::Env, well, that really depends on what class it is. Possible whatever method .country is doesn't actually call [].
Really, if you call MyBigApp::Env['country'] and stub MyBigApp::Env to receive [] with 'country', it should work.
In regards to your concern about changing your running application's behavior from the tests ... what kind of tests are these?! Running unit tests against a live production application would be very odd. How do you imagine it would change your production app's code? The Env hash just lives in memory right?
Anyway, you should never have to worry about your tests changing the experience for an 'end user'. Always run tests on a completely quarantined envionment, meaning don't use the same database. Actually, the test database is usually wiped after each test.
Just wanted to suggest a non-stubbing alternative. For example:
def code_under_test
key = 'country'
# ... maybe lots of code
value = MyBigApp::Env[key] # deep inside some classes
# ... lots more code
"This is the #{value}"
end
MyBigApp::Env is hard-coded deep in the code, and the need for stubbing reveals that dependency, and the benefits of OOP encapsulation are lost.
It'd be much easier if this were the case:
def code_under_test(config_vars = MyBigApp::Env)
"This is the #{config_vars['country']}"
end
it 'should return my country value' do
value = previous_code_under_test('country' => 'TEST VALUE')
expect(value).to eq("This is the TEST VALUE")
end
No stubbing required, just plain old method calls.
I have a simple test in an RSpec controller spec that checks to see if the correct message was passed to a delayed job:
it 'sends a message to NotifyFollowersJob with relevant person and split_time data' do
allow(NotifyFollowersJob).to receive(:perform_later)
post :import, params: request_params
split_time_ids = SplitTime.all.ids
person_id = SplitTime.first.effort.person_id
expect(NotifyFollowersJob).to have_received(:perform_later)
.with(person_id: person_id,
split_time_ids: split_time_ids)
end
The test usually passes, but sometimes it fails because the split_time_ids (an Array) are sometimes reversed. I do not care what order the split_time_ids are passed to NotifyFollowersJob, so the test should pass regardless of the order.
If I were testing the contents of the Array alone, I could write:
expect(split_time_ids).to match_array(SplitTime.all.ids)
But I can't figure out how to get similar functionality where the Array is a value of one of several arguments.
Any RSpec masters out there care to give me some guidance?
One idea is to make a block to check the message and pass it to your allow method.
Like this:
allow(NotifyFollowersJob).to receive(:perform_later) do |arg|
# Handle your args to avoid fails because of its order
expect(...)
end
And you should keep the expect(NotifyFollowersJob).to have_received(:perform_later) (without the with verification) just to make sure it is still being called.
It looks like your controller action runs SplitTime.all.ids query and passes them to the worker as args.
If you don't care about the order, maybe you can stub this query, to get the same result all the time?
In addition, it'll make your test faster.
Many times one outcome may have two different consequences that need to be tested with a test double. For example if a network connection is successful I'd like to log a message, and also pass the resource to another object that will store it internally. On the other hand it feels unclean to put these two in one test. For example this code fails:
describe SomeClass do
let(:logger) { double('Logger') }
let(:registry) { double('Registry') }
let(:cut) { SomeClass.new }
let(:player) { Player.new }
describe "#connect" do
context "connection is successful" do
it "should log info" do
logger.should_receive(:info).with('Player connected successfully')
cut.connect player
end
it "should register player" do
registry.should_receive(:register).with(player)
cut.connect player
end
end
end
end
I could specify in each test that the function in the other one might get called, but that looks like unnecessary duplication. In that case I'd rather make this one test.
I also don't like that it's never explicit in the test that a method should NOT be called.
Does anyone know about an alternative that has an explicit 'should_not_receive' message instead of automatically rejecting calls that are not explicitly specified?
RSpec supports should_not_receive, which is equivalent to should_receive(...).exactly(0).times as discussed in this message from the original author of RSpec.
I setup a mock object and told it to expect a check for nil and to return false:
status = double('status')
status.should_receive(:nil?).and_return(false)
I only expect the call to nil? to occur once, but I got an error in my rspec test, saying that status received nil? twice.
Is there a way to get rspec to show where/how each call occurred?
adding the '--backtrace' option did not work.
Try something like this:
status.should_receive(:nil?).twice { puts caller; false }
This tells rspec to allow two invocations and call the associated block each time. Thecaller method generates a full backtrace which you should be able to analyze onstdout. We also returnfalse to stay on the code-path we're testing.
If the two backtraces are hard to distinguish and you're only interested in the second (unexpected) invocation, then set up two successive expectations:
status.should_receive(:nil?).and_return(false)
status.should_receive(:nil?) { puts caller; false }
Here the double will return false on the first invocation and call the block on the second.
Reference for setting responses on expectations:
https://github.com/rspec/rspec-mocks#setting-responses