Background: My project's continuous integration build runs RSpec in several parallel runs. Specs are partitioned across parallel runs by spec file. That means long spec files dominate test suite run time. So I want to know the time each spec file takes to run (not just the time each example takes to run).
How can I get RSpec to tell me the time each spec file takes to run? Several of RSpec's stock formatters tell me the time each example takes to run, but they don't sum the time for each spec.
I'm using RSpec 3.2.
I addressed this need by writing my own RSpec formatter. Put the following class in spec/support, make sure it's required, and run rspec like so:
rspec --format SpecTimeFormatter --out spec-times.txt
class SpecTimeFormatter < RSpec::Core::Formatters::BaseFormatter
RSpec::Core::Formatters.register self, :example_started, :stop
def initialize(output)
#output = output
#times = []
end
def example_started(notification)
current_spec = notification.example.file_path
if current_spec != #current_spec
if #current_spec_start_time
save_current_spec_time
end
#current_spec = current_spec
#current_spec_start_time = Time.now
end
end
def stop(_notification)
save_current_spec_time
#times.
sort_by { |_spec, time| -time }.
each { |spec, time| #output << "#{'%4d' % time} seconds #{spec}\n" }
end
private
def save_current_spec_time
#times << [#current_spec, (Time.now - #current_spec_start_time).to_i]
end
end
I have a class like this.
class Time
def has_same_hours?(t)
self.strftime("%Y%m%d%H") == t.strftime("%Y%m%d%H")
end
end
class MyLogger
DATA_DIR = 'data'
def initialize
#time_current_hour = Time.now
#io = nil
update_io_to_current_hour
end
def update_io_to_current_hour
#io = open output_filename, "a+" if #io.nil?
return if #time_current_hour.has_same_hours? Time.now
#io.close
#io = open output_filename, "a+"
#time_current_hour = Time.now
end
def output_filename(time = Time.now)
"#{DATA_DIR}/#{time.strftime('%Y_%m_%d_%H')}.txt"
end
end
When update_io_to_current_hour is called, the file IO should be changed if hour is different compare to #time_current_hour.
I want to write RSpec test for it. This is what I wrote.
describe Logger do
let(:logger){ Logger.new }
describe "#update_io_to_current_hour" do
context "when the hour changes" do
before{
#time_now = Time.parse("2010/4/10 19:00")
#time_current = Time.parse("2010/4/10 18:59")
Time.stub(:now).and_return(#time_now)
logger.stub(:time_current_hour).and_return(#time_current)
}
it "should change file io" do
expect{logger.update_io_to_current_hour}.to change{ logger.instance_variable_get :#io }
end
end
context "when the hour doesn't changes" do
before{
#time_now = Time.parse("2010/4/10 18:59")
#time_current = Time.parse("2010/4/10 18:58")
Time.stub(:now).and_return(#time_now)
logger.stub(:time_current_hour).and_return(#time_current)
}
it "should not change file io" do
expect{logger.update_io_to_current_hour}.not_to change{ logger.instance_variable_get :#io }
end
end
end
end
Second test passes and first not. It looks like file io is never changed whatever stubbed to Time object.
What am I doing wrong? How can I write the test properly?
A couple of points:
logger.stub(:time_current_hour)
The class has no method named :time_current_hour, only an instance variable. There is rarely a good reason to test the values of instance variables; that is an implementation detail. You want to test behavior. In any case this stub is ineffective. Also
logger.instance_variable_get :#io
Now you are reaching right into the guts of your object and inspecting its internal values. Have you no regard for its privacy? :)
I think this would be a lot easier if you simply tested the value of :output_filename. When the hour changes, the filename changes. When the hour is the same, the filename is the same.
Starting using rspec I have difficulties trying to test threaded code.
Here is a simplicfication of a code founded, and I made it cause i need a Queue with Timeout capabilities
require "thread"
class TimeoutQueue
def initialize
#lock = Mutex.new
#items = []
#new_item = ConditionVariable.new
end
def push(obj)
#lock.synchronize do
#items.push(obj)
#new_item.signal
end
end
def pop(timeout = :never)
timeout += Time.now unless timeout == :never
#lock.synchronize do
loop do
time_left = timeout == :never ? nil : timeout - Time.now
if #items.empty? and time_left.to_f >= 0
#new_item.wait(#lock, time_left)
end
return #items.shift unless #items.empty?
next if timeout == :never or timeout > Time.now
return nil
end
end
end
alias_method :<<, :push
end
But I can't find a way to test it using rspec. Is there any effective documentation on testing threaded code? Any gem that can helps me?
I'm a bit blocked, thanks in advance
When unit-testing we don't want any non-deterministic behavior to affect our tests, so when testing threading we should not run anything in parallel.
Instead, we should isolate our code, and simulate the cases we want to test, by stubbing #lock, #new_item, and perhaps even Time.now (to be more readable I've taken the liberty to imagine you also have attr_reader :lock, :new_item):
it 'should signal after push' do
allow(subject.lock).to receive(:synchronize).and_yield
expect(subject.new_item).to receive(:signal)
subject.push('object')
expect(subject.items).to include('object')
end
it 'should time out if taken to long to enter synchronize loop' do
#now = Time.now
allow(Time).to receive(:now).and_return(#now, #now + 10.seconds)
allow(subject.items).to receive(:empty?).and_return true
allow(subject.lock).to receive(:synchronize).and_yield
expect(subject.new_item).to_not receive(:wait)
expect(subject.pop(5.seconds)).to be_nil
end
etc...
I am writing an automated testing program which will test some web programs that are sometimes slow to load certain AJAX calls. For instance the user will click 'Query' which will result in a HTML 'loading' overlay for anywhere from 15 to 90 seconds. When the search completes, it will then update a table on the same page with the results.
So obviously I can increase the waiting time individually like so:
browser.td(:id => 'someId').when_present.some_action #=> will wait 30 seconds
browser.td(:id => 'someId').when_present(90).some_action #=> will wait *90* seconds
But is there a way to modify (in my case increase) the time so Watir-Webdriver always waits 90 seconds on .when_present like so:
browser.some_default = 90
browser.td(:id => 'someId').when_present.some_action #=> will wait *90* seconds
A few words of warning: Client timeout will not affect when_present. Nor will implicit wait.
Update: This monkey patch has been merged into watir-webdriver and so will no longer be needed in watir-webdriver v0.6.5. You will be able to set the timeout using:
Watir.default_timeout = 90
The wait methods are defined similar to:
def when_present(timeout = 30)
message = "waiting for #{selector_string} to become present"
if block_given?
Watir::Wait.until(timeout, message) { present? }
yield self
else
WhenPresentDecorator.new(self, timeout, message)
end
end
As you can see, the default timeout of 30 seconds is hard-coded. Therefore, there is no easy way to change it everywhere.
However, you could monkey patch the wait methods to use a default time and set it to what you want. The following monkey patch will set the default timeout to 90 seconds.
require 'watir-webdriver'
module Watir
# Can be changed within a script with Watir.default_wait_time = 30
#default_wait_time = 90
class << self
attr_accessor :default_wait_time
end
module Wait
class << self
alias old_until until
def until(timeout = Watir.default_wait_time, message = nil, &block)
old_until(timeout, message, &block)
end
alias old_while while
def while(timeout = Watir.default_wait_time, message = nil, &block)
old_while(timeout, message, &block)
end
end # self
end # Wait
module EventuallyPresent
alias old_when_present when_present
def when_present(timeout = Watir.default_wait_time, &block)
old_when_present(timeout, &block)
end
alias old_wait_until_present wait_until_present
def wait_until_present(timeout = Watir.default_wait_time)
old_wait_until_present(timeout)
end
alias old_wait_while_present wait_while_present
def wait_while_present(timeout = Watir.default_wait_time)
old_wait_while_present(timeout)
end
end # EventuallyPresent
end # Watir
Include the patch after the watir webdriver code is loaded.
What's the best way to set Time.now for the purpose of testing time-sensitive methods in a unit test?
I really like the Timecop library. You can do time warps in block form (just like time-warp):
Timecop.travel(6.days.ago) do
#model = TimeSensitiveMode.new
end
assert #model.times_up!
(Yes, you can nest block-form time travel.)
You can also do declarative time travel:
class MyTest < Test::Unit::TestCase
def setup
Timecop.travel(...)
end
def teardown
Timecop.return
end
end
I have some cucumber helpers for Timecop here. They let you do things like:
Given it is currently January 24, 2008
And I go to the new post page
And I fill in "title" with "An old post"
And I fill in "body" with "..."
And I press "Submit"
And we jump in our Delorean and return to the present
When I go to the home page
I should not see "An old post"
Personally I prefer to make the clock injectable, like so:
def hello(clock=Time)
puts "the time is now: #{clock.now}"
end
Or:
class MyClass
attr_writer :clock
def initialize
#clock = Time
end
def hello
puts "the time is now: #{#clock.now}"
end
end
However, many prefer to use a mocking/stubbing library. In RSpec/flexmock you can use:
Time.stub!(:now).and_return(Time.mktime(1970,1,1))
Or in Mocha:
Time.stubs(:now).returns(Time.mktime(1970,1,1))
I'm using RSpec and I did this: Time.stub!(:now).and_return(2.days.ago) before I call Time.now. In that way I'm able to control the time I used for that particular test case
Using Rspec 3.2, the only simple way I found to fake Time.now return value is :
now = Time.parse("1969-07-20 20:17:40")
allow(Time).to receive(:now) { now }
Now Time.now will always return the date of Apollo 11 landing on the moon.
Source: https://www.relishapp.com/rspec/rspec-mocks/docs
Do the time-warp
time-warp is a library that does what you want. It gives you a method that takes a time and a block and anything that happens in the block uses the faked time.
pretend_now_is(2000,"jan",1,0) do
Time.now
end
Don't forget that Time is merely a constant that refers to a class object. If you're willing to cause a warning, you can always do
real_time_class = Time
Time = FakeTimeClass
# run test
Time = real_time_class
If you have ActiveSupport included, you could use:
travel_to Time.zone.parse('2010-07-05 08:00')
http://api.rubyonrails.org/classes/ActiveSupport/Testing/TimeHelpers.html
Also see this question where I put this comment as well.
Depending upon what you are comparing Time.now to, sometimes you can change your fixtures to accomplish the same goal or test the same feature. For example, I had a situation where I needed one thing to happen if some date was in the future and another to happen if it was in the past. What I was able to do was include in my fixtures some embedded ruby (erb):
future:
comparing_date: <%= Time.now + 10.years %>
...
past:
comparing_date: <%= Time.now - 10.years %>
...
Then in your tests then you choose which one to use to test the different features or actions based upon the time relative to Time.now.
Had the same issue, I had to fake time for a spec for a specific day and time just did that:
Time.stub!(:now).and_return(Time.mktime(2014,10,22,5,35,28))
this will give you:
2014-10-22 05:35:28 -0700
This kind of works and allows for nesting:
class Time
class << self
attr_accessor :stack, :depth
end
def self.warp(time)
Time.stack ||= []
Time.depth ||= -1
Time.depth += 1
Time.stack.push time
if Time.depth == 0
class << self
alias_method :real_now, :now
alias_method :real_new, :new
define_method :now do
stack[depth]
end
define_method :new do
now
end
end
end
yield
Time.depth -= 1
Time.stack.pop
class << self
if Time.depth < 0
alias_method :new, :real_new
alias_method :now, :real_now
remove_method :real_new
remove_method :real_now
end
end
end
end
It could be slightly improved by undefing the stack and depth accessors at the end
Usage:
time1 = 2.days.ago
time2 = 5.months.ago
Time.warp(time1) do
Time.real_now.should_not == Time.now
Time.now.should == time1
Time.warp(time2) do
Time.now.should == time2
end
Time.now.should == time1
end
Time.now.should_not == time1
Time.now.should_not be_nil
Depending upon what you are comparing Time.now to, sometimes you can change your fixtures to accomplish the same goal or test the same feature. For example, I had a situation where I needed one thing to happen if some date was in the future and another to happen if it was in the past. What I was able to do was include in my fixtures some embedded ruby (erb):
future:
comparing_date: <%= Time.now + 10.years %>
...
past:
comparing_date: <%= Time.now - 10.years %>
...
Then in your tests then you choose which one to use to test the different features or actions based upon the time relative to Time.now.
i just have this in my test file:
def time_right_now
current_time = Time.parse("07/09/10 14:20")
current_time = convert_time_to_utc(current_date)
return current_time
end
and in my Time_helper.rb file i have a
def time_right_now
current_time= Time.new
return current_time
end
so when testing the time_right_now is overwritten to use what ever time you want it to be.
I allways extract Time.now into a separate method that I turn into attr_accessor in the mock.
The recently-released Test::Redef makes this and other fakery easy, even without restructuring the code in a dependency-injection style (especially helpful if you're using other peoples' code.)
fake_time = Time.at(12345) # ~3:30pm UTC Jan 1 1970
Test::Redef.rd 'Time.now' => proc { fake_time } do
assert_equal 12345, Time.now.to_i
end
However, be careful of other ways to obtain time that this will not fake out (Date.new, a compiled extension that makes its own system call, interfaces to things like external database servers which know current timestamps, etc.) It sounds like the Timecop library above might overcome these limitations.
Other great uses include testing things like "what happens when I'm trying to use this friendly http client but it decides to raise this an exception instead of returning me a string?" without actually setting up the network conditions which lead to that exception (which may be tricky). It also lets you check the arguments to redef'd functions.
My own solution https://github.com/igorkasyanchuk/rails_time_travel - a gem with UI, so you don't need to hardcode any datetime in the code. Just change it from the UI.
It might be also very useful for you QA's team, or testing app on the staging.