I'm writing a new rspec test case using Capybara (SitePrism actually, which uses Capybara) and I've run into an apparently known issue: https://github.com/jnicklas/capybara/issues/1396. Essentially, due to a change in one or the other, RSpec and Capybara now both have methods named all, and when I try to use SitePrism to find a group of elements or SitePrism sections, Capybara invokes the wrong method and returns something of the type RSpec::Matchers::BuiltIn::All rather than the expected array of Capybara or SitePrism objects.
For some reason, all my old tests, including many with very similar usage of sections and elements constructs, work perfectly fine. I'm having a really hard time finding differences between them that would account for one failing and the other succeeding. I briefly tried rolling back either Capybara or RSpec to just try to make the problem go away for the moment, but it seemed silly, trying to pinpoint when the problem was introduced, when existing test cases that have been running every day never broke.
Can anyone advise me on why one works and the other fails? Here is what these two test cases have in common:
Both spec files require the same spec_helper.rb file.
The spec_helper.rb file has both require rspec and require capybara.
Each spec file uses require_relative to require a page object used in each test.
Each page object file has a sections :table_rows, <SECTION CLASS>, <ROW CSS> declaration. They may have different names, classes, and CSS Selectors, but they're the same basic construct.
In each spec file, methods are invoked on the page objects that reference table_rows.
Referencing table_rows in one of these older test cases works just fine, but I'm getting the name collision error in the new test case. Anyone know why that might be, so I can fix the new test case?
Failing that, does anyone know how I can separate things so that the page object requires capybara but not rspec and the spec file requires rspec but not capybara to prevent the collision? I don't know much about Ruby package management, but it seems like in order to run an rspec test case that uses a page object, it's all going to be mixed together. I'm not sure of a way to avoid requiring them both.
Failing that, does anyone know which versions of either of these two packages I could theoretically use to avoid the issue? I did experiments with rspec 3.2 and capybara 2.4, and neither seemed to work, and I gave up there, because I remembered that no matter how far back in time I go, the test cases I've already written were working fine, and it just seemed silly to try to solve it this way.
Require capybara after Rspec or always call page.all rather than just all
Not the best approach, but you can monkey patch the SitePrism::Page class. Here you replace 'all' by 'page.all':
SitePrism::Page.class_eval do
def find_all(*find_args)
page.all(*find_args)
end
end
Related
I've been working to diagnose a test failure that only occurs on my master branch. Following is the relevant code, in simplified form.
There's a service:
class Service
attr_reader :traces
def initialize
#traces = []
end
def do_work
#traces << Thread.current.backtrace
# ... actual work ...
end
end
And a class that makes use of the service:
class Widget
def get_cached_service
puts("Getting a cached service!")
puts("Do I already have one? #{!!#service}")
#service ||= make_service
end
def make_service
puts("Making a service!")
Service.new
end
end
I have a test (that lives in a file widget_spec.rb) that fails intermittently. This test creates an instance of Widget and calls get_cached_service. I see the Getting a cached service! message on the console, followed by Do I already have one? false, but I don't see the Making a service! message.
Furthermore, when I examine the traces attribute of the returned Service object, I find stack traces originating from other tests in my project (eg. foo_spec.rb, bar_spec.rb, etc).
In a few different places I find code like:
allow_any_instance_of(Widget)
.to receive(:make_service).and_return(whatever)
The other tests whose stack traces I find are likely stubbing make_service like this. But it appears that the stubbing is not being undone after those tests, as should always happen according to my understanding.
Is there any reason, other than a bug in rspec, that could cause a stub not to be reset at the end of a test?
The stub is almost certainly being cleared, but you’ve cached the fake instance in get_cached_service. Nothing clears the cached value in #service, and RSpec (rightfully) doesn’t know about it. As such, stubbing make_service is not enough if tests call get_cached_service. You have a few options:
Always stub get_cached_service instead of, or in addition to, make_service
Provide a way to clear the cached value which is called after each test.
Make the caching configurable in some way, or a wrapper around the actual implementation, such that the caching does not occur in test code.
I realise this is quite late to answer, but for posterity for anyone who reads this:
Use rspec bisect to figure out if there is a consistent test ordering that causes failure, then start ripping code out until you're left with only the bit that breaks.
I can't remember a case where RSpec is at fault - almost invariably, somewhere there is a class variable that isn't getting cleared, or someone is manually playing with a class with something like define_method. Occasionally it might be happening in a gem.
Make sure everything is cleared after every test in your spec_helper - clear the Rails cache, clear ActionMailer deliveries, return from Timecop freezes, etc.
Anything directly RSpec-related should clear itself in theory, because it's designed to integrate into RSpec, and is probably the least likely explanation in general.
I am having a very strange problem regarding method dispatch to the wrong class using Minitest.
I am helping maintain the dnsruby library (https://github.com/alexdalitz/dnsruby), and we have a particular test that intermittently fails (https://github.com/alexdalitz/dnsruby/blob/master/test/tc_resolv.rb#L56).
By setting a breakpoint in pry, I discovered that when it fails it is using the getname method on the internal Ruby Resolv class rather than the Dnsruby::Resolv class specified in the code. Both ::Resolv and Dnsruby::Resolv seem to point to the Ruby internal Resolv class:
[1] pry(#<TestResolv>)> ::Resolv.object_id
=> 70320518250220
[2] pry(#<TestResolv>)> Dnsruby::Resolv.object_id
=> 70320518250220
The error does not occur when the test is run on its own (ruby test/tc_resolv.rb). When combined with other tests (using ruby test/ts_online.rb and modifying its list of test files that are run), it often fails but sometimes not. The only factor I have found that seems to be correlated with the failure is the amount of test code that is run; the more tests are run, the more likely it is to fail.
When the test succeeds, ::Resolv is not even defined:
[1] pry(#<TestResolv>)> ::Resolv.object_id
NameError: uninitialized constant Resolv
I searched our codebase for require 'resolv' but did not find it. Perhaps another library we're using is requiring it. Even so, though, shouldn't the Dnsruby:: in Dnsruby::Resolv specify our customized Resolv class?
How can we fix this?
The problem was solved by removing an include Dnsruby in the top level that should not have been there. (Thanks to Ruby community member https://github.com/matt-glover for spotting this!)
More detail is at the Github issue at https://github.com/alexdalitz/dnsruby/issues/112, and the fix is in the pull request at https://github.com/alexdalitz/dnsruby/pull/121/files.
One thing that confuses me...I would have expected that include to result in the Dnsruby::Resolv class being pulled into the top level, but it seems that the reverse happened.
I know best practice is to make tests run independently.
I am using Rspec to run Selenium Webdriver tests. If I create and teardown users-groups-other models for every specific test case, the test run time goes up significantly (30 mins or so).
So I separated the spec files into create_..., delete_..., and would like to call these spec files inside one spec file. Is this possible and is it a good or bad way of setting up the tests?
Something like:
create_users_spec.rb
create_user_groups_spec.rb
create_dashboard_spec.rb
==> run some tests....
delete_dashboard_spec.rb
delete_user_groups_spec.rb
delete_users_spec.rb
run_all_spec.rb => run all of the spec files above, so you can see that
it needs to run in a certain sequence
You really want to be able to run tests independently. Having to run all tests, even "just" all tests in a given file, seems to me like an unacceptable delay in the development process. It also makes in hard to use tools like Guard that run tests for you based on a file watcher.
If there is really something that needs to run before, for example, every request spec, you can do something like:
# spec_helper.rb
RSpec.configure do |config|
config.before :all, type: :request do
# do stuff
end
end
But also look at what you are trying to do. If you are just setting up model/database data for the tests, then you want fixtures, or better yet, factories. Look into FactoryGirl.
Perhaps organizing your tests with context and using before(:each) and before(:all) judiciously would yield a performance boost. Simplifying the database, or using a library like FactoryGirl can also help.
I am new to automation and I have started using Watir Webdriver to automate a website. However, there are certain pieces of code which can be reused for multiple test cases. How can I group the reusable pieces of code into a method which I can call in every test case ? Could you please provide references or examples ?
You are interested in methods.
Here is an example:
def automation_snippet
#browser.text_field(:id => 'field_id').set 'foo'
end
Later you will probably be interested in classes, page objects, and modules.
This is basic ruby stuff. You will need to make sure that whatever method you want to use is in scope, too.
So, keywords to start with are methods and scope. If you are using Cucumber, then you can define methods in any step definition file and they will be available in all your other Cucumber tests.
I've been looking for some tutorials on creating ruby gems, but all seem to be overly complicated. Essentially, all I want is to make a gem that (trivial example alert) allows the user to place a method add_one(x) in their code, once the gem is installed.
A lot of tutorials explain how you can call the gem's methods with Classname.method(), but that's really not what I want. RyanB has a good example here, but again I can't get it to work - mainly because of the 3 lines of code at the bottom of uniquify.rb that read.
class ActiveRecord::Base
include Uniquify
end
In my case, my main module is called AddOne, so how would I go about doing this? Again, sorry for the trivial example, I'm just looking to get started so that my gems will allow the user to call plain ol' methods, without specifying Modules or Classes. In Ryan's example, he's able to simple call uniquify() in the code.
In the example given, he can call uniquify as a method because he is calling it inside a class which includes ActiveRecord::Base. (He calls uniquify inside the class Product < ActiveRecord::Base...end block) There isn't any real magic here.
To my understanding, what you desire is easily achievable, but isn't a good idea, 'best practice' considered. You can define anything you want in your gem. For instance, I could package a file called say_hello.rb inside a gem called salutations.
say_hello.rb
def say_hello
puts "Hello everybody"
end
By requiring the packaged gem salutations I would gain access to anything defined therein. However, it might lead to unexpected consequences. Do what you need to do, but just understand that it might not be the best idea. Hence, you build things into classes or modules to contain them, and to prevent conflicts with other code. Good luck.