I have a Ruby program that fails at runtime, but works when I test it with RSpec. I know the cause of the bug and how to fix it (see below), but I can't figure out how to build a failing RSpec test which proves the existence of the bug.
Imagine the following Ruby:
foobar.rb
class Foobar
attr_reader :fruit
def initialize
#fruit = Set.new ["Apple", "Banana", "Kiwi"]
end
end
The above code uses a Set, but it fails to "require 'set'". This causes it to fail at runtime:
$ irb
> require './foobar.rb'
> f = Foobar.new
NameError: uninitialized constant Foobar::Set
Before fixing the oversight, I wanted to build a simple RSpec test that proves the bug. My test looks like this:
foobar_spec.rb
require 'rspec'
require './foobar.rb'
describe Foobar do
it "can be initialized" do
expect { Foobar.new }.to_not raise_error
end
end
Running the test, I was surprised to see that it passes:
$ rspec foobar_spec.rb
.
Finished in 0.00198 seconds
1 example, 0 failures
After a little digging, I learned that RSpec loads Set for itself. This has the consequence of making Set available to the code it tests, and in my case concealing a bug.
I had the idea of "unloading/unrequiring" Set in my test. The closest I came was this code:
Object.send(:remove_const, :Set)
That indeed causes the test to fail, but unfortunately it also prevents Set from being loaded again by a future 'require', meaning it continued to fail even after I added require 'set' inside foobar.rb.
Is there a better way to unload gems at runtime? If not, what can I do to make this test fail as it should?
require 'rspec'
describe 'foobar.rb' do
it "can instantiate Foobar" do
`ruby -e 'Foobar.new' -r./foobar.rb`
$?.exitstatus.should == 0
end
end
works for the one case you mentioned. That said, I wouldn't recommend this approach. To cover all the cases where a class is referenced, you'd need to run all your specs this way, since the class reference could appear anywhere in your code.
Related
I am trying to run a very basic test with Terminal and Sublime Text 3. My simple test runs, but fails (undefined local variable or method 'x')
My folder hierarchy looks like this:
spec_helper.rb looks like this:
require_relative '../test'
require 'yaml'
test_spec.rb is extremely basic
require 'spec_helper.rb'
describe "testing ruby play" do
it "finds if x is equal to 5" do
x.should eql 5
end
end
and my test.rb file has x = 5 That's it.
Will a variable only be recognizable if it's part of a class? And do I need to call a new class every time I run my test?
From the docs
require(name) → true or false
Loads the given name, returning true if successful and false if the feature is already
loaded.
[snip]
Any constants or globals within the loaded source file will be
available in the calling program’s global namespace. However, local
variables will not be propagated to the loading environment.
You could use a constant in your required file:
X = 5
...
X.should eql 5 # => passes
But you probably want to do something entirely different here. Perhaps you could expand on the question and explain what you are trying to accomplish.
I'm trying to write a custom tool that runs ruby unit tests with my customizations.
What I need it to do is to load a certain TestCase from given file(through require or whatever), and then run it after doing some calculations and initializations.
Problem is, the moment I require "test/unit" and a test case, it runs immediately.
What can I do with this?
Thanks.
Since you're running 1.9 and test/unit in 1.9 is merely a wrapper for MiniTest, the following approach should work:
implement your own custom Runner
set MiniTest's runner to your custom runner
Something like (shameless plug from EndOfLine Custom Test Runner, adjusted to Ruby 1.9):
fastfailrunner.rb:
require 'test/unit'
class FastFailRunner19 < MiniTest::Unit
def _run args = []
puts "fast fail runner"
end
end
~
example_test.rb:
require 'test/unit'
class ExampleTest < Test::Unit::TestCase
def test_assert_equal
assert_equal 1, 1
end
def test_lies
assert false
end
def test_exceptions
raise Exception, 'Beware the Jubjub bird, and shun the frumious Bandersnatch!'
end
def test_truth
assert true
end
end
run.rb:
require_relative 'fast_fail_runner'
require_relative 'example_test'
MiniTest::Unit.runner= FastFailRunner19.new
If you run this with
ruby run.rb
the custom FastFailRunner19 will be used, which does nothing.
What about reading file content as a regular text file and doing eval on its content after you initialize/calculate things you say? It may not be sufficient for your needs and may require manual setup and execution of testing framework.
Like that (I put heredoc instead of reading file). Basically content is just a string containing your test case code.
content = <<TEST_CASE
class YourTestCase
def hello
puts 'Hello from eval'
end
end
YourTestCase.new.hello
TEST_CASE
eval content
Note: Altough I'd rather not use eval if there is another way. One should be extra careful when evaling code from string manually in any language.
You could collect the test cases you want to deferred its executions and store them in an array. Afterwards you would create a block execution code. For instance:
test_files = ['test/unit/first_test.rb'] #=> Testcases you want to run
test_block = Proc.new {spec_files.each {|f|load f} } #=> block storing the actual execution of those tests.
Once you're ready to call those testcases you just do test_block.call.
To generalize a bit, when thinking about deferring or delaying code executions, closures are a very elegant and flexible alternative.
I'd like to test class and gem loading. Have a look at the following stupid test case:
require 'rubygems'
require 'shoulda'
class SimpleTest < Test::Unit::TestCase
context 'This test' do
should 'first load something' do
require 'bundler'
assert Object.const_defined? :Bundler
end
should 'second have it reset again' do
assert !Object.const_defined?(:Bundler)
end
teardown do
# This works, but is tedious and unclean
#Object.send :remove_const, :Bundler rescue nil
# Instead I want something like this ;)
magic_reset
end
end
end
How about creating a subclass of Test::Unit::TestCase which runs the test method in a forked process?
class ForkingTestCase < Test::Unit::TestCase
def run(...)
fork do
super.run(...)
# somehow communicate the result back to the parent process
# this is the hard part
end
end
end
If this can be implemented, it should then just be a matter of changing the base class of your test case.
AFAIK, you cannot unload a file that you have loaded. You need to start a separate Ruby process for every test. (Or a separate Ruby instance if you are running on a Ruby implementation which supports multiple instances in the same process.)
Try using Kernel#load with wrap set to true:
load(filename, wrap=false) → true
Loads and executes the Ruby program in the file filename. If the
filename does not resolve to an absolute path, the file is searched
for in the library directories listed in $:. If the optional wrap
parameter is true, the loaded script will be executed under an
anonymous module, protecting the calling program’s global namespace.
In no circumstance will any local variables in the loaded file be
propagated to the loading environment.
each time you want to do a test of bundler, load it into a new anonymous module, do your tests on the bundler class within that module, and then go onto your next test.
If your code refers to the Bundler constant, then you'd have to set and unset that constant, though.
I haven't tried it myself, but can't see why it wouldn't work.
Try keeping track of what constants were defined before your tests started, and what constants are defined after your test finished, and remove the constants that were defined during the test.
I guess this isn't so much telling you how to call magic_reset as how to implement magic_reset.
edit: this problem only happens sometimes
This only appears to happen when I run the test from within TextMate (even when I specify the ruby to run it from by hand with a shebang). If I run it from the terminal then everything is peachy…
Here's some code:
require 'test/unit'
require 'shoulda'
class TestingTest < Test::Unit::TestCase
context "My thing" do
should "always have this test fail, and give me this message" do
assert false
end
end
end
I'm expecting it to tell me something like:
1) Failure:
test: My thing should always have this test fail, and give me this message (TestingTest)
# etc
An assert message, if one was given
But I'm getting:
1) Failure:
test:8
Failed assertion, no message given.
So what am I missing? The example code above is as simple as I think I can make it and I can't see the problem!
try inheriting from ActiveSupport::TestCase instead of the Test::Unit::TestCase
With Test::Unit, I can run:
ruby path/to/test.rb --name=test_name_that_i_want_to_run
Thus far, I have not been able to figure out how to do this with test/spec specifications. I am wondering if the way that specifications are automatically named does not allow me to do something like this.
Take the following spec for example:
require 'rubygems'
require 'spec'
describe 'tests' do
it 'should be true' do
1.should == 1
end
it 'should be false' do
1.should_not == 2
end
end
You can execute a single spec by using the -e flag and providing the portion specified by the it block. e.g. ruby my_spec.rb -e 'should be false'
After contacting the gem maintainer, Christian Neukirchen, I found out how to do this, so I am documenting it here for future reference.
specrb path/to/test.rb --name ".*should behave this way.*"
I needed to use the specrb test runner, an extended version Test::Unit's test runner, rather than just the ruby command.
You can also do this with the ruby command:
ruby path/to/test.rb -n "/should behave this way/"