I know that in RSpec you can easily mark an example as incomplete by omitting the block:
it 'is not a complete example'
But is it possible to get RSpec to fail any example which does not contain any expectations?
IE. RSpec will happily pass this example:
it 'does not have any expectations but still passes' do
end
I want to emulate the behavior of PHPUnit which fails any test not containing any assertions.
There isn't a built-in way, but it's a feature I'd like to see added at some point.
In the meantime, it's not too hard to add yourself:
http://blog.sorah.jp/2012/12/17/rspec-warn-for-no-expectations
Related
What is the best way of testing a whole script? I want to test if a script returns (prints) a correct result. So far what I have is another script that more or less does just:
# test.rb
expected = 'test'
output = `script.rb --many params`
if output != expected
puts "Error"
end
In this example, the script I want to test is called script.rb.
Cheers.
Using test-unit you end up with code like this:
class TestScript < Test::Unit::TestCase
def test_output
assert_equal 'test', `script --options`
end
end
You'll probably want to use the open3 library when dealing with output from programs as that gives you more control over how that's handled. You could write your own wrapper for that as well, like assert_output_equal:
def assert_output_equal(output, command, message = nil, &block)
Open3.popen3(command) do |stdin, stdout, stderr, wait_thr|
assert_equal output, stdout.read, message, &block
end
end
You can add on to that to test for the status from your process, ensuring it succeeded, and that there wasn't any output on STDERR and such.
When it comes to testing things that produce a lot of textual output (e.g. JSON data, formatted lists) I find it's best to create files in a place like test/expect and compare the file to the output rather than defining the text in the source. This makes it easy to create the files in the first place:
old_script > test/expected/old_script.default
old_script --option > test/expected/old_script.with-option
It also makes your version control diffs easier to read when something needs to be adjusted in there.
Define Test Fixtures
What is the best way of testing a whole script? I want to test if a script returns (prints) a correct result.
You need a test fixture that includes a known input value and an expected result. In other words:
Given a known input value,
When you run the program
Then you will receive your expected answer as output.
Note that testing an entire program this way can tell you if you got the right results from a set of inputs, but it will generally not provide insight into where the problem is. However, many forms of acceptance testing using this "black box" approach to testing, so it's certainly a reasonable approach as long as you understand the limitations.
Possible Frameworks
The list of possible testing frameworks is not infinite, but if you don't want to write your own test harness from scratch you may want to try one of the more popular ones. Some possibilities to kick-start your own research include:
Bats
Cucumber
Aruba
There are of course plenty of others, but per your original question you will probably want to focus on ATDD or BDD testing tools, rather than unit testing, if you want to test the entire script rather than its component parts.
After looking at:
How do I stub/mock a call to the command line with rspec?
The answer provided is:
require "rubygems"
require "spec"
class Dummy
def command_line
system("ls")
end
end
describe Dummy do
it "command_line should call ls" do
d = Dummy.new
d.should_receive("system").with("ls")
d.command_line
end
end
My question is: How does that actually test anything?
By making a method to that says "call the ls command on the system", and then writing a test that says "my method should call the ls command on the system", how does that provide any benefit?
If the method were to change, I would have to change the test as well, but I'm not sure I see the added benefit.
The approach you are describing is known as the "mockist" or "London" school of unit testing. It's benefits include
That the act of constructing such a test creates an incentive to design units which are not excessively complex in terms of their dependencies or conditional logic
That such tests execute very quickly, which can be very important for large systems
That such tests can be reasonably built to provide maximal coverage for the unit under test
That such tests provide a "vise" of sorts around your code such that inadvertent changes will result in a failing test
Of course, such tests have their limitations, which is why they are often paired with system level tests which test whether a collection of units operating together achieve some higher order outcome.
I have a function which receives a complex argument (an HTML string). I want to check multiple conditions about this string, i.e.:
receiver.should_receive(:post_data).with(json_content).with(id_matching(5))
Multiple with arguments doesn't work, any alternatives? I'm happy to define custom matchers if it's possible to make a compound one in some way.
Obviously I could run the same test multiple times and test different things about the result, however this is an integration test which takes several seconds to run, so I don't want to make it even slower.
Thanks
EDIT:
At time of writing, the accepted answer (use a custom matcher with custom description), appears to be the best option. However it isn't perfect, ideally with would support a concept of 'this was an item of the expected type, but wasn't the one we expected', instead of a pure binary match.
Maybe you don't even need a custom matcher and the block form is sufficient for you.
receiver.should_receive(:post_data) do |*args|
json_content = args.first
json_content.should_not be_empty
json_content.should include "some string"
end
See RSpec Mocks documentation, section Arbitrary Handling
You need to provide a custom matcher, but you can readily define your error reporting so that you can give specifics about what failed and why. See https://github.com/dchelimsky/rspec/wiki/Custom-Matchers .
In particular, the custom matcher would be supplied as the argument to with, as mentioned in the last sentence of the first paragraph of the "Argument Matchers" section of https://github.com/rspec/rspec-mocks.
As for error reporting, there are no custom failure methods that apply to this use case, but the description method of the custom matcher is used generate the string shown as the "expected" value and, though not its purpose, can be defined to output anything you want regarding the failed match.
When running RSpec tests in Ruby on Rails 2.3 with ActiveRecord, the database gets rolled back to the state after a before :all block after each example (it block).
However, I want to spec the lifecycle of an object, which means going through a number of examples one by one, changing the state and testing postconditions. This is impossible with the rollback behaviour.
So to clarify:
describe MyModel
before :all { #thing = MyModel.create }
it "should be settable" do
lambda { #thing.a_number = 42 }.should_not raise_exception
end
it "should remember things" do
#thing.a_number.should == 42
# this fails because the database was rolled back ☹
end
end
Is there some way to persist changes made in examples?
I agree with normalocity, in this case it looks like you would be better off with a single spec containing two assertions.
There are cases in which it is helpful to turn off rollbacks, e.g. for higher level tests with Capybara and Selenium, in which case you can use the use_transactional_fixtures configuration option. You can put thi
RSpec.configure do |config|
config.use_transactional_fixtures = false
end
Well, that depends on what you're trying to do. If you're testing the life cycle (a series of things that happen over time), that's more the realm of integration tests, which you can build more in tools such as Cucumber, etc. Spec is more designed to do small tests of small bits of code.
It's technically possible for you to simply write a long spec test, with multiple .should statements, and so long as all of them pass, then you've effectively got the kind of test you're describing. However, that's not really, in my experience, what spec is designed to give you.
I guess what I'm saying is, don't try to prevent the rollback - that's not what it's there to do. Either use a tool more designed to do the kinds of tests you're looking to build, or write a longer test that has multiple .should statements.
I'm building a tool that automates a process then runs some tests on it's own results then goes to do some other stuff.
In trying to clean up my code I have created a separate file that just has the test cases class. Now before I can run these tests, I have to pass the class a couple of parameters/objects before they can be run. Now the problem is that I can't seem to find a way to pass a parameter/object to the test class.
Right now I am thinking to generate a Yaml file and read it in the test class but it feels "wrong" to use a temporary file for this. If anyone has a nicer solution that would be great!
**************Edit************
Example Code of what I am doing right now:
#!/usr/bin/ruby
require 'test/unit/ui/console/testrunner'
require 'yaml'
require 'TS_SampleTestSuite'
automatingSomething()
importantInfo = getImportantInfo()
File.open('filename.yml', 'w') do |f|
f.puts importantInfo.to_yaml
end
Test::Unit::UI::Console::TestRunner.run(TS_SampleTestSuite)
Now in the example above TS_SampleTestSuite needs importantInfo, so the first "test case" is a method that just reads in the information from the Yaml file filname.yml.
I hope that clears up some confusion.
Overall, it looks like you're not really using the unit tests in a very rubyish way, but I'll leave that aside for a minute.
Your basic problem is that you have some setup that needs to happen before the tests run. The normal way to do that is with a setup method within the test unit case itself.
class UserTest < TestUnit::TestCase
def setup
# do your important calculation
end
def test_success
#.. assert some things
end
end
I would give some thought to what code it is that you're actually testing here, and see if you can break it down and test it in a more granular way, with lots more tests.
First, I agree with Cameron, this code definitely does not adhere to the Ruby way, though I'll also sidestep that for now.
The fastest way to get up and running with this, especially if this data is pretty much immutable (that is to say, your tests won't be altering it in anyway), is to just assign the value to a constant. So instead of naming your variable importantInfo, you name it IMPORTANT_INFO. Then it will be available to you in your tests. It's definitely not a pretty solution, and I think it couuld even be considered a test smell that you need that sort of global setup, but it's there for you.
Alternatively, you could look at stubbing out the importantInfo, which I actually think would provide for much cleaner and more readable tests.