What is the best way of testing a whole script? I want to test if a script returns (prints) a correct result. So far what I have is another script that more or less does just:
# test.rb
expected = 'test'
output = `script.rb --many params`
if output != expected
puts "Error"
end
In this example, the script I want to test is called script.rb.
Cheers.
Using test-unit you end up with code like this:
class TestScript < Test::Unit::TestCase
def test_output
assert_equal 'test', `script --options`
end
end
You'll probably want to use the open3 library when dealing with output from programs as that gives you more control over how that's handled. You could write your own wrapper for that as well, like assert_output_equal:
def assert_output_equal(output, command, message = nil, &block)
Open3.popen3(command) do |stdin, stdout, stderr, wait_thr|
assert_equal output, stdout.read, message, &block
end
end
You can add on to that to test for the status from your process, ensuring it succeeded, and that there wasn't any output on STDERR and such.
When it comes to testing things that produce a lot of textual output (e.g. JSON data, formatted lists) I find it's best to create files in a place like test/expect and compare the file to the output rather than defining the text in the source. This makes it easy to create the files in the first place:
old_script > test/expected/old_script.default
old_script --option > test/expected/old_script.with-option
It also makes your version control diffs easier to read when something needs to be adjusted in there.
Define Test Fixtures
What is the best way of testing a whole script? I want to test if a script returns (prints) a correct result.
You need a test fixture that includes a known input value and an expected result. In other words:
Given a known input value,
When you run the program
Then you will receive your expected answer as output.
Note that testing an entire program this way can tell you if you got the right results from a set of inputs, but it will generally not provide insight into where the problem is. However, many forms of acceptance testing using this "black box" approach to testing, so it's certainly a reasonable approach as long as you understand the limitations.
Possible Frameworks
The list of possible testing frameworks is not infinite, but if you don't want to write your own test harness from scratch you may want to try one of the more popular ones. Some possibilities to kick-start your own research include:
Bats
Cucumber
Aruba
There are of course plenty of others, but per your original question you will probably want to focus on ATDD or BDD testing tools, rather than unit testing, if you want to test the entire script rather than its component parts.
Related
I've developed a set of Ruby scripts. Each of them should be 'self-contained', so a user can run it on its own. But also I would like to use them to build other scripts, I mean for example to use its methods but also to run it as a whole, without doing ` script.rb`.
So far what I have is just a couple of scripts (separate files) where I have no classes, just a couple of methods. The processing of taking user input and running those methods is outside of any functions. I see that this model may be not right.
My question is, what should I do now to keep every script self contained but also to allow other scripts to use it? Should every script just contain a class with a main method that I would run object.main?
Or maybe my approach of writing a simple scripts, no classes is also good?
If I start a new script, should I always go the objective way?
When I write a one off script, I often wrap it in a class. You've pointed out some advantages of doing this including reuse and cleaner documentation.
I find that there are several levels of polish for scripts depending on how they are going to be used. If the script is run once and never used again, I may not wrap it in a class. If it's important (taking backups of production systems), it's probably worth putting it in full gem form and writing tests. Somewhere in the middle is the single purpose class. Generally this means you're taking the code that's not in a method and putting it in the class constructor.
This:
#!ruby
def amethod(i)
i+1
end
ARGF.each do |l|
if l.chomp.to_i > 0
puts amethod(l.chomp.to_i)
end
end
Becomes:
#!ruby
class OneAdder
def amethod(i)
i+1
end
def initialize
ARGF.each do |l|
if l.chomp.to_i > 0
puts amethod(l.chomp.to_i)
end
end
end
end
OneAdder.new
I'm trying to create a Ruby command line interface with the following options:
ruby cli.rb -g # Prints a list of people sorted by gender
ruby cli.rb -n # Prints a list of people sorted by last name
ruby cli.rb -b # Prints a list of people sorted by birth date
ruby cli.rb -a LastName FirstName DateOfBirth # Add a new person to the list of people
I have already created a Directory class that stores a list of people and also has methods to sort people and add people. The next step is to build a CLI around this class.
How can I create this in a way that can be tested with RSpec? My initial attempt involved a looped gets.chomp instead of running a Ruby file with flags, but this loop does not play well with RSpec. The examples I've found online just use Ruby's OptionParser in a simple script and not inside an actual class. Also, how would I handle creating ARGV parameters in RSpec? I'm just looking for tips on the general structure of this CLI class so that I can write tests.
I do not want to use any gems for this.
I'm not sure what the context of your problem is, so I'm just going to give you a little brain dump, the merit of which is yours to determine:
A cold hard fact: In the wild, I've seen few people write unit tests for command line arguments. The reason for this probably has to do with the fact that a well-written command line tool is a lot like a very skinny router (in the MVC sense). It looks at the arguments, and routes them to the proper function (eg, "-g" routes to something like MyObject.print_by_gender). And if you use a battle-tested options parser (like OptionParser) on the incoming side and a well-tested piece of code that the router calls (eg, MyObject.print_by_gender, assuming that's the name of a function, and that you've tested it well), then there's very little to test in the command line file itself.
That context in place, let's consider your problem. You want to test an object that interacts with the external world in 2 ways. Namely, stdin and stdout. This means that you could create an integration-style test, and actually pass thing in on stdin and test the output the command creates on stdout - as you mentioned - or you can mock stdin and stdout. In my experience, mocking external things like stdin and stdout are a better approach. That said, I'll recommend these two SO questions/answers to explain how to mock STDIN (this and this).
I'm writing a command-line application in Ruby. I'm rather familiar with Rspec 2, as it's used in some Rails applications I'm working on. I'm writing a command-line application and I'm attempting to use Rspec 2 for BDD.
How can I test the user interface layer of my application? I need to give the application interactive input, or check for certain output printed using puts. Also, is there any way to suppress the STDOUT output? When I run autotest, the output of my application gets printed between the status outputs of rspec, making it extremely hard to read.
Here's an example that should help you. I use it in one of my apps that I test with Minitest, but it should be easy to translate to Rspec.
def setup
$stdout = StringIO.new
...
end
This gets rid of the app's output in between the test results, since stdout will be written to a StringIO object. The same way you can also test if something specific got output, since you can check the StringIO object against regular expressions.
I'm used to making calls such as:
new_count.should eql(10)
on variables, but how can I do something similar with a class method such as File.directory?(my_path)?
Every combination of File.should be_directory(my_path) that I've tried leads to a method missing, as Ruby tries to find "be_directory" on my current object, rather than matching it against File.
I know I can turn it around and write
File.directory?(my_path).should == true
but that gives a really poor message when it fails.
Any ideas?
Hmm, maybe I have an idea.
File is part of Ruby proper, so it may have elements written in C. Some of Ruby's meta-programming tools break down when dealing with classes imported from C, that could explain Rspec's failure to make .should behave as expected.
If that's true, there is no real solution here. I'd suggest using the MockFS library:
http://mockfs.rubyforge.org/
This downside to MockFS is using it everywhere you'd normally use File, Dir and FileUtils:
require 'mockfs'
def move_log
MockFS.file_utils.mv( '/var/log/httpd/access_log', '/home/francis/logs/' )
end
The upside, especially if your code is file-intensive, is the ability to spec really complex scenarios out, and have them run without actually touching the slow filesystem. Everything happens in memory. Faster, more complete specs.
Hope this helps, Good luck!
I'm not sure why be_directory wouldn't work for you. What version of rspec are you using? You can also use rspec's predicate_matchers method, when a predicate exists, but it doesn't read nicely as be_predicate.
Here's what I tried:
describe File, "looking for a directory" do
it "should be directory" do
File.should be_directory("foo")
end
predicate_matchers[:find_the_directory_named] = :directory?
it "should find directory" do
File.should find_the_directory_named("foo")
end
end
And that gave me the following output (run with spec -fs spec.rb):
File looking for a directory
- should be directory
- should find directory
Finished in 0.004895 seconds
2 examples, 0 failures
I'm building a tool that automates a process then runs some tests on it's own results then goes to do some other stuff.
In trying to clean up my code I have created a separate file that just has the test cases class. Now before I can run these tests, I have to pass the class a couple of parameters/objects before they can be run. Now the problem is that I can't seem to find a way to pass a parameter/object to the test class.
Right now I am thinking to generate a Yaml file and read it in the test class but it feels "wrong" to use a temporary file for this. If anyone has a nicer solution that would be great!
**************Edit************
Example Code of what I am doing right now:
#!/usr/bin/ruby
require 'test/unit/ui/console/testrunner'
require 'yaml'
require 'TS_SampleTestSuite'
automatingSomething()
importantInfo = getImportantInfo()
File.open('filename.yml', 'w') do |f|
f.puts importantInfo.to_yaml
end
Test::Unit::UI::Console::TestRunner.run(TS_SampleTestSuite)
Now in the example above TS_SampleTestSuite needs importantInfo, so the first "test case" is a method that just reads in the information from the Yaml file filname.yml.
I hope that clears up some confusion.
Overall, it looks like you're not really using the unit tests in a very rubyish way, but I'll leave that aside for a minute.
Your basic problem is that you have some setup that needs to happen before the tests run. The normal way to do that is with a setup method within the test unit case itself.
class UserTest < TestUnit::TestCase
def setup
# do your important calculation
end
def test_success
#.. assert some things
end
end
I would give some thought to what code it is that you're actually testing here, and see if you can break it down and test it in a more granular way, with lots more tests.
First, I agree with Cameron, this code definitely does not adhere to the Ruby way, though I'll also sidestep that for now.
The fastest way to get up and running with this, especially if this data is pretty much immutable (that is to say, your tests won't be altering it in anyway), is to just assign the value to a constant. So instead of naming your variable importantInfo, you name it IMPORTANT_INFO. Then it will be available to you in your tests. It's definitely not a pretty solution, and I think it couuld even be considered a test smell that you need that sort of global setup, but it's there for you.
Alternatively, you could look at stubbing out the importantInfo, which I actually think would provide for much cleaner and more readable tests.