I'm some rspec tests for a command line ruby based application. I'm trying to build my test suite allow for testing of missing command line parameters. Specifically, I'd like to stub out what ARGV[0]..ARGV[N] would appear to the application. I've seen similar posts mention ENV.stub; however, I don't see how I can simulate "nameless" args and a given order.
Any help would be appreciated. Thanks.
I simply did this in my test case located in spec:
it "does something" do
arg = "/tmp"
exe = File.expand_path('../bin/site_checker', File.dirname(__FILE__))
stdin, stdout, stderr = Open3.popen3("#{exe} -Ilib #{arg}")
stdout.readlines.should be_empty
end
Related
I have a Ruby project with a UNIX executable file called parse located in a bin subfolder in my project root directory.
At the moment it's just this:
#!/usr/bin/env ruby
# frozen_string_literal: true
puts 'hello world'
The file can be executed on the command line when this command is run from the project root directory: bin/parse
It works fine, but I also want to write a passing Rspec test for it.
I have this spec file:
RSpec.describe "end-to-end application behaviour" do
subject { system('bin/parse') }
it 'prints the expected messsage to stdout' do
expect { subject }.to output(
'hello world'
).to_stdout
end
end
When I run it I get the test failure:
expected block to output "hello world" to stdout, but output nothing
This is the location of my spec file relative to my project root: spec/integration/parse_spec.rb
I tried placing require and require_relative statements in that spec file with the paths to the parse executable, in case that would help, but I just kept getting:
LoadError: cannot load such file
Does anyone know how I can write a test in that file that will pass and prove the parse executable behaviour works?
Don't Use the RSpec Output Matcher
RSpec has a built-in output matcher than can test both where output goes, as well as its contents. However, it's testing where your Ruby output goes, not whether some external application is using standard input or standard error. You're going to have to make some different assumptions about your code.
You can avoid driving yourself nuts by comparing strings rather than testing the underlying shell or your output streams. For example, consider:
RSpec.describe "parse utility output" do
it "prints the right string on standard output" do
expect(`echo hello world`).to start_with("hello world")
end
it "shows nothing on standard output when it prints to stderr" do
expect(`echo foo >&2 > /dev/null`).to be_empty
end
end
Just replace the echo statements with the correct invocation of parse for your system, perhaps by setting PATH directly in your shell, using a utility like direnv, or by modifying ENV["PATH"] in your spec or spec_helper.
As a rule of thumb, RSpec isn't really meant for testing command-line applications. If you want to do that, consider using the Aruba framework to exercise your command-line applications. It's best to use RSpec to test the results of methods or the output of commands, rather than trying to test basic functionality. Of course, your mileage may vary.
Use to_stdout_from_any_process instead of to_stdout:
expect { subject }.to output('hello world').to_stdout_from_any_process
I assume this is very newbie stuff but I'm learning Ruby by doing, and I'm developing a small CLI tool that receives a couple of parameters in order to do its stuff properly. This is my current workflow:
I want to test (using Minitest) all the possible flows:
Exits with 0 and help message is shown if ARGV.count != 2
Exits with 1 if first param is not correct
Exits with 1 if second param is not correct
Exits with 1 if both params are not correct
Exits with 0 and does stuff if all params are correct
Now, if I run tests the only thing I see is the help output as there is no parameter being passed.
So, a couple of questions:
How can I pass arguments to the main program in tests?
How can I test the output? (I'm using puts)
Thanks!
nice diagram!
you can either use helpers like aruba https://github.com/cucumber/aruba
or dig into ruby internals in order to bend it to your will!
# test.rb
pseudoIO = StringIO.new
$stdout = pseudoIO
puts "hi #{ARGV.join(', ')}"
ARGV.replace ["file1"]
puts "now its #{ARGV.join(', ')}"
abort "captured: #{pseudoIO.string}"
output should be
ruby text.rb "whutup"
# => captured: hi whutup
# => now its file1
I am trying to execute rspec from ruby, and get the status or number of failures from a method or something like that. Actually I am running something like this:
system("rspec 'myfilepath'")
but I only can get the string returned by the function. Is there any way to do this directly using objects?
I think the best way would be using RSpec's configuration and Formatter. This would not involve parsing the IO stream, also gives much richer result customisation programmatically.
RSpec 2:
require 'rspec'
config = RSpec.configuration
# optionally set the console output to colourful
# equivalent to set --color in .rspec file
config.color = true
# using the output to create a formatter
# documentation formatter is one of the default rspec formatter options
json_formatter = RSpec::Core::Formatters::JsonFormatter.new(config.output)
# set up the reporter with this formatter
reporter = RSpec::Core::Reporter.new(json_formatter)
config.instance_variable_set(:#reporter, reporter)
# run the test with rspec runner
# 'my_spec.rb' is the location of the spec file
RSpec::Core::Runner.run(['my_spec.rb'])
Now you can use the json_formatter object to get result and summary of a spec test.
# gets an array of examples executed in this test run
json_formatter.output_hash
An example of output_hash value can be found here:
RSpec 3
require 'rspec'
require 'rspec/core/formatters/json_formatter'
config = RSpec.configuration
formatter = RSpec::Core::Formatters::JsonFormatter.new(config.output_stream)
# create reporter with json formatter
reporter = RSpec::Core::Reporter.new(config)
config.instance_variable_set(:#reporter, reporter)
# internal hack
# api may not be stable, make sure lock down Rspec version
loader = config.send(:formatter_loader)
notifications = loader.send(:notifications_for, RSpec::Core::Formatters::JsonFormatter)
reporter.register_listener(formatter, *notifications)
RSpec::Core::Runner.run(['spec.rb'])
# here's your json hash
p formatter.output_hash
Other Resources
Detailed work through
Gist example
I suggest you to take a look into rspec source code to find out the answer. I think you can start with example_group_runner
Edit: Ok here is the way:
RSpec::Core::Runner::run(options, err, out)
Options - array of directories, err & out - streams. For example
RSpec::Core::Runner.run(['spec', 'another_specs'], $stderr, $stdout)
Your problem is that you're using the Kernel#system method to execute your command, which only returns true or false based on whether or not it can find the command and run it successfully. Instead you want to capture the output of the rspec command. Essentially you want to capture everything that rspec outputs to STDOUT. You can then iterate through the output to find and parse the line which will tell you how many examples were run and how many failures there were.
Something along the following lines:
require 'open3'
stdin, stdout, stderr = Open3.popen3('rspec spec/models/my_crazy_spec.rb')
total_examples = 0
total_failures = 0
stdout.readlines.each do |line|
if line =~ /(\d*) examples, (\d*) failures/
total_examples = $1
total_failures = $2
end
end
puts total_examples
puts total_failures
This should output the number of total examples and number of failures - adapt as needed.
This one prints to console and at the same time captures the message. The formatter.stop is just a stub function, I don't know what it is for normally, I had to include it to use DocumentationFormatter. Also the formatter output contains console coloring codes.
formatter = RSpec::Core::Formatters::DocumentationFormatter.new(StringIO.new)
def formatter.stop(arg1)
end
RSpec.configuration.reporter.register_listener(formatter, :message, :dump_summary, :dump_profile, :stop, :seed, :close, :start, :example_group_started)
RSpec::Core::Runner.run(['test.rb','-fdocumentation'])
puts formatter.output.string
Both examples are going to STDOUT, but cucumber only sees the first one. The second scenario fails with:
Then the stdout should contain "test" # aruba-0.4.11/lib/aruba/cucumber.rb:82
expected "" to include "test" (RSpec::Expectations::ExpectationNotMetError)
features/test.feature:13:in `Then the output should contain "test"'
The features:
Scenario: echo test
Given a blank slate
When I run `echo "test"`
The stdout should contain "test"
Scenario: puts test
Given a blank slate
When I start the program
The stdout should contain "test"
The step definitions:
When /^I start the program$/ do
TestModule::Main.new.start
end
The code:
module TestModule
class Main
def initialize
end
def start
$stdout.puts "test"
end
end
end
I'm not that familiar with Aruba, but a quick peek into it's source code suggests that the assertions it makes against STDOUT (or any output) only apply to processes that it started itself, and not all content that's been written to STDOUT. The code that you invoke yourself, in the second scenario, is outside of the control of Aruba, so it's output won't be tracked.
If you think about it, it couldn't really work any other way - if Aruba captured all STDOUT for assertions, then it would contain Cucumber's own test output as well...
It looks like you're trying to test your program in-process without using Aruba to invoke a separate Ruby process. If that's the case I'd suggest modifying the program to make it possible to pass in a STDOUT replacement e.g.
def initialize(output=$stdout)
Then when you start the code:
When /^I start the program$/ do
TestModule::Main.new(#output).start
end
And you can change your assertion:
Then the stdout should contain "(.+)" do |string|
#output.should include string
end
I have a command line (NON-RAILS) application written in pure Ruby that I'm driving out through Cucumber and RSpec. It follows the typical application hierarchy of lib, bin, spec, and feature directories.
Up until now, I've followed the traditional process of writing a failing Cucumber feature/scenario, dropping down to RSpec to drive out the supporting lib files, then getting the scenario to pass.
Unfortunately, this doesn't seem to be as straight forward when driving out the main application entry point in "bin/my_application.rb". The main issue for me is that I'm not describing a class in RSpec, it's a sequential Ruby script for managing the application's classes and initialization via command line parameters and options.
"bin/my_application.rb" is just a small shell-executed wrapper for parsing command line options and passing them onto my main application class as initializer options. I'd still like to test the behavior of the bin script (e.g. MyApp.should_receive(option_a).with(parameter)).
Any suggestions/thoughts/advice? Is this a normal test strategy for driving out command line Ruby script behavior?
Thanks in advance.
Not sure I fully comprehend what you're asking, but I'd say that if you want to use RSpec to test your parameter passing it should be easy enough to do. Say you have your wrapper script:
# my_application.rb
command = Command.new
command.foo = true if ARGV[0]
command.bar = true if ARGV[1]
command.baz = false if ARGV[2]
command.make_dollars(1000000)
Just mix it up and make it a class suitable for testing.
# command_runner.rb
class CommandRunner
def run(args, command = Command.new)
command.foo = true if args[0]
command.bar = true if args[1]
command.baz = false if args[2]
command.make_dollars(1000000)
end
end
# my_application.rb
CommandRunner.new.run(ARGV)
Now the only thing you don't have tested is your default parameter on the run command and the one line in the file my_application.rb
Hope that helps.
Brandon