How to test stdin for a CLI using rspec - ruby

I'm making a small Ruby program and can't figure out how to write RSpec specs that simulate multiple user command line inputs (the functionality itself works). I think this StackOverflow answer probably covers ground that is closest to where I am, but it's not quite what I need. I am using Thor for the command line interface, but I don't think this is an issue with anything in Thor.
The program can read in commands either from a file or the command line, and I've been able to successfully write tests to read in an execute them. Here's some code:
cli.rb
class CLI < Thor
# ...
method_option :filename, aliases: ['-f'],
desc: "name of the file containing instructions",
banner: 'FILE'
desc "execute commands", "takes actions as per commands"
def execute
thing = Thing.new
instruction_set do |instructions|
instructions.each do |instruction|
command, args = parse_instruction(instruction) # private helper method
if valid_command?(command, args) # private helper method
response = thing.send(command, *args)
puts format(response) if response
end
end
end
end
# ...
no_tasks do
def instruction_set
if options[:filename]
yield File.readlines(options[:filename]).map { |a| a.chomp }
else
puts usage
print "> "
while line = gets
break if line =~ /EXIT/i
yield [line]
print "> "
end
end
end
# ..
end
I've tested successfully for executing commands contained in a file with this code:
spec/cli_spec.rb
describe CLI do
let(:cli) { CLI.new }
subject { cli }
describe "executing instructions from a file" do
let(:default_file) { "instructions.txt" }
let(:output) { capture(:stdout) { cli.execute } }
context "containing valid test data" do
valid_test_data.each do |data|
expected_output = data[:output]
it "should parse the file contents and output a result" do
cli.stub(:options) { { filename: default_file } } # Thor options hash
File.stub(:readlines).with(default_file) do
StringIO.new(data[:input]).map { |a| a.strip.chomp }
end
output.should == expected_output
end
end
end
end
# ...
end
and the valid_test_data referred to above is in the following form:
support/utilities.rb
def valid_test_data
[
{
input: "C1 ARGS\r
C2\r
C3\r
C4",
output: "OUTPUT\n"
}
# ...
]
end
What I want to do now is exactly the same thing but instead of reading each command from the 'file' and executing it, I want to somehow simulate a user typing in to stdin. The code below is utterly wrong, but I hope it can convey the direction I want to go.
spec/cli_spec.rb
# ...
# !!This code is wrong and doesn't work and needs rewriting!!
describe "executing instructions from the command line" do
let(:output) { capture(:stdout) { cli.execute } }
context "with valid commands" do
valid_test_data.each do |data|
let(:expected_output) { data[:output] }
let(:commands) { StringIO.new(data[:input]).map { |a| a.strip } }
it "should process the commands and output the results" do
commands.each do |command|
cli.stub!(:gets) { command }
if command == :report
STDOUT.should_receive(:puts).with(expected_output)
else
STDOUT.should_receive(:puts).with("> ")
end
end
output.should include(expected_output)
end
end
end
end
I've tried using cli.stub(:puts) in various places, and generally rearranging this code around a lot, but can't seem to get any of my stubs to put data in stdin. I don't know if I can parse the set of inputs I expect from the command line in the same way as I do with a file of commands, or what code structure I should be using to solve this issue. If someone who has spec-ed up command-line apps could chime in, that would be great. Thanks.

Rather than stubbing the universe, I think a few bits of indirection would help you write a unit test for this code. The simplest thing you can do is to allow the IO object for output to be injected, and default it to STDOUT:
class CLI < Thor
def initialize(stdout=STDOUT)
#stdout = stdout
end
# ...
#stdout.puts # instead of raw puts
end
Then, in your tests, you can use a StringIO to examine what was printed:
let(:stdout) { StringIO.new }
let(:cli) { CLI.new(stdout) }
Another option is to use Aruba or something like it, and write full-on integration tests, where you actually run your program. This has other challenges as well (such as being nondestructive and making sure not to squash/use system or user files), but will be a better test of your app.
Aruba is Cucumber, but the assertions can use RSPec matchers. Or, you can use Aruba's Ruby API (which is undocumented, but public and works great) to do this without the hassle of Gherkin.
Either way, you will benefit from making your code a bit more test-friendly.

I ended up finding a solution that I think fairly closely mirrors the code for executing instructions from a file. I overcame the main hurdle by finally realizing that I could write cli.stub(:gets).and_return and pass it in the array of commands I wanted to execute (as parameters thanks to the splat * operator), and then pass it the "EXIT" command to finish. So simple, yet so elusive. Many thanks go to this StackOverflow question and answer for pushing me over the line.
Here is the code:
describe "executing instructions from the command line" do
let(:output) { capture(:stdout) { cli.execute } }
context "with valid commands" do
valid_test_data.each do |data|
let(:expected_output) { data[:output] }
let(:commands) { StringIO.new(data[:input]).map { |a| a.strip } }
it "should process the commands and output the results" do
cli.stub(:gets).and_return(*commands, "EXIT")
output.should include(expected_output)
end
end
end
# ...
end

Have you looked at Aruba? It lets you write Cucumber feature tests for command line programs. You can define the input to your CLI that way.
You can write you feature definitions with RSpec, so it's not completely new.

You can stub all Thor::Actions with Rspec's allow_any_instance_of
here is one example:
it "should import list" do
allow_any_instance_of(Thor::Actions).to receive(:yes?).and_return(true)
end

Related

How to test OptionParser with Rspec - RSpec options are stored in ARGV array during testing

I am learning ruby and trying to write a unit test with rspec for the following method:
def get()
options = {}
OptionParser.new do |opt|
opt.banner = 'Usage: validate-gitlab-ci [options]'
opt.on('-f', '--yaml YAML-PATH', 'Path to .gitlab-ci.yml') { |o| options[:yamlFile] = o }
opt.on('-l', '--base-url GitLab url', 'GitLab API url') { |o| options[:baseUrl] = o + API_PATH }
opt.on('-t', '--timeout[TIMEOUT]', Integer, 'Api timeout in seconds') { |o| options[:timeout] = o || 10 }
opt.on('-v', '--version', 'Program version') { |o| options[:version] = o }
end.parse!
validateUrl!(options[:baseUrl])
validateYamlFile!(options[:yamlFile])
#baseUrl = options[:baseUrl]
#pathToYamlFile = options[:yamlFile]
end
The code for my unit test so far is:
RSpec.describe Gitlab::Lint::Client::Args do
describe "#get" do
context "when arguments are valid" do
it "sets baseUrl and pathToYamlFile" do
io = StringIO.new
io.puts "glab-lint --base-url=https://example.com --yaml=valid.ym\n"
io.rewind
$stdin = io
args = Gitlab::Lint::Client::Args.new
args.get()
expect(args.baseUrl).to.eq("https://example.com")
end
end
end
end
I am trying to mock STDIN for OptionParser. However, upon executing the test the following error is displayed:
OptionParser::InvalidOption:
invalid option: --pattern
This is raised by the end.parse! line in the get() method
Has anyone managed to test OptionsParser with stdin mocked?
Update
I think what is happening is that some RSpec options, e.g. --pattern?? are being captured in STDIN and passed to script??? Or .... RSpec is consuming the stdin options??
Reading this post seems to suggest that the desired functionality is not possible with RSpec....if this is indeed true then I will migrate over to using alternative test frameworks in future for CLI projects that use ARGV. There is a workaround suggested here but that suggests using environment variables for capturing commmand line arguments. In this case that would require further refactoring of the software under test, purely to suit the capabilities of the RSpec test framework!!
If I add a puts statement to display the contents of ARGV in the test script it confirms this is the case, with this output:
--pattern
spec/**{,/*/**}/*_spec.rb
[--base-url=https://gitlab.com --yaml=valid.ym]
So.....as a complete newbie to RSpec.....my options are:
Update the signature of the get method to accept an args array:
def get(args)
options = {}
OptionParser.new do |opt|
...
end.parse!(args)
end
This delays the issue with testing the code that reads from ARGV further up the call hierarchy
Modify ARGV shifting the first two arguments out of the array and then after the test has completed restore ARGV to original state. Looks like something similar has already been tried here without success.
Some other configuration that I am not aware of as a newbie to RSpec
Investigate alternative options, e.g. minitest, that maybe do not modify the ARGV array??
Further information regarding options 3 and 4 appreciated....
You can use RSpec to mock STDIN. For example:
STDIN.should_receive(:read).and_return("glab-lint --base-url=https://example.com --yaml=valid.yml")
Alternatively, you can invoke your actual command line program using backticks or system, and assert on the response.

Writing a Rspec test for an error condition finished with exit

I am writing a command line interface to a Ruby gem and I have this method exit_error, which acts as an exit error point to all validations performed while processing.
def self.exit_error(code,possibilities=[])
puts #errormsgs[code].colorize(:light_red)
if not possibilities.empty? then
puts "It should be:"
possibilities.each{ |p| puts " #{p}".colorize(:light_green) }
end
exit code
end
where #errormsgs is a hash whose keys are the error codes and whose values are the corresponding error messages.
This way I may give users customized error messages writing validations like:
exit_error(101,#commands) if not valid_command? command
where:
#errormsgs[101] => "Invalid command."
#commands = [ :create, :remove, :list ]
and the user typing a wrong command would receive an error message like:
Invalid command.
It should be:
create
remove
list
At the same time, this way I may have bash scripts detecting exactly the error code who caused the exit condition, and this is very important to my gem.
Everything is working fine with this method and this strategy as a whole. But I must confess that I wrote all this without writing tests first. I know, I know... Shame on me!
Now that I am done with the gem, I want to improve my code coverage rate. Everything else was done by the book, writing tests first and code after tests. So, it would be great having tests for these error conditions too.
It happens that I really don't know how to write Rspec tests to this particular situation, when I use exit to interrupt processing. Any suggestions?
Update => This gem is part of a "programming environment" full of bash scripts. Some of these scripts need to know exactly the error condition which interrupted the execution of a command to act accordingly.
For example:
class MyClass
def self.exit_error(code,possibilities=[])
puts #errormsgs[code].colorize(:light_red)
if not possibilities.empty? then
puts "It should be:"
possibilities.each{ |p| puts " #{p}".colorize(:light_green) }
end
exit code
end
end
You could write its rspec to be something like this:
describe 'exit_error' do
let(:errormsgs) { {101: "Invalid command."} }
let(:commands) { [ :create, :remove, :list ] }
context 'exit with success'
before(:each) do
MyClass.errormsgs = errormsgs # example/assuming that you can #errormsgs of the object/class
allow(MyClass).to receive(:exit).with(:some_code).and_return(true)
end
it 'should print commands of failures'
expect(MyClass).to receive(:puts).with(errormsgs[101])
expect(MyClass).to receive(:puts).with("It should be:")
expect(MyClass).to receive(:puts).with(" create")
expect(MyClass).to receive(:puts).with(" remove")
expect(MyClass).to receive(:puts).with(" list")
MyClass.exit_error(101, commands)
end
end
context 'exit with failure'
before(:each) do
MyClass.errormsgs = {} # example/assuming that you can #errormsgs of the object/class
allow(MyClass).to receive(:exit).with(:some_code).and_return(false)
end
# follow the same approach as above for a failure
end
end
Of course this is an initial premise for your specs and might not just work if you copy and paste the code. You will have to do a bit of a reading and refactoring in order to get green signals from rspec.

Mocking popen3 block form ruby

I am developing some test cases in Ruby using rspec.
I am attempting to mock the popen3 function.
However, while still keeping the blocking form, I am unable to capture the expected output information:
Class MyClass
def execute_command
Open3.popen3(command) do |stdin, stdout, stderr, wait_thr|
output['wait_thr'] = wait_thr.value
while line = stderr.gets
output['stderr'] += line
end
end
return output
end
end
To mock out the function, I am doing the following:
it 'should do something'
response = []
response << 'stdin'
response << 'stdout'
response << 'test'
response << 'exit 0'
# expect
allow(Open3).to receive(:popen3).with(command).and_yield(response)
# when
output = myClassInstance.execute_script
#then
expect(output['wait_thr'].to_s).to include('exit 0')
Mocking out the function doesn't enter the "do" code and I'm left with an empty data structure.
I was wondering how I could properly do this?
Thanks!
To add some more context to Chris Reisor's answer, this is the approach that worked for me:
I have a piece of code that reads as shown here.
Open3.popen2e(*cmd) do |_, stdout_and_stderr, wait_thr|
while (line = stdout_and_stderr.gets)
puts line
end
raise NonZeroExitCode, "Exited with exit code #{wait_thr.value.exitcode}" unless wait_thr.value.success?
end
And my testing setup looks like shown below.
let(:wait_thr) { double }
let(:wait_thr_value) { double }
let(:stdout_and_stderr) { double }
before do
allow(wait_thr).to receive(:value).and_return(wait_thr_value)
allow(wait_thr_value).to receive(:exitcode).and_return(0)
allow(wait_thr_value).to receive(:success?).and_return(true)
allow(stdout_and_stderr).to receive(:gets).and_return('output', nil)
allow(Open3).to receive(:popen2e).and_yield(nil, stdout_and_stderr, wait_thr)
end
I think you needed to put "*response" instead of "response."
allow(Open3).to receive(:popen3).with(command).and_yield(*response)
That will send 4 string args to and_yield ("arity of 4"), rather than one arg which is an array.

How can I test rspec user input and output with Highline?

I'd like to test response to user input. That input is queried using Highline:
def get_name
return HighLine.new.ask("What is your name?")
end
I'd like to do something similar to this question and put it in my test:
STDOUT.should_receive(:puts).with("What is your name?")
STDIN.should_receive(:read).and_return("Inigo Montoya")
MyClass.new.get_name.should == "Inigo Montoya"
What is the correct way to do this with Highline?
The best way to find out how to test Highline is to see how the author tests his package.
class TestHighLine < Test::Unit::TestCase
def setup
#input = StringIO.new
#output = StringIO.new
#terminal = HighLine.new(#input, #output)..
end
..
def test_agree
#input << "y\nyes\nYES\nHell no!\nNo\n"
#input.rewind
assert_equal(true, #terminal.agree("Yes or no? "))
assert_equal(true, #terminal.agree("Yes or no? "))
assert_equal(true, #terminal.agree("Yes or no? "))
assert_equal(false, #terminal.agree("Yes or no? "))
....
#input.truncate(#input.rewind)
#input << "yellow"
#input.rewind
assert_equal(true, #terminal.agree("Yes or no? ", :getc))
end
def test_ask
name = "James Edward Gray II"
#input << name << "\n"
#input.rewind
assert_equal(name, #terminal.ask("What is your name? "))
....
assert_raise(EOFError) { #terminal.ask("Any input left? ") }
end
Etc., as shown in his code. You can find this information in the highline source paying close attention to the setup, which I have highlighted in the link.
Notice how he uses the STDIN IO pipe to act in the place of typing the keys on the keyboard.
What this indicates, really, is that you don't need to use highline to test that kind of thing. The setup in his tests are really key here. Along with his use of StringIO as an object.
Highline already has it's own tests to make sure that it outputs to STDOUT and reads from STDIN. There is no reason to write these types of tests. It's the same reason that you would not write ActiveRecord tests that make sure attributes can be saved and read from the database.
However...
it would be extremely useful if there were a framework for Highline that works in a similar fashion as Capybara for web forms...
something that actually drives the input from the UI and tests the logic of your command-line utility.
For example, the following kind of hypothetical test would be nice:
run 'my_utility.rb'
highline.should :show_menu
select :add
highline.should(:ask).with_prompt("name?")
enter "John Doe"
output.should == "created new user"
I have published HighLine::Test - it allows you to run your tests in one process and your application in another (in the same way as you would with browser-based tests with Selenium).
GitHub link: https://github.com/joeyates/highline-test
gem: 'highline-test'
I worked on this DSL to try to solve this problem:
https://github.com/bonzofenix/cancun
require 'spec_helper'
describe Foo do
include Cancun::Highline
before { init_highline_test }
describe '#hello' do
it 'says hello correctly' do
execute do
Foo.new.salute
end.and_type 'bonzo'
expect(output).to include('Hi bonzo')
end

How to share data between implementation and description of a spec?

I wonder if there's any good way to reuse data between implementation and description of a spec... More particularly, I'd like to be able do something like the following:
describe "#some_method" do
let(:arg1) { "Sample String 1" }
let(:arg2) { "Sample String 2" }
context "with '#{arg1}', its result" do
specify { some_method(arg1).should == 1 }
end
context "with '#{arg2}', its result" do
specify { some_method(arg2).should == 2 }
end
end
Of course, this code won't work - arg1 and arg2 are not accessible outside of spec bodies.
Is it possible to achieve the similar result without using global variables or external classes?
Update:
I'm interested in the output of the spec. Something like this:
#some_method
with 'Sample String 1' its result
should == 1
with 'Sample String 2' its result
should == 2
The answer is that you don't use dynamic descriptions. The RSpec way to do this would be
describe "#some_method" do
it "extracts the number correctly" do
some_method("Sample String 1").should == 1
some_method("Sample String 2").should == 2
end
end
It is no problem to hard-code test data in your specs. If you want more complete output, you can use a custom matcher
require 'rspec'
class Test
def some_method(str)
str[/[0-9]+/].to_i
end
end
RSpec::Matchers.define :return_value_for_argument do |result, arg|
match do |actual|
actual.call(arg) == result
end
description do
"return #{result.inspect} for argument #{arg.inspect}"
end
end
describe Test do
let(:test) { Test.new }
describe "#some_method" do
subject { test.method(:some_method) }
it { should return_value_for_argument 1, "str 1" }
end
end
When doing API testing, I find it incredibly useful to be able to see the path, params, and response of each test. I have used the very useful tips given by Jeff Nyman to store things in the example.metatadata[:thing_i_want_to_store_like_url] of each test and with a custom formatter, print it out.
So my tests output look something like this:
that jonathan does not know it exists
:path : /user/20
:params: {}
=> response: {"error"=>{"message"=>"error", "code"=>404}}
that jonathan cannot edit
:path : /user/20/update
:params: {:name=>"evil_name"}
=> response: {"error"=>{"message"=>"error", "code"=>404}}
It's not appropriate to cite specific arguments in your descriptions. Your descriptions should provide a human-readable description of the desired behavior, without reference to specific arguments in most cases.

Resources