I have a Ruby project with a UNIX executable file called parse located in a bin subfolder in my project root directory.
At the moment it's just this:
#!/usr/bin/env ruby
# frozen_string_literal: true
puts 'hello world'
The file can be executed on the command line when this command is run from the project root directory: bin/parse
It works fine, but I also want to write a passing Rspec test for it.
I have this spec file:
RSpec.describe "end-to-end application behaviour" do
subject { system('bin/parse') }
it 'prints the expected messsage to stdout' do
expect { subject }.to output(
'hello world'
).to_stdout
end
end
When I run it I get the test failure:
expected block to output "hello world" to stdout, but output nothing
This is the location of my spec file relative to my project root: spec/integration/parse_spec.rb
I tried placing require and require_relative statements in that spec file with the paths to the parse executable, in case that would help, but I just kept getting:
LoadError: cannot load such file
Does anyone know how I can write a test in that file that will pass and prove the parse executable behaviour works?
Don't Use the RSpec Output Matcher
RSpec has a built-in output matcher than can test both where output goes, as well as its contents. However, it's testing where your Ruby output goes, not whether some external application is using standard input or standard error. You're going to have to make some different assumptions about your code.
You can avoid driving yourself nuts by comparing strings rather than testing the underlying shell or your output streams. For example, consider:
RSpec.describe "parse utility output" do
it "prints the right string on standard output" do
expect(`echo hello world`).to start_with("hello world")
end
it "shows nothing on standard output when it prints to stderr" do
expect(`echo foo >&2 > /dev/null`).to be_empty
end
end
Just replace the echo statements with the correct invocation of parse for your system, perhaps by setting PATH directly in your shell, using a utility like direnv, or by modifying ENV["PATH"] in your spec or spec_helper.
As a rule of thumb, RSpec isn't really meant for testing command-line applications. If you want to do that, consider using the Aruba framework to exercise your command-line applications. It's best to use RSpec to test the results of methods or the output of commands, rather than trying to test basic functionality. Of course, your mileage may vary.
Use to_stdout_from_any_process instead of to_stdout:
expect { subject }.to output('hello world').to_stdout_from_any_process
Related
I've got a Thor based ruby CLI project to which I'm trying to add unit testing. It started as a pet project and is now rather large and important internally. Anyway, I have never really started unit testing, only grafted onto existing examples, and so I'm very much a noob when it comes to rspec.
I'm starting with a rather basic example, making sure that if I give it a file that doesn't exist, that it replies with an explanatory error and exits.
Here's the ways I have tried to compound them:
context 'with incorrect filename' do
it 'should fail cleanly' do
expect do
subject.format('bad_file_name')
end.to output(' ERROR Unable to load file bad_file_name, exiting...').to_stdout.and raise_error(SystemExit)
end
end
which works but doesn't capture the stdout properly output here.. also tried:
context 'with incorrect filename' do
it 'should fail cleanly' do
expect do
expect do
subject.format('fixtures/invalid.yaml')
end.to output(' ERROR Unable to load file bad_file_name, exiting...').to_stdout
end.to raise_error(SystemExit)
end
end
Which looks like it works, except (as this example shows) it's not actually testing the output because that output wouldn't match.
So what's the right way to check both an error that was raised and the output to stdout at the same time?
this works for me
expect { whatever }.to output('stderr message').to_stderr
.and output(Regexp.new('stdout message')).to_stdout
.and raise_error('Error message')
I'm some rspec tests for a command line ruby based application. I'm trying to build my test suite allow for testing of missing command line parameters. Specifically, I'd like to stub out what ARGV[0]..ARGV[N] would appear to the application. I've seen similar posts mention ENV.stub; however, I don't see how I can simulate "nameless" args and a given order.
Any help would be appreciated. Thanks.
I simply did this in my test case located in spec:
it "does something" do
arg = "/tmp"
exe = File.expand_path('../bin/site_checker', File.dirname(__FILE__))
stdin, stdout, stderr = Open3.popen3("#{exe} -Ilib #{arg}")
stdout.readlines.should be_empty
end
Both examples are going to STDOUT, but cucumber only sees the first one. The second scenario fails with:
Then the stdout should contain "test" # aruba-0.4.11/lib/aruba/cucumber.rb:82
expected "" to include "test" (RSpec::Expectations::ExpectationNotMetError)
features/test.feature:13:in `Then the output should contain "test"'
The features:
Scenario: echo test
Given a blank slate
When I run `echo "test"`
The stdout should contain "test"
Scenario: puts test
Given a blank slate
When I start the program
The stdout should contain "test"
The step definitions:
When /^I start the program$/ do
TestModule::Main.new.start
end
The code:
module TestModule
class Main
def initialize
end
def start
$stdout.puts "test"
end
end
end
I'm not that familiar with Aruba, but a quick peek into it's source code suggests that the assertions it makes against STDOUT (or any output) only apply to processes that it started itself, and not all content that's been written to STDOUT. The code that you invoke yourself, in the second scenario, is outside of the control of Aruba, so it's output won't be tracked.
If you think about it, it couldn't really work any other way - if Aruba captured all STDOUT for assertions, then it would contain Cucumber's own test output as well...
It looks like you're trying to test your program in-process without using Aruba to invoke a separate Ruby process. If that's the case I'd suggest modifying the program to make it possible to pass in a STDOUT replacement e.g.
def initialize(output=$stdout)
Then when you start the code:
When /^I start the program$/ do
TestModule::Main.new(#output).start
end
And you can change your assertion:
Then the stdout should contain "(.+)" do |string|
#output.should include string
end
Can I replace an executable (accessed via a system call from ruby) with an executable that expects certain input and supplies the expected output in a consistent amount of time? I'm mainly operating on Mac OSX 10.6 (Snow Leopard), but I also have access to Linux and Windows. I'm using MRI ruby 1.8.7.
Background: I'm looking at doing several DNA sequence alignments, one in each thread. When I try using BioRuby for this, either BioRuby or ruby's standard library's tempfile sometimes raise exceptions (which is better than failing silently!).
I set up a test that reproduces the problem, but only some of the time. I assume the main sources of variability between tests are the threading, the tempfile system, and the executable used for alignment (ClustalW). Since ClustalW probably isn't malfunctioning, but can be a source of variability, I'm thinking that eliminating it may aid reproducibility.
For those thinking select isn't broken - that's what I'm wondering too. However, according to the changelog, there was concern about tempfile's thread safety in August 2009. Also, I've checked on the BioRuby mailing list whether I'm calling the BioRuby code correctly, and that seems to be the case.
I really don't understand what the problem is or what exactly are you after, can't you just write something like
#!/bin/sh
#Test for input (syntax might be wrong, but you get the idea)
if [ $* ne "expected input" ]; then
echo "expected output for failure"
exit -1
fi
#have it work in a consistent amount of time
$CONSISTENT_AMOUNT_OF_TIME = 20
sleep $CONSISTENT_AMOUNT_OF_TIME
echo "expected output"
You can. In cases where I'm writing a functional test for program A, I may need to "mock" a program, B, that A runs via system. What I do then is to make program B's pathname configurable, with a default:
class ProgramA
def initialize(argv)
#args = ParseArgs(argv)
#config = Config.new(#args.config_path || default_config_path)
end
def run
command = [
program_b_path,
'--verbose',
'--do_something_wonderful',
].join(' ')
system(command)
...
end
def program_a_path
#config.fetch('program_b_path', default_program_b_path)
end
end
Program A takes a switch, "--config PATH", which can override the default config file path. The test sets up a configuration file in /tmp:
program_b_path: /home/wayne/project/tests/mock_program_b.rb
And passes to program A that configuration file:
program_a.rb --config /tmp/config.yaml
Now program A will run not the real program B, but the mock one.
Have you tried the Mocha gem? It's used a lot for testing, and you describe it perfectly. It "fakes" the method call of an object (which includes just about anything in ruby), and returns the result you want without actually running the method. Take this example file:
# test.rb
require 'rubygems'
require 'mocha'
self.stubs(:system).with('ls').returns('monkey')
puts system('ls')
Running this script outputs "monkey" because I stubbed out the system call. You can use this to bypass parts of an application you don't want test, to factor out irrelevant parts.
I am using rspec for my test in a ruby project, and I want to spec that my program should not output anything when the -q option is used. I tried:
Kernel.should_not_receive :puts
That did not result in a failed test when there was output to the console.
How do I verify the absents of text output?
puts uses $stdout internally. Due to the way it works, the easiest way to check is to simply use: $stdout.should_not_receive(:write)
Which checks nothing is written to stdout as expected.
Kernel.puts (as above) would only result in a failed test when it
is explictely called as such (e.g. Kernel.puts "Some text"), where
as most cases it's call in the scope of the current object.
The accepted answer above is incorrect. It "works" because it doesn't receive a :write message but it might have received a :puts message.
The correct line should read:
$stdout.should_not_receive(:puts)
Also you need to make sure you put the line before the code that will write to STDIO. For instance:
it "should print a copyright message" do
$stdout.should_receive(:puts).with(/copyright/i)
app = ApplicationController.new(%w[project_name])
end
it "should not print an error message" do
$stdout.should_not_receive(:puts).with(/error/i)
app = ApplicationController.new(%w[project_name])
end
That's an actual working RSpec from a project