I've got a Thor based ruby CLI project to which I'm trying to add unit testing. It started as a pet project and is now rather large and important internally. Anyway, I have never really started unit testing, only grafted onto existing examples, and so I'm very much a noob when it comes to rspec.
I'm starting with a rather basic example, making sure that if I give it a file that doesn't exist, that it replies with an explanatory error and exits.
Here's the ways I have tried to compound them:
context 'with incorrect filename' do
it 'should fail cleanly' do
expect do
subject.format('bad_file_name')
end.to output(' ERROR Unable to load file bad_file_name, exiting...').to_stdout.and raise_error(SystemExit)
end
end
which works but doesn't capture the stdout properly output here.. also tried:
context 'with incorrect filename' do
it 'should fail cleanly' do
expect do
expect do
subject.format('fixtures/invalid.yaml')
end.to output(' ERROR Unable to load file bad_file_name, exiting...').to_stdout
end.to raise_error(SystemExit)
end
end
Which looks like it works, except (as this example shows) it's not actually testing the output because that output wouldn't match.
So what's the right way to check both an error that was raised and the output to stdout at the same time?
this works for me
expect { whatever }.to output('stderr message').to_stderr
.and output(Regexp.new('stdout message')).to_stdout
.and raise_error('Error message')
Related
I have a Ruby project with a UNIX executable file called parse located in a bin subfolder in my project root directory.
At the moment it's just this:
#!/usr/bin/env ruby
# frozen_string_literal: true
puts 'hello world'
The file can be executed on the command line when this command is run from the project root directory: bin/parse
It works fine, but I also want to write a passing Rspec test for it.
I have this spec file:
RSpec.describe "end-to-end application behaviour" do
subject { system('bin/parse') }
it 'prints the expected messsage to stdout' do
expect { subject }.to output(
'hello world'
).to_stdout
end
end
When I run it I get the test failure:
expected block to output "hello world" to stdout, but output nothing
This is the location of my spec file relative to my project root: spec/integration/parse_spec.rb
I tried placing require and require_relative statements in that spec file with the paths to the parse executable, in case that would help, but I just kept getting:
LoadError: cannot load such file
Does anyone know how I can write a test in that file that will pass and prove the parse executable behaviour works?
Don't Use the RSpec Output Matcher
RSpec has a built-in output matcher than can test both where output goes, as well as its contents. However, it's testing where your Ruby output goes, not whether some external application is using standard input or standard error. You're going to have to make some different assumptions about your code.
You can avoid driving yourself nuts by comparing strings rather than testing the underlying shell or your output streams. For example, consider:
RSpec.describe "parse utility output" do
it "prints the right string on standard output" do
expect(`echo hello world`).to start_with("hello world")
end
it "shows nothing on standard output when it prints to stderr" do
expect(`echo foo >&2 > /dev/null`).to be_empty
end
end
Just replace the echo statements with the correct invocation of parse for your system, perhaps by setting PATH directly in your shell, using a utility like direnv, or by modifying ENV["PATH"] in your spec or spec_helper.
As a rule of thumb, RSpec isn't really meant for testing command-line applications. If you want to do that, consider using the Aruba framework to exercise your command-line applications. It's best to use RSpec to test the results of methods or the output of commands, rather than trying to test basic functionality. Of course, your mileage may vary.
Use to_stdout_from_any_process instead of to_stdout:
expect { subject }.to output('hello world').to_stdout_from_any_process
I'm not terribly familiar with Ruby testing and my searches haven't yielded an answer to my specific question.
Currently, I have an app that raises StandardError to exit on certain conditions. The drawback to this is that the exit code is always 1.
I want to use exit() to provide unique codes to the exit conditions.
Currently, the project spec tests the StandardError mechanism as follows:
it "should raise an exception if no unignored project coverage file files were found" do
fixtures_project.ignore_list = ["*fixturesTests*", "*fixtures*"]
expect {fixtures_project.coverage_files}.to raise_error(StandardError)
end
I want to do something like assert($?.exit == 102), but I still need the fixtures_project.coverage_files to fire before hand in order to get the exit code.
I've tried variations of assert_equal(102, fixtures_project.coverage_files, "Hello there") inside the it/end with no luck. I'm sure this is probably simple Ruby, but I haven't grokked this bit yet.
You were right in trying to use $? object. Just call $?.exitstatus to get your exit status.
it "should raise an exception if no unignored project coverage file files were found" do
fixtures_project.ignore_list = ["*fixturesTests*", "*fixtures*"]
expect {fixtures_project.coverage_files}.to raise_error(StandardError)
expect($?.exitstatus).to eq(102)
end
I want output line, which was failed during rspec comparasion inside example, but I don't know how to do it best.
For example I have test like this:
require 'rspec'
describe 'My behaviour' do
it 'should do something' do
test_string = 'test'
expect(test_string).to eq('failed_test')
end
after :each do |example|
puts example.metadata[:expect_line]
end
end
And I want to outputed line in after :each be
"expect(test_string).to eq('failed_test')"
I know, I have acces to example.metadata[:location] which return something like "./spec/test_spec.rb:4" and I can parse it and extract line, but is there already something like I need hided in whole example structure?
Update:
I just understand. that example.metadata[:location] return not failed line, but actually line in whitch it started, so it have no use for me :(
So question still exist - how to get failed line?
This information isn't hidden anywhere in the example structure that I'm aware of. RSpec's default output shows the line that failed:
Failures:
1) My Description should fail
Failure/Error: expect(1).to eq(2)
expected: 2
got: 1
If we look at how rspec itself gets this in rspec/core/formatters/exception_presenter.rb and rspec/core/formatters/snippet_extractor.rb, it appears that they go through the backtrace of the example exception to find the spec file and extract the line (similar to what you mentioned). If there was an easier way to pull that out of the example, I would think RSpec itself would use that :)
Let's say I have a rakefile like this:
file 'file1' => some_dependencies do
sh 'external tool I do not have control over, which sometimes fail to create the file'
???
end
task :default => 'file1' do
puts "everything's OK"
end
Now if I put nothing in place of ???, I get the OK message, even if the external tool fails to generate file. What is the proper way to informing rake, that 'file1' task has failed and it should abort (hopefully presenting a meaningful message - like which task did fail) - the only think I can think of now is raising an exception there, but that just doesn't seem right.
P.S The tool always returns 0 as exit code.
Use the raise or fail method as you would for any other Ruby script (fail is an alias for raise). This method takes a string or exception as an argument which is used as the error message displayed at termination of the script. This will also cause the script to return the value 1 to the calling shell. It is documented here and other places.
You can use abort("message") to gracefully fail rake task.
It will print message to stdout and exit with code 1.
Exit code 1 is a failure in Unix-like systems.
See Kernel#abort for details.
I am using rspec for my test in a ruby project, and I want to spec that my program should not output anything when the -q option is used. I tried:
Kernel.should_not_receive :puts
That did not result in a failed test when there was output to the console.
How do I verify the absents of text output?
puts uses $stdout internally. Due to the way it works, the easiest way to check is to simply use: $stdout.should_not_receive(:write)
Which checks nothing is written to stdout as expected.
Kernel.puts (as above) would only result in a failed test when it
is explictely called as such (e.g. Kernel.puts "Some text"), where
as most cases it's call in the scope of the current object.
The accepted answer above is incorrect. It "works" because it doesn't receive a :write message but it might have received a :puts message.
The correct line should read:
$stdout.should_not_receive(:puts)
Also you need to make sure you put the line before the code that will write to STDIO. For instance:
it "should print a copyright message" do
$stdout.should_receive(:puts).with(/copyright/i)
app = ApplicationController.new(%w[project_name])
end
it "should not print an error message" do
$stdout.should_not_receive(:puts).with(/error/i)
app = ApplicationController.new(%w[project_name])
end
That's an actual working RSpec from a project