I'm testing a generator, which outputs a lot of stuff to STDOUT. I want to suppress this, and there are lots of answers for that.
But I want to still be able to use pry. Right now, I have to disable the suppression if I need to pry into the test state.
I was using this code. It bypassed pry entirely:
def suppress_output(&block)
#original_stderr = $stderr
#original_stdout = $stdout
$stderr = $stdout = StringIO.new
yield(block)
$stderr = #original_stderr
$stdout = #original_stdout
#original_stderr = nil
#original_stdout = nil
end
I replaced it with this. It stops at the pry, but continues to suppress output, so you can't do anything:
def suppress_output(&block)
orig_stderr = $stderr.clone
orig_stdout = $stdout.clone
$stderr.reopen File.new("/dev/null", "w")
$stdout.reopen File.new("/dev/null", "w")
yield(block)
rescue Exception => e
$stdout.reopen orig_stdout
$stderr.reopen orig_stderr
raise e
ensure
$stdout.reopen orig_stdout
$stderr.reopen orig_stderr
end
Is there any way to have my cake and eat it too?
I'd still like an answer to this question if someone can think of a way. This isn't the only time I've had to suppress STDOUT in tests, and the scenarios haven't always been the same as this one.
However, it occurred to me today that in this case, the easier solution is to change the code, rather than the testing setup.
The generators are using Thor, which is very powerful, but has very opaque documentation past the basics and hasn't really been updated in years. When I dug around in the docs, I found there is some muting capability.
By calling add_runtime_options! in my main Cli < Thor class, I get a global --quiet option. This suppresses a lot of output, but not everything I need. #say still prints. #run itself is muted, but whatever shell commands I pass it to run are not.
Overwriting these methods takes care of the rest of my issues:
no_commands do
def quiet?
!!options[:quiet]
end
def run(command, config = {})
config[:capture] ||= quiet?
super(command, config)
end
def say(message = "", color = nil, force_new_line = (message.to_s !~ /( |\t)\Z/))
super(message, color, force_new_line) unless quiet?
end
end
I don't currently have a use-case where I would only want to suppress some things, so making it all-or-nothing works for now.
Now, I have to explicitly create the Cli instances in my tests with quiet: true, but I can run RSpec without unwanted output and still use pry.
I found this solution by Chris Hough to work in my case, adding the following configuration to spec/spec_helper:
RSpec.configure do |config|
config.before(:each) do
allow($stdout).to receive(:write)
end
end
This replaced an :all before block, which was setting the following (and reversing the assignment in an after block):
$stderr = File.open(File::NULL, "w")
$stdout = File.open(File::NULL, "w")
The fix still suppresses output while allowing Pry to function as expected.
Related
I'm attempting to test the output of my gem that is using GLI however I'm running into issues. In my test setup I'm redirecting $stdout and $stderr to StringIO instances however when I run my code I'm still getting a message printed to my screen rather than being redirected to my custom objects.
begin
old_stdout, old_stderr, $stdout, $stderr = $stdout, $stderr, StringIO.new, StringIO.new
#exit_code = described_class.run([command] + command_args)
#stdout = $stdout.string
#stderr = $stderr.string
ensure
$stdout = old_stdout
$stderror = old_stderr
end
I would expect that described_class.run([command] + command_args) which ends up calling GLI's exit_now! method would be redirected to $stderr however this does not seem to be the case.
Is there a way to "trace" where/how a string was printed to the screen?
It looks like you're using RSpec based on described_class. If you're just trying to test stdout/stderr, then there's a matcher for that.
it "does something" do
expect {
described_class.run([command] + command_args)
}.to output(/some string match/).to_stdout
end
The owner of the GLI project made some suggestions here about how to test GLI setup, perhaps that will help as well.
I have a ruby class like this:
require 'logger'
class T
def do_something
log = Logger.new(STDERR)
log.info("Here is an info message")
end
end
And a test script line this:
#!/usr/bin/env ruby
gem "minitest"
require 'minitest/autorun'
require_relative 't'
class TestMailProcessorClasses < Minitest::Test
def test_it
me = T.new
out, err = capture_io do
me.do_something
end
puts "Out is '#{out}'"
puts "err is '#{err}'"
end
end
When I run this test, both out and err are empty strings. I see the message printed on stderr (on the terminal). Is there a way to make Logger and capture_io to play nicely together?
I'm in a straight Ruby environment, not Ruby on Rails.
The magic is to use capture_subprocess_io
out, err = capture_subprocess_io do
do_stuff
end
MiniTest's #capture_io temporarily switches $stdout and $stderr for StringIO objects to capture output written to $stdout or $stderr. But Logger has its own reference to the original standard error stream, which it will write to happily. I think you can consider this a bug or at least a limitation of MiniTest's #capture_io.
In your case, you're creating the Logger inside the block to #capture_io with the argument STDERR. STDERR still points to the original standard error stream, which is why it doesn't work as expected.
Changing STDERR to $stderr (which at that points does point to a StringIO object) works around this problem, but only if the Logger is actually created in the #capture_io block, since outside that block it points to the original standard error stream.
class T
def do_something
log = Logger.new($stderr)
log.info("Here is an info message")
end
end
Documentation of capture_subprocess_io
Basically Leonard's example fleshed out and commented with working code and pointing to the docs.
Captures $stdout and $stderr into strings, using Tempfile to ensure that subprocess IO is captured as well.
out, err = capture_subprocess_io do
system "echo Some info" # echos to standard out
system "echo You did a bad thing 1>&2" # echos to standard error
end
assert_match %r%info%, out
assert_match %r%bad%, err
NOTE: This method is approximately 10x slower than #capture_io so only use it when you need to test the output of a subprocess.
See Documentation
This is an old question, but one way we do this is to mock out the logger with an expects. Something like
logger.expects(:info).with("Here is an info message")
This allows us to assert the code under test without changing how logger works out of the box.
As an example of capture_io, we have a logger implementation to allow us to pass in hashes and output them to json. When we test that implementation we use capture_io. This is possible because we initialize the logger implementation in our subject line with $stdout.
subject { CustomLogging.new(ActiveSupport::Logger.new($stdout)) }
in the test
it 'processes a string message' do
msg = "stuff"
out, err = capture_io do
subject.info(msg)
end
out.must_equal "#{msg}\n"
end
You need to provide a different StringIO object while initializing Logger.new to capture the output, rather than the usual: STDERR which actually points to the console.
I modified the above two files a bit and made into a single file so that you can copy and test easily:
require 'logger'
require 'minitest'
class T
def do_something(io = nil)
io ||= STDERR
log = Logger.new io
log.info("Here is an info message")
end
end
class TestT < Minitest::Test
def test_something
t = T.new
string_io = StringIO.new
t.do_something string_io
puts "Out: #{string_io.string}"
end
end
Minitest.autorun
Explanation:
Method do_something will function normally in all other code when used without the argument.
When a StringIO method is provided, it uses that instead of the typical STDERR thus enabling to capture output like into a file or in this case for testing.
I am testing the class which put on the console some messages (with puts, p warnings and etc.). I am just wondering if there is any ability to suppress this output during RSpec tests ?
I suppress puts output in my classes by redirecting $stout to a text file. That way, if I need to see the output for any reason, it is there but it doesn't muddy up my test results.
#spec_helper.rb
RSpec.configure do |config|
config.before(:all, &:silence_output)
config.after(:all, &:enable_output)
end
public
# Redirects stderr and stout to /dev/null.txt
def silence_output
# Store the original stderr and stdout in order to restore them later
#original_stderr = $stderr
#original_stdout = $stdout
# Redirect stderr and stdout
$stderr = File.new(File.join(File.dirname(__FILE__), 'dev', 'null.txt'), 'w')
$stdout = File.new(File.join(File.dirname(__FILE__), 'dev', 'null.txt'), 'w')
end
# Replace stderr and stdout so anything else is output correctly
def enable_output
$stderr = #original_stderr
$stdout = #original_stdout
#original_stderr = nil
#original_stdout = nil
end
EDIT:
In response to the comment by #MyronMarston, it probably would be smarter to just insert the methods directly into before and after as blocks.
#spec_helper.rb
RSpec.configure do |config|
original_stderr = $stderr
original_stdout = $stdout
config.before(:all) do
# Redirect stderr and stdout
$stderr = File.new(File.join(File.dirname(__FILE__), 'dev', 'null.txt'), 'w')
$stdout = File.new(File.join(File.dirname(__FILE__), 'dev', 'null.txt'), 'w')
end
config.after(:all) do
$stderr = original_stderr
$stdout = original_stdout
end
end
It looks a little cleaner and keeps methods off of main.
Also, note that if you are using Ruby 2.0, you can use __dir__ instead of File.dirname(__FILE__).
EDIT2
Also it should be mentioned, that you can forward to true os /dev/null by using File::NULL as it was introduced in Ruby v 1.9.3. (jruby 1.7)
Then the code snippet will look as following:
#spec_helper.rb
RSpec.configure do |config|
original_stderr = $stderr
original_stdout = $stdout
config.before(:all) do
# Redirect stderr and stdout
$stderr = File.open(File::NULL, "w")
$stdout = File.open(File::NULL, "w")
end
config.after(:all) do
$stderr = original_stderr
$stdout = original_stdout
end
end
Try stubbing methods that make the output in a before block, e.g.
before do
IO.any_instance.stub(:puts) # globally
YourClass.any_instance.stub(:puts) # or for just one class
end
This is explicit, so you won't miss anything you don't want to miss. If you don't care about any output and the method above doesn't work you can always stub the IO object itself:
before do
$stdout.stub(:write) # and/or $stderr if needed
end
An Rspec3.0 Version would be => in spec_helper.rb
RSpec.configure do |c|
c.before { allow($stdout).to receive(:puts) }
end
it will act as before(:each)
but :each is default, so no need to write it explicitly
Tested with rspec-core (~> 3.4.0)
In describe block you could do
# spec_helper.rb
def suppress_log_output
allow(STDOUT).to receive(:puts) # this disables puts
logger = double('Logger').as_null_object
allow(Logger).to receive(:new).and_return(logger)
end
# some_class_spec.rb
RSpec.describe SomeClass do
before do
suppress_log_output
end
end
This way you have the advantage of toggling log output for specific tests. Note, this will not suppress rspec warnings, or messages from rspec.
Another way to disable warnings coming from gems:
add config.warnings = false to spec_helper
If you wanted to suppress only certain logger methods, like error, info, or warn you could do
allow_any_instance_of(Logger).to receive(:warn).and_return(nil)
To disable warnings coming from the rspec gem
allow(RSpec::Support).to receive(:warning_notifier).and_return(nil)
but this is generally discouraged because it is meant as a way to let you know you are doing something smelly in your tests.
If you want to suppress output for a single test, there is a more concise way:
it "should do something with printing" do
silence_stream(STDOUT) do
foo.print.should be_true
end
end
You may want to change STDOUT to STDERR if your test prints an error.
Updated answer for Rails 5, in a one-off situation:
before do
RSpec::Mocks.with_temporary_scope do
allow(STDOUT).to receive(:puts)
end
end
You can make this into a method in spec_helper if you'll be doing this a lot.
After trying all of these examples, I ended up using this varation which does not silence or mute binding.pry
# frozen_string_literal: true
RSpec.configure do |config|
config.before(:each) do
allow($stdout).to receive(:puts)
allow($stdout).to receive(:write)
end
end
It can be useful to inject an IO object defaulting to STDOUT. This also makes it easier to assert on the output if you want to.
E.g.
def my_method(arg, io: STDOUT)
io.puts "hello"
arg.reverse
end
And then in your test:
# Suppress it.
my_method("hi", io: StringIO.new)
# Assert on it.
io = StringIO.new
my_method("hi", io: io)
output = io.tap(&:rewind).read
expect(output).to include("hello")
You could have the object itself supress based on the environment:
class Foo
def call
puts("blah blah")
# ...
end
private
def puts(msg)
return if ENV['APP_ENV'] == 'test'
super
end
end
I'm trying to get my head around test-driven design, specifically RSpec. But I'm having trouble with some of the examples from The RSpec Book.
In the book, we test for output on $STDOUT like this:
output = double('output')
game = Game.new
output.should_receive(:puts).with('Welcome to Codebreaker!')
game.start()
Well, that works after a fashion. But why on earth should I care if the Game object uses the puts() method? If I change it to print(), should it really break the test? And, more importantly, isn't this against the one of the principals of TDD - that I should be testing what the method does (the design) rather than how it does it (the implementation)?
Is there some way I can write a test that just tests what ends up on $STDOUT, without looking at what method puts it there?
Create a display class with the ability to write the status out.
You production code will make use of this display object so you are free to change how you write to STDOUT. There will be just one place for this logic while your tests rely on the abstraction.
For example:
output = stub('output')
game = Game.new(output)
output.should_receive(:display).with('Welcome to Codebreaker!')
game.start()
While your production code will have something such as
class Output
def display(message)
# puts or whatever internally used here. You only need to change this here.
end
end
I'd make this test pass by doing the following:
def start
#output.display('Welcome to Codebreaker!')
end
Here the production code doesn't care how the output is displayed. It could be any form of display now the abstraction is in place.
All of the above theory is language agnostic, and works a treat. You still mock out things you don't own such as third party code, but you are still testing you are performing the job at hand via your abstraction.
take a look at this post. Nick raised questions about the same example, and a very interesting conversation follows in the comments. Hope you find it helpful.
Capture $stdout and test against that instead of trying to mock the various methods that might output to stdout. After all, you want to test stdout and not some convoluted method for mimicking it.
expect { some_code }.to match_stdout( 'some string' )
Which uses a custom Matcher (rspec 2)
RSpec::Matchers.define :match_stdout do |check|
#capture = nil
match do |block|
begin
stdout_saved = $stdout
$stdout = StringIO.new
block.call
ensure
#capture = $stdout
$stdout = stdout_saved
end
#capture.string.match check
end
failure_message_for_should do
"expected to #{description}"
end
failure_message_for_should_not do
"expected not to #{description}"
end
description do
"match [#{check}] on stdout [#{#capture.string}]"
end
end
RSpec 3 has changed the Matcher API slightly.
failure_message_for_should is now failure_message
failure_message_for_should_not is now failure_message_when_negated
supports_block_expectations? has been added to make errors clearer for blocks.
See Charles' answer for the complete rspec3 solution.
The way I'd test it is with a StringIO object. It acts like a file, but doesn't touch the filesystem. Apologies for the Test::Unit syntax - feel free to edit to RSpec syntax.
require "stringio"
output_file = StringIO.new
game = Game.new(output_file)
game.start
output_text = output_file.string
expected_text = "Welcome to Codebreaker!"
failure_message = "Doesn't include welcome message"
assert output_text.include?(expected_text), failure_message
I came across this blog post which helped me solve this issue:
Mocking standard output in rspec.
He sets up before/after blocks, and I ended up doing them inside the actual rspec itself, for some reason I couldn't get it to work from my spec_helper.rb as recommended.
Hope it helps!
An updated version of Matt's answer for RSpec 3.0:
RSpec::Matchers.define :match_stdout do |check|
#capture = nil
match do |block|
begin
stdout_saved = $stdout
$stdout = StringIO.new
block.call
ensure
#capture = $stdout
$stdout = stdout_saved
end
#capture.string.match check
end
failure_message do
"expected to #{description}"
end
failure_message_when_negated do
"expected not to #{description}"
end
description do
"match [#{check}] on stdout [#{#capture.string}]"
end
def supports_block_expectations?
true
end
end
When you pass the :verbose flag to a FileUtils command, the command gets printed to STDOUT. Is there a way to capture the command so it can be logged or used elsewhere?
If you look at the source for FileUtils it uses the following method for doing its verbose output:
def fu_output_message(msg) #:nodoc:
#fileutils_output ||= $stderr
#fileutils_label ||= ''
#fileutils_output.puts #fileutils_label + msg
end
i.e. it is writing the messages to #fileutils_output and by default it is using $stderr. There doesn't seem to be a method to alter #fileutils_output but you could add one:
module FileUtils
def FileUtils.fileutils_output=(new_out)
#fileutils_output = new_out
end
end
Then if you wanted to capture the commands into a file you could do:
my_fu_log = open('fu_log.log', 'w')
FileUtils.fileutils_output = my_fu_log
# FileUtils operations with :verbose => true here
my_fu_log.close
FileUtils.fileutils_output = $stderr # restore writing to stderr if you want
or if you wanted to get them in a string you could do:
log = StringIO.new
FileUtils.fileutils_output = log
# FileUtils operations with :verbose => true here
# commands are in log.string
Also, there is a module FileUtils::Verbose which basically includes FileUtils (so has all the same methods) but defaults the options to :verbose => true so if you wanted to capture lots of commands you could use this instead of specifying the option each time. (you would need to add the fileutils_output= method to this module in the same way as above.)
Alternatives
As Joshua says in the comments below, an alternative is to reassign $stderr but as he says this does mean that everything written to stderr (not just by FileUtils) is redirected. If all the FileUtils operations are happening in one go without anything else in between then this might not be an issue. So something along the lines of:
orig_stderr = $stderr # keep reference to original stderr
$stderr = my_fu_log
# use FileUtils here
$stderr = orig_stderr # restore stderr
Finally, you could reopen FileUtils and override fu_output_message(msg) itself if you need more control.
To add to Mike's answer (since I can't comment), I created this wrapper def if you want to get the output as a string:
def fileutil_out
out = StringIO.new
FileUtils.fileutils_output = out
yield
return out.string
end
mylog.info fileutil_out { FileUtils.chmod_R(0664, 'file.txt', :verbose => isVerbose) }
I ended up not using it since I wanted to revert back to #fileutils_output ||= $stderr afterwards.