I wonder how to capture the output of $stdout in a threaded environment in ruby.
A few details. I use the capture for logging purpose and I have sidekiq to process jobs in the background so the threads. Ive coded:
#previous_stdout, $stdout = $stdout, StringIO.new
self.stdout = $stdout.string
It throws me (for certain threads):
WARN: undefined method `string' for #<IO:<STDOUT>>
If you are logging things yourself, keep a StringIO log per job and write to it:
#log = StringIO.new
#log.write 'debug message'
Sidekiq also offers logging options:
https://github.com/mperham/sidekiq/wiki/Logging
Otherwise you could try something real hacky like overwriting the printing methods on Kernel and synchronizing it so you never accidentally write to the wrong $stdout. Something like:
require 'stringio'
module Kernel
##io_semaphore = Mutex.new
[ :printf, :p, :print, :puts ].each do |io_write|
hidden_io_write = "__#{io_write}__"
alias_method hidden_io_write, io_write
define_method(io_write) do |*args|
##io_semaphore.synchronize do
$stdout = Thread.current[:log] || STDOUT
self.__send__(hidden_io_write, *args)
$stdout = STDOUT
end
end
end
end
threads = 3.times.map do
Thread.new do
Thread.current[:log] = log = StringIO.new
sleep(rand)
puts "testing..."
log.string
end
end
logs = threads.map(&:value)
p logs
# => ["testing...\n", "testing...\n", "testing...\n"]
Related
I would like to capture stderr into a variable in memory, without using a file on the filesystem. (This is because if the process is kill -9'ed, the file, even if it is a Tempfile, will not be deleted.)
There is a solution for this at How do I temporarily redirect stderr in Ruby?, but its strategy is to assign a StringIO to $stderr. This will not work if the value of $stderr was copied prior to the reassignment, nor will it work if STDERR is used. As evidence:
#!/usr/bin/env ruby
stderr_sav = $stderr
$stderr = File.open(File::NULL, 'w')
$stderr.puts 'Using $stderr'
stderr_sav.puts 'Using stderr_sav'
STDERR.puts 'Using STDERR'
# Outputs:
# Using stderr_sav
# Using STDERR
In contrast, using reopen works:
#!/usr/bin/env ruby
stderr_sav = $stderr
$stderr.reopen(File.new(File::NULL, 'w'))
$stderr.puts 'Using $stderr'
stderr_sav.puts 'Using stderr_sav'
STDERR.puts 'Using STDERR'
# Outputs:
# [nothing]
Unfortunately, passing a StringIO to reopen does not work:
`reopen': no implicit conversion of StringIO into String (TypeError)
Is there any way to accomplish the capturing of $stderr without using a file?
How about a proxy which temporary delegate IO#puts to StringIO#puts
stderr_sav = $stderr
# delegate
$stdclone = $stderr.clone
$temp = StringIO.new
[:puts].each do |m|
if $temp.respond_to?(m)
$stderr.define_singleton_method "#{m}" do |*args, &block|
$temp.send(m, *args, &block)
end
end
end
# test
$stderr.puts 'secret1' # nothing
STDERR.puts 'secret2' # nothing
stderr_sav.puts 'secret3' # nothing
new_strerr = IO.new(1)
new_strerr.puts "not secret" # not secret
$stdclone.puts $temp.string # secret 1 -> 3
# reset
$stderr = $stdclone
$stderr.puts "secret4" # secret4
I'm testing a generator, which outputs a lot of stuff to STDOUT. I want to suppress this, and there are lots of answers for that.
But I want to still be able to use pry. Right now, I have to disable the suppression if I need to pry into the test state.
I was using this code. It bypassed pry entirely:
def suppress_output(&block)
#original_stderr = $stderr
#original_stdout = $stdout
$stderr = $stdout = StringIO.new
yield(block)
$stderr = #original_stderr
$stdout = #original_stdout
#original_stderr = nil
#original_stdout = nil
end
I replaced it with this. It stops at the pry, but continues to suppress output, so you can't do anything:
def suppress_output(&block)
orig_stderr = $stderr.clone
orig_stdout = $stdout.clone
$stderr.reopen File.new("/dev/null", "w")
$stdout.reopen File.new("/dev/null", "w")
yield(block)
rescue Exception => e
$stdout.reopen orig_stdout
$stderr.reopen orig_stderr
raise e
ensure
$stdout.reopen orig_stdout
$stderr.reopen orig_stderr
end
Is there any way to have my cake and eat it too?
I'd still like an answer to this question if someone can think of a way. This isn't the only time I've had to suppress STDOUT in tests, and the scenarios haven't always been the same as this one.
However, it occurred to me today that in this case, the easier solution is to change the code, rather than the testing setup.
The generators are using Thor, which is very powerful, but has very opaque documentation past the basics and hasn't really been updated in years. When I dug around in the docs, I found there is some muting capability.
By calling add_runtime_options! in my main Cli < Thor class, I get a global --quiet option. This suppresses a lot of output, but not everything I need. #say still prints. #run itself is muted, but whatever shell commands I pass it to run are not.
Overwriting these methods takes care of the rest of my issues:
no_commands do
def quiet?
!!options[:quiet]
end
def run(command, config = {})
config[:capture] ||= quiet?
super(command, config)
end
def say(message = "", color = nil, force_new_line = (message.to_s !~ /( |\t)\Z/))
super(message, color, force_new_line) unless quiet?
end
end
I don't currently have a use-case where I would only want to suppress some things, so making it all-or-nothing works for now.
Now, I have to explicitly create the Cli instances in my tests with quiet: true, but I can run RSpec without unwanted output and still use pry.
I found this solution by Chris Hough to work in my case, adding the following configuration to spec/spec_helper:
RSpec.configure do |config|
config.before(:each) do
allow($stdout).to receive(:write)
end
end
This replaced an :all before block, which was setting the following (and reversing the assignment in an after block):
$stderr = File.open(File::NULL, "w")
$stdout = File.open(File::NULL, "w")
The fix still suppresses output while allowing Pry to function as expected.
I use one rb file with rufus/scheduler on Windows. The script is executed on a comupter start up and it runs in a cmd window.
How can I log everything that ruby outputs to the screen to a file? I still want to be able to see the output on the screen. So I want the logging on top of current behaviour.
Windows 7 64 bit
ruby 1.9.3p194 (2012-04-20) [i386-mingw32]
If you just want the script to send output to the file instead of the console use IO#reopen to redirect stdout and stderr.
def redirect_console(filename)
$stdout.reopen(filename,'w')
$stderr.reopen(filename,'w')
end
redirect_console('/my/console/output/file')
If you need to direct to one or more output streams, use a proxy object and method_missing to send to them
class TeeIO
def initialize(*streams)
raise ArgumentError, "Can only tee to IO objects" unless streams.all? { |e| e.is_a? IO }
#streams = streams
end
def method_missing(meth, *args)
# HACK only returns result of first stream
#streams.map {|io| io.send(meth, *args) }.first
end
def respond_to_missing?(meth, include_all)
#streams.all? {|io| io.respond_to?(meth, include_all) }
end
end
def tee_console(filename)
tee_to = File.open(filename, 'w')
tee_to.sync = true # flush after each write
$stdout = TeeIO.new($stdout, tee_to)
$stderr = TeeIO.new($stderr, tee_to)
end
tee_console('/my/console/output/file')
I am testing the class which put on the console some messages (with puts, p warnings and etc.). I am just wondering if there is any ability to suppress this output during RSpec tests ?
I suppress puts output in my classes by redirecting $stout to a text file. That way, if I need to see the output for any reason, it is there but it doesn't muddy up my test results.
#spec_helper.rb
RSpec.configure do |config|
config.before(:all, &:silence_output)
config.after(:all, &:enable_output)
end
public
# Redirects stderr and stout to /dev/null.txt
def silence_output
# Store the original stderr and stdout in order to restore them later
#original_stderr = $stderr
#original_stdout = $stdout
# Redirect stderr and stdout
$stderr = File.new(File.join(File.dirname(__FILE__), 'dev', 'null.txt'), 'w')
$stdout = File.new(File.join(File.dirname(__FILE__), 'dev', 'null.txt'), 'w')
end
# Replace stderr and stdout so anything else is output correctly
def enable_output
$stderr = #original_stderr
$stdout = #original_stdout
#original_stderr = nil
#original_stdout = nil
end
EDIT:
In response to the comment by #MyronMarston, it probably would be smarter to just insert the methods directly into before and after as blocks.
#spec_helper.rb
RSpec.configure do |config|
original_stderr = $stderr
original_stdout = $stdout
config.before(:all) do
# Redirect stderr and stdout
$stderr = File.new(File.join(File.dirname(__FILE__), 'dev', 'null.txt'), 'w')
$stdout = File.new(File.join(File.dirname(__FILE__), 'dev', 'null.txt'), 'w')
end
config.after(:all) do
$stderr = original_stderr
$stdout = original_stdout
end
end
It looks a little cleaner and keeps methods off of main.
Also, note that if you are using Ruby 2.0, you can use __dir__ instead of File.dirname(__FILE__).
EDIT2
Also it should be mentioned, that you can forward to true os /dev/null by using File::NULL as it was introduced in Ruby v 1.9.3. (jruby 1.7)
Then the code snippet will look as following:
#spec_helper.rb
RSpec.configure do |config|
original_stderr = $stderr
original_stdout = $stdout
config.before(:all) do
# Redirect stderr and stdout
$stderr = File.open(File::NULL, "w")
$stdout = File.open(File::NULL, "w")
end
config.after(:all) do
$stderr = original_stderr
$stdout = original_stdout
end
end
Try stubbing methods that make the output in a before block, e.g.
before do
IO.any_instance.stub(:puts) # globally
YourClass.any_instance.stub(:puts) # or for just one class
end
This is explicit, so you won't miss anything you don't want to miss. If you don't care about any output and the method above doesn't work you can always stub the IO object itself:
before do
$stdout.stub(:write) # and/or $stderr if needed
end
An Rspec3.0 Version would be => in spec_helper.rb
RSpec.configure do |c|
c.before { allow($stdout).to receive(:puts) }
end
it will act as before(:each)
but :each is default, so no need to write it explicitly
Tested with rspec-core (~> 3.4.0)
In describe block you could do
# spec_helper.rb
def suppress_log_output
allow(STDOUT).to receive(:puts) # this disables puts
logger = double('Logger').as_null_object
allow(Logger).to receive(:new).and_return(logger)
end
# some_class_spec.rb
RSpec.describe SomeClass do
before do
suppress_log_output
end
end
This way you have the advantage of toggling log output for specific tests. Note, this will not suppress rspec warnings, or messages from rspec.
Another way to disable warnings coming from gems:
add config.warnings = false to spec_helper
If you wanted to suppress only certain logger methods, like error, info, or warn you could do
allow_any_instance_of(Logger).to receive(:warn).and_return(nil)
To disable warnings coming from the rspec gem
allow(RSpec::Support).to receive(:warning_notifier).and_return(nil)
but this is generally discouraged because it is meant as a way to let you know you are doing something smelly in your tests.
If you want to suppress output for a single test, there is a more concise way:
it "should do something with printing" do
silence_stream(STDOUT) do
foo.print.should be_true
end
end
You may want to change STDOUT to STDERR if your test prints an error.
Updated answer for Rails 5, in a one-off situation:
before do
RSpec::Mocks.with_temporary_scope do
allow(STDOUT).to receive(:puts)
end
end
You can make this into a method in spec_helper if you'll be doing this a lot.
After trying all of these examples, I ended up using this varation which does not silence or mute binding.pry
# frozen_string_literal: true
RSpec.configure do |config|
config.before(:each) do
allow($stdout).to receive(:puts)
allow($stdout).to receive(:write)
end
end
It can be useful to inject an IO object defaulting to STDOUT. This also makes it easier to assert on the output if you want to.
E.g.
def my_method(arg, io: STDOUT)
io.puts "hello"
arg.reverse
end
And then in your test:
# Suppress it.
my_method("hi", io: StringIO.new)
# Assert on it.
io = StringIO.new
my_method("hi", io: io)
output = io.tap(&:rewind).read
expect(output).to include("hello")
You could have the object itself supress based on the environment:
class Foo
def call
puts("blah blah")
# ...
end
private
def puts(msg)
return if ENV['APP_ENV'] == 'test'
super
end
end
I'm attempting to copy stdout to a file for logging purposes. I also want it to display in the Ruby console of the IDE I'm using.
I inserted this code into my script and it redirects $stdout to the my.log file:
$stdout.reopen("my.log", "w")
Does anyone know of a gem or technique to copy the contents of $stdout to a file and not redirect it to a file? Also, I am not using Rails just Ruby.
Something like this might help you:
class TeeIO < IO
def initialize orig, file
#orig = orig
#file = file
end
def write string
#file.write string
#orig.write string
end
end
Most of the methods in IO that do output ultimately use write, so you only have to override this one method. You can use it like this:
#setup
tee = TeeIO.new $stdout, File.new('out.txt', 'w')
$stdout = tee
# Now lots of example uses:
puts "Hello"
$stdout.puts "Extending IO allows us to expicitly use $stdout"
print "heres", :an, :example, "using", 'print', "\n"
48.upto(57) do |i|
putc i
end
putc 10 #newline
printf "%s works as well - %d\n", "printf", 42
$stdout.write "Goodbye\n"
After this example, the following is written identically to both the console and to the file:
Hello
Extending IO allows us to expicitly use $stdout
heresanexampleusingprint
0123456789
printf works as well - 42
Goodbye
I won't claim this technique is fail proof, but it should work for simple uses of stdout. Test it for your use.
Note that you don't have to use reopen on $stdout unless you want to redirect output from a child process or an uncooperative extension. Simply assigning a different IO object to it will work for most uses.
RSpec
The RSpec command line takes a reference to $stdout before you can get any code to run to reassign it, so this doesn't work. reopen still works in this case as you're changing the actual object pointed to by both $stdout and the reference that RSpec has, but this doesn't give you output to both.
One solution is to monkey-patch $stdout like this:
$out_file = File.new('out.txt', 'w')
def $stdout.write string
$out_file.write string
super
end
This works, but as with all monkey patching, be careful. It would be safer to use your OS's tee command.
If you are using Linux or Mac OS, the tee command available in the OS makes it easy to do this. From its man page:
NAME
tee -- pipe fitting
SYNOPSIS
tee [-ai] [file ...]
DESCRIPTION
The tee utility copies standard input to standard output, making a copy in zero or more files. The output is unbuffered.
So something like:
echo '>foo bar' | tee tmp.out
>foo bar
echos the output to STDOUT and to the file. Catting the file gives me:
cat tmp.out
>foo bar
Otherwise, if you want to do it inside your code, it's a simple task:
def tee_output(logfile)
log_output = File.open(logfile, 'w+')
->(o) {
log_output.puts o
puts o
}
end
tee = tee_output('tmp.out')
tee.call('foo bar')
Running it:
>ruby test.rb
foo bar
And checking the output file:
>cat tmp.out
foo bar
I'd use "w+" for my file access to append to the output file, rather than over-write it.
CAVEAT: This opens the file and leaves it open during the life of the code after you've called the tee_output method. That bothers some people, but, personally, it doesn't bother me because Ruby will close the file when the script exits. In general we want to close files as soon as we're done with them, but in your code, it makes more sense to open it and leave it open, than to repeatedly open and close the output file, but your mileage might vary.
EDIT:
For Ruby 1.8.7, use lambda instead of the new -> syntax:
def tee_output(logfile)
log_output = File.open(logfile, 'w+')
lambda { |o|
log_output.puts o
puts o
}
end
tee = tee_output('tmp.out')
tee.call('foo bar')
I know it's a old question, but I found myself in the same situation. I wrote a Multi-IO Class which extends File and overrode write puts and close methods, I also made sure its thread safe:
require 'singleton'
class MultiIO < File
include Singleton
##targets = []
##mutex = Mutex.new
def self.instance
self.open('/dev/null','w+')
end
def puts(str)
write "#{str}\n"
end
def write(str)
##mutex.synchronize do
##targets.each { |t| t.write str; flush }
end
end
def setTargets(targets)
raise 'setTargets is a one-off operation' unless ##targets.length < 1
targets.each do |t|
##targets.push STDOUT.clone if t == STDOUT
##targets.push STDERR.clone if t == STDERR
break if t == STDOUT or t == STDERR
##targets.push(File.open(t,'w+'))
end
self
end
def close
##targets.each {|t| f.close}
end
end
STDOUT.reopen MultiIO.instance.setTargets(['/tmp/1.log',STDOUT,STDERR])
STDERR.reopen STDOUT
threads = []
5.times.each do |i|
threads.push(
Thread.new do
10000.times.each do |j|
STDOUT.puts "out#{i}:#{j}"
end
end
)
end
5.times.each do |i|
threads.push(
Thread.new do
10000.times.each do |j|
STDERR.puts "err#{i}:#{j}"
end
end
)
end
threads.each {|t| t.join}
Tiny improvement from matts answer:
class TeeIO < IO
def initialize(ios)
#ios = ios
end
def write(string)
#ios.each { |io| io.write string }
end
end