How do you test whether a Ruby destructor will be called? - ruby

I have created a class which I want to hang on to a file descriptor and close it when the instance is GC-ed.
I created a class that looks something like this:
class DataWriter
def initialize(file)
# open file
#file = File.open(file, 'wb')
# create destructor
ObjectSpace.define_finalizer(self, self.class.finalize(#file))
end
# write
def write(line)
#file.puts(line)
#file.flush
end
# close file descriptor, note, important that it is a class method
def self.finalize(file)
proc { file.close; p "file closed"; p file.inspect}
end
end
I then tried to test the destructor method like so:
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
data_writer = DataWriter.new('/tmp/example.txt')
expect(DataWriter).to receive(:finalize)
data_writer = nil
GC.start
end
end
end
When running this test, even though the "file closed" is printed along with the file.inspect, the test fails with the following output:
1) DataWriter it should call its destructor calls the destructor
Failure/Error: expect(DataWriter).to receive(:finalize)
(DataWriter (class)).finalize(*(any args))
expected: 1 time with any arguments
received: 0 times with any arguments
# ./spec/utils/data_writer_spec.rb:23:in `block (3 levels) in <top (required)>'

finalize is called in initialize, returns the proc, and is never called again, so you can't expect it to be called at finalization time. It's the proc that's called when the instance is finalized. To check that, have the proc call a method instead of doing the work itself. This passes:
class DataWriter
# initialize and write same as above
def self.finalize(file)
proc { actually_finalize file }
end
def self.actually_finalize(file)
file.close
end
end
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
data_writer = DataWriter.new('/tmp/example.txt')
expect(DataWriter).to receive(:actually_finalize)
data_writer = nil
GC.start
end
end
end

even though the "file closed" is printed along with the file.inspect, the test fails with the following output
I threw your code into a single file and ran it. It appears that the finalize code isn't being cleaned up until rspec exits given the output I'm receiving:
Failures:
F
1) DataWriter it should call its destructor calls the destructor
Failure/Error: expect(DataWriter).to receive(:finalize)
(DataWriter (class)).finalize(*(any args))
expected: 1 time with any arguments
received: 0 times with any arguments
# /scratch/data_writer.rb:27:in `block (3 levels) in <top (required)>'
Finished in 0.01066 seconds (files took 0.16847 seconds to load)
1 example, 1 failure
Failed examples:
rspec /scratch/data_writer.rb:25 # DataWriter it should call its destructor calls the destructor
"file closed"
"#<File:/tmp/example.txt (closed)>"
As to the why of it, I can't tell right now. Dave is right you're asserting on something that's already happened, so your test is never going to pass. You can observe this by changing your test to:
it 'calls the destructor' do
expect(DataWriter).to receive(:finalize).and_call_original
data_writer = DataWriter.new('/tmp/example.txt')
data_writer = nil
GC.start
end

IMHO you should not rely on the finalizer to run exactly when GC runs. They will run, eventually. But perhaps only when the Process finishes. As far as I can tell this is also dependent on the Ruby implementation and the GC implementation.
1.8 has different behavior than 1.9+, Rubinius and JRuby might be different as well.
Making sure that a resource is released can be achieved by a block, which will also take care that the resource is released as soon as not needed anymore.
Multiple APIs have the same style in Ruby:
File.open('thing.txt', 'wb') do |file| # file is passed to block
# do something with file
end # file will be closed when block ends
Instead of doing this (as you showed in your gist)
(1..100_000).each do |i|
File.open(filename, 'ab') do |file|
file.puts "line: #{i}"
end
end
I'd do it this way:
File.open(filename, 'wb') do |file|
(1..100_000).each do |i|
file.puts "line: #{i}"
end
end

I rewrote my working solution below, bit I have not ran this code.
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
# creating pipe for IPC to get result from child process
# after it garbaged
# http://ruby-doc.org/core-2.0.0/IO.html#method-c-pipe
rd, wr = IO.pipe
# forking
# https://ruby-doc.org/core-2.1.2/Process.html#method-c-fork
if fork
wr.close
called = rd.read
Process.wait
expect(called).to eq('/tmp/example.txt')
rd.close
else
rd.close
# overriding DataWriter.actually_finalize(file)
DataWriter.singleton_class.class_eval do
define_method(:actually_finalize) do |arg|
wr.write arg
wr.close
end
end
data_writer = DataWriter.new('/tmp/example.txt')
data_writer = nil
GC.start
end
end
end
end
The main thing is I catch that GC.start call performs real work exactly when exiting from process. I have tried blocks and threads but in my case (ruby 2.2.4p230 # Ubuntu x86_64) it works only when a process finished.
I suggest, that there may exist a better way to get results from child process, but I used inter-process communication (IPC).
And I have not got result with building the rspec expectation on destructor method call in form like expect(DataWriter).to receive(:actually_finalize).with('/tmp/example.txt') - I don't know why, but I suppose that wrappers created by Rspec have been garbaged or infringed before calling of destructor of a class.
Hope this helps!

Related

Ruby check whether program is currently being closed

How can I check whether current script in Ruby is being closed?
Particularly, in case program is closing, I want to set #reconnect to false, not to allow web-socket reconnect any more. I tried Signal.trap("TERM"), but it doesn't seem to work.
#reconnect is an instance variable inside WebsocketClient class, i can't directly change it in my script outside class.
class WebsocketClient
def ws_closed(event)
$logger.warn "WS CLOSED"
Signal.trap("TERM") {
#stop = true
#reconnect = false
}
unless $reauth
if #stop
EM.stop
elsif #reconnect
$logger.warn "Reconnecting..."
EM.add_timer(#reconnect_after){ connect! }
end
end
end
end
at_exit {
$logger.fatal "Application terminated. Shutting down gracefully..."
# ...
# Do some exit work...
# ...
exit!
}
Output on CTRL-C
01-02-2018 12:00:54.59 WARN > WS CLOSED
01-02-2018 12:00:54.595 WARN > Reconnecting...
01-02-2018 12:00:54.596 FATAL > Application terminated. Shutting down gracefully..
See Below taken from my answer Here but seems more pertinent to your question than the one it is currently attached to:
Your best bet is probably a bit easier than signal trapping. The Kernel Module actually offers you an #at_exit method that will be executed just prior to the program actually exiting.
Usage: (from Kernel#at_exit Docs)
def do_at_exit(str1)
at_exit { print str1 }
end
at_exit { puts "cruel world" }
do_at_exit("goodbye ")
exit
"produces:"
goodbye cruel world
as you can see you can define multiple handlers which will be executed in reverse order when the program exits.
Since Kernel is included in Object you can handle Object specifics as well like
class People
at_exit {puts "The #{self.name} have left"}
end
exit
# The People have left
or even on instances
p = People.new
p.send(:at_exit, &->{puts "We are leaving"})
# We are leaving
# The People have left
Additionally for more specific Object based implementations you can take a look at ObjectSpace.define_finalizer.
example of usage:
class Person
def self.finalize(name)
proc {puts "Goodbye Cruel World -#{name}"}
end
def initialize(name)
#name = name
ObjectSpace.define_finalizer(self, self.class.finalize(#name))
end
end
Usage:
p = Person.new("engineersmnky")
exit
# Goodbye Cruel World -engineersmnky
This may not be specifically what you want as this will fire when an Object is garbage collected as well (not great for ephemeral objects) but if you have objects that should exist throughout the entire application this could still be used similar to an at_exit . Example
# requiring WeakRef to allow garbage collection
# See: https://ruby-doc.org/stdlib-2.3.3/libdoc/weakref/rdoc/WeakRef.html
require 'weakref' #
p1 = Person.new("Engineer")
p2 = Person.new("Engineer's Monkey")
p2 = WeakRef.new(p2)
GC.start # just for this example
# Goodbye Cruel World -Engineer's Monkey
#=> nil
p2
#=> WeakRef::RefError: Invalid Reference - probably recycled
exit
# Goodbye Cruel World -Engineer
As you can see the defined finalizer for p2 fired because the Person was gc'd but the program has not exited yet. p1's finalizer waited until exit to fire because it retained its reference throughout the application.

How do I get ruby to print a full backtrace that includes arguments passed to functions?

Sometimes backtrace is enough to diagnose problem. But sometimes reason of crash is not obvious without knowledge what was passed to function.
Getting information what was passed to function that caused crash would be quite useful, especially in cases where reproducing is not obvious because it was caused by for example exception in network connection, weird user input or because program is depends on randomisation or processes data from external sensor.
Lets say that there is following program
def handle_changed_input(changed_input)
raise 'ops' if changed_input =~ /magic/
end
def do_something_with_user_input(input)
input = "#{input.strip}c"
handle_changed_input(input)
end
input = gets
do_something_with_user_input(input)
where user typed "magic" as input. Normally one has
test.rb:2:in `handle_changed_input': ops (RuntimeError)
from test.rb:7:in `do_something_with_user_input'
from test.rb:11:in `<main>'
as output. What one may do to show also what was passed to function? Something like
test.rb:2:in `handle_changed_input("magic")': ops (RuntimeError)
from test.rb:7:in `do_something_with_user_input("magi\n")'
from test.rb:11:in `<main>'
It would be useful in many situations (and not truly useful where parameters are not representable as strings of reasonable legth, there is a good reason why it is not enabled by default).
How one may add this functionality? It is necessary that program works as usually during normal operation and preferably there is no additional output before crash.
I tried for example
def do_something_with_user_input(input)
method(__method__).parameters.map do |_, name|
puts "#{name}=#{binding.local_variable_get(name)}"
end
raise 'ops' if input =~ /magic/
end
input = gets
found in Is there a way to access method arguments in Ruby? but it would print on every single entrance to function what both would flood output and make program significantly slower.
I don't have a complete solution but... But you can get method arguments of all called methods in controlled environment with TracePoint class from Ruby core lib.
Look at the example:
trace = TracePoint.new(:call) do |tp|
puts "===================== #{tp.method_id}"
b_self = tp.binding.eval('self')
names = b_self.method(tp.method_id).parameters.map(&:last)
values = names.map { |name| tp.binding.eval(name.to_s) }
p names.zip(values)
end
trace.enable
def method_a(p1, p2, p3)
end
method_a(1, "foobar", false)
#=> ===================== method_a
#=> [[:p1, 1], [:p2, "foobar"], [:p3, false]]
To print exception backtraces, Ruby uses the C function exc_backtrace from error.c (exc_backtrace on github). Unless you patch Ruby with the functionality you need, I don't think there a way to change exception backtrace outputs.
Here is a snippet (trace.rb) you might find useful:
set_trace_func -> (event, file, line, id, binding, classname) do
if event == 'call' && meth = binding.eval('__method__')
params = binding.method(meth).parameters.select{|e| e[0] != :block}
values = params.map{|_, var| [var, binding.local_variable_get(var)]}
printf "%8s %s:%-2d %15s %8s %s\n", event, file, line, id, classname, values.inspect
else
printf "%8s %s:%-2d %15s %8s\n", event, file, line, id, classname
end
end
def foo(a,b = 0)
bar(a, foo: true)
end
def bar(c, d = {})
puts "!!!buz!!!\n"
end
foo('lol')
The output of that snippet is:
c-return /path/to/trace.rb:1 set_trace_func Kernel
line /path/to/trace.rb:12
c-call /path/to/trace.rb:12 method_added Module
c-return /path/to/trace.rb:12 method_added Module
line /path/to/trace.rb:16
c-call /path/to/trace.rb:16 method_added Module
c-return /path/to/trace.rb:16 method_added Module
line /path/to/trace.rb:20
call /path/to/trace.rb:12 foo Object [[:a, "lol"], [:b, 0]]
line /path/to/trace.rb:13 foo Object
call /path/to/trace.rb:16 bar Object [[:c, "lol"], [:d, {:foo=>true}]]
line /path/to/trace.rb:17 bar Object
c-call /path/to/trace.rb:17 puts Kernel
c-call /path/to/trace.rb:17 puts IO
c-call /path/to/trace.rb:17 write IO
!!!buz!!!
c-return /path/to/trace.rb:17 write IO
c-return /path/to/trace.rb:17 puts IO
c-return /path/to/trace.rb:17 puts Kernel
return /path/to/trace.rb:18 bar Object
return /path/to/trace.rb:14 foo Object
I hope that helps you as much as it helped me.
I think that it is possible. The code below is not perfect and would require some additional work, but it caputers the primary idea of a stacktrace with argument values. Please note, that in order to know the call site, I am zipping the original stacktrace with the entry sites catched by trace function. To distinguishe these entries I use '>' and '<' respectively.
class Reporting
def self.info(arg1)
puts "*** #{arg1} ***"
end
end
def read_byte(arg1)
Reporting.info(arg1)
raise Exception.new("File not found")
end
def read_input(arg1)
read_byte(arg1)
end
def main(arg1)
read_input(arg1)
end
class BetterStacktrace
def self.enable
set_trace_func -> (event, file, line, id, binding, classname) do
case event
when 'call'
receiver_type = binding.eval('self.class')
if receiver_type == Object
meth = binding.eval('__method__')
params = binding.method(meth).parameters.select{|e| e[0] != :block}
values = params.map{|_, var| [var, binding.local_variable_get(var)]}
self.push(event, file, line, id, classname, values)
else
self.push(event, file, line, id, classname)
end
when 'return'
self.pop
when 'raise'
self.push(event, file, line, id, classname)
Thread.current[:_keep_stacktrace] = true
end
end
end
def self.push(event, file, line, id, classname, values=nil)
Thread.current[:_saved_stacktrace] = [] unless Thread.current.key?(:_saved_stacktrace)
unless Thread.current[:_keep_stacktrace]
if values
values_msg = values.map(&:last).join(", ")
msg = "%s:%d:in `%s(%s)'" % [file, line, id, values_msg]
else
msg = "%s:%d:in `%s'" % [file, line, id]
end
Thread.current[:_saved_stacktrace] << msg
end
end
def self.pop()
Thread.current[:_saved_stacktrace] = [] unless Thread.current.key?(:_saved_stacktrace)
unless Thread.current[:_keep_stacktrace]
value = Thread.current[:_saved_stacktrace].pop
end
end
def self.disable
set_trace_func nil
end
def self.print_stacktrace(calls)
enters = Thread.current[:_saved_stacktrace].reverse
calls.zip(enters).each do |call, enter|
STDERR.puts "> #{enter}"
STDERR.puts "< #{call}"
end
Thread.current[:_saved_stacktrace] = []
end
end
BetterStacktrace.enable
begin
main(10)
rescue Exception => ex
puts "--- Catched ---"
puts ex
BetterStacktrace.print_stacktrace(ex.backtrace)
end
BetterStacktrace.disable
begin
main(10)
rescue Exception
puts "--- Catched ---"
puts ex
puts ex.backtrace
end
The output of the above code is as follows:
*** 10 ***
--- Catched ---
File not found
> work/tracing_with_params.rb:10:in `read_byte'
< work/tracing_with_params.rb:10:in `read_byte'
> work/tracing_with_params.rb:8:in `read_byte(10)'
< work/tracing_with_params.rb:14:in `read_input'
> work/tracing_with_params.rb:13:in `read_input(10)'
< work/tracing_with_params.rb:18:in `main'
> work/tracing_with_params.rb:17:in `main(10)'
< work/tracing_with_params.rb:82:in `<main>'
*** 10 ***
--- Catched ---
File not found
work/tracing_with_params.rb:10:in `read_byte'
work/tracing_with_params.rb:14:in `read_input'
work/tracing_with_params.rb:18:in `main'
work/tracing_with_params.rb:82:in `<main>'
EDIT:
The calls to class functions are not recorded. This has to be fixed in order for the stacktrace printing function not to get invalid output.
Moreover I used the STDERR as output to easily get one or the other output. You can change it if you wish.
MAX_STACK_SIZE = 200
tracer = proc do |event|
if event == 'call' && caller_locations.length > MAX_STACK_SIZE
fail "Probable Stack Overflow"
end
end
set_trace_func(tracer)

Rspec. The tested code is automatically started after test

I have a problem with the testing the Sensu Plugin.
Everytime when I start rspec to test plugin it test it, but anyway at the end of test, the original plugin is started automatically. So I have in my console:
Finished in 0 seconds (files took 0.1513 seconds to load)
1 example, 0 failures
CheckDisk OK: # This comes from the plugin
Short explanation how my system works:
Plugin call system 'wmic' command, processes it, checks the conditions about the disk parameters and returns the exit statuses (ok, critical, etc)
Rspec mocks the response from system and sets into the input of plugin. At the end rspec checks the plugin exit status when the mocked input is given.
My plugin looks like that:
require 'rubygems' if RUBY_VERSION < '1.9.0'
require 'sensu-plugin/check/cli'
class CheckDisk < Sensu::Plugin::Check::CLI
def initialize
super
#crit_fs = []
end
def get_wmic
`wmic volume where DriveType=3 list brief`
end
def read_wmic
get_wmic
# do something, fill the class variables with system response
end
def run
severity = "ok"
msg = ""
read_wmic
unless #crit_fs.empty?
severity = "critical"
end
case severity
when /ok/
ok msg
when /warning/
warning msg
when /critical/
critical msg
end
end
end
Here is my test in Rspec:
require_relative '../check-disk.rb'
require 'rspec'
def loadFile
#Load template of system output when ask 'wmic volume(...)
end
def fillParametersInTemplate (template, parameters)
#set mocked disk parameters in template
end
def initializeMocks (options)
mockedSysOutput = fillParametersInTemplate #loadedTemplate, options
po = String.new(mockedSysOutput)
allow(checker).to receive(:get_wmic).and_return(po) #mock system call here
end
describe CheckDisk do
let(:checker) { described_class.new }
before(:each) do
#loadedTemplate = loadFile
def checker.critical(*_args)
exit 2
end
end
context "When % of free disk space = 10 >" do
options = {:diskName => 'C:\\', :diskSize => 1000, :diskFreeSpace => 100}
it 'Returns ok exit status ' do
begin
initializeMocks options
checker.run
rescue SystemExit => e
exit_code = e.status
end
expect(exit_code).to eq 0
end
end
end
I know that I can just put "exit 0" after the last example, but this is not a solution because when I will try to start many spec files it will exit after the first one. How to start only test, without running the plugin? Maybe someone can help me and show how to handle with such problem?
Thank you.
You can stub the original plugin call and optionally return a dummy object:
allow(SomeObject).to receive(:method) # .and_return(double)
you can put it in the before block to make sure that all assertions will share the code.
Another thing is that you are using rescue blocks to catch the situation when your code aborts with an error. You should use raise_error matcher instead:
expect { run }.to raise_error(SystemExit)

EventMachine Rspec connection and send_data test not working

I'm attempting to update the em-irc library to make it work with current versions of Ruby, as well as update it with some new features. I'm trying to make the spec work to my changes, but it's not working as I expect.
One of the tests that's not working, regardless of the changes I introduce, is the send_data context.
subject do
EventMachine::IRC::Client.new
end
...
context 'send_data' do
let(:connection) { mock('Connection') }
before do
subject.stub(:conn => connection)
subject.stub(:connected => true)
end
it 'should return false if not connected' do
subject.stub(:connected => false)
subject.send_data("NICK jch").should == false
end
it 'should send message to irc server' do
connection.should_receive(:send_data).with("NICK jch\r\n")
subject.send_data("NICK jch")
end
end
Which references this function in my code:
def send_data(message)
return false unless #connected
message = message + "\r\n"
#conn.send_data(message)
trigger 'send', message
end
The first test works; when subject is not connected, send_data returns false. However, the second test fails because mock('Connection') never receives the send_data calls. This is the failure I receive:
1) EventMachine::IRC::Client send_data should send message to irc server
Failure/Error: connection.should_receive(:send_data).with("NICK jch\r\n")
(Mock "Connection").send_data("NICK jch\r\n")
expected: 1 time with arguments: ("NICK jch\r\n")
received: 0 times with arguments: ("NICK jch\r\n")
# ./spec/lib/em-irc/client_spec.rb:80:in `block (3 levels) in <top (required)>'
I've tried a couple changes but none of them seem to be working. I don't see why connection isn't receiving send_data calls even though I'm calling send_data on that mocked connection. It was working in the previous version of the library, with the only difference being I use let(:connection){...} rather than #connection = mock('Connection').
in rspec, you need to have tests run inside of an event loop. I achieve that with this monkey patch:
RSpec::Core::Example.class_eval do
alias ignorant_run run
def run(example_group_instance, reporter)
result = false
Fiber.new do
EM.run do
df = EM::DefaultDeferrable.new
df.callback do |test_result|
result = test_result
# stop if we are still running.
# We won't be running if something inside the test
# stops the run loop.
EM.stop if EM.reactor_running?
end
test_result = ignorant_run example_group_instance, reporter
df.set_deferred_status :succeeded, test_result
end
end.resume
result
end
end

Collecting exceptions in ruby script

I'm writing a script which collects data from various url's. I want to collect errors from begin rescue blocks into an array to output them when the program runs in verbose mode. With normal use, a failed connection is ignored and the script moves on to the next url.
I thought the best way to do this would be to create an array errArray = Array.new at the top of the script to hold errors, and then do:
rescue Exception => e
errArray << e.message
in various functions to log errors. The die function outputs the array using p unless it is empty. However, I get the error
Undefined local variable or method 'errArray'
Any help (and constructive criticism) appreciated.
EDIT: die function:
def die(e)
p errorArray unless errorArray.empty?
# Some other irrelevant code
end
errArray is not global variable and therefore methods have no access to it. You can declare it as a global variable by $err_array.
However the best solution would be create a simple class:
class ExceptionCollector
def collect
yield
rescue => e
errors << e.message
end
def errors
#errors ||= []
end
end
And then simple:
$logger = ExceptionCollector.new
$logger.collect do
# this may raise an exception
end
def foo
$logger.collect do
# another exception
end
end
$logger.errors #=> list of errors

Resources