I'm attempting to update the em-irc library to make it work with current versions of Ruby, as well as update it with some new features. I'm trying to make the spec work to my changes, but it's not working as I expect.
One of the tests that's not working, regardless of the changes I introduce, is the send_data context.
subject do
EventMachine::IRC::Client.new
end
...
context 'send_data' do
let(:connection) { mock('Connection') }
before do
subject.stub(:conn => connection)
subject.stub(:connected => true)
end
it 'should return false if not connected' do
subject.stub(:connected => false)
subject.send_data("NICK jch").should == false
end
it 'should send message to irc server' do
connection.should_receive(:send_data).with("NICK jch\r\n")
subject.send_data("NICK jch")
end
end
Which references this function in my code:
def send_data(message)
return false unless #connected
message = message + "\r\n"
#conn.send_data(message)
trigger 'send', message
end
The first test works; when subject is not connected, send_data returns false. However, the second test fails because mock('Connection') never receives the send_data calls. This is the failure I receive:
1) EventMachine::IRC::Client send_data should send message to irc server
Failure/Error: connection.should_receive(:send_data).with("NICK jch\r\n")
(Mock "Connection").send_data("NICK jch\r\n")
expected: 1 time with arguments: ("NICK jch\r\n")
received: 0 times with arguments: ("NICK jch\r\n")
# ./spec/lib/em-irc/client_spec.rb:80:in `block (3 levels) in <top (required)>'
I've tried a couple changes but none of them seem to be working. I don't see why connection isn't receiving send_data calls even though I'm calling send_data on that mocked connection. It was working in the previous version of the library, with the only difference being I use let(:connection){...} rather than #connection = mock('Connection').
in rspec, you need to have tests run inside of an event loop. I achieve that with this monkey patch:
RSpec::Core::Example.class_eval do
alias ignorant_run run
def run(example_group_instance, reporter)
result = false
Fiber.new do
EM.run do
df = EM::DefaultDeferrable.new
df.callback do |test_result|
result = test_result
# stop if we are still running.
# We won't be running if something inside the test
# stops the run loop.
EM.stop if EM.reactor_running?
end
test_result = ignorant_run example_group_instance, reporter
df.set_deferred_status :succeeded, test_result
end
end.resume
result
end
end
Related
I'm maintaining a standalone test automation suite written in Rspec & Capybara and SitePrism (No Rails).
Recently I started integrating it with Testrail for reporting - I used rspec-testrail gem for that but I had to modify it a bit, because I also wanted to send Slack notifications with test progress and report.
Anyway, the integration works smoothly, except the fact that example processing is relying on example having an exception and lack of exception causes setting the test status as Passed on Testrail.
As it appears, after :each nor after :example in Rspec.configure doesn't guarantee that the example has finished running.
I also tried around :example and around :each as described here, but to no avail.
I inspected contents of example.metadata and it looks that example.metadata[:execution_result] has only started_at variable, but a finished example would have also finished_at and status variables, according to the docs
My suspicion (after reading the relish docs) is that :aggregated_failures is the cause of different metadata structure and multiple expects running in threads that are later merged into one backtrace.
Do you know how can I wait for the example to finish or how to hook into the state where it's finished?
Or maybe I should create a custom formatter where I would hook after example notifications printed to the console (I would like to keep the stacktrace there).
My code is as follows:
Test (both assertions are failing):
require 'spec_helper'
feature 'Sign in' do
let(:login_page) { LoginPage.new }
let(:user) { { email: ENV['ADMIN_EMAIL'], password: ENV['ADMIN_PASSWORD'] } }
scenario 'is successful and user is redirected to dashboard for user with correct credentials', testrail_id: 5694 do
login_page.load
login_page.form.email.set(user[:email])
login_page.form.password.set(user[:password])
login_page.form.submit_btn.click
expect(login_page.sidebar).to have_jobs(text: "some_nonexistenttext")
login_page.logout
expect(current_url).to have_content "google.com"
end
end
Console output from the above test:
Failures:
1) Sign in is successful and user is redirected to dashboard for user with correct credentials
Got 2 failures:
1.1) Failure/Error: expect(login_page.sidebar).to have_jobs(text: "blala")
expected #has_jobs?({:text=>"some_nonexistenttext"}) to return true, got false
# ./spec/auth/login_spec.rb:13:in `block (3 levels) in <top (required)>'
1.2) Failure/Error: expect(current_url).to have_content "google.com"
expected to find text "google.com" in "https://example.com/"
# ./spec/auth/login_spec.rb:15:in `block (3 levels) in <top (required)>'
Finished in 53.91 seconds (files took 1.45 seconds to load)
4 examples, 1 failure
Failed examples:
rspec ./spec/auth/login_spec.rb:8 # Sign in is successful and user is redirected to dashboard for user with correct credentials
Spec helper:
require 'rubygems'
require 'capybara/rspec'
require 'selenium-webdriver'
require 'site_prism'
require 'slack-notifier'
require_relative '../config'
require_relative '../lib/testrail'
...
RSpec.configure do |config|
config.define_derived_metadata do |meta|
meta[:aggregate_failures] = true
end
config.example_status_persistence_file_path = 'examples.txt'
config.before :all do
testrail_initialize_test_run!
end
config.after :example, testrail_id: proc { |value| !value.nil? } do |example|
RSpec::Testrail.process(example)
end
end
processing method (slightly modified from original)
def process(example)
if example.exception
status = 5
message = example.exception.message
slack_notifier.publish(message_text "Failed")
elsif example.skipped? || example.pending?
puts 'Example skipped or pending'
status = 10
message = 'Pending or not implemented'
else
status = 1
message = ''
end
client.send_post("add_result_for_case/#{testrun['id']}/#{example.metadata[:testrail_id]}",
status_id: status,
comment: message)
end
So basically all I had to was to use a reporter listener and process notifications inside it :)
config.reporter.register_listener RSpec::Testrail::Listener.new, :start, :example_failed, :example_passed, :example_pending, :stop
I have a problem with the testing the Sensu Plugin.
Everytime when I start rspec to test plugin it test it, but anyway at the end of test, the original plugin is started automatically. So I have in my console:
Finished in 0 seconds (files took 0.1513 seconds to load)
1 example, 0 failures
CheckDisk OK: # This comes from the plugin
Short explanation how my system works:
Plugin call system 'wmic' command, processes it, checks the conditions about the disk parameters and returns the exit statuses (ok, critical, etc)
Rspec mocks the response from system and sets into the input of plugin. At the end rspec checks the plugin exit status when the mocked input is given.
My plugin looks like that:
require 'rubygems' if RUBY_VERSION < '1.9.0'
require 'sensu-plugin/check/cli'
class CheckDisk < Sensu::Plugin::Check::CLI
def initialize
super
#crit_fs = []
end
def get_wmic
`wmic volume where DriveType=3 list brief`
end
def read_wmic
get_wmic
# do something, fill the class variables with system response
end
def run
severity = "ok"
msg = ""
read_wmic
unless #crit_fs.empty?
severity = "critical"
end
case severity
when /ok/
ok msg
when /warning/
warning msg
when /critical/
critical msg
end
end
end
Here is my test in Rspec:
require_relative '../check-disk.rb'
require 'rspec'
def loadFile
#Load template of system output when ask 'wmic volume(...)
end
def fillParametersInTemplate (template, parameters)
#set mocked disk parameters in template
end
def initializeMocks (options)
mockedSysOutput = fillParametersInTemplate #loadedTemplate, options
po = String.new(mockedSysOutput)
allow(checker).to receive(:get_wmic).and_return(po) #mock system call here
end
describe CheckDisk do
let(:checker) { described_class.new }
before(:each) do
#loadedTemplate = loadFile
def checker.critical(*_args)
exit 2
end
end
context "When % of free disk space = 10 >" do
options = {:diskName => 'C:\\', :diskSize => 1000, :diskFreeSpace => 100}
it 'Returns ok exit status ' do
begin
initializeMocks options
checker.run
rescue SystemExit => e
exit_code = e.status
end
expect(exit_code).to eq 0
end
end
end
I know that I can just put "exit 0" after the last example, but this is not a solution because when I will try to start many spec files it will exit after the first one. How to start only test, without running the plugin? Maybe someone can help me and show how to handle with such problem?
Thank you.
You can stub the original plugin call and optionally return a dummy object:
allow(SomeObject).to receive(:method) # .and_return(double)
you can put it in the before block to make sure that all assertions will share the code.
Another thing is that you are using rescue blocks to catch the situation when your code aborts with an error. You should use raise_error matcher instead:
expect { run }.to raise_error(SystemExit)
I have created a class which I want to hang on to a file descriptor and close it when the instance is GC-ed.
I created a class that looks something like this:
class DataWriter
def initialize(file)
# open file
#file = File.open(file, 'wb')
# create destructor
ObjectSpace.define_finalizer(self, self.class.finalize(#file))
end
# write
def write(line)
#file.puts(line)
#file.flush
end
# close file descriptor, note, important that it is a class method
def self.finalize(file)
proc { file.close; p "file closed"; p file.inspect}
end
end
I then tried to test the destructor method like so:
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
data_writer = DataWriter.new('/tmp/example.txt')
expect(DataWriter).to receive(:finalize)
data_writer = nil
GC.start
end
end
end
When running this test, even though the "file closed" is printed along with the file.inspect, the test fails with the following output:
1) DataWriter it should call its destructor calls the destructor
Failure/Error: expect(DataWriter).to receive(:finalize)
(DataWriter (class)).finalize(*(any args))
expected: 1 time with any arguments
received: 0 times with any arguments
# ./spec/utils/data_writer_spec.rb:23:in `block (3 levels) in <top (required)>'
finalize is called in initialize, returns the proc, and is never called again, so you can't expect it to be called at finalization time. It's the proc that's called when the instance is finalized. To check that, have the proc call a method instead of doing the work itself. This passes:
class DataWriter
# initialize and write same as above
def self.finalize(file)
proc { actually_finalize file }
end
def self.actually_finalize(file)
file.close
end
end
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
data_writer = DataWriter.new('/tmp/example.txt')
expect(DataWriter).to receive(:actually_finalize)
data_writer = nil
GC.start
end
end
end
even though the "file closed" is printed along with the file.inspect, the test fails with the following output
I threw your code into a single file and ran it. It appears that the finalize code isn't being cleaned up until rspec exits given the output I'm receiving:
Failures:
F
1) DataWriter it should call its destructor calls the destructor
Failure/Error: expect(DataWriter).to receive(:finalize)
(DataWriter (class)).finalize(*(any args))
expected: 1 time with any arguments
received: 0 times with any arguments
# /scratch/data_writer.rb:27:in `block (3 levels) in <top (required)>'
Finished in 0.01066 seconds (files took 0.16847 seconds to load)
1 example, 1 failure
Failed examples:
rspec /scratch/data_writer.rb:25 # DataWriter it should call its destructor calls the destructor
"file closed"
"#<File:/tmp/example.txt (closed)>"
As to the why of it, I can't tell right now. Dave is right you're asserting on something that's already happened, so your test is never going to pass. You can observe this by changing your test to:
it 'calls the destructor' do
expect(DataWriter).to receive(:finalize).and_call_original
data_writer = DataWriter.new('/tmp/example.txt')
data_writer = nil
GC.start
end
IMHO you should not rely on the finalizer to run exactly when GC runs. They will run, eventually. But perhaps only when the Process finishes. As far as I can tell this is also dependent on the Ruby implementation and the GC implementation.
1.8 has different behavior than 1.9+, Rubinius and JRuby might be different as well.
Making sure that a resource is released can be achieved by a block, which will also take care that the resource is released as soon as not needed anymore.
Multiple APIs have the same style in Ruby:
File.open('thing.txt', 'wb') do |file| # file is passed to block
# do something with file
end # file will be closed when block ends
Instead of doing this (as you showed in your gist)
(1..100_000).each do |i|
File.open(filename, 'ab') do |file|
file.puts "line: #{i}"
end
end
I'd do it this way:
File.open(filename, 'wb') do |file|
(1..100_000).each do |i|
file.puts "line: #{i}"
end
end
I rewrote my working solution below, bit I have not ran this code.
RSpec.describe DataWriter do
context 'it should call its destructor' do
it 'calls the destructor' do
# creating pipe for IPC to get result from child process
# after it garbaged
# http://ruby-doc.org/core-2.0.0/IO.html#method-c-pipe
rd, wr = IO.pipe
# forking
# https://ruby-doc.org/core-2.1.2/Process.html#method-c-fork
if fork
wr.close
called = rd.read
Process.wait
expect(called).to eq('/tmp/example.txt')
rd.close
else
rd.close
# overriding DataWriter.actually_finalize(file)
DataWriter.singleton_class.class_eval do
define_method(:actually_finalize) do |arg|
wr.write arg
wr.close
end
end
data_writer = DataWriter.new('/tmp/example.txt')
data_writer = nil
GC.start
end
end
end
end
The main thing is I catch that GC.start call performs real work exactly when exiting from process. I have tried blocks and threads but in my case (ruby 2.2.4p230 # Ubuntu x86_64) it works only when a process finished.
I suggest, that there may exist a better way to get results from child process, but I used inter-process communication (IPC).
And I have not got result with building the rspec expectation on destructor method call in form like expect(DataWriter).to receive(:actually_finalize).with('/tmp/example.txt') - I don't know why, but I suppose that wrappers created by Rspec have been garbaged or infringed before calling of destructor of a class.
Hope this helps!
I am working on wrapping the ruby-mqtt gem into a class which implements a subscribe and publish method. The subscribe method connects to the server and listens in a separate thread because this call is synchronous.
module PubSub
class MQTT
attr_accessor :host, :port, :username, :password
def initialize(params = {})
params.each do |attr, value|
self.public_send("#{attr}=", value)
end if params
super()
end
def connection_options
{
remote_host: self.host,
remote_port: self.port,
username: self.username,
password: self.password,
}
end
def subscribe(name, &block)
channel = name
connect_opts = connection_options
code_block = block
::Thread.new do
::MQTT::Client.connect(connect_opts) do |c|
c.get(channel) do |topic, message|
puts "channel: #{topic} data: #{message.inspect}"
code_block.call topic, message
end
end
end
end
def publish(channel = nil, data)
::MQTT::Client.connect(connection_options) do |c|
c.publish(channel, data)
end
end
end
end
I have a test that I have written using rspec to test the class but it does not pass.
mqtt = ::PubSub::MQTT.new({host: "localhost",port: 1883})
block = lambda { |channel, data| puts "channel: #{channel} data: #{data.inspect}"}
block.should_receive(:call).with("channel", {"some" => "data"})
thr = mqtt.subscribe("channel", &block)
mqtt.publish("channel", {"some" => "data"})
When I run the following ruby-mqtt-example I have now problems at all.
uri = URI.parse ENV['CLOUDMQTT_URL'] || 'mqtt://localhost:1883'
conn_opts = {
remote_host: uri.host,
remote_port: uri.port,
username: uri.user,
password: uri.password,
}
# Subscribe example
Thread.new do
puts conn_opts
MQTT::Client.connect(conn_opts) do |c|
# The block will be called when you messages arrive to the topic
c.get('test') do |topic, message|
puts "#{topic}: #{message}"
end
end
end
# Publish example
puts conn_opts
MQTT::Client.connect(conn_opts) do |c|
# publish a message to the topic 'test'
loop do
c.publish('test', 'Hello World')
sleep 1
end
end
So my question is, what am I doing wrong when I simply create a class and separate out the publish and subscribe logic? My guess is that it has something to do with Threading in the function call but I can't seem to figure it out. Any help is much appreciated.
UPDATE
I believe I know why the test is not passing and it is because when I pass a lambda in to subscribe expecting it to receive a call it actually will not receive the call when it exits the method or until publish is called. So I would like to rephrase the question to: How do I test that a block is called within a thread? If someone answers, "you don't", then the question is: How do you test that block is being called in an infinite loop like in the example of calling get within ruby-mqtt gem.
The RSpec expectations machinery will work fine with threads, as evidenced by the following example, which passes:
def foo(&block)
block.call(42)
end
describe "" do
it "" do
l = lambda {}
expect(l).to receive(:call).with(42)
Thread.new { foo(&l) }.join
end
end
The join waits for the thread(s) to finish before going further.
I am trying to run some functional test on a small server I have created. I am running Ruby 1.9.2 and RSpec 2.2.1 on Mac OS X 10.6. I have verified that the server works correctly and is not causing the problems I am experiencing. In my spec, I am attempting to spawn of a process to start the server, run some examples, and then kill the process running the server. Here is the code for my spec:
describe "Server" do
describe "methods" do
let(:put) { "put foobar beans 5\nhowdy" }
before(:all) do
#pid = spawn("bin/server")
end
before(:each) do
#sock = TCPSocket.new "127.0.0.1", 3000
end
after(:each) do
#sock.close
end
after(:all) do
Process.kill("HUP", #pid)
end
it "should be valid for a valid put method" do
#sock.send(put, 0).should == put.length
response = #sock.recv(1000)
response.should == "OK\n"
end
#more examples . . .
end
end
When I run the spec, it appears that the before(:all) and after(:all) blocks are run and the server processes is killed before the examples are run, because I get the following output:
F
Failures:
1) Server methods should be valid for a valid put method
Failure/Error: #sock = TCPSocket.new "127.0.0.1", 3000
Connection refused - connect(2)
# ./spec/server_spec.rb:11:in `initialize'
# ./spec/server_spec.rb:11:in `new'
# ./spec/server_spec.rb:11:in `block (3 levels) in <top (required)>'
When I comment out the call to Process.kill, the server is started and the tests are run, but the server remains running, which means I have to go manually kill it.
It seems like I am misunderstanding what the after(:all) method is supposed to do, because it is not being run in the order I thought it would. Why is this happening? What do I need to do so that my specs
Are you sure the server is ready to accept connections? Maybe something like this would help:
before(:each) do
3.times do
begin
#sock = TCPSocket.new "127.0.0.1", 2000
break
rescue
sleep 1
end
end
raise "could not open connection" unless #sock
end