I'm trying out the whole BDD approach and would like to test the AMQP-based aspect of a vanilla Ruby application I am writing. After choosing Minitest as the test framework for its balance of features and expressiveness as opposed to other aptly-named vegetable frameworks, I set out to write this spec:
# File ./test/specs/services/my_service_spec.rb
# Requirements for test running and configuration
require "minitest/autorun"
require "./test/specs/spec_helper"
# External requires
# Minitest Specs for EventMachine
require "em/minitest/spec"
# Internal requirements
require "./services/distribution/my_service"
# Spec start
describe "MyService", "A Gateway to an AMQP Server" do
# Connectivity
it "cannot connect to an unreachable AMQP Server" do
# This line breaks execution, commented out
# include EM::MiniTest::Spec
# ...
# (abridged) Alter the configuration by specifying
# an invalid host such as "l0c#alho$t" or such
# ...
# Try to connect and expect to fail with an Exception
MyApp::MyService.connect.must_raise EventMachine::ConnectionError
end
end
I have commented out the inclusion of the em-minitest-spec gem's functionality which should coerce the spec to run inside the EventMachine reactor, if I include it I run into an even sketchier exception regarding (I suppose) inline classes and such: NoMethodError: undefined method 'include' for #<#<Class:0x3a1d480>:0x3b29e00>.
The code I am testing against, namely the connect method within that Service is based on this article and looks like this:
# Main namespace
module MyApp
# Gateway to an AMQP Server
class MyService
# External requires
require "eventmachine"
require "amqp"
# Main entry method, connects to the AMQP Server
def self.connect
# Add debugging, spawn a thread
Thread.abort_on_exception = true
begin
#em_thread = Thread.new {
begin
EM.run do
#connection = AMQP.connect(#settings["amqp-server"])
AMQP.channel = AMQP::Channel.new(#connection)
end
rescue
raise
end
}
# Fire up the thread
#em_thread.join
rescue Exception
raise
end
end # method connect
end
end # class MyService
The whole "exception handling" is merely an attempt to bubble the exception out to a place where I can catch/handle it, that didn't help either, with or without the begin and raise bits I still get the same result when running the spec:
EventMachine::ConnectionError: unable to resolve server address, which actually is what I would expect, yet Minitest doesn't play well with the whole reactor concept and fails the test on ground of this Exception.
The question then remains: How does one test EventMachine-related code using Minitest's spec mechanisms? Another question has also been hovering around regarding Cucumber, also unanswered.
Or should I focus on my main functionality (e.g. messaging and seeing if the messages get sent/received) and forget about edge cases? Any insight would truly help!
Of course, it can all come down to the code I wrote above, maybe it's not the way one goes about writing/testing these aspects. Could be!
Notes on my environment: ruby 1.9.3p194 (2012-04-20) [i386-mingw32] (yes, Win32 :>), minitest 3.2.0, eventmachine (1.0.0.rc.4 x86-mingw32), amqp (0.9.7)
Thanks in advance!
Sorry if this response is too pedantic, but I think you'll have a much easier time writing the tests and the library if you distinguish between your unit tests and your acceptance tests.
BDD vs. TDD
Be careful not to confuse BDD with TDD. While both are quite useful, it can lead to problems when you try to test every edge case in an acceptance test. For example, BDD is about testing what you're trying to accomplish with your service, which has more to do with what you're doing with the message queue than connecting to the queue itself. What happens when you try to connect to a non-existent message queue fits more into the realm of a unit test in my opinion. It's also worth pointing out that your service shouldn't be responsible for testing the message queue itself, since that's the responsibility of AMQP.
BDD
While I'm not sure what your service is supposed to do exactly, I would imagine your BDD tests should look something like:
start the service (can do this in a separate thread in the tests if you need to)
write something to the queue
wait for your service to respond
check the results of the service
In other words, BDD (or acceptance tests, or integration tests, however you want to think about them) can treat your app as a black box that is supposed to provide certain functionality (or behavior). The tests keep you focused on your end goal, but are more meant for ensuring one or two golden use cases, rather than the robustness of the app. For that, you need to break down into unit tests.
TDD
When you're doing TDD, let the tests guide you somewhat in terms of code organization. It's difficult to test a method that creates a new thread and runs EM inside that thread, but it's not so hard to unit test either of these individually. So, consider putting the main thread code into a separate function that you can unit test separately. Then you can stub out that method when unit testing the connect method. Also, instead of testing what happens when you try to connect to a bad server (which tests AMQP), you can test what happens when AMQP throws an error (which is your code's responsibility to handle). Here, your unit test can stub out the response of AMQP.connect to throw an exception.
Related
Using Rspec and Capybara after recently adding in a debounce to most of my pages the test now fail randomly.
Now locally these are passing fine but on Semaphore 2.0 I am getting the random failings on shorter tests.
We use WebMock to stub the request in remoteFetch() and it seems that this is removed on shorter tests. As this is called afterwards, the stub doesn't exist and the test fails
function debouncedFetch(ids) {
store.idsToFetch.push(ids);
$timeout.cancel(store.fetchTimeoutFn);
store.fetchTimeoutFn = $timeout(() => { remoteFetch(store.idsToFetch); }, 200);
}
I have tried putting the debounce/timeout to 0 still with no joy.
Is there a way to check if tests/rootscope have finished or destroyed or something and not run the remoteFetch function.
Or get the test to wait for this function to run
Assuming you're using the default Capybara configuration where Capybara manages the running of the app under test it will wait for all network connections to be closed during the test reset in an after block. Since you're cleaning up your WebMock in an after block it's possible it's occurring before the Capybara registered block. To fix that you can change the order they're defined in or defined your WebMock cleanup with append_after rather than after so it's guaranteed to run after the Capybara session reset.
It turns out that the after(each:) { WebMock.reset! } as part of the gem is called before Capybara.reset_sessions!
This causes a race condition in the code. the way around this is to change the order and make sure in your spec_helper that require 'webmock/rspec' is called before require 'rspec/rails'
This ensures the order of the hooks are setup in the right order.
hope this helps someone else
I think the question does not pin-point to the real problem, I have difficulties to nail it down precisely and concisely.
I have a gem that implements i.e. MySQL-database "queries" (also inserts, updates...)
module DBGEM::Query
def self.client settings=DBGEM.settings
##client ||= Mysql2::Client.new settings
end
def query_this
client.query(...)
end
def process_insert_that list_of_things
list_of_things.each do |thing|
# process
client.query(...)
end
end
Furthermore, this gem is used by a sinatra app sitting on a forking webserver like puma.
Within the sinatra-app i can now
get '/path' do
happy = DBGEM::Query.query_this
# process happy
great = DBGEM::Query.process_insert_that 1..20
# go on
end
I like that API and this code should open only one database connection.
But as far as I understood, because the code within the 'get' definition is not guaranteed to be the only one accessing the DBGEM::Query stuff at that time, weird things could happen (through race-conditions, shared internal state?).
Is there a clever way to keep the nice syntax and the connection sharing without boilerplate object creation (query = DBGEM::Query.new() #...) wrapping the stuff in a block (DBGEM::Query.process do |query| #...)?
The example above is obviously simplified. The sinatra handling might be more involved, the Queries actually done in a Service object etc.pp. Also, afaiu in a forking webserver environment, the GC would destroy the client (closing the connection - thats how mysql2 is implemented).
I think that the connection will not be closed every time.
##client is shared between DBGEM::Query object itself (in Ruby modules and classes are also objects) and all the instances of that object (to be precise: all the instances of classes to which that object is mixed in).
So, this variable will live as long as the DBGEM::Query object will live.
You can check out when DBGEM::Query object will be garbage collected, by defining finalizer logging a text and observe the server console.
module DBGEM::Query
ObjectSpace.define_finalizer(self, proc { print 'garbage collected' })
..
end
Im not sure, however I guess that DBGEM::Query object will be garbage collected only when you stop the server.
As it goes for weird "things could happen", I believe you mean potential conflicts, race conditions, situations where you create double records, or update the same record nearly at the same time overwriting something, etc. And when that happen you lose data integrity.
IMHO you can't prevent it by allowing only one client instance. I'd suggest aiming for solid database design (unique constrains, indexes, foreign keys, validations) which can raise errors when race condition occure and then handling that errors in your application.
When I write tests that involve subscribing to events on the Eventstream or watching actors and listning for "Terminated", the tests work fine running them 1 by 1 but when I run the whole testsuite those tests fail.
Tests also works if each of those tests are in a separate test class with Xunit.
How come?
A repo with those kind of tests: https://github.com/Lejdholt/AkkaTestError
Took a look at your repository. I can reproduce the problems you are describing.
It feels like a bug in the TestKit, some timing issue somewhere. But its hard to pin down.
Also, not all unit test frameworks are created equally. The testkit uses its own TaskDispatcher to enable the testing of what are normally inherently asynchronous processed operations.
This sometimes causes some conflicts with the testframework being used. Is also coincidentally why akka.net also moved to XUnit for their own CI process.
I have managed to fix your problem, by not using the TestProbe. Although im not sure if the problem lies with the TestProbe per say, or the fact that your where using an global reference (your 'process' variable).
I suspect that the testframework, while running tests in parrallel, might be causing some wierd things to happen with your testprobe reference.
Example of how i changed one of your tests:
[Test]
public void GivenAnyTime_WhenProcessTerminates_ShouldLogStartRemovingProcess()
{
IProcessFactory factory = Substitute.For<IProcessFactory>();
var testactor = Sys.ActorOf<FakeActor>("test2");
processId = Guid.NewGuid();
factory.Create(Arg.Any<IActorRefFactory>(), Arg.Any<SupervisorStrategy>()).Returns(testactor);
manager = Sys.ActorOf(Props.Create(() => new Manager(factory)));
manager.Tell(new StartProcessCommand(processId));
EventFilter.Info("Removing process.")
.ExpectOne(() => Sys.Stop(testactor));
}
It should be fairly self explanatory on how you should change your other test.
The FakeActor is nothing more then an empty ReceiveActor implementation.
I have an ActiveJob that's supposed to load a piece of data from an external system over HTTP. When that job completes, I want to queue a second job that does some postprocessing and then submits the data to a different external system.
I don't want the first job to know about the second job, because
encapsulation
reusability
it's none of the first job's business, basically
Likewise, I don't want the first job to care what happens next if the data-loading fails -- maybe the user gets notified, maybe we retry after a timeout, maybe we just log it and throw up our hands -- again it could vary based on the details of the exception, and there's no need for the job to include the logic for that or the connections to other systems to handle it.
In Java (which is where I have the most experience), I could use something like Guava's ListenableFuture to add success and failure callbacks after the fact:
MyDataLoader loader = new MyDataLoader(someDataSource)
ListenableFuture<Data> future = executor.submit(loader);
Futures.addCallback(future, new FutureCallback<Data>() {
public void onSuccess(Data result) {
processData(result);
}
public void onFailure(Throwable t) {
handleFailure(t);
}
});
ActiveJob, though, doesn't seem to provide this sort of external callback mechanism -- as best I can make out from relevant sections in "Active Job Basics", after_perform and rescue_from are only meant to be called from within the job class. And after_peform isn't meant to distinguish between success and failure.
So the best I've been able to come up with (and I'm not claiming it's very good) is to pass a couple of lambdas into the job's perform method, thus:
class MyRecordLoader < ActiveJob::Base
# Loads data expensively (hopefully on a background queue) and passes
# the result, or any exception, to the appropriate specified lambda.
#
# #param data_source [String] the URL to load data from
# #param on_success [-> (String)] A lambda that will be passed the record
# data, if it's loaded successfully
# #param on_failure [-> (Exception)] A lambda that will be passed any
# exception, if there is one
def perform(data_source, on_success, on_failure)
begin
result = load_data_expensively_from data_source
on_success.call(result)
rescue => exception
on_failure.call(exception)
end
end
end
(Side note: I have no idea what the yardoc syntax is for declaring lambdas as parameters. Does this look correct, or, failing that, plausible?)
The caller would then have to pass these in:
MyRecordLoader.perform_later(
some_data_source,
method(:process_data),
method(:handle_failure)
)
That's not terrible, at least on the calling side, but it seems clunky, and I can't help but suspect there's a common pattern for this that I'm just not finding. And I'm somewhat concerned that, as a Ruby/Rails novice, I'm just bending ActiveJob to do something it was never meant to do in the first place. All the ActiveJob examples I'm finding are 'fire and forget' -- asynchronously "returning" a result doesn't seem to be an ActiveJob use case.
Also, it's not clear to me that this will work at all in the case of a back-end like Resque that runs the jobs in a separate process.
What's "the Ruby way" to do this?
Update: As hinted at by dre-hh, ActiveJob turned out not to be the right tool here. It was also unreliable, and overcomplicated for the situation. I switched to Concurrent Ruby instead, which fits the use case better, and which, since the tasks are mostly IO-bound, is fast enough even on MRI, despite the GIL.
ActiveJob is not an async Library like a future or promise.
It is just an interface for performing tasks in a background. The current thread/process receives no result of this operation.
For example when using Sidekiq as ActiveJob queue, it will serialize the parameters of the perform method into the redis store. Another daemon process running within the context of your rails app will be watching the redis queue and instantiate your worker with the serialized data.
So passing callbacks might be alright, however why having them as methods on another class. Passing callbacks would make sense if those are dynamic (changing on different invocation). However as you have them implemented on the calling class, consider just moving those methods into your job worker class.
Part of my command-line Ruby program involves checking if there is an internet connection before any commands are processed. The actual check in the program is trivial (using Socket::TCPSocket), but I'm trying to test this behaviour in Cucumber for an integration test.
The code:
def self.has_internet?(force = nil)
if !force.nil? then return force
begin
TCPSocket.new('www.yelp.co.uk', 80)
return true
rescue SocketError
return false
end
end
if has_internet? == false
puts("Could not connect to the Internet!")
exit 2
end
The feature:
Scenario: Failing to log in due to no Internet connection
Given the Internet is down
When I run `login <email_address> <password>`
Then the exit status should be 2
And the output should contain "Could not connect to the Internet!"
I obviously don't want to change the implementation to fit the test, and I require all my scenarios to pass. Clearly if there is actually no connection, the test passes as it is, but my other tests fail as they require a connection.
My question: How can I test for this in a valid way and have all my tests pass?
You can stub your has_internet? method and return false in the implementation of the Given the Internet is down step.
YourClass.stub!(:has_internet?).and_return(false)
There are three alternative solutions I can think of:
have the test temporarily monkeypatch TCPSocket.initialize (or maybe Socket#connect, if that's where it ends up) to pretend the internet is down.
write (suid) a script that adds/removes an iptables firewall rule to disable the internet, and have your test call the script
use LD_PRELOAD on a specially written .so shared library that overrides the connect C call. This is harder.
Myself, I would probably try option 1, give up after about 5 minutes and go with option 2.
maybe a bit late for you :), but have a look at
https://github.com/mmolhoek/vcr-uri-catcher
I made this to test network failures, so this should do the trick for you.