Testing multiple hosts with the same test using serverspec - ruby

Question

The Advanced Tips section of the Serverspec site shows an example of testing multiple hosts with the same test set. I've built an example of my own (https://gist.github.com/neilhwatson/81249ad393800a76a8ad), but there are problems.
The first problem is that the tests stop at the first failure rather than proceeding through the lot and keeping a tally. The second is that the failure output does not indicate on which host the failure occurred. What can I do to fix these problems and produce a final report for all hosts?

Answer 1

For the first issue, ServerSpec by default will run all your tests. However, since you have a loop that executes a Rake task for each environment, the first environment to have a failure causes the task to fails and so an exception is raised and the rest of your tasks don't run.
I've forked your gist and updated the Rake task to surround it with a begin/rescue.
...
begin
desc "Run serverspec to #{host}"
RSpec::Core::RakeTask.new(host) do |t|
ENV['TARGET_HOST'] = host
t.pattern = "spec/base,cfengine3/*_spec.rb"
end
rescue
end
...
For the second problem, it doesn't look like ServerSpec will output which environment the tests are running in. But since the updated Gist shows that the host gets set in the spec_helper.rb we can use that to add an RSpec configuration that sets up an after(:each) and only output the host on errors. The relevant code changes are in a fork of the gist, but basically you'll just need the below snippet in your spec_helper.rb:
RSpec.configure do |c|
c.after(:each) do |example|
if example.exception
puts "Failed on #{host_run_on}"
end
end
end