How does one configure Cucumber / Aruba to use SimpleCov? - ruby

Question

Ruby-2.0.0p247
ActiveRecord-4.0.1
Cucumber 1.3.10
Aruba-0.5.3
SimpleCove-0.8.2
We use Cucumber with Aruba in a NON-RAILS project that nonetheless uses ActiveRecord. Our cucumber features exercise the code both in-process and out-of-process. The out-of-process code is executed using the same loader sequences as in production through a startup stub in bin:
#!/usr/bin/env ruby
require 'bundler/setup'
Bundler.require
require 'pathname'
my_dir = Pathname.new(
File.join( File.dirname(
__FILE__ ), '../', 'lib/' ) ).realpath.to_s + '/'
require my_dir + File.basename( __FILE__ )
HllThForexRssFetch::Main.new( ARGV ).execute
#EOF
Our features/support/env.rb file contains this:
$ cat features/support/env.rb
# Must load and start simplecov before any application code
require 'simplecov'
SimpleCov.start do
add_filter "/features/"
add_filter "/libexec"
add_filter "/lib/hll_active_record/"
add_filter "/test/"
add_filter "/tmp/"
end
SimpleCov.command_name( "Cucumber Features" )
# Do not use cucumber/rails in standalone projects
#require 'cucumber/rails'
. . .
When our step definitions call the external bin/file through aruba's run command the step definitions work properly and the tests complete as expected but the code coverage is not merged with the rest of the run. What I am seeking are instructions for how to set up simplecov to report the code coverage of the out-of-process tests together with the portions that are run in-process by cucumber directly.
How does one do this?

Answer 1

I have an environment similar to yours, this is how I got it working:
Assuming a directory tree like:
project
|- bin
| |- binary
|- lib
| |- ...
|- spec
| |- ...
|- features
| |- support
| | |- env.rb
| |- ...
Fist check this issue https://github.com/colszowka/simplecov/issues/234
It describes that the binary should start simplecov. It's hackish but I added this header to my binary (project/bin/binary):
if ENV['COVERAGE']
require 'simplecov'
# As described in the issue, every process must have an unique name:
SimpleCov.command_name "binary #{Process.pid}"
# When running with aruba simplecov was using /tmp/aruba as the root folder.
# This is to force using the project folder
SimpleCov.root(File.join(File.expand_path(File.dirname(__FILE__)), '..'))
SimpleCov.start do
filters.clear
# Because simplecov filters everything outside of the SimpleCov.root
# This should be added, cf.
# https://github.com/colszowka/simplecov#default-root-filter-and-coverage-for-things-outside-of-it
add_filter do |src|
!(src.filename =~ /^#{SimpleCov.root}/) unless src.filename =~ /project/
end
# Ignoring test folders and tmp for Aruba
add_filter '/spec/'
add_filter '/test/'
add_filter '/features/'
add_filter '/tmp/'
end
end
Then in the calling of binary inside cucumber the COVERAGE environment variable should be set.
In the feature/support/env.rb at the before clause:
require 'simplecov'
SimpleCov.command_name 'Cucumber'
Before do
# This is using the aruba helper,
# cf. https://github.com/cucumber/aruba/blob/master/lib/aruba/api.rb
set_env('COVERAGE', 'true')
# This could also be accomplished with the "I set the environment variables to:" step
end
If in your environment you have two Frameworks (like RSpec and Cucumber in this example) don't forget to https://github.com/colszowka/simplecov#merging-results

Answer 2

Using the above, setting the command_name based on the PID of the aruba
process causes SimpleCov to accumulate a very large result set and pollutes
the results with old runs.
I had better luck setting the command name as follows:
# As described in the issue, every process must have an unique name:
SimpleCov.command_name ARGV.join(' ')
Which causes only the latest run with the same arguments to get included in
the results.