I am currently working to integrate rcov with our Hudson server.
I am able to run rcov via rake and get the results with out issue (see rake file below). When I run things via Hudson - calling the rake task - the tests with coverage are clearly generated and displayed in the console.
But at almost the end, the following shows up in the console:
** Execute test:coverage
Recording test results
No test report files were found. Configuration error?
Build wasn't successful, skipping rcov coverage report
Finished: FAILURE
Rake task
desc 'Aggregate code coverage for unit, functional and integration tests'
task :coverage => "test:coverage:clean"
%w[unit functional].each do |target|
namespace :coverage do
Rcov::RcovTask.new(target) do |t|
t.libs << "test"
t.test_files = FileList["test/#{target}/*_test.rb"]
t.output_dir = "coverage"
t.verbose = true
t.rcov_opts << '--rails --aggregate coverage.data'
end
end
task :coverage => "test:coverage:#{target}"
end
Going to coverage/ I see output file. The coverage.data file is present as well.
Any thoughts on what might be going wrong here? Am I missing something blindingly obvious?
Hudson will not pull the coverage results if the build has failed further up the chain. I ran into this issue when one test was failing, and could not figure out why coverage was not being reported.
Related
I have situation where I want to compare code coverage of last Jenkins build from current Jenkins build so if current Jenkins build code coverage is low then last one , build should unstable or failed
Is that possible with any Jenkins plugin or do I need to do some stuff with SimpleCov?
You can use the RUbyMetrics Jenkins plugin: https://wiki.jenkins.io/display/JENKINS/RubyMetrics+plugin
along with with simplecov-json and simplecov-rcov gems.
So you need to add this to your rspec_helper:
require 'simplecov'
require 'simplecov-json'
require 'simplecov-rcov'
SimpleCov.formatters = [
SimpleCov::Formatter::HTMLFormatter,
SimpleCov::Formatter::JSONFormatter,
SimpleCov::Formatter::RcovFormatter
]
SimpleCov.start
I have a suite of tests that are launched in continuous integration using parallel test. After the deploys the environment is a little unstable, so when the suite is finished, the failed tests are relaunched (by this moment the environment is stable again).
Basically, there is only one jenkins build, and it will execute this:
task :jenkins1_with_rerun do
selenium_successful = system "bundle exec parallel_cucumber features/web/ -o \"-p jenkins1\" -n 3"
p 'start rerun'
rerun_successful = run_rake_task("features:jenkins_rerun")
unless selenium_successful || rerun_successful
raise 'Cucumber tests failed'
end
end
The first execution launches the suite with parallel tests. Here is the cucumber profile:
jenkins1:
<%= std_opts %>
HEADLESS=true
--tags #part1
--tags ~#wip
--format junit
--out log/junit
--format ParallelTests::Cucumber::FailuresLogger
--out parallel_cucumber_failures.log
After this is finished. The second execution starts, launching the failed tests recorded in a file. The cucumber profile for this execution is this:
jenkins_rerun:
ENVIRONMENT=master
SCREENSHOT=true
HEADLESS=true
--format pretty
--no-source
--format junit
--out log/junit_rerun
#parallel_cucumber_failures.log
Now. Everything works. The only problem is that I use the junit report in jenkins to make graphs of failed and successful tests. I may have reports like this:
-------------------Tests---KO---OK
First execution---75----10----65
Rerun-------------10-------0----10
This is 100% green, because all the problems were caused by instability after the deploy. So I want the junit report in jenkins to said that 75 tests have been launched, 75 OK, 0 KO, or something equivalent.
Right now, the junit report of the first execution says that of 75 tests, I have 10 KOs; and the second junit says that of 10 tests, there are 0 KOs.
What may be a good solution to this? It possible to mix the results of both Junit reports?
I will also accept to be able to display in jenkins both junit reports, each one with a graph. But I think that jenkins only allows to show one junit report graph.
I finally resolved this problem.
The solution was to create a "parser" that searches (with RegEx) through the Junit files generated by the rerun, and then goes to the Jnuit files of the original/first run and rewrites the results of those tests that have passed during the rerun.
In the end, if everything is ok in the rerun, the junit graph is 100% green and there are not alerts of failures.
I was wondering whether it's possible to obtain information about test execution order somehow.
I have a project in Maven and all the tests are passing. After I migrated the project to Gradle, one of the tests started failing. The test itself is working: when I execute gradle test -Dtest.single=..., it passes. However, when I run the tests for the whole project, the test fails.
It's possible that some tests that run before the failing test do not release resources correctly and therefore the test fails. But I need to somehow find out which tests are causing this problem.
Tell Gradle to log more events about test processing. There is a documentation how to do that http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.testing.logging.TestLoggingContainer.html
The beforeTest method can be used to output details about each test before it is run, based on a TestDescriptor for each test:
test {
beforeTest { testDescriptor ->
println "${testDescriptor.className} > ${testDescriptor.name} STARTED"
}
}
I am trying to use the simplecov gem in a Ruby project. However I have failed miserably. Here is what I've done until now.
My project structure is:
ProjectFolder
- lib
- test
I have my tests in test and source code in lib. I have created a test_helper.rb in the test directory and added the following.
require 'simplecov'
SimpleCov.start
Then I put `require 'test/test_helper.rb' in every test file. What happened was it sometimes created some report, and sometimes didn't. And when it did it was inconsistent.
All the tutorials I found was for Rails so I turn to StackOverflow once more, to show me the way.
Rcov/SimpleCov will only report the coverage of the tests ran.
For a full coverage report, you must ensure that the full test suite is ran as the last test, to build a full coverage report.
You will also want to make sure it is the first require in your test_helper.rb file.
From the documentation:
Note: If SimpleCov starts after your application code is already
loaded (via require), it won't be able to track your files and their
coverage! The SimpleCov.start must be issued before any of your
application code is required!
I've started with watir and cucumber a while ago and have been trying to run my project with Jenkins. I have a watir Project (watir,cucumber) in C:\project_name. I have rake file in the project also and when I run it in CMD everything goes ok and tests are succesfully executed --> C:\project_name>rake
Rakefile I'm running looks like this.
require 'rubygems'
require 'cucumber'
require 'cucumber/rake/task'
Cucumber::Rake::Task.new(:features) do |t|
t.profile = 'default'
end
task :default => :features
I have tried to execute Rake file with Jenkins but unable to make it work. So far what I have done with Jenkins (Running in localhost:8080)
1) Plugins installed eg. cucumber-reports, Jenkins Rake plugin
2) Tried to add New job --> Build a free-style software project
I really don't have idea how to make Rake file to work with Jenkins. I was hoping to get these nice reports (cucumber-reporting) available through Jenkins.
If someone can help me with this it would be more than perfect.