My Maven project has multiple surefire executions configured to make sure different test groups get rund with slight modifications to the classpath etc. (see the pom.xml). When running the build job, the tests are executed as expected.
The build job running the Maven Sonar plugin only runs the default test execution and skips the other test configurations which effectively leads to Sonar reporting 0% test coverage. How can I tweak the Maven Sonar plugin to not skip the additional test executions. The relevant section from the log is here, the full log here.
I'd be interested in why the additional executions are skipped as well.
The objective of SonarQube is not to run your unit tests, especially when the configuration is not standard. For you information, our long-term goal is to remove such a feature and only allow one mode: reuseReports.
Therefore you should run your unit tests first and reuse those generated reports while running the SonarQube analysis. See http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Unit+Tests+for+Java+Project#CodeCoveragebyUnitTestsforJavaProject-ReusingExistingReports.
Related
I use Maven to build a multi-component project. One of the components is a web app with HTML and JavaScript. Maven invokes NPM and Karma to run that component's unit and integration tests, written in QUnit. This yields two independent Cobertura coverage reports that I then combine into one with istanbul-combine.
As a final step, I'd like the Maven build to fail if that combined Cobertura report doesn't satisfy my coverage requirements. But how?
I've tried the Cobertura Maven Plugin, but it seems to be unfit to simply read an existing report and apply its check configuration. First, it needlessly repeats the build's tests ("Invokes the execution of the lifecycle phase test prior to executing itself."), then it finds that it cannot instrument the non-Java resources, then it simply stops, without any log output whatsoever.
Are there other plugins that fit? Could changing the report format help? (Doesn't have to be Cobertura.)
In my company we develope eclipse plugins continuously built by a Jenkins CI Server (with tycho & maven) which also executes the tests (Unit Tests and SWTBot Tests). As a post build action we send the test data (JaCoCo & Surefire reports) via maven to a SonarQube server for analysis. Our sources and tests are each located in an own module.
In src/com.mycompany.projectA/ we have the source from Project A.
In test/com.mycompany.projectA.tests.ut/ we have the unit tests from Project A.
In test/com.mycompany.projectA.tests.swtbot/ we have the swtbot tests from Project A.
In order to allow Sonar to find the Jacoco and Surefire reports for the tests of each source plugin we set the report paths in our pom.xml:
<sonar.jacoco.reportPath>${project.basedir}/../../test/${project.artifactId}.tests.ut/target/jacoco.exec</sonar.jacoco.reportPath>
<sonar.surefire.reportsPath>${project.basedir}/../../test/${project.artifactId}.tests.ut/target/surefire-reports</sonar.surefire.reportsPath>
But this only allows one test per plugin. To also see the SwtBot test coverage in SonarQube, we added the SWTBot tests as integration tests (yeah I know, not very neat...)
<sonar.jacoco.itReportPath>${project.basedir}/../../test/${project.artifactId}.tests.swtbot/target/jacoco.exec</sonar.jacoco.itReportPath>
Like this, we can at least see the unit test coverage as well as the SWTBot test coverage in SonarQube. But as we are only able to set one report path for the surefire reports, those are ignored by Sonar and we can not see how many SWTBot test passed/failed.
Is there a convenient way to add multiple locations of surefire reports to be considered in the analysis?
This is not possible yet, but there's a ticket open for that: https://jira.codehaus.org/browse/SONAR-4101
Feel free to watch it and vote for it.
Maybe you can append the jacoco.exec of each project together and put the integrated jacoco report to Sonar. The following link is a good example to integrate the jacoco report.
http://www.lordofthejars.com/2012/07/jacoco-in-maven-multi-module-projects.html
I have a large build where tests sometimes prevent creation of artifacts because they stall or terminate the build due to make module timeout. I'd like my build server to:
build and publish artifacts
run unit / integration tests
At present unit and integration tests execute during each module's test and integration test phases in the typical maven way. This is great for developers but places risk of lack of build. The build server can easily setup a build/publish step by skipping the tests and I'd like to run a testing only test that skips process-resources through package phases just running the tests. However, I'm fairly certain that maven will traverse all the previous stages again.
I'd like to avoid lengthening an already quite long build by needlessly rebuilding and repackaging my artifacts.
Has anyone split a maven build like this so the desktop developers get the benefit of test integration and the build server / continuous integration system get's the benefit of pushing risky tests to the end of the process so we always produce artifacts?
Am I trying to break the build up in the wrong fashion and should consider changing a monolithic end-to-end build in a series of smaller component builds?
Thanks
Peter
Here's how you can do this:
You can run maven and skip all tests to install/deploy your artifacts
Then you can run it again but this time only run the tests
Like so:
mvn install -DskipTests
mvn surefire:test failsafe:integration-test
If you have to do this all on one command line line, I'm sure you can figure something out by having two profiles. One that skips tests and one that only runs tests.
You could put the tests into their own modules which depend on the modules they test then use profiles to include/exclude the test modules as desired.
Does sonar analyse code in test directory of Maven project ?
In the project I'm working on I don't think it does but I'm unsure if this has been turned off or is enabled by default ?
for sure not every metrics is applied to tests. sonar counts number of tests (but it's probably from junit's report) but checkstyle etc will not analyse tests - unfortunately. at least not with default settings
Actually Sonar checks unit test files against PMD unit test rules and of course creates violations on test files if any of them(rules) are broken. By default, however, these rules are disabled so you need to enable them in profile administration
I read Maven Failsafe plugin is designed specifically to run integration tests. Currently I'm working on a multi-module project and integration tests are in its own separate module, written in TestNg and run using Surefire plugin. We don't have conflicts with unit tests since only integration tests are run in the test phase in that module. And to set up the environment before the tests, and clean it after tests are run, #BeforeSuite #AfterSuite TestNg annotations are used. So there's no need to make use of pre-integration-test phase, integration-test phase, post-integration-test phase utilized by Failsafe plugin.
Are there any more benefits I'm missing out on, by not using the Failsafe plugin?
Are there better ways to do my current requirement using Failsafe plugin?
Can I do my server startup, shut down, file unzipping etc. in the pre-integration-test, post-integration-test phases without writing a maven plugin?
If you already have your own test setup/teardown in your suites, which from the looks of it you do, there is not much you can gain from the FailSafe plugin.
The FailSafe plugin is useful in situations where the Setup of your System Under Test is costly or takes a long time such as starting up a Servlet or a distributed system. The way the FailSafe plugin comes handy in these situations is that you can set up this environment in the pre-integration-test phase. This plugin also doesn't stop the execution of the Maven build when a test fails, which allows you to clean up all of your artifacts during the post-integration-test phase, after which it checks the status of your tests and passes or fails the build accordingly during the verify phase.
Failsafe has one big feature vs Surefire: When a test fails, it does not immediately abort. Instead it lets the clean-up code run (which typically takes down the Jetty server).
Addressing your third question as it isn't really answered, imho.
Can I do my server startup, shut down, file unzipping etc. in the pre-integration-test, post-integration-test phases without writing a maven plugin?
Taken from this answer to "Maven Failsafe Plugin: how to use the pre- and post-integration-test phases"
It boils down to: pre-integration-test and post-integration-test do nothing per default. You can bind a plugin specific for your task to those phases. Finding a specific plugin depends on what you're trying to do.
Another important thing to point out is default naming conventions used by the maven-failsafe-plugin: It runs test-classes with names starting or ending with IT (as integration test class)