Currently, we are using Sonar to check the quality of our production code. I would Like to check the quality of the test Code either.
How can I do it ?
I tried to change the "sonar.sources" to include test, but in this case i have an error : test folders are defined twice, one in test and one in source.
May be I can configure sonar / maven / jenkins to run 2 analysis : first one, the src code, with the test coverage, thant the tests (only quality, no coverage). But I also need to have everything run in a single jenkins job and displayed in a single Sonar Project.
And I don't know where to configure (pom ? sonar ? jenkins ?) I found some things about profiles, but it seems to be deprecated for my version.
I'm using Sonar 5.1, run from a jenkins job.
our Sonar properties in the pom are :
<project.testresult.directory>${project.build.directory}/test-results</project.testresult.directory>
<run.addResources>false</run.addResources>
<sonar-maven-plugin.version>2.6</sonar-maven-plugin.version>
<sonar.exclusions>src/main/webapp/assets/**/*.*,
src/main/webapp/bower_components/**/*.*,
src/main/webapp/dist/**/*.*</sonar.exclusions>
<sonar.jacoco.itReportPath>${project.testresult.directory}/coverage/jacoco/jacoco-it.exec</sonar.jacoco.itReportPath>
<sonar.jacoco.reportPath>${project.testresult.directory}/coverage/jacoco/jacoco.exec</sonar.jacoco.reportPath>
<sonar.java.codeCoveragePlugin>jacoco</sonar.java.codeCoveragePlugin>
<sonar.javascript.jstestdriver.reportsPath>${project.testresult.directory}/karma</sonar.javascript.jstestdriver.reportsPath>
<sonar.javascript.lcov.reportPath>${project.testresult.directory}/coverage/report-lcov/lcov.info</sonar.javascript.lcov.reportPath>
<sonar.sources>${project.basedir}/src/main/</sonar.sources>
<sonar.surefire.reportsPath>${project.testresult.directory}/surefire-reports</sonar.surefire.reportsPath>
<sonar.tests>${project.basedir}/src/test/</sonar.tests>
In the latest versions of the Java plugin there are some rules to check tests, but they are test-specific rules. I.E. they check if your #Test methods contain assertions &etc. You simply need to add those rules to your profile.
If, however, you're talking about running "code rules" on your tests, then your best bet is probably to define another project where the tests are treated as sources. Since you've already noted the difficulties of doing that with Maven, I would use SonarQube Scanner for this separate, second analysis.
Related
I'm using Gradle sonarqube plugin and I need to exclude all test sources from the sonar analysis (main goal is to filter out unit test classes from the issues/code smells report)
To do so, I have used the dedicated sonar.test.exclusions property as follow, to exclude the whole src/test directory from analysis
sonarqube {
properties {
property("sonar.exclusions" , "")
property("sonar.test.exclusions" , "src/test/**/*.java")
// other sonar properties, omitted
}
}
This works as expected (test sources are filtered out) BUT : when this property is set, sonar does not compute/report number of unit tests correctly.
See simple example for a very basic project: 2 main source files, 1 test source file containing 2 Junit tests (and also containing some issues I don"t want to see in report)
Without exclusions:
Sonar properly reports my 2 unit tests, but it also includes code smells from the unit test class
With exclusions:
Now, code smells from the unit test are properly filtered, but I lost the Unit test count information
Notes:
using Gradle 6.7, sonarqube plugin version 3.0, and sonar server Community EditionVersion 8.4.2
also tried with the property sonar.exclusions : same problem
all other sonar properties are correctly set and have same values in both scenarios : specially sonar.tests, sonar.java.test.binaries, sonar.junit.reportPaths, sonar.jacoco.reportPath
Any idea how to configure the sonarqube plugin, to exclude properly test sources, while keeping Unit tests information available?
I agree with Chriki's answer but only the first part : using sonar.test.exclusions is not the good way.
I disagree with the last part : using sonar.issue.ignore.multicriteria is totally possible with Gradle
https://www.baeldung.com/sonar-exclude-violations#using-sonar-projectproperties
Try something like this (not tested from my end though) :
sonarqube {
properties {
property "sonar.issue.ignore.multicriteria", "e1"
property "sonar.issue.ignore.multicriteria.e1.resourceKey", "src/test/java/**/*"
property "sonar.issue.ignore.multicriteria.e1.ruleKey", "*"
}
}
What you’re after doesn’t seem to be possible from a Gradle configuration. Let me elaborate on how I came to that conclusion.
In a (admittedly very) old thread on the Sonarqube mailing list from 2013, somebody asked the same question (albeit for Maven). A Sonarqube consultant has answered it as follows:
When you exclude a file (or a test file) with sonar.exclusions (or sonar.test.exclusions), the file is just ignored by SonarQube. The source code is not imported, no metrics are computed on this file (such as number of tests).
To ignore some specific issues on specific files, see http://docs.codehaus.org/display/SONAR/Narrowing+the+Focus#NarrowingtheFocus-IgnoreIssues. To ignore some files to be taking into account for code coverage computation, see http://docs.codehaus.org/display/SONAR/Narrowing+the+Focus#NarrowingtheFocus-IgnoreCodeCoverage and so on.
The current documentation link corresponding to the first (now broken) one from the quote is this: https://docs.sonarqube.org/latest/project-administration/narrowing-the-focus/#header-3 (or for version 8.4) It’s about ignoring “issues on certain components and against certain coding rules” – which is what you’re after, if I’m not mistaken. However, these docs state right at the beginning:
Note that the properties below can only be set through the web interface because they are multi-valued.
Here “web interface” is meant as opposed to a (Gradle) build configuration for example. In fact, the previously mentioned Sonarqube consultant explicitly states this for Maven, too:
The property is sonar.issue.ignore.multicriteria. I think I read somewhere that you have to do these through the GUI because they are multi-valued? Is that the case? If not, how would I write it as an entry in the pom.xml? We like to keep all config in the one place (pom.xml).
It's unfortunately not possible.
The mailing list discussion is a bit longer and gives more insight, so it may be worth a read in full. It also sheds some more light on the sonar.test.exclusions property for which I could otherwise not find any good documentation.
I have a Spring project (Apache CXF, Spring, Hibernate, Maven ...) hosted on BitBucket and I'm trying to use Bamboo as my CI server. My idea is deploying the code directly to Heroku from Bamboo so that deploying time is automated.
I made a plan with a couple of tasks to achieve this. First I have a Source Code Checkout task and a builder task. Both of them are working, code is compiling and test are passing, I can see that in the task log. The problem is that Bamboo doesn't seem to recognize the tests (it marks the task are testless).
I have also tried to create a new JUnit test task and it's even worst. Log shows that everything is working properly but Bamboo marks the plan as a failure after the test task is executed.
Any ideas?
Not sure which version of Bamboo you're using, but in the version that we have, you have to turn on unit test result evaluation on the Builder tab. Please see the attached screenshot, and make sure that this is enabled, and the directory setting is pointing to the directory where Maven Surefire creates the test results (in XML format).
I'm trying to decide how to create a set of Acceptance Tests for a Java-EE web application.
Here's the setup: Maven is used to generate a WAR file and deploy it into Glassfish. On deployment, the MySQL database schema is automatically updated from model classes using Hibernate ("hbm2ddl=auto" option).
The Acceptance Tests need to test the deployed code by invoking various methods and checking the results are as expected(*). We wrote an additional set of packages to hook into an existing system so the Acceptance Tests should show how these can be integrated into the existing codebase.
(*) This may sound more like Unit/Integration Testing but they are Acceptance Tests in the sense that they should prove what we did works and they need to be run after deployment so there is a database in place.
From the above, my current thinking is to use JUnit to check expected values etc. The bit I'm struggling with is how to invoke these tests after deployment. "deploy" is Maven's last phase so not sure if this is possible?
Just because that phase is called deploy doesn't mean that you have to use it for deploying your application for testing. In fact, it should only be used for "deploying" the artifact to a maven repository. Read through the description of the Maven lifecycle phases and you'll see that there are some phases dedicated to your use case:
pre-integration-test
integration-test
post-integration-test
Have a look at the Cargo Maven plugin. It's made to deploy your WAR file to various containers for testing. They definitely show demos of use cases like the one you describe on your site. I would expect that ultimately, you can be using Cargo to deploy to your container ( from one of the earlier phases like pre-integration-test )
Note, Jenkins also has a plugin that is a wrapper around the Cargo plugin. So you might do what you need via Jenkins. Also note, you don't need to run your Jenkins build job as mvn clean deploy. You could have one build job that just runs the integration tests, and fires another "deploy" job only when it succeeds.
If you really need to do stuff after deployment, then you can either run failsafe, and by implication JUnit) as part of the deploy phase.
What I usually do, if to have seperate module. So, you can have one maven project, which contains your project and a separate 'deployment test' project. Then, building the parent project will build and run your war and then run the deployment tests. You can use junit as normal.
The second fits better into jenkins because you'll still have a single project as well.
I'm using Maven 2 w/ cargo to deploy to different remote tomcats depending on the maven profile used.
I also have integration tests (junit w/ remote webservice calls) that I would like to automatically run.
Question: How do I setup my test code to be able to read either the cargo.hostname (preferred, changed property value based on maven profile) or the maven profile actived so it knows which url to go run the tests against?
Specifically, this is in java in the test case.
Thanks!
Either you define a fixed value for the cargo.hostname (not the best; but sometimes it workds well, cause using a fixed test server) or better define an appropriate property in Maven for it and put the information also into a properties file which will be filtered by the build process in the src/test/resources folder which can be read before the real integration tests.
I want to launch SonarQube analysis with Jenkins for a Maven 2 project. I first used the goal sonar:sonar in the build configuration.
But I just found the SonarQube plugin for Jenkins. Why use it? Is it a better practice and why?
You can of course do a simple mvn sonar:sonar, this will work.
On the other side, there's the SonarQube plugin for Jenkins that will make the configuration easier. For instance you will be able to define information about your SonarQube server (URL, DB user and password) or your multiple SonarQube servers in a single place (the configuration section of Jenkins) so that you don't have to repeat it everywhere.
The plugin also offers the ability to run a SonarQube analysis on the fly (without Maven): you just have to provide some mandatory properties (like sonar.projectKey and sonar.projectVersion for instance) and the plugin will start the Java Standalone Runner transparently for you (this is helpful mostly for other languages than Java which don't rely on Maven for their build).
So if you're just making some tests, you don't really need this plugin. But if you're setting up a production instance of Jenkins, then it's best to use the SonarQube plugin.