I have set up SonarQube, Maven and Cobertura to upload unit test coverage at every build run. I want to use the differential views to get coverage for the delta (new code only).
The coverage itself is successfully uploaded, and I can see the delta in lines of code. However, I can't get the "Coverage on new Code" to work.
I tried different values for the leak period, including previous_analysis and previous_version (changing the version from 1.0 to 1.1 in the new analysis). I also used -Dsonar.projectDate to simulate a past date of analysis but still no luck.
Any thoughts?
The key requirement to obtain new_code -related information/metrics is to leverage SonarQube SCM integration. And to benefit from that you have to install a compatible SCM Plugin applicable to your project (e.g. Git Plugin, SVN Plugin etc.).
"Coverage on New Code" is only displayed if you are using/activating SCM support (SVN, Git, ...).
sonar.scm.disabled=false
Adding this sonar.java.binaries=classes directory, most likely target/classes
helped me in fixing this issue. No JaCoCo analysis of project coverage can be done since there is no class files
Related
I'm using gitlab-ci pipeline, it will run a new docker container with following commands:
mvn --batch-mode verify sonar:sonar
-Dsonar.analysis.mode=preview
-Dsonar.gitlab.project_id=$CI_PROJECT_ID
-Dsonar.gitlab.commit_sha=$CI_COMMIT_SHA
-Dsonar.gitlab.only_issue_from_commit_file=true
-Dsonar.gitlab.ref_name=$CI_COMMIT_REF_NAME
-Dsonar.host.url=xxx
-Dsonar.test.inclusions="/src/test/java/**/*.java"
-Dsonar.login=xxx
It becomes quite slow for my project, almost run for 20mins.
I found it will scan all files instead of only the commited files.
Is there anything wrong with my configurations?
In some past SonarQube versions we changed the preview mode to only scan changed files. But it has some drawback, like not being able to properly detect cross file issues.
In recent SonarQube versions, the preview mode is deprecated in favor of pull request analysis. But even this PR analysis feature is scanning all files, for the same reasons (cross file issues, coverage measures, duplication detections, ...).
We stopped trying to do partial analysis, and are instead trying to optimize full analysis duration. How big is your project? Is SonarQube analysis time long compared to your regular build (compile + tests)? If yes, then I suggest you report you case on the SonarSource community forum.
I recently started using SonarQube on my Maven Java Project. The problem is that I modified a single line in a Class which causes Sonar failing to load the project into the database. Why does Sonar know that my project is checked into a SVN and why does mvn sonar:sonar fails with "can not blame XXX on line YYY" if a class is not committed into the SVN?
I am the only one working on this and I dont want to check in every single change or experimental code snipet. How can I turn this feature off?
Thank you!
SCM integration (which detects that your projects uses SVN) allows SonarQube to track changes to source code, e.g to compute coverage on new code, new issues etc.
A standard analysis is supposed to be performed from commited code (e.g by a continuous integration server) to serve as a reference.
In your case, I would advise you to use the incremental mode: this will allow you to perform an analysis on your local code changes, which will not be persisted to the central server, but which will show you which issues your new code introduced/fixed (this is the analysis mode used by the IDE integration plugins).
I am analysing a Java project that has been unit tested and Cobertura coverage.xml reported. I am using SonarQube 4.1.1 and latest Sonar Runner. I have successfully imported Cobertura coverage results to Sonar 3.2 and Ant analyzer, but with this new version I am running into problems. In the new Sonar analysis execution (through Jenkins) I see no reference in logs that it would have started any Cobertura engine or anything. My settings in Runner Jenkins project:
sonar.dynamicAnalysis=reuseReports
sonar.java.coveragePlugin=cobertura
sonar.cobertura.reportPath=[mypath]/coverage.xml
sonar.junit.reportsPath=[mypath]/
No mentioning of Cobertura in the analysis output (except my own property values) and SonarQube page shows "-" in coverage report. Unit test results are shown fine.
I have also added all source, bin, and test directories. Any ideas? Thanks.
Update
I wonder if the reason why Cobertura coverage is not reported on SonarQube page, is because in Jenkins my SonarQube project clones (Clone plugin) the workspace from a previous Project build? If the coverage.xml file contains static paths, then maybe it goes wrong somehow.
Ok the solution was quite obvious. SonarCube 4.1.1 does not come with Cobertura preinstalled, so I installed it and now it works :) Maybe it was preinstalled in 3.2 version, I can't remember.
For any "reuse reports" feature in SonarQube (may it be for code coverage, test execution or any other third-party tool that generates a intermediate report), the report must be generated based on the same source files (in terms of file system location) than the ones that will be used for the SonarQube analysis. Otherwise the SonarQube plugins won't be able to match paths from the reports with paths of the analyzed sources.
So you guessed it right: cloning the workspace and running the SonarQube analysis on this clone is the reason why the coverage can't be computed.
The root cause of this problem is that SonarQube does not support Cobertura format and you need OpenCover, or dotCover format. Using reportgenerator is one of the solutions but it's unnecessary complicated: you can make "dotnet test" command to return the report in Opencover format by passing special parameter and then pick up the Opencover report.
Use
dotnet test --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura,opencover
to get two reports: one in OpenCover format for SonarQube and default Cobertura for Azure DevOps.
Now you can specify the path to OpenCover report in SonarQubePrepare task.
There is also detailed article on this here
Does Sonar offer any way to raise alerts and fail a build when the trend for certain metrics is bad?
Background: In our legacy project using a static threshold for example for code coverage ("red alert when coverage is below 80%") does not make much sense. But we would like to make sure that the coverage does not go down any further.
Please do not give any advice on lowering the bar by using a less restrictive rule set. This is no option in our case.
There is a build breaker plug-in that will fail the build if you breach a Warning or Error threshold setup in the quality profile.
Plug-in details are here:
http://docs.sonarqube.org/display/PLUG/Build+Breaker+Plugin
Not aware of any functionality that enables you to a metric trend.
We use Sonar as the second last step in our release process. The build breaker ensures that releases do not breach predetermined quality criteria.
We tried exactly the same, using the build breaker plugin. After a while, it showed to be too unflexible (and configuring Sonar is a mess), so we moved from sonar to Jenkins/Hudson plugins like Cobertura (for code coverage) or PMD for code style:
https://wiki.jenkins-ci.org/display/JENKINS/PMD+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/Cobertura+Plugin
With these plugins, very fine-granular settings are possible, to set for example the build to yellow at <70% code coverage or to red by <50%; even the weather-symbol for each build is setable.
In the meanwhile we scripted our own buildbreaker that gets excecuted within our build. We use Groovy to query the REST API of Sonar to retrieve a certain set of metrics (including their historical values). The retrieval of metrics is provided by a build plugin that is provided for our whole division.
Each team can parameterize their build with a set of rules regarding those metrics that have to be verified for their project. Of course, the rules are also provides as Groovy snippets :-)
Typical are:
Number of (major|critical|blocker) violations is less or equal than in previous build
No new duplicates
Coverage not lower than in previous build
Bad findings can then be used for breaking the build or just for reporting.
I've seen a number of posts on making a Maven-backed Jenkins build fail for a given project if a coverage threshold isn't met i.e. coverage must be at least 80% or the build fails.
I'm wondering if there is a way to configure Jenkins to fail a build if the coverage is lower than the last build i.e. if the coverage for build N is 20%, and N+1 is 19%, then the build fails. I don't want to put in an explicit threshold, but I want the coverage to stay steady or get higher over time.
I haven't tried it, but assuming you are using maven cobertura plugin, I believe it can be configured to fail as documented here. Would jenkins not honour the failure?
I also see an open feature request for this.
Save last coverage percentage on the side. Use Groovy script (run via Groovy Plugin) that taps into Jenkins Java API to compare current coverage level with previous one and fail the build if it drops. If you seriously decide to go this way I can provide more details and code samples.
Since the 1.6 version of Cobertura plugin, the issue has been fixed, and a new option appeared that allow the plugin to make the build failed if coverage is lower than previous build.
This should be configurable in the configuration section of the plugin, that calculates the coverage. E.g. for the PMD plugin you can configure limits for unstable or failed build on the Job configuration page in the section "Build settings".
In NCover I specify a certain percent through a config file. You could create a python script (or similar to alter that value if you wanted). Other than that I do not think there are any config variables