I am calling Sonar from my Jenkins job
I noticed that the coverage of the classes in my domain module all show as having 0% test coverage even though the domains are being used in the other modules which have high test coverage. I am using cobertura within Sonar for measuring the coverage
Can anyone offer any insights into how to get Sonar/Cobertura to recognise the test coverage for the classes in the domain module
Thanks
Damien
By default, Sonar computes unit test coverage. So if your domain classes are not unit-tested in the module where they are defined, this is normal that you don't get unit test coverage for those.
You can have a look at how to add integration test coverage in Sonar: see https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/combined%20ut-it/combined-ut-it-multimodule-maven-jacoco
But that'd be best to have "real" unit tests for your domain classes to test their logic in an isolated context.
Edit
Link to sonar-exmaple-multiple-modules
Related
Scenario
Compile code & run unit tests
Execute sonar scan on code
Once the scan completes, code coverage for unit tests + other analysis gets stored in sonar (lets call report1)
All good till now. Now once the application is deployed, we are run some integrations tests and manual tests are also performed.
All these tests will generate code coverage.
Now we have to merge unit test, integrations & manual tests code coverage and store it on sonar under same analysis report (basically update report1)
Any thoughts/suggestions for best practice or solution for same on sonar?
It's pretty simple. Don't run the SonarQube scan until you've produced all the data you want to integrate into the report.
However, I would also point out that it's generally not worthwhile to generate code coverage for anything but unit tests. If you can't reach it with a unit test, I don't see how you could reach it with any other test.
We have a large Maven project with about 50 modules. Today we have a good setup where we get a combined JaCoCo coverage for the whole project. We also use the final binary generated by JaCoCo for custom coverage logic.
We would like to implement a more strict coverage policy where only tests in the same module contribute to the overall coverage. We want every module to be responsible for its own tests and coverage.
For example. Module A depends on module B. A has code that makes use of code in B. When tests in A is run, we do not want to have coverage from any class in B recorded.
Is this setup possible? Trying around, I find that JaCoCo has many options for filtering when generating the final report, but I'd like to do this in the first step, when the coverage is collected.
Eventually, I'd like to end up with a JaCoCo exec file in each module's target directory that only has classes recorded from its own module. It would be perfectly fine if we had to do some configuration on a module-by-module basis like having an explicit package expression for inclusion.
Trying around, I find that JaCoCo has many options for filtering when generating the final report, but I'd like to do this in the first step, when the coverage is collected.
There is also options for filtering during collection - see includes and excludes options of JaCoCo Java Agent and corresponding options of prepare-agent in jacoco-maven-plugin.
Build of JaCoCo itself uses them exactly for this.
I'm trying to get separate Jacoco coverage figures in SonarQube for unit & integration tests.
The docs describes using 'sonar.jacoco.itReportPath', to distinguish integration tests, but it's deprecated and if I use the new 'sonar.jacoco.reportPaths' then it all shows up as unit tests.
How do you get separate coverage figures in a non-deprecated way?
Actually I've just found an Jira Feature via Release Notes for Version 6.2, basically you can't get separate coverage metrics for unit and integration tests anymore.
Of course I've spent half the day digging in to documentation and only just found it minutes after posting the question :D
I have collected jacoco code coverage on integration test cases. Now I want to upload integration code coverage in Sonar 6.3. Since these are integration coverage results, source code is not available. Is it possible to show integration code coverage results into Sonar by any api/any other methos. If it possible, please share the relevant documentation. It would help me a lot.
You cannot update an existing analysis. Your only option is to run the integration tests before analysis, and feed the report into the analysis.
How to visualize Integration Tests in Sonar ?
Currently I only see :
the global coverage (UT + IT)
the number of UT
I read somewhere we had to configure a widget in the GUI : I didn't see any option which could do that.
Documentation states :
If coverage by unit tests is not computed with JaCoCo, overall
coverage = integration test coverage.
But in my case I see that the coverage is changing when I change my UTs (or ITs). Moreover I see jacoco reports in the targets :
jacoco.exec
jacoco-it.exec
I finaly tryed the official Sonar samples : it is the same ! I didn't find any sample with a clear separation between :
Unit test coverage
Integration test coverage
Overall test coverage
And sorry but Sonar documentation and samples have to be improved...
Context : sonar6.2, java8, spring boot, modular project, maven, surefire & failsafe
Starting in SonarQube 6.2, all test results are merged into simply "coverage". This was done on the theory that by and large, most people don't care how their code is covered, only that it is covered.
With this change, some math inconsistencies w/r/t how Overall Coverage was calculated from unit tests versus integration tests were eliminated, and additionally the ability to feed many different coverage reports was added. (Some people have unit, integration, smoke, ... tests).
Regarding
I read somewhere we had to configure a widget in the GUI : I didn't see any option which could do that.
In 6.2 dashboards were dropped, so there are no widgets to be configured. Coverage is shown automatically on the project home page.
Merging unit testing coverage and integration testing coverage numbers is not a good idea.
Modern unit tests that mock out all dependencies and environmental factors and have been proven to run on a developers local machine cannot by definition fail in a continuous integration environment. This makes them next to useless to run in a CI environment. Therefore, 100% unit test coverage but 0% integration test coverage means no meaningful testing is actually happening in the CI build.