I'm trying to get separate Jacoco coverage figures in SonarQube for unit & integration tests.
The docs describes using 'sonar.jacoco.itReportPath', to distinguish integration tests, but it's deprecated and if I use the new 'sonar.jacoco.reportPaths' then it all shows up as unit tests.
How do you get separate coverage figures in a non-deprecated way?
Actually I've just found an Jira Feature via Release Notes for Version 6.2, basically you can't get separate coverage metrics for unit and integration tests anymore.
Of course I've spent half the day digging in to documentation and only just found it minutes after posting the question :D
Related
Generally we find code coverage with unit tests using Jacoco and unit test cases however My Dev manager asking if we can use the existing functional automation tests to find the code coverage of the spring boot application. My functional tests using testng/restassured and stored in a separate code repo, not with in development source code repo.
Please suggest how can be this done if it is really possible. Appreciate if anyone can guide and share links for reference.
Thanks.
Scenario
Compile code & run unit tests
Execute sonar scan on code
Once the scan completes, code coverage for unit tests + other analysis gets stored in sonar (lets call report1)
All good till now. Now once the application is deployed, we are run some integrations tests and manual tests are also performed.
All these tests will generate code coverage.
Now we have to merge unit test, integrations & manual tests code coverage and store it on sonar under same analysis report (basically update report1)
Any thoughts/suggestions for best practice or solution for same on sonar?
It's pretty simple. Don't run the SonarQube scan until you've produced all the data you want to integrate into the report.
However, I would also point out that it's generally not worthwhile to generate code coverage for anything but unit tests. If you can't reach it with a unit test, I don't see how you could reach it with any other test.
I have collected jacoco code coverage on integration test cases. Now I want to upload integration code coverage in Sonar 6.3. Since these are integration coverage results, source code is not available. Is it possible to show integration code coverage results into Sonar by any api/any other methos. If it possible, please share the relevant documentation. It would help me a lot.
You cannot update an existing analysis. Your only option is to run the integration tests before analysis, and feed the report into the analysis.
How to visualize Integration Tests in Sonar ?
Currently I only see :
the global coverage (UT + IT)
the number of UT
I read somewhere we had to configure a widget in the GUI : I didn't see any option which could do that.
Documentation states :
If coverage by unit tests is not computed with JaCoCo, overall
coverage = integration test coverage.
But in my case I see that the coverage is changing when I change my UTs (or ITs). Moreover I see jacoco reports in the targets :
jacoco.exec
jacoco-it.exec
I finaly tryed the official Sonar samples : it is the same ! I didn't find any sample with a clear separation between :
Unit test coverage
Integration test coverage
Overall test coverage
And sorry but Sonar documentation and samples have to be improved...
Context : sonar6.2, java8, spring boot, modular project, maven, surefire & failsafe
Starting in SonarQube 6.2, all test results are merged into simply "coverage". This was done on the theory that by and large, most people don't care how their code is covered, only that it is covered.
With this change, some math inconsistencies w/r/t how Overall Coverage was calculated from unit tests versus integration tests were eliminated, and additionally the ability to feed many different coverage reports was added. (Some people have unit, integration, smoke, ... tests).
Regarding
I read somewhere we had to configure a widget in the GUI : I didn't see any option which could do that.
In 6.2 dashboards were dropped, so there are no widgets to be configured. Coverage is shown automatically on the project home page.
Merging unit testing coverage and integration testing coverage numbers is not a good idea.
Modern unit tests that mock out all dependencies and environmental factors and have been proven to run on a developers local machine cannot by definition fail in a continuous integration environment. This makes them next to useless to run in a CI environment. Therefore, 100% unit test coverage but 0% integration test coverage means no meaningful testing is actually happening in the CI build.
I am calling Sonar from my Jenkins job
I noticed that the coverage of the classes in my domain module all show as having 0% test coverage even though the domains are being used in the other modules which have high test coverage. I am using cobertura within Sonar for measuring the coverage
Can anyone offer any insights into how to get Sonar/Cobertura to recognise the test coverage for the classes in the domain module
Thanks
Damien
By default, Sonar computes unit test coverage. So if your domain classes are not unit-tested in the module where they are defined, this is normal that you don't get unit test coverage for those.
You can have a look at how to add integration test coverage in Sonar: see https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/combined%20ut-it/combined-ut-it-multimodule-maven-jacoco
But that'd be best to have "real" unit tests for your domain classes to test their logic in an isolated context.
Edit
Link to sonar-exmaple-multiple-modules