Scenario
Compile code & run unit tests
Execute sonar scan on code
Once the scan completes, code coverage for unit tests + other analysis gets stored in sonar (lets call report1)
All good till now. Now once the application is deployed, we are run some integrations tests and manual tests are also performed.
All these tests will generate code coverage.
Now we have to merge unit test, integrations & manual tests code coverage and store it on sonar under same analysis report (basically update report1)
Any thoughts/suggestions for best practice or solution for same on sonar?
It's pretty simple. Don't run the SonarQube scan until you've produced all the data you want to integrate into the report.
However, I would also point out that it's generally not worthwhile to generate code coverage for anything but unit tests. If you can't reach it with a unit test, I don't see how you could reach it with any other test.
Related
I am trying to get code coverage for rest API endpoints written in node, but my functional(not unit) test cases calling these node end points are written in ruby and cucumber. Which approach should I use in this situation?
Thanks, appreciate your help!!
As long as you run an instrumented version of the classes, it shouldn't matter how they are executed. The coverage logging will still be executed.
My solution would be to
Make sure that the classes under test are properly instrumented using Cobertura or similar
Run the test suit
Collect all coverage files and generate a coverage report
This blog post describes a solution where a multi module Maven project is measured. It is somewhat related to your problem.
We've been adding an increasing number of Groovy (Spock) unit tests to our existing suite of Java (JUnit) tests.
We've configured things correctly to get Spock code coverage listed in Sonar, but the "Unit Test Success" listings - Tests, Failures, Errors, Skipped tests - only shows for the Java tests.
What configuration do we need to add for the Spock tests to report correctly?
Thank you
Those results in Java are fed by the Unit Test execution report, which is separate from the coverage report. The docs tell you how to feed that data into an analysis.
I have collected jacoco code coverage on integration test cases. Now I want to upload integration code coverage in Sonar 6.3. Since these are integration coverage results, source code is not available. Is it possible to show integration code coverage results into Sonar by any api/any other methos. If it possible, please share the relevant documentation. It would help me a lot.
You cannot update an existing analysis. Your only option is to run the integration tests before analysis, and feed the report into the analysis.
How to visualize Integration Tests in Sonar ?
Currently I only see :
the global coverage (UT + IT)
the number of UT
I read somewhere we had to configure a widget in the GUI : I didn't see any option which could do that.
Documentation states :
If coverage by unit tests is not computed with JaCoCo, overall
coverage = integration test coverage.
But in my case I see that the coverage is changing when I change my UTs (or ITs). Moreover I see jacoco reports in the targets :
jacoco.exec
jacoco-it.exec
I finaly tryed the official Sonar samples : it is the same ! I didn't find any sample with a clear separation between :
Unit test coverage
Integration test coverage
Overall test coverage
And sorry but Sonar documentation and samples have to be improved...
Context : sonar6.2, java8, spring boot, modular project, maven, surefire & failsafe
Starting in SonarQube 6.2, all test results are merged into simply "coverage". This was done on the theory that by and large, most people don't care how their code is covered, only that it is covered.
With this change, some math inconsistencies w/r/t how Overall Coverage was calculated from unit tests versus integration tests were eliminated, and additionally the ability to feed many different coverage reports was added. (Some people have unit, integration, smoke, ... tests).
Regarding
I read somewhere we had to configure a widget in the GUI : I didn't see any option which could do that.
In 6.2 dashboards were dropped, so there are no widgets to be configured. Coverage is shown automatically on the project home page.
Merging unit testing coverage and integration testing coverage numbers is not a good idea.
Modern unit tests that mock out all dependencies and environmental factors and have been proven to run on a developers local machine cannot by definition fail in a continuous integration environment. This makes them next to useless to run in a CI environment. Therefore, 100% unit test coverage but 0% integration test coverage means no meaningful testing is actually happening in the CI build.
I am calling Sonar from my Jenkins job
I noticed that the coverage of the classes in my domain module all show as having 0% test coverage even though the domains are being used in the other modules which have high test coverage. I am using cobertura within Sonar for measuring the coverage
Can anyone offer any insights into how to get Sonar/Cobertura to recognise the test coverage for the classes in the domain module
Thanks
Damien
By default, Sonar computes unit test coverage. So if your domain classes are not unit-tested in the module where they are defined, this is normal that you don't get unit test coverage for those.
You can have a look at how to add integration test coverage in Sonar: see https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/combined%20ut-it/combined-ut-it-multimodule-maven-jacoco
But that'd be best to have "real" unit tests for your domain classes to test their logic in an isolated context.
Edit
Link to sonar-exmaple-multiple-modules