I wonder if there's any tool that can convert go test -coverprofile=cover.out into the formats that Jenkins can accept? I found some tools like go-junit-report and go2xunit, but they actually just convert output from go test -v, which is not the coverage report.
I want to know the detailed test coverage in Jenkins directly. Basically, I want to see the output from go tool cover -func=cover.out and go tool cover -html=cover.out in Jenkins webpage.
There are several go tools for converting coverage data from go test to Cobertura for Jenkins: gocover-cobertura, or gocov with gocov-xml.
You can use gocover-cobertura as follows:
$ go get github.com/t-yuki/gocover-cobertura
$ go test -coverprofile=cover.out example.com/demo/...
ok example.com/demo 0.008s coverage: 0.0% of statements
ok example.com/demo/cmd/demo 0.020s coverage: 23.4% of statements
$ gocover-cobertura < cover.out > coverage.xml
$ head coverage.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
<coverage line-rate="0.35415787" branch-rate="0" version="" timestamp="1520609235359" lines-covered="839" lines-valid="2369" branches-covered="0" branches-valid="0" complexity="0">
<sources>
<source>/usr/local/go/src</source>
<source>/Users/wilfred/workspace/go-scratch/src</source>
</sources>
<packages>
<package name="example.com/demo/cmd/demo" line-rate="0.4848485" branch-rate="0" complexity="0">
<classes>
Note that you'll need Go 1.10+ to run -coverprofile against multiple packages in one run.
https://github.com/AlekSi/gocov-xml
This is what we need. Use Coberuta plugin to generate the coverage profile.
There isn't a dedicated plugin for Go coverage reports, nor really for generic code coverage.
For reports like this, I use the HTML Publisher Plugin to publish .html files created during a build.
Related
I have some jest tests and I can determine the coverage with
jest --coverage
Also see Code coverage for Jest
I automatically execute the tests on a build server (gitlab runner) and want that my tests fail if the coverage is below a certain limit.
In python there is a flag --cov-fail-under that can be used with pytest, e.g.
pytest --cov src --cov-fail-under=90 --cov-report=term
Unfortunately, I could not find a corresponding option for jest.
=>What is the recommended way to check the total coverage?
Should I write some extra script to evaluate the generated json coverage file or is there an easier solution like a specific reporter to use?
Not listed under CLI-Options, but there is coverageThreshold, which can be used in package.json or within an extra jest configuration file:
https://jestjs.io/docs/en/configuration#coveragethreshold-object
I have a Go project with the following structure:
foo/foo.go
foo/foo_test.go
main.go
As you notice, there is no test for main.go.
I collect the coverage report using the following command:
go test ./foo ./ -coverprofile=coverage.txt -covermode=atomic
Here ./foo and ./ show where to look for packages.
Problem: I send the coverage report to codecov.io which shows that my code is 100% covered with tests. But this is not true as my main.go has no tests at all.
It seems like the system only counts those packages that explicitly specify test files.
Question: How to fix the coverage report in the way that it will count information about untested packages?
Note: You can find my project on GitHub and the real statistic is here. The project has a different structure, but the issue persists (wrong coverage).
The -coverpkg flag may be used to specify the packages that are used as the basis of coverage analyis.
Quoting from Command go: Testing flags:
-coverpkg pattern1,pattern2,pattern3
Apply coverage analysis in each test to packages matching the patterns.
The default is for each test to analyze only the package being tested.
See 'go help packages' for a description of package patterns.
Sets -cover.
So in your specific example this will do it:
go test -coverpkg=.,./foo -coverprofile=coverage.txt -covermode=atomic . ./foo
To apply it for a complete module / project, you may use:
go test -coverpkg=./... -coverprofile=coverage.txt -covermode=atomic ./...
Another option is to place an "empty" test file into the folders of packages that do not currently have a test file. That way they will be naturally included in default coverage analysis, but obviously nothing will be covered from them.
See related discussion on github:
cmd/go: go test -cover & go test -coverprofile should always output a coverage #24570
try this one:
go test -coverpkg=./... -race -coverprofile=coverage.txt -covermode=atomic ./..
I am using Shippable as my CI and my project is based on Go 1.11. All the unit tests are written in BDD style using Ginkgo and Gomega. In my test pipeline, I have something like:
gocov test ./... | gocov-xml > shippable/codecoverage/coverage.xml
go test -v ./... | go-junit-report > shippable/testresults/junitresults.xml
This will create the coverage file in Cobertura format and unit test report in JUnit format.
In my project, we have multiple test suites. I am planning to use Ginkgo CLI to perform coverage and unit test instead of gocov and go test. Something like:
ginkgo -r -cover -outputdir=./shippable/codecoverage/ -coverprofile=coverage.txt
ginkgo -r -focus="\[Unit\]" -outputdir=./shippable/testresults/ -coverprofile=unit.txt
Now the problem is that I am unable to convert the coverage.txt file to equivalent Cobertura format XML file (which shippable will require) and unit.txt file to equivalent JUnit XML file.
I've seen how to generate the JUnit file from a test suite but in my project, we have multiple test suite, which will result in multiple JUnit files, which I don't want.
Any idea, how can I convert and use them?
To create the Cobertura format:
Nevermind, figured out myself.
Generate the cover using covermode as set
ginkgo -r -cover -covermode=set -outputdir=shippable/codecoverage/ -coverprofile=coverage.out
Now, we need to remove duplicate entries of mode: set
awk '!seen[$0]++' shippable/codecoverage/coverage.out > shippable/codecoverage/coverage-fix.out
Finally, convert to Cobertura
bash gocov convert shippable/codecoverage/coverage-fix.out | gocov-xml > shippable/codecoverage/coverage.xml
I have a package with only two Go file: one defines the main function and the other is for the tests.
Now assume that I have cd into this package and run the following command:
$ go test -cover
PASS
coverage: 41.8% of statements
ok github.com/suzaku/dummage 0.010s
As you can see, this works correctly.
But I want to generate a HTML report, so after some googling I use the following command:
$ go test -run=Coverage -coverprofile=c.out github.com/suzaku/dummage
ok github.com/suzaku/dummage 0.010s coverage: 1.8% of statements
Note that this time the coverage drops to 1.8%.
What can I do to fix this?
Are you sure you need that -run=Coverage flag in your go test? This means it will only run tests that match Coverage. If you just want to generate a cover profile for that tests, run go test -coverprofile c.out github.com/suzaku/dummage. Then you may run go tool cover -html c.out to see the HTML report.
If you added -run=Coverage on purpose, then it's expected behavior - the amount of code that runs during -run=Coverage is much less than while running all tests, and the test coverage is calculated for the entire package.
I have below problems with SonarRunner.
SonarQube along with Sonar runner unable to pull junit format xml reports
Unit Tests or Test Coverage widget doesn't show up. It says No Data.
I am following instructions described here
I manually created report file in XML format as described, but still no luck.
Below is the XML file - TEST-Firefox_210_Mac_OS.com.company.BarTest.xml,
<testsuite name="Firefox_210_Mac_OS.com.company.BarTest" errors="0" failures="0" tests="3" time="0.0">
<testcase classname="Firefox_210_Mac_OS.com.company.BarTest" name="testfullName" time="0.0"/>
</testsuite>
To pull the Unit Test execution report to show on sonar dashboard. I have used sample git project from here
and below is my sonar-project.properties,
# project metadata (required)
sonar.projectKey=org.codehaus.sonar:javascript-sonar-runner-jstestdriver
sonar.projectName=JavaScript project with Sonar Runner reusing reports generated by JsTestDriver
sonar.projectVersion=1.0
# path to source directories (required)
sonar.sources=C:/Sonar/sonar-runner-dist-2.4/sonar-runner-2.4/projects/sources
# path to tests source directories (required)
sonar.tests=C:/Sonar/sonar-runner-dist-2.4/sonar-runner-2.4/projects/tests
sonar.javascript.jstestdriver.reportsPath=C:/Sonar/sonar-runner-dist-2.4/sonar-runner-2.4/projects/target/TEST-Firefox_210_Mac_OS.com.company.BarTest.xml
sonar.sourceEncoding=UTF-8
below is my jsTestDriver.conf
server: http://localhost:9876
load:
- C:/Sonar/sonar-runner-dist-2.4/sonar-runner-2.4/projects/sources/*.js
- C:/Sonar/sonar-runner-dist-2.4/sonar-runner-2.4/projects/sources/com/company/*.js
test:
- C:/Sonar/sonar-runner-dist-2.4/sonar-runner-2.4/projects/tests/*.js
- C:/Sonar/sonar-runner-dist-2.4/sonar-runner-2.4/projects/tests/com/company/*.js
plugin:
- name: "coverage"
jar: "coverage-1.3.5.jar"
module: "com.google.jstestdriver.coverage.CoverageModule"
My sonarqube is running on port : 9000 and below is the screen shot. As you see SonarRunner and jsTestDriver just doing a code analysis and not showing any unit tests.
SonarQube doc website says jsTestDriver will run the javascript unit
tests and copy the results in target folder in XML format
SonarQube doesnt run your Unit Tests, it just gathers the reports generated from your manuall run or other tools automatic run (like: Jenkins).
From SonarQube doc:
Prior to the SonarQube analysis, execute your unit tests in order to
generate XML report. The JUnit like XML format supported is the one
generated by js-test-driver
Then I manually created report file in XML format as described, but
still no luck.
If you created manually the report files and reports are still not showing, check your paths if ok.Check if sonar is really reading data from C:/Sonar/sonar-runner-dist-2.4/sonar-runner-2.4/projects/ path