I have some jest tests and I can determine the coverage with
jest --coverage
Also see Code coverage for Jest
I automatically execute the tests on a build server (gitlab runner) and want that my tests fail if the coverage is below a certain limit.
In python there is a flag --cov-fail-under that can be used with pytest, e.g.
pytest --cov src --cov-fail-under=90 --cov-report=term
Unfortunately, I could not find a corresponding option for jest.
=>What is the recommended way to check the total coverage?
Should I write some extra script to evaluate the generated json coverage file or is there an easier solution like a specific reporter to use?
Not listed under CLI-Options, but there is coverageThreshold, which can be used in package.json or within an extra jest configuration file:
https://jestjs.io/docs/en/configuration#coveragethreshold-object
Related
I'm setting up GitLab CI.
We use Ginkgo tests for BDD.
Ginkgo creates a report per each folder where tests are located.
This create a problem with collecting all reports and publishing it as a single test report file.
Is it possible to configure GinkGo in a such way so I could take all test in a single test report file?
What I understand is your reports lies in each test folder:
Example
testScripts
- test_1_directory (contains test spec and result files)
- test_2_directory (contains test spec and result files)
- test_3_directory (contains test spec and result files)
I'm not sure this might exactly help you but can give it a try
In you job add reports paths as mentioned below:
artifacts:
reports:
junit:
- ./packages/e2e/goProject/testScripts/**/**.xml
Assuming .xml are report generated.
At the end all test will be displayed in pipeline's test section
I have a Go project with the following structure:
foo/foo.go
foo/foo_test.go
main.go
As you notice, there is no test for main.go.
I collect the coverage report using the following command:
go test ./foo ./ -coverprofile=coverage.txt -covermode=atomic
Here ./foo and ./ show where to look for packages.
Problem: I send the coverage report to codecov.io which shows that my code is 100% covered with tests. But this is not true as my main.go has no tests at all.
It seems like the system only counts those packages that explicitly specify test files.
Question: How to fix the coverage report in the way that it will count information about untested packages?
Note: You can find my project on GitHub and the real statistic is here. The project has a different structure, but the issue persists (wrong coverage).
The -coverpkg flag may be used to specify the packages that are used as the basis of coverage analyis.
Quoting from Command go: Testing flags:
-coverpkg pattern1,pattern2,pattern3
Apply coverage analysis in each test to packages matching the patterns.
The default is for each test to analyze only the package being tested.
See 'go help packages' for a description of package patterns.
Sets -cover.
So in your specific example this will do it:
go test -coverpkg=.,./foo -coverprofile=coverage.txt -covermode=atomic . ./foo
To apply it for a complete module / project, you may use:
go test -coverpkg=./... -coverprofile=coverage.txt -covermode=atomic ./...
Another option is to place an "empty" test file into the folders of packages that do not currently have a test file. That way they will be naturally included in default coverage analysis, but obviously nothing will be covered from them.
See related discussion on github:
cmd/go: go test -cover & go test -coverprofile should always output a coverage #24570
try this one:
go test -coverpkg=./... -race -coverprofile=coverage.txt -covermode=atomic ./..
I have a package with only two Go file: one defines the main function and the other is for the tests.
Now assume that I have cd into this package and run the following command:
$ go test -cover
PASS
coverage: 41.8% of statements
ok github.com/suzaku/dummage 0.010s
As you can see, this works correctly.
But I want to generate a HTML report, so after some googling I use the following command:
$ go test -run=Coverage -coverprofile=c.out github.com/suzaku/dummage
ok github.com/suzaku/dummage 0.010s coverage: 1.8% of statements
Note that this time the coverage drops to 1.8%.
What can I do to fix this?
Are you sure you need that -run=Coverage flag in your go test? This means it will only run tests that match Coverage. If you just want to generate a cover profile for that tests, run go test -coverprofile c.out github.com/suzaku/dummage. Then you may run go tool cover -html c.out to see the HTML report.
If you added -run=Coverage on purpose, then it's expected behavior - the amount of code that runs during -run=Coverage is much less than while running all tests, and the test coverage is calculated for the entire package.
I am building a CI pipeline with Bamboo and I run automated tests using Appium. I have successfully run the Appium server and test scripts but I would like to have results in for example xml file so I could pass it to Bamboo. Do you know any way to achieve this?
it might be a bit late to respond on your question, but I'm using Appium with node.js and mocha to write my tests. To create a proper xml with the results I'm doing:
mocha --recursive -R xunit test/ > test-reports.xml
(test/ is the folder containing all my test files). This is generating a proper xml. I haven't tested it with Bamboo but I assume it's capable of reading XUnit test reports.
I use QF test tool (http://www.qfs.de/en/qftest/) to run my integrated UI based tests . Is there any tool which can get code coverage of qft test suites ?
Note : I use Sonar (jacoco plugin) to get code coverage for Junit tests .
I googled a lot and couldn't find any relevant documentation for this . So any links to documentation or example would be helpfull
oYes, this is possible. I'm using QF-Test with Jenkins CI, Sonar and JaCoCo.
To keep it short, in QF-Test go to the step which invokes the SUT and add the -javaagent: parameter to the program
e.g.:
-javaagent:/path/to/mvnlib/org.jacoco.agent-0.6.4.201312101107-runtime.jar=destfile=/usr/share/tomcat6/.jenkins/jobs/Integration_Build/workspace/your.program.test/jacoco/jacoco-qf.exec,includes=your.packages.*,output=file
Configure Jenkins (with Jacoco Plugin) to look for jacoco-qf.exec file.
PS: If you use regular Junit Tests you should combine both the coverage of QF-Test and Junit by this Ant script:
<jacoco:merge destfile="${jacoco.file}">
<fileset dir="${jacoco.report.dir}" includes="*.exec"/>
</jacoco:merge>