I am using Shippable as my CI and my project is based on Go 1.11. All the unit tests are written in BDD style using Ginkgo and Gomega. In my test pipeline, I have something like:
gocov test ./... | gocov-xml > shippable/codecoverage/coverage.xml
go test -v ./... | go-junit-report > shippable/testresults/junitresults.xml
This will create the coverage file in Cobertura format and unit test report in JUnit format.
In my project, we have multiple test suites. I am planning to use Ginkgo CLI to perform coverage and unit test instead of gocov and go test. Something like:
ginkgo -r -cover -outputdir=./shippable/codecoverage/ -coverprofile=coverage.txt
ginkgo -r -focus="\[Unit\]" -outputdir=./shippable/testresults/ -coverprofile=unit.txt
Now the problem is that I am unable to convert the coverage.txt file to equivalent Cobertura format XML file (which shippable will require) and unit.txt file to equivalent JUnit XML file.
I've seen how to generate the JUnit file from a test suite but in my project, we have multiple test suite, which will result in multiple JUnit files, which I don't want.
Any idea, how can I convert and use them?
To create the Cobertura format:
Nevermind, figured out myself.
Generate the cover using covermode as set
ginkgo -r -cover -covermode=set -outputdir=shippable/codecoverage/ -coverprofile=coverage.out
Now, we need to remove duplicate entries of mode: set
awk '!seen[$0]++' shippable/codecoverage/coverage.out > shippable/codecoverage/coverage-fix.out
Finally, convert to Cobertura
bash gocov convert shippable/codecoverage/coverage-fix.out | gocov-xml > shippable/codecoverage/coverage.xml
Related
We are on latest 8.X WebdriverIO version and latest Jasmine 4.X at this moment, using latest Node 18.X.
In my Wdio.conf.js file I have suite:
all: [ 'path/test1.js', 'path/test2.js'],
test1.js has a Describe with value "SMOKE test 1", while test2.js has "test 2".
I want to run --suite=all but exclude all files that DOESN'T contain "SMOKE" in the title, or run --suite=all but include only tests which contains "SMOKE" in the title.
Is this possible, and how? I'm having real troubles finding examples for this case. Can we use grep for this example somehow? I don't have an idea how can we do it.
Basically, the idea is to run only smoke test from some suite. We are not using Mocha and Cucumber runners which have additional tagging, so we are basically running suites all the time. I don't want to create more suites and divide tests in separate suites, we have to many test files.
I tried following the documentation, but I'm having troubles using their example:
grep -r -l --include "*.js" "myText" | wdio wdio.conf.js
I have a Go project with the following structure:
foo/foo.go
foo/foo_test.go
main.go
As you notice, there is no test for main.go.
I collect the coverage report using the following command:
go test ./foo ./ -coverprofile=coverage.txt -covermode=atomic
Here ./foo and ./ show where to look for packages.
Problem: I send the coverage report to codecov.io which shows that my code is 100% covered with tests. But this is not true as my main.go has no tests at all.
It seems like the system only counts those packages that explicitly specify test files.
Question: How to fix the coverage report in the way that it will count information about untested packages?
Note: You can find my project on GitHub and the real statistic is here. The project has a different structure, but the issue persists (wrong coverage).
The -coverpkg flag may be used to specify the packages that are used as the basis of coverage analyis.
Quoting from Command go: Testing flags:
-coverpkg pattern1,pattern2,pattern3
Apply coverage analysis in each test to packages matching the patterns.
The default is for each test to analyze only the package being tested.
See 'go help packages' for a description of package patterns.
Sets -cover.
So in your specific example this will do it:
go test -coverpkg=.,./foo -coverprofile=coverage.txt -covermode=atomic . ./foo
To apply it for a complete module / project, you may use:
go test -coverpkg=./... -coverprofile=coverage.txt -covermode=atomic ./...
Another option is to place an "empty" test file into the folders of packages that do not currently have a test file. That way they will be naturally included in default coverage analysis, but obviously nothing will be covered from them.
See related discussion on github:
cmd/go: go test -cover & go test -coverprofile should always output a coverage #24570
try this one:
go test -coverpkg=./... -race -coverprofile=coverage.txt -covermode=atomic ./..
I am building a CI pipeline with Bamboo and I run automated tests using Appium. I have successfully run the Appium server and test scripts but I would like to have results in for example xml file so I could pass it to Bamboo. Do you know any way to achieve this?
it might be a bit late to respond on your question, but I'm using Appium with node.js and mocha to write my tests. To create a proper xml with the results I'm doing:
mocha --recursive -R xunit test/ > test-reports.xml
(test/ is the folder containing all my test files). This is generating a proper xml. I haven't tested it with Bamboo but I assume it's capable of reading XUnit test reports.
I use QF test tool (http://www.qfs.de/en/qftest/) to run my integrated UI based tests . Is there any tool which can get code coverage of qft test suites ?
Note : I use Sonar (jacoco plugin) to get code coverage for Junit tests .
I googled a lot and couldn't find any relevant documentation for this . So any links to documentation or example would be helpfull
oYes, this is possible. I'm using QF-Test with Jenkins CI, Sonar and JaCoCo.
To keep it short, in QF-Test go to the step which invokes the SUT and add the -javaagent: parameter to the program
e.g.:
-javaagent:/path/to/mvnlib/org.jacoco.agent-0.6.4.201312101107-runtime.jar=destfile=/usr/share/tomcat6/.jenkins/jobs/Integration_Build/workspace/your.program.test/jacoco/jacoco-qf.exec,includes=your.packages.*,output=file
Configure Jenkins (with Jacoco Plugin) to look for jacoco-qf.exec file.
PS: If you use regular Junit Tests you should combine both the coverage of QF-Test and Junit by this Ant script:
<jacoco:merge destfile="${jacoco.file}">
<fileset dir="${jacoco.report.dir}" includes="*.exec"/>
</jacoco:merge>
I have inherited a Go project that consists of a lot of common files, a library of sorts, two executables, and theoretically a test suite. The test suite is being written after the fact. But I dislike the only way I've found of setting up is rather unpalatable
I'm using Ginkgo, and this is my starting directory structure
component1/component1.go
component2/component2.go
cmd1/cmd1.go
cmd2/cmd2.go
project_suite_test.go
component1_test.go
Each cmd?.go file will be compiled into a separate executable.
What I would like is a multi-file test suite, usually one file per component. Where do I put the files so that go test will find and run all of them, without leaving them here in the root of the project?
ginkgo init and ginkgo bootstrap will set up your tests. ginkgo -r will run all your tests recursively.
Reason:
Ginkgo command will only work if you have actually bootstrap your project via ginkgo.
Options:
To use that you have to go to your test dir in terminal and run
ginkgo init : To Initialise project:
ginkgo bootstrap : This will generate new file with test suite config
ginkgo or ginkgo test : this will now be able to run tests based on your new generated file because that's what it is trying to search.
Alternatively:
If you like to keep your tests in a sub-folder, say test, then running
go test ./...
will attempt to run tests in every folder, even those that do not contain any test, thus having a ? in the subsequent report for non-test folders.
Running
go test ./.../test
instead will target only your test folders, thus having a clean report focused on your tests folders only.
you can alternatively use 'go run $(ls *.go)' to run all the files in a given folder.
Notice you have regular expression within () braces.
In-case you want to run test in different path update path as per your desired dir in the regular expression
You can use go test ./... in the root and it will go into child folders and execute the tests:
component1/component1.go
component1/component1_test.go
component2/component2.go
component2/component2_test.go
cmd1/cmd1.go
cmd1/cmd1_test.go
cmd2/cmd2.go
cmd2/cmd2_test.go