I'm using simplecov to generate test reports for my ruby project in gitlab. The reports are getting generate successfully but the output is not in proper format. while in my localhost, it's generating output in the proper format. Any idea why this is happening?
Here's how my gitlab-ci.yml step looks like :
rspec:
stage: rspec
script:
- bundle exec rspec
allow_failure: false
artifacts:
name: test_report
paths:
- coverage/index.html
when: always
SimpleCov generates a bunch of files apart from the index.html, I guess there are some styles as well, so probably you can try adding to the gitlab artifacts the folder instead of only adding the index.html
Related
I have a maven project. Inside it I have a _models/ folder where I have some .puml files.
I would like to generate the .pngs from those plantuml files each time I commit.
This is my .gitlab-ci.yml
stages:
- test
- build
- uml
.maven:
image: maven:3.6.3-jdk-11
test:
extends: .maven
script:
- mvn test
build:
extends: .maven
script:
- mvn package -B
uml:
image: miy4/plantuml
script:
- plantuml -charset UTF-8 ./_models/sequence_diagram.puml
- plantuml -charset UTF-8 ./_models/class_diagram.puml
- plantuml -charset UTF-8 ./_models/object_diagram.puml
The pipeline passes but the diagrams.png are not created in the _models folder. Do you know why?
In the GitLab pipelines it says: No diagram found but when I run locally those commands everything works fine
Thanks :)
Recently I added test coverage to our CI/CD pipeline, so we have the % of coverage outputted after each job. Currently our .gitlab-ci.yml looks like this:
run tests:
stage: test
image: python:3
script:
- pip install pytest pytest-cov
- coverage run -m pytest
- coverage report
- coverage xml
coverage: '/TOTAL.*\s([.\d]+)%/'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
Coverage is a gitlab CI keyword and it pulls the total % coverage from coverage report using regex on the total coverage that is outputted to the terminal. I want to save the coverage to a variable so I can compare it to a threshold number and fail the pipeline if it goes below the threshold. I tried:
variables:
coverage_percent: '/TOTAL.*\s([.\d]+)%/'
after_script:
- echo $coverage_percent
But $coverage_percent just gets saved as exact string '/TOTAL.*\s([.\d]+)%/' - it doesn't perform a regex on the terminal like the coverage command does. How can I either a) save the result of coverage: '/TOTAL.*\s([.\d]+)%/' or b) perform regex with a variable in the pipeline?
you could try using the gitlab api to pull the value out and perform a comparison, then based on the result of that fail your pipeline.
https://www.reddit.com/r/gitlab/comments/agqhnt/using_gitlab_api_to_access_code_coverage/ has an example of how to do this.
Depending on how often the pipelines value (the bit after /pipelines/) in the URL changes this might work for you :)
I'm setting up GitLab CI.
We use Ginkgo tests for BDD.
Ginkgo creates a report per each folder where tests are located.
This create a problem with collecting all reports and publishing it as a single test report file.
Is it possible to configure GinkGo in a such way so I could take all test in a single test report file?
What I understand is your reports lies in each test folder:
Example
testScripts
- test_1_directory (contains test spec and result files)
- test_2_directory (contains test spec and result files)
- test_3_directory (contains test spec and result files)
I'm not sure this might exactly help you but can give it a try
In you job add reports paths as mentioned below:
artifacts:
reports:
junit:
- ./packages/e2e/goProject/testScripts/**/**.xml
Assuming .xml are report generated.
At the end all test will be displayed in pipeline's test section
I have a test project that is built with Maven (Java). I can either execute the test from IntelliJ manually or from the command line by writing mvn test.
I put the project on CircleCI and it generated a yml file. And it was able to execute the tests on pipelines as well without any issue at first
Then I made something stupid. Initially tests were in this root: src/main/java/api/test. But I decided to move the test class to this root: src/test/java/api.
I did this change because src/test/java was the actual test folder created automatically when you create a Maven project. (The other test folder was created by me manually so I thought this was not the best practice and therefore decided to move the test class to src/test/java. Basically what I did is, I created a package named api under src/test/java and moved the test class to this package. After that I deleted src/main/java/api/test as it is empty now.
After this change, I didn't observe any issue. I could still run the tests from IntelliJ or from the command line by mvn test command. But after I commit my changes, I just checked the pipelines and saw this:
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.793
My project is still built and tested on CircleCI but it obviously does not execute the test class. I dont know why it is happening. I checked the yml file and there is nothing related with paths and it was working before. Here is my yml file:
# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1
jobs:
# Below is the definition of your job to build and test your app, you can rename and customize it as you want.
build-and-test:
# These next lines define a Docker executor: https://circleci.com/docs/2.0/executor-types/
# You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
# Be sure to update the Docker image tag below to openjdk version of your application.
# A list of available CircleCI Docker Convenience Images are available here: https://circleci.com/developer/images/image/cimg/openjdk
docker:
- image: cimg/openjdk:11.0
steps:
# Checkout the code as the first step.
- checkout
# Use mvn clean and package as the standard maven build phase
- run:
name: Build
command: mvn -B -DskipTests clean package
# Then run your tests!
- run:
name: Test
command: mvn test
workflows:
# Below is the definition of your workflow.
# Inside the workflow, you provide the jobs you want to run, e.g this workflow runs the build-and-test job above.
# CircleCI will run this workflow on every commit.
# For more details on extending your workflow, see the configuration docs: https://circleci.com/docs/2.0/configuration-reference/#workflows
sample:
jobs:
- build-and-test
As I said the same yml file was working fine before I moved the test class. Now it probably can not locate my test file. What could be the problem here and how can I solve this? Any help is appreciated.
I'm trying to collect coverage info and publish it to SonarCloud for my C# project, using GitHub Actions as my CI pipeline. The execution is very simple, basically trying to execute tests for all projects, merging all coverage files:
run: |
.\.sonar\scanner\dotnet-sonarscanner begin /k:"LanguageDev_Yoakke" /o:"languagedev" /d:sonar.login="${{ secrets.SONAR_TOKEN }}" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.cs.opencover.reportsPaths="CoverageResults/coverage.opencover.xml" /d:"sonar.verbose=true"
dotnet build
dotnet test /p:CollectCoverage=true /p:CoverletOutput="../CoverageResults/" /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat=\"opencover,json\" /maxcpucount:1
.\.sonar\scanner\dotnet-sonarscanner end /d:sonar.login="${{ secrets.SONAR_TOKEN }}"
Note, that I do need to output in both opencover and json formats. coverlet only seems to merge properly is json is among the output formats, so it can then convert that to opencover in the end.
The problem is, this generates no coverage information on the CI - downloading artifacts show no generated folder or files. However, if just pass /p:CoverletOutputFormat=opencover, all coverage information is generated on the CI - but not merged properly because of no json output.
Locally, the command
dotnet test /p:CollectCoverage=true /p:CoverletOutput="../CoverageResults/" /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat=\"opencover,json\" /maxcpucount:1
just works and generates the proper coverage XML file.
What could be the problem, why does it not work with both coverage formats specified for the CI? Initially I thought that I do not know about escaping quotes in YAML but this does not seem to be the case here.