I have a CI running that does mocha test executions in parallel. A test execution creates a coverage report coverage.json - generated by nyc. I want to have a final merged report from all of the coverage.jsons in lcov format, but I can't manage to generate one - I always end up having an empty text summary. What I've tried:
Running nyc report:
nyc report --temp-dir=nyc/coverage_dir --reporter=text --reporter=lcov
gives an empty report and text summary
First running nyc merge and then nyc report
nyc merge ./coverage_dir coverage.json
nyc report --report-dir=temp --reporter=text --reporter=lcov
This combination does seem to merge the coverage.jsons files, at least judging from the file size, but the report step still displays an empty text summary.
Trying the above with different version, i.e. nyc 15.1.0 and nyc 14.1.1
Specifying the -t flag and --report-dir options outlined in this answer
Changing to a parent directory and rerunning the commands as specified by this answer.
All resulting in an empty text summary of the final report.
I solved it in this way.
Copy all the json files in the default folder:
mkdir .nyc_output
cp project_a/coverage/coverage-final.json .nyc_output/project_a.json
cp project_b/coverage/coverage-final.json .nyc_output/project_b.json
Then generate the report:
nyc report --reporter=text --reporter=lcov --report-dir=destination-dir
Related
I am using cypress and integrated it with allure, My automation is getting triggered by github-actions and I would to have history trend for it.
The Process which I followed to generate the history trend is:
run test execution which will generate folder allure-results
run command allure generate --clean it will generate folder allure-report
copy history folder from folder allure-report to folder allure-results
run command allure generate --clean it will generate folder allure-report
open index.html inside folder allure-report
go to TREND section and see the history-trend
again run test execution
again copy the history folder from allure-report to allure-report
run command allure generate --clean
open index.html inside allure-report
go to TREND section and see the history-trend
For history-trend in X axis, I can see build number is 1 for every run, I was expecting it to increased every time based on you last execution build value at right most
would be helpful to know what I am missing here.
I'm just starting with Allure, and I had similar problem. Right now, I'm using docker from #frank-escobar - that I found here: https://stackoverflow.com/a/62155628/3497625
The history has increasing build numbers. Maybe look into sources, and find out how it is done.
What I can see there is an executor.xml file in the new allure-reports dir, that has context like:
{ "reportName": "default", "buildName": "default #17", "buildOrder": "17", "name": "Automatic Execution", "reportUrl": "../17/index.html", "buildUrl": "", "type": "another" }
The correct way would be to:
Run the tests
allure generate allure-results --clean -o allure-report
mkdir -p allure-results/history && cp -r allure-report/history/* allure-results/history/ || true
allure serve - to open up the reports.
allure-results/ directory contains the history(which you want).
allure-report/ directory contains just the last run.
You need to execute at least 2 runs for the history to "be working".
Recently I added test coverage to our CI/CD pipeline, so we have the % of coverage outputted after each job. Currently our .gitlab-ci.yml looks like this:
run tests:
stage: test
image: python:3
script:
- pip install pytest pytest-cov
- coverage run -m pytest
- coverage report
- coverage xml
coverage: '/TOTAL.*\s([.\d]+)%/'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
Coverage is a gitlab CI keyword and it pulls the total % coverage from coverage report using regex on the total coverage that is outputted to the terminal. I want to save the coverage to a variable so I can compare it to a threshold number and fail the pipeline if it goes below the threshold. I tried:
variables:
coverage_percent: '/TOTAL.*\s([.\d]+)%/'
after_script:
- echo $coverage_percent
But $coverage_percent just gets saved as exact string '/TOTAL.*\s([.\d]+)%/' - it doesn't perform a regex on the terminal like the coverage command does. How can I either a) save the result of coverage: '/TOTAL.*\s([.\d]+)%/' or b) perform regex with a variable in the pipeline?
you could try using the gitlab api to pull the value out and perform a comparison, then based on the result of that fail your pipeline.
https://www.reddit.com/r/gitlab/comments/agqhnt/using_gitlab_api_to_access_code_coverage/ has an example of how to do this.
Depending on how often the pipelines value (the bit after /pipelines/) in the URL changes this might work for you :)
I'm setting up GitLab CI.
We use Ginkgo tests for BDD.
Ginkgo creates a report per each folder where tests are located.
This create a problem with collecting all reports and publishing it as a single test report file.
Is it possible to configure GinkGo in a such way so I could take all test in a single test report file?
What I understand is your reports lies in each test folder:
Example
testScripts
- test_1_directory (contains test spec and result files)
- test_2_directory (contains test spec and result files)
- test_3_directory (contains test spec and result files)
I'm not sure this might exactly help you but can give it a try
In you job add reports paths as mentioned below:
artifacts:
reports:
junit:
- ./packages/e2e/goProject/testScripts/**/**.xml
Assuming .xml are report generated.
At the end all test will be displayed in pipeline's test section
I'm having a problem with the Sonarqube report. It is showing 42.5% for test coverage, but "-" Unit Tests.
I created the report with coverage.py and with the command
coverage run --branch --omit 'venv/*,tests/*' -m unittest.
I've set up my coverage path with the following line:
sonar.python.coverage.reportPaths=coverage.xml
I couldn't find a solution online and don't know if it's a problem while generating the report or importing it to Sonarqube. Here is a picture of the report:
Thanks in advance!
I'm using Protractor and jasmine-allure-reporter
And trying to run multiple specs that are defined in the config.js file.
specs: ['spec1.js','spec2.js']
spec1.js contains 3 tests and spec2.js contains single test.But the jasmine-allure-reporter displays all the four test cases(3+1) together and there is no specification about the spec files (spec file name). How can I display the test cases separately under each spec file-name in one HTML.
Please help me on this.
I am generating report using command "allure generate allure-results --clean -o allure-report || true"
allure-results >> Location where xml files are generated and
allure-report >> where html report is generated
Looking at the allure reporter repository, it doesn't look like it's supported. They set the outDir but do not expose a way to consolidateAll like how jasmine-reporter does. See jasmine-reporter GitHub. If you decide to switch to jasmine-reporter, consolidating XML files is pretty simple. See the Protractor cookbook for an example.