I have a file with N>10 tests defined in BDD in a feature file.
For debugging purposes, I need to execute only one of the tests many times, but I am not finding any other way than commenting out the other tests.
I am trying now to use the #focus tag in front of the test, in a way that cypress IDE actually marks the rest of the tests as sync skip; aborting execution, but the test that I need to run actually says No commands were issued in this test..
This is the example .feature file I'm playing with:
#e2eTests
#cases
Feature: Test feature
Scenario: Test 1
Given foo
When bar
Then foobar
#focus
Scenario: Test 2
Given bar
When foo
Then barfoo!
And when running npm run cypress and selecting this feature, I get:
What am I doing wrong?
Thanks
I would recommend using cypress-grep if you are looking for a flaky test, that needs repeating n times.
After installation and configuration (see here: Burning Tests with cypress-grep) in terminal indicate which test needs to be repeated and how many times (cypress 7.6) ->
npx cypress run --spec cypress/folder/with/test/testFile.spec.ts, --env grep="name/description of specific test", burn= >>numberOfIterations<<
Example:
npx cypress run --spec cypress/integration/testExample.spec.ts, --env grep="A popup should appear after login",burn=250
Related
I run Cypress from the terminal using an npm script. I would like to run a series of checks before any tests in any spec are executed (e.g. to ensure env variables are set as expected).
If any check fails, I'd like to log some debug info and Cypress to mark each spec as failed.
Is something like this possible without having to have a custom script that executes before Cypress is started?
I've played around with the support file, but logging to the terminal and failing all test specs seems problematic.
Yes you can use Before Run API
on('before:run', (details) => {
/* ... */
})
I am using Cypress for end to end testing, and I would like to be able to see all run test suites, in the browser, even after they are run. Currently, after each test suite is completed (test which are stored in separate files), the browser reloads and I cannot see previously run tests, and after the final test suite, the browser closes. Is there an option to change this behavior so that I can run all test files, have all the results visible in the browser and that the browser doesn't close at the end?
I am currently running tests using this command: ./node_modules/.bin/cypress run --headed --spec 'cypress/integration/tests/*'
where /tests is the folder where I currently have my files.
I have added --no-exit but in this case cypress doesn't move to the next test file and only the first one runs.
A workaround solution could be to generate reports with Mochawsome, for each Test Spec, and then merge and view those rendered reports. The reports will contain the results from the tests, test bodies, any errors that occurred and some other bits of information.
If you read through the page in the link it shows you how to generate individual reports then combine them together, and then render them as HTML. These can then be viewed in the browser.
This command can be used to install what's needed npm install --save-dev mochawesome mochawesome-merge mochawesome-report-generator
and then add the Reporter configuration to the cypress.json:
{
"reporter": "mochawesome",
"reporterOptions": {
"reportDir": "cypress/results",
"overwrite": false,
"html": false,
"json": true
}
}
Keep in mind that it may not give you the level of detail that is contained in the Cypress Dashboard in the browser, for example, what was yielded from a request.
Cypress has a lot of possible command with a lot of possible config too.
Read this.
And if you use npm just run like this :
npm run cypress:open
and in your package.json :
"scripts": {
"cypress:open": "cypress open"
}
I am using protractor with jasmine framework. I need solution to stop the execution of running the suite if previous suite fails.
I have three set of test suites as below:
1. Health check - Tests all the web services are returning 200 response.
2. Smoke test - Checks the basic features of front end are looking good.
3. Regression test - Tests all the features.
My requirement is, If health check test case fails then do not run smoke test cases.
This can be achieved through following two ways:
1. Jenkins
2. Using process.exit(1) in the script
But, these two are not fit to my need.
Is there any way to achieve my need through protractor or jasmine way?
You could achieve it using several scripts in package.json. For example:
...
"scripts": {
"health": "launch here only health",
"smoke": "launch here only smoke",
"regression": "launch here only regression",
"test": "npm run health && npm run smoke && npm run regression",
},
...
You will launch all tests using npm run test. If in some command appears error the others will not be execute.
If a test wants to assert the file permission error, for example, writing to the root of file system, "go test" returns an syscall.EACCES error, while "bazel test" returns an syscall.EPERM. How to make the test passes in both "bazel test" and "go test"?
An example can be found here.
You can disable the sandbox by using bazel --spawn_strategy=standalone test //.... I suspect this will work around the problem.
However, you may want to consider whether writing to / is the behavior that you want to test. If you need to run code on a different operating system or inside a Docker container, you'll get different behavior in this case, so you could think about testing a more predictable code path, or mocking out the file access layer to isolate your tests from it.
I followed this guide to get coverage report integrated in WebStorm.
https://www.jetbrains.com/help/webstorm/mocha.html
When I click "Run Mocha test with coverage" the coverage window shows up after running the test. But I don't see any coverage reported. It is not even showing 0%, it is not showing any numbers at all in the Statistics column.
Any help will be appreciated.
Some more background:
WebStorm 2017.2.3
I'm referring Mocha v3.2.0 in package.json
The code is written in ES2015.
The mocha Run configuration uses these environment variables
BABEL_ENV=test and these extra mocha options: --compilers
js:babel-core/register
Thanks in advance
Further information: when I run with coverage in WebStorm I see this command being output to the console:
/Users/nikolaschou/.nvm/versions/node/v8.1.2/bin/node /Users/nikolaschou/Dev/dsg/bifrostshop/node_modules/nyc/bin/nyc.js --reporter=lcovonly --extension=.ts --extension=.js --exclude=test/unitmocha//.spec. --exclude=test/unitmocha//.test. --report-dir=/private/var/folders/q7/kn0zjzks5dz0q2bx0kpg2yhw0000gn/T/mocha-intellij-coverage- /Users/nikolaschou/Dev/dsg/bifrostshop/node_modules/mocha/bin/_mocha --compilers js:babel-core/register --ui bdd --reporter "/Applications/WebStorm 2.app/Contents/plugins/NodeJS/js/mocha-intellij/lib/mochaIntellijReporter.js" --recursive /Users/nikolaschou/Dev/dsg/bifrostshop/test/unitmocha
Add a file called .nycrc in the project root with these contents:
{ "exclude": [] }
This overrides nyc default config and win is restored.
In my case I had this problem because mocha did not exit properly. As the coverage reports seem to be generated on program exit, they are never generated.
Mocha 4 especially has changed behaviour from Mocha 3 as it does "no longer automatically kill itself via process.exit() when it thinks it should be done running".
However you can force that old behaviour with the --exit option.
The better option is probably to examine what keeps your program from terminating and perform a clean shut down. (Maybe with some help of wtfnode for example)
Readings:
Mocha Won't Force Exit
mocha 4 doesn't exit unlike mocha 3