I followed this guide to get coverage report integrated in WebStorm.
https://www.jetbrains.com/help/webstorm/mocha.html
When I click "Run Mocha test with coverage" the coverage window shows up after running the test. But I don't see any coverage reported. It is not even showing 0%, it is not showing any numbers at all in the Statistics column.
Any help will be appreciated.
Some more background:
WebStorm 2017.2.3
I'm referring Mocha v3.2.0 in package.json
The code is written in ES2015.
The mocha Run configuration uses these environment variables
BABEL_ENV=test and these extra mocha options: --compilers
js:babel-core/register
Thanks in advance
Further information: when I run with coverage in WebStorm I see this command being output to the console:
/Users/nikolaschou/.nvm/versions/node/v8.1.2/bin/node /Users/nikolaschou/Dev/dsg/bifrostshop/node_modules/nyc/bin/nyc.js --reporter=lcovonly --extension=.ts --extension=.js --exclude=test/unitmocha//.spec. --exclude=test/unitmocha//.test. --report-dir=/private/var/folders/q7/kn0zjzks5dz0q2bx0kpg2yhw0000gn/T/mocha-intellij-coverage- /Users/nikolaschou/Dev/dsg/bifrostshop/node_modules/mocha/bin/_mocha --compilers js:babel-core/register --ui bdd --reporter "/Applications/WebStorm 2.app/Contents/plugins/NodeJS/js/mocha-intellij/lib/mochaIntellijReporter.js" --recursive /Users/nikolaschou/Dev/dsg/bifrostshop/test/unitmocha
Add a file called .nycrc in the project root with these contents:
{ "exclude": [] }
This overrides nyc default config and win is restored.
In my case I had this problem because mocha did not exit properly. As the coverage reports seem to be generated on program exit, they are never generated.
Mocha 4 especially has changed behaviour from Mocha 3 as it does "no longer automatically kill itself via process.exit() when it thinks it should be done running".
However you can force that old behaviour with the --exit option.
The better option is probably to examine what keeps your program from terminating and perform a clean shut down. (Maybe with some help of wtfnode for example)
Readings:
Mocha Won't Force Exit
mocha 4 doesn't exit unlike mocha 3
Related
After running Hardhat tests in the console with npx hardhat test I decided that being able to set break points would help me iterate faster.
How can I get Webstorm to run the underlying functions started by npx hardhat test so that I can use the built in Debugger?
I've since discovered that hardhat runs mocha under the hood.
To debug in WebStorm you can:
delete your existing configurations
create a new mocha configuration
set any configurations in 'Node options'. Note: since I'm forking the main net it takes a while for tests to start so I added the --timeout 10000 because mocha's default timeout is only 2000ms
select the mocha package, WebStorm doesn't select it by default
set your test file pattern
add const {ethers} = require('hardhat'); to your test file because it is no longer injected by hardhat during run time.
If the green debug icon does not appear I had success in closing and reopening WebStorm.
At this point I could successfully set break points in my test file but not in the MyContract.sol file. This is not surprising given that the contract is compiled before its run.
Create or open the package.json file for your Hardhat project.
Add a test NPM run script and save the file. Your package.json should look something like this.
{
"name": "hardhat-project",
"scripts": {
"test": "hardhat test"
},
"devDependencies": {
"#nomiclabs/hardhat-ethers": "2.0.2",
"#nomiclabs/hardhat-waffle": "2.0.1",
"chai": "4.3.4",
"ethereum-waffle": "3.4.0",
"ethers": "5.4.4",
"hardhat": "2.6.0"
}
}
In the left gutter of the editor pane, a little play icon should appear, click it and then click Debug "test".
I go through the instructions in a little more detail here, but this is the general idea. https://allendefibank.medium.com/how-to-debug-solidity-contracts-in-webstorm-hardhat-2ea0d3c4d582
If you're use typescript you need to import ts-mocha instead of mocha
I have a file with N>10 tests defined in BDD in a feature file.
For debugging purposes, I need to execute only one of the tests many times, but I am not finding any other way than commenting out the other tests.
I am trying now to use the #focus tag in front of the test, in a way that cypress IDE actually marks the rest of the tests as sync skip; aborting execution, but the test that I need to run actually says No commands were issued in this test..
This is the example .feature file I'm playing with:
#e2eTests
#cases
Feature: Test feature
Scenario: Test 1
Given foo
When bar
Then foobar
#focus
Scenario: Test 2
Given bar
When foo
Then barfoo!
And when running npm run cypress and selecting this feature, I get:
What am I doing wrong?
Thanks
I would recommend using cypress-grep if you are looking for a flaky test, that needs repeating n times.
After installation and configuration (see here: Burning Tests with cypress-grep) in terminal indicate which test needs to be repeated and how many times (cypress 7.6) ->
npx cypress run --spec cypress/folder/with/test/testFile.spec.ts, --env grep="name/description of specific test", burn= >>numberOfIterations<<
Example:
npx cypress run --spec cypress/integration/testExample.spec.ts, --env grep="A popup should appear after login",burn=250
Normally Ruby test-unit will display a summary of tests run after they are finished, something like this:
Finished in 0.117158443 seconds.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10 tests, 10 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
298.74 tests/s, 0.00 assertions/s
This was working, but now something has changed and when the unit tests are run it shows the dots but then stops. I tried re-organizing some test file into different directories and made absolutely sure to change the filepaths in the testrunner. Also, the dots do not match the number of tests/assertions.
Loaded suite test
Started
.................$prompt> // <<-- does not put newline here.
I notice that if I run the testrunner from another directory, the summary will show, but it will cause errors with the test dependencies. I should be able to run the testrunner from the same directory. This is an example of the testrunner I am using: https://test-unit.github.io/test-unit/en/file.how-to.html. What are the reasons that this would not display at the end?
Seems like it could be an issue with not having the test-unit.yml file in the same directory as which you run the script.
See here in the code or in the same section in that document you posted.
See how it's configured here, for example:
runner: console
console_options:
color_scheme: inverted
color_schemes:
inverted:
success:
name: red
bold: true
failure:
name: green
bold: true
This part of the code documentation really stuck out:
# ## Test Runners
#
# So, now you have this great test class, but you still
# need a way to run it and view any failures that occur
# during the run. There are some test runner; console test
# runner, GTK+ test runner and so on. The console test
# runner is automatically invoked for you if you require
# 'test/unit' and simply run the file. To use another
# runner simply set default test runner ID to
# Test::Unit::AutoRunner:
Maybe you need to have that runner specified in your YML file?
Without seeing how you are calling your script and your directory organization, it will be hard to tell what is causing that issue but I think it begins with it not reading that yaml file for instance.
If all else fails, let me recommend two great unit testing libraries for Ruby if you feel compelled to switch to a more widely-used library:
minitest
RSpec
Edit: You could also revert your directories to be in the same order as before and hardcode in your Gemfile for that test-unit gem the last version that worked for you, like `"test-unit": "3.4.0".
I am using Cypress for end to end testing, and I would like to be able to see all run test suites, in the browser, even after they are run. Currently, after each test suite is completed (test which are stored in separate files), the browser reloads and I cannot see previously run tests, and after the final test suite, the browser closes. Is there an option to change this behavior so that I can run all test files, have all the results visible in the browser and that the browser doesn't close at the end?
I am currently running tests using this command: ./node_modules/.bin/cypress run --headed --spec 'cypress/integration/tests/*'
where /tests is the folder where I currently have my files.
I have added --no-exit but in this case cypress doesn't move to the next test file and only the first one runs.
A workaround solution could be to generate reports with Mochawsome, for each Test Spec, and then merge and view those rendered reports. The reports will contain the results from the tests, test bodies, any errors that occurred and some other bits of information.
If you read through the page in the link it shows you how to generate individual reports then combine them together, and then render them as HTML. These can then be viewed in the browser.
This command can be used to install what's needed npm install --save-dev mochawesome mochawesome-merge mochawesome-report-generator
and then add the Reporter configuration to the cypress.json:
{
"reporter": "mochawesome",
"reporterOptions": {
"reportDir": "cypress/results",
"overwrite": false,
"html": false,
"json": true
}
}
Keep in mind that it may not give you the level of detail that is contained in the Cypress Dashboard in the browser, for example, what was yielded from a request.
Cypress has a lot of possible command with a lot of possible config too.
Read this.
And if you use npm just run like this :
npm run cypress:open
and in your package.json :
"scripts": {
"cypress:open": "cypress open"
}
I am using protractor with jasmine framework. I need solution to stop the execution of running the suite if previous suite fails.
I have three set of test suites as below:
1. Health check - Tests all the web services are returning 200 response.
2. Smoke test - Checks the basic features of front end are looking good.
3. Regression test - Tests all the features.
My requirement is, If health check test case fails then do not run smoke test cases.
This can be achieved through following two ways:
1. Jenkins
2. Using process.exit(1) in the script
But, these two are not fit to my need.
Is there any way to achieve my need through protractor or jasmine way?
You could achieve it using several scripts in package.json. For example:
...
"scripts": {
"health": "launch here only health",
"smoke": "launch here only smoke",
"regression": "launch here only regression",
"test": "npm run health && npm run smoke && npm run regression",
},
...
You will launch all tests using npm run test. If in some command appears error the others will not be execute.