Given the following test.js
var someCriticalFunction = function() {
throw 'up';
};
describe('test 1', () => {
it('is true', () => {
expect(true).toBe(true);
})
});
describe('test 2', () => {
it('is ignored?', () => {
someCriticalFunction();
expect(true).toBe(false);
})
});
Running Jest will output
Using Jest CLI v0.9.2, jasmine2
PASS __tests__/test.js (0.037s)
1 test passed (1 total in 1 test suite, run time 0.795s)
Which pretty much gives you the impression everything is great. So how do I prevent shooting my own foot while writing tests? I just spent an hour writing tests and now I wonder how many of them actually run? Because I will certainly not count all of them and compare with the number in the report.
Is there a way to make Jest fail as loud as possible when an error is thrown as part of the test suite, not the tests itself (like preparation code)? Putting it inside beforeEach doesn't change anything.
This is fixed in 0.9.3 jest-cli https://github.com/facebook/jest/pull/812
Which for some reason is unpublished again (I had it installed minutes ago). So anyway, throwing strings is now caught. You should never throw strings anyway and thanks to my test suite I found one of the last places I actually still threw strings...
Related
Is it possible to run specific part of the test in Cypress over and over again without execution of whole test case? I got error in the second part of test case and first half of it takes 100s. It means I have to wait 100s every time to get to the point where the error occurs. I would like rerun test case just few steps before error occurs. So once again, my question is: Is it possible to do in Cypress? Thanks
Workaround #1
If you are using cucumber in cypress you can modify your scenario to a Scenario Outline that will execute Nth times with a scenario tag:
#runMe
Scenario Outline: Visit Google Page
Given that google page is displayed
Examples:
| nthRun |
| 1 |
| 2 |
| 3 |
| 4 |
| 100 |
After that run the test in the terminal by running through tags:
./node_modules/.bin/cypress-tags run -e TAGS='#runMe'
Reference: https://www.npmjs.com/package/cypress-cucumber-preprocessor?activeTab=versions#running-tagged-tests
Workaround #2
Cypress does have retry capability but it would only retry the scenario during failure. You can force your scenario to fail to retry it Nth times with a scenario tag:
In your cypress.json add the following configuration:
{
"retries": {
// Configure retry attempts for `cypress run`
// Default is 0
"runMode": 99,
// Configure retry attempts for `cypress open`
// Default is 0
"openMode": 99
}
}
Reference: https://docs.cypress.io/guides/guides/test-retries#How-It-Works
Next is In your feature file, add an unknown step definition on the last step of your scenario to make it fail:
#runMe
Scenario: Visit Google Page
Given that google page is displayed
And I am an uknown step
Then run the test through tags:
./node_modules/.bin/cypress-tags run -e TAGS='#runMe'
For a solution that doesn't require adding a change to the config file, you can pass retries as a param to specific tests that are known to be flakey for acceptable reasons.
https://docs.cypress.io/guides/guides/test-retries#Custom-Configurations
Meaning you can write (from docs)
describe('User bank accounts', {
retries: {
runMode: 2,
openMode: 1,
}
}, () => {
// The per-suite configuration is applied to each test
// If a test fails, it will be retried
it('allows a user to view their transactions', () => {
// ...
}
it('allows a user to edit their transactions', () => {
// ...
}
})```
am trying to skip all other tests from all spec files if one test fails and found a working solution over here Is there a reliable way to have Cypress exit as soon as a test fails?. However, this looks to be working only if the test fails in it() assertions. How can we skip the tests if somethings fails in beforeach()
For eg:
before(() => {
cy.get('[data-name="email-input"]').type(email);
cy.get('[data-name="password-input"]').type(email);
cy.get('[data-name="account-save-btn"]').click();
});
And if something goes wrong (for eg: CypressError: Timed out retrying: Expected to find element: '[data-name="email-input"]', but never found it.) in above code then stop/ skip all tests in all spec files.
Just in case anyone is looking answer for the same question. I have found a solution and would like to share.
To implement the solution I have used a cookie that I will set to value true if something fails and before executing each test cypress will check the value of cookie. If the value of cookie is true it skips the test.
Cypress.on('fail', error => {
document.cookie = "shouldSkip=true" ;
throw error;
});
function stopTests() {
cy.getCookie('shouldSkip').then(cookie => {
if (cookie && typeof cookie === 'object' && cookie.value === 'true') {
Cypress.runner.stop();
}
});
}
beforeEach(stopTests);
Also to note: Tests should be written in it() block and avoid using before() to write tests
As of Cypress 10, tests don't run if before or beforeEach hook fails.
On a clean node project with Jest 24.9.0, this test suite
test('a test', () => {
expect('foo').toBe('foo');
});
test('b test', () => {
throw 'foo';
});
outputs the following (abbreviated):
√ a test
× b test (1ms)
● b test
The non-ASCII characters, especially the bullet, cause a significant slow-down in an Emacs buffer on Windows 10.
How can I instruct Jest to output only ASCII characters in its decorations?
Jest documentation is silent on the issue; grepping node_modules folder for these characters turned out nothing promising (directly related to Jest).
I tried various reporters, but not to my satisfaction. It's easiest to simply filter the report:
package.json
{
"scripts": {
"test": "jestwrapper"
}
}
bin/jestwrapper
#!/usr/bin/bash
jest 2>&1 | perl -C -Mutf8 -lpe'tr/√✔✓✘✕●/+++xx-/'
I am going to create end to end(e2e) test using protractor with jasmine and angular 6. I have written some test cases almost 10 cases. That's all working fine, but always some cases become fails. And its failed because of jasmine timeout. I have configure timeout value like below. But I am not getting consistant result. sometimes a test cases is success but at next run it will goes to success or fail. I have searched on google but I have not found any useful solution.
I have defined some common properties for wait
waitForElement(element: ElementFinder){
browser.waitForAngularEnabled(false);
browser.wait(() => element.isPresent(), 100000, 'timeout: ');
}
waitForUrl(url: string){
browser.wait(() => protractor.ExpectedConditions.urlContains(url), 100000, 'timeout')
}
And protractor.conf.js file I have defined that
jasmineNodeOpts: {
showColors: true,
includeStackTrace: true,
defaultTimeoutInterval: 20000,
print: function () {
}
}
I am getting below error
- Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
- Failed: stale element reference: element is not attached to the page document
(Session info: chrome=76.0.3809.100)
(Driver info: chromedriver=76.0.3809.12 (220b19a666554bdcac56dff9ffd44c300842c933-refs/branch-heads/3809#{#83}),platform=Windows NT 10.0.17134 x86_64)
I have got the solution:
I have configured waiting timeout 100000 ms for individual element find where whole script timeout was 20000 ms. So I have follow below process:
Keep full spec timeout below than sum of all elements find timeouts. I have configured defaultTimeoutInterval at jasmineNodeOpts greater than sum of value for all test cases timeout. And then add a large value to allScriptsTimeout: 2000000 inside of export.config. Its resolved my problem.
NB: I gave this answer because I think it may help others who will face this kind of problem.
I'm using mocha with chai.assert for my tests. Errors are caught and reported, but they don't show a file/line number where they happen. I'm used to having location information with tests in other languages, it's otherwise hard to figure out which assert failed.
Is there some way to get location information with mocha/chai/assert?
From version 1.9.1 onwards, if you set the includeStack flag to true, you'll get a stack trace on assertion failures:
var chai = require("chai");
chai.config.includeStack = true;
var assert = chai.assert;
describe("test", function () {
it("blah", function () {
assert.isTrue(false);
});
});
In versions prior to 1.9.1 you had to set chai.Assertion.includeStack = true. From 1.9.1 onwards this method of getting stack traces is deprecated. It is still available in 1.10.0 but may be removed in 1.11.0 or 2.0.0. (See here for details.)
The example above will show a stack trace where assert.isTrue fails. Like this:
AssertionError: expected false to be true
at Assertion.<anonymous> (.../node_modules/chai/lib/chai/core/assertions.js:193:10)
at Assertion.Object.defineProperty.get (.../node_modules/chai/lib/chai/utils/addProperty.js:35:29)
at Function.assert.isTrue (.../node_modules/chai/lib/chai/interface/assert.js:242:31)
at Context.<anonymous> (.../test.js:7:16)
[... etc ...]
(I've truncated the trace to what is only relevant and truncated the paths.) The last frame shown in what I've included above is the one where the error happened (.../test.js:7:16). I do not think that chai allows having only the file name and line number of the assertion call.
chai.Assertion.includeStack is now deprecated. Use chai.config instead
var chai = require("chai");
chai.config.includeStack = true;
var assert = chai.assert;