Catching MochaJS timeouts - mocha.js

This question has been asked before, but the answer given was to re-write the specific code that was used in that case.
The Mocha documentation only mentions changing the duration of timeouts, and not the behaviour on timeout.
In my case, I want to test code that under certain conditions does not have any side effects. The obvious way to do this is to call the code, wait for Mocha to time out, and then run some assertions on my spies, e.g.:
this.onTimeout(function() {
assert.equal(someObject.spiedMethod.called, false);
});
someAsyncFunction();
Is this possible, or do I have to resort to setting my own timeout, e.g.:
// 1. Disable the mocha timeout
this.timeout(0);
// 2. create my own timeout, which should fire before the mocha timeout
setTimeout(function() {
assert.equal(someObject.spiedMethod.called, false);
done();
}, 2000);
someAsyncFunction();

Mocha's own timeouts are designed only to detect when tests are considered to be slow beyond anything reasonable. An asynchronous computation could be buggy in such a way that causes it to never terminate. Rather than wait forever, Mocha lets you decide that after X time it should fail the test. As soon as the timeout is hit, Mocha gives up on the test that timed out.
So you have to set your own timeouts.

Was facing a similar problem & also didn't find a way to do that in mochajs.
However, if you want to run some code on test timeout, you could structure your code like this:
describe('something', function() {
const TEST_TIMEOUT = 10000
this.timeout(TEST_TIMEOUT)
beforeEach('set timeout callback', function() {
setTimeout(function() {console.log('hi')}, TEST_TIMEOUT)
})
})
A few caveats:
1- I'm not sure if it'd affect a test's result. But if it were to, I'd setTimeout with TEST_TIMEOUT minus some time to make sure my assertions run before the mochajs timeout fires.
2- On the contrary, if the goal of the callback is not to do some assertions but to run some code (e.g., cleanup or logging), I'd set the timeout to be TEST_TIMEOUT plus some time.
Starting with mocha v4 (or with --no-exit flag in previous versions), as long as you have a timeout (or other handlers/the process didn't terminate), mocha will not force exit as it used to, so you can be comfortable your code will run.

Related

Cypress cy.wait() only waits for the first networkcall, need to wait for all calls

I would like to wait until the webpage is loaded with items. Each is getting retreived with a GET.
And I would like to wait on all these items until the page is loaded fully. I already made a interceptions for these. Named: 4ItemsInEditorStub
I have tried cy.wait('#4ItemsInEditorStub.all')
But this gives an timeout error at the end.
How can I let Cypress wait untill all "4ItemsInEditorStub" interceptions are completed?
Trying to wait on alias.all won't work -- Cypress has no idea what .all means in this context, or what value it should have. Even after your 4 expected calls are completed, there could be a fifth call after that (Cypress doesn't know). alias.all should only be used with cy.get(), to retrieve all yielded calls by that alias.
Instead, if you know that it will always be four calls, you can just wait four times.
cy.wait('4ItemsInEditorStub')
.wait('4ItemsInEditorStub')
.wait('4ItemsInEditorStub')
.wait('4ItemsInEditorStub');
You can either hard code a long enough wait (ie. cy.wait(3_000)) to cover the triggered request time and then use cy.get('#4ItemsInEditorStub.all')
cy.wait(10_000)
cy.get('#4ItemsInEditorStub.all')
// do some checks with the calls
or you can use unique intercepts and aliases to wait on all 4
cy.intercept('/your-call').as('4ItemsInEditorStub1')
cy.intercept('/your-call').as('4ItemsInEditorStub2')
cy.intercept('/your-call').as('4ItemsInEditorStub3')
cy.intercept('/your-call').as('4ItemsInEditorStub4')
cy.visit('')
cy.wait([
'#4ItemsInEditorStub1',
'#4ItemsInEditorStub2',
'#4ItemsInEditorStub3',
'#4ItemsInEditorStub4',
])
There is a package cypress-network-idle that makes the job simple
cy.waitForNetworkIdlePrepare({
method: 'GET',
pattern: '**/api/item/*',
alias: 'calls',
})
cy.visit('/')
// now wait for the "#calls" to finish
cy.waitForNetworkIdle('#calls', 2000) // no further requests after 2 seconds
Installation
# install using NPM
npm i -D cypress-network-idle
# install using Yarn
yarn add -D cypress-network-idle
In cypress/support/e2e.js
import 'cypress-network-idle'
Network idle testing looks good, but you might find it difficult to set the right time period, which may change each time you run (depending on network speed).
Take a look at my answer here Test that an API call does NOT happen in Cypress.
Using a custom command you can wait for a maximum number of calls without failing if there are actually less calls.
For example, if you have 7 or 8 calls, setting the maximum to 10 ensures you wait for all of them
Cypress.Commands.add('maybeWaitAlias', (selector, options) => {
const waitFn = Cypress.Commands._commands.wait.fn
return waitFn(cy.currentSubject(), selector, options)
.then((pass) => pass, (fail) => fail)
})
cy.intercept(...).as('allNetworkCalls')
cy.visit('/');
// up to 10 calls
Cypress._.times(10, () => {
cy.maybeWaitAlias('#allNetworkCalls', {timeout:1000}) // only need short timeout
})
// get array of all the calls
cy.get('#allNetworkCalls.all')
.then(calls => {
console.log(calls)
})

Cypress retries making test always pass

I am attempting to use the Cypress (at 7.6.0) retries feature as per https://docs.cypress.io/guides/guides/test-retries#How-It-Works
but for some reason it seems to be not working, in that a test that is guaranteed to always fail:
describe('Deliberate fail', function() {
it('Make an assertion that will fail', function() {
expect(true).to.be.false;
});
});
When run from the command line with the config retries set to 1,
npx cypress run --config retries=1 --env server_url=http://localhost:3011 -s cypress/integration/tmp/deliberate_fail.js
it seems to pass, with the only hint that something is being retried being the text "Attempt 1 of 2" and the fact that a screenshot has been made:
The stats on the run look also to be illogical:
1 test
0 passing
0 failing
1 skipped (but does not appear as skipped in summary)
Exactly the same behavior when putting the "retries" option in cypress.json, whether as a single number or options for runMode or openMode.
And in "open" mode, the test does not retry but just fails.
I am guessing that I'm doing something face-palmingly wrong, but what?
I think your problem is that you are not testing anything. Cypress will retry operations that involve the DOM, because the DOM takes time to render. Retrying is more efficient than a straight wait, because it might happen quicker.
So I reckon because you are just comparing 2 literal values, true and false, Cypress says, "Hey, there is nothing to retry here, these two values are never going to change, I'm outta here!"
I was going to say, if you set up a similar test with a DOM element, it might behave as you are expecting, but in fact it will also stop after the first attempt, because when it finds the DOM element, it will stop retrying. The purpose of the retry is to allow the element to be instantiated rather than retrying because the value might be different.
I will admit that I could be wrong in this, but I have definitely convinced myself - what do you think?
Found the cause .. to fix the problem that Cypress does not abort a spec file when one of the test steps (the it() steps) fails, I had the workaround for this very old issue https://github.com/cypress-io/cypress/issues/518 implemented
//
// Prevent it just running on if a step fails
// https://github.com/cypress-io/cypress/issues/518
//
afterEach(function() {
if (this.currentTest.state === 'failed') {
Cypress.runner.stop()
}
});
This means that a describe() will stop on fail, but does not play well with the retry apparently.
My real wished-for use case is to retry at the describe() level, but that may ne expensive as the above issue just got resolved but the solution is only available to those on the Business plan at $300/mo. https://github.com/cypress-io/cypress/issues/518#issuecomment-809514077

Cypress: how to wait for all requests to finish

I am using cypress to test our web application.
In certain pages there are different endpoint requests that are executed multiple times. [ e.g. GET /A GET /B GET /A].
What would be the best practise in cypress in order to wait for all requests to finish and guarantee that page has been fully loaded.
I don't want to use a ton cy.wait() commands to wait for all request to be processed. (there are a lot of different sets of requests in each page)
You can use the cy.route() feature from cypress. Using this you can intercept all your Get requests and wait till all of them are executed:
cy.server()
cy.route('GET', '**/users').as('getusers')
cy.visit('/')
cy.wait('#getusers')
I'm sure this is not recommended practice but here's what I came up with. It effectively waits until there's no response for a certain amount of time:
function debouncedWait({ debounceTimeout = 3000, waitTimeout = 4000 } = {}) {
cy.intercept('/api/*').as('ignoreMe');
let done = false;
const recursiveWait = () => {
if (!done) {
// set a timeout so if no response within debounceTimeout
// send a dummy request to satisfy the current wait
const x = setTimeout(() => {
done = true; // end recursion
fetch('/api/blah');
}, debounceTimeout);
// wait for a response
cy.wait('#ignoreMe', { timeout: waitTimeout }).then(() => {
clearTimeout(x); // cancel this wait's timeout
recursiveWait(); // wait for the next response
});
}
};
recursiveWait();
}
According to Cypress FAQ there is no definite way. But I will share some solutions I use:
Use the JQuery sintax supported by cypress
$('document').ready(function() {
//Code to run after it is ready
});
The problem is that after the initial load - some action on the page can initiate a second load.
Select an element like an image or select and wait for it to load. The problem with this method is that some other element might need more time.
Decide on a maindatory time you will wait for the api requests (I personaly use 4000 for my app) and place a cy.wait(mandatoryWaitTime) where you need your page to be loaded.
I faced the same issue with our large Angular application doing tens of requests as you navigate through it.
At first I tried what you are asking: to automatically wait for all requests to complete. I used https://github.com/bahmutov/cypress-network-idle as suggested by #Xiao Wang in this post. This worked and did the job, but I eventually realized I was over-optimizing my tests. Tests became slow. Test was waiting for all kinds of calls to finish, even those that weren't needed at that point in time to finish (like 3rd party analytics etc).
So I'd suggest not trying to wait for everything at a step, but instead finding the key API calls (you don't need to know the full path, even api/customers is enough) in your test step, use cy.intercept() and create an alias for it. Then use cy.wait() with your alias. The result is that you are waiting only when needed and only for the calls that really matter.
// At this point, there are lots of GET requests that need to finish in order to continue the test
// Intercept calls that contain a GET request with a request path containing /api/customer/
cy.intercept({ method: 'GET', url: '**/api/customer/**' }).as("customerData");
// Wait for all the GET requests with path containing /api/customer/ to complete
cy.wait("#customerData");
// Continue my test knowing all requested data is available..
cy.get(".continueMyTest").click()

After setting an it() timeout, my next test does not load

I'm interested in setting spec level timeouts so my automation can resume past failed specs quicker, rather than having to wait for the global jasmine timeout to trigger.
I've recently discovered one can supply the timeout argument in milliseconds as an argument to the spec's it() function.
Using that, I DO get the timeout error, but the next test does not load, I'm just stuck with:
"A Jasmine spec timed out. Resetting the WebDriver Control Flow."
Here's what my code looks like:
it('Verify login', function () {
baseFuncs.login();
browser.driver.sleep(3000);
},12000);
I suspect for some reason after this spec timeout, the global jasmine.DEFAULT_TIMEOUT_INTERVAL timeout is still waiting, and respected?

AJAX (XmlHttpRequest) timeout length by browser

I've been scouring the web trying to find a straight answer to this. Does anyone know the default timeout lengths for ajax request by browser? Also by version if it's changed?
According to the specs, the timeout value defaults to zero, which means there is no timeout. However, you can set a timeout value on the XHR.timeout property; the value is in milliseconds.
Sources:
http://www.w3.org/TR/2011/WD-XMLHttpRequest2-20110816/#the-timeout-attribute
http://msdn.microsoft.com/en-us/library/cc304105(v=vs.85).aspx
I don't think browsers have a timeout for AJAX, there is only synchronous or asynchronous requests; synchronous - first freezes the JavaScript execution until the request returns,
asynchronous - does not freeze JavaScript execution, it simply takes the request out of the execution flow, and if you have a callback function it will execute the the function in parallel with the running scripts (similar to a thread)
**sync flow:**
running JS script
|
ajax
(wait for response)
|
execute callback
|
running JS script
**async flow:**
running JS script
|
ajax --------------------
| |
running JS script execute callback
I did a modest amount of testing. To test I loaded my website, stopped the local server and then attempted an AJAX request. I set the timeout to something low like 1000ms until I could ensure I had minimal code (you must put the xhr.timeout after open and before send).
Once I got it working my initial goal was to determine the appropriate amount of time to allow however I was surprised how quickly the timeout would be outright ignored by browsers. My goal reformed in to trying to determine what the maximum timeout could be before error handling was no longer viable.That means past these fairly short spans of time your timeout handler script will not work at all. What I found was pretty pathetic.
Chrome 60: 995ms, 996ms will throw a dirty evil error in to the console.
Firefox 52 ESR: ~3000ms, position of mouse or other issue may cause no response around or just under three seconds.
So...
xhr.open(method,url,true);
xhr.timeout = 995;//REALLY short
xhr.send(null);
xhr.ontimeout = function ()
{
//Code will only execute if at or below *effective* timeouts list above.
//Good spot to make a second attempt.
}
So if your timeout is set higher than 995ms Chrome will ignore your code and puke on your nice clean empty console that you worked hard to keep clean. Firefox is not much better and there are unreliable requests that just timeout for well beyond any patience I have and in doing so ignore the ontimeout handler.
Browser does have a timeout value, behavior depends upon browser chrome has timeout value of 5 minutes and after 5 minutes it does resend ajax call

Resources