I'm interested in setting spec level timeouts so my automation can resume past failed specs quicker, rather than having to wait for the global jasmine timeout to trigger.
I've recently discovered one can supply the timeout argument in milliseconds as an argument to the spec's it() function.
Using that, I DO get the timeout error, but the next test does not load, I'm just stuck with:
"A Jasmine spec timed out. Resetting the WebDriver Control Flow."
Here's what my code looks like:
it('Verify login', function () {
baseFuncs.login();
browser.driver.sleep(3000);
},12000);
I suspect for some reason after this spec timeout, the global jasmine.DEFAULT_TIMEOUT_INTERVAL timeout is still waiting, and respected?
Related
I have an 'afterEach' hook for each test. This will write some value to json.
The value for 'TestDetails' is available globally in in the test file.
afterEach(function () {
cy.readFile('cypress/dataFiles/data1.json').then(function (t1) {
t1["testDetails"] = testDetailsValue;
cy.writeFile('cypress/dataFiles/data1.json', JSON.stringify(t1));
})
})
The first test is login test and launching the browser and navigating to login page takes more than 10 seconds.
When test is executed, the below error is displayed:
CypressError: `cy.readFile("cypress/dataFiles/data1.json")` timed out after waiting `4000ms`.
Because this error occurred during a `after each` hook we are skipping the remaining tests in the current suite:
Due to this error, unable to write the value in json in after each hook.
Could anyone please help on this?
I think that means that your file doesn't exist. See https://docs.cypress.io/api/commands/readfile#Existence
I am attempting to use the Cypress (at 7.6.0) retries feature as per https://docs.cypress.io/guides/guides/test-retries#How-It-Works
but for some reason it seems to be not working, in that a test that is guaranteed to always fail:
describe('Deliberate fail', function() {
it('Make an assertion that will fail', function() {
expect(true).to.be.false;
});
});
When run from the command line with the config retries set to 1,
npx cypress run --config retries=1 --env server_url=http://localhost:3011 -s cypress/integration/tmp/deliberate_fail.js
it seems to pass, with the only hint that something is being retried being the text "Attempt 1 of 2" and the fact that a screenshot has been made:
The stats on the run look also to be illogical:
1 test
0 passing
0 failing
1 skipped (but does not appear as skipped in summary)
Exactly the same behavior when putting the "retries" option in cypress.json, whether as a single number or options for runMode or openMode.
And in "open" mode, the test does not retry but just fails.
I am guessing that I'm doing something face-palmingly wrong, but what?
I think your problem is that you are not testing anything. Cypress will retry operations that involve the DOM, because the DOM takes time to render. Retrying is more efficient than a straight wait, because it might happen quicker.
So I reckon because you are just comparing 2 literal values, true and false, Cypress says, "Hey, there is nothing to retry here, these two values are never going to change, I'm outta here!"
I was going to say, if you set up a similar test with a DOM element, it might behave as you are expecting, but in fact it will also stop after the first attempt, because when it finds the DOM element, it will stop retrying. The purpose of the retry is to allow the element to be instantiated rather than retrying because the value might be different.
I will admit that I could be wrong in this, but I have definitely convinced myself - what do you think?
Found the cause .. to fix the problem that Cypress does not abort a spec file when one of the test steps (the it() steps) fails, I had the workaround for this very old issue https://github.com/cypress-io/cypress/issues/518 implemented
//
// Prevent it just running on if a step fails
// https://github.com/cypress-io/cypress/issues/518
//
afterEach(function() {
if (this.currentTest.state === 'failed') {
Cypress.runner.stop()
}
});
This means that a describe() will stop on fail, but does not play well with the retry apparently.
My real wished-for use case is to retry at the describe() level, but that may ne expensive as the above issue just got resolved but the solution is only available to those on the Business plan at $300/mo. https://github.com/cypress-io/cypress/issues/518#issuecomment-809514077
I'm using Puppeteer sharp to render reports and part of that is executing user provided javascript to prepare the data for the report.
I use AddScriptTagAsync to add the scripts to the page then call the user provided script before rendering the report.
If the user provided javascript has an issue that causes an infinite loop (for example) then my call to EvaluateExpressionAsync could await forever:
await page.EvaluateExpressionAsync<dynamic>($"Prepare({DataObject});")
I can't pass a cancellation token to EvaluateExpressionAsync so I can't control it and there appear to be no timeouts available for this method.
I would like to limit it to a controllable number of seconds and then have it timeout.
Any suggestions on how to do this would be very much appreciated.
You would use WaitForExpressionAsync.
The idea of this method is executing the expression over a period of time until the result is truthy.
But if you make sure that your expression will always return a truthy value, WaitForExpressionAsync will timeout using the timeout you pass as an option.
This question has been asked before, but the answer given was to re-write the specific code that was used in that case.
The Mocha documentation only mentions changing the duration of timeouts, and not the behaviour on timeout.
In my case, I want to test code that under certain conditions does not have any side effects. The obvious way to do this is to call the code, wait for Mocha to time out, and then run some assertions on my spies, e.g.:
this.onTimeout(function() {
assert.equal(someObject.spiedMethod.called, false);
});
someAsyncFunction();
Is this possible, or do I have to resort to setting my own timeout, e.g.:
// 1. Disable the mocha timeout
this.timeout(0);
// 2. create my own timeout, which should fire before the mocha timeout
setTimeout(function() {
assert.equal(someObject.spiedMethod.called, false);
done();
}, 2000);
someAsyncFunction();
Mocha's own timeouts are designed only to detect when tests are considered to be slow beyond anything reasonable. An asynchronous computation could be buggy in such a way that causes it to never terminate. Rather than wait forever, Mocha lets you decide that after X time it should fail the test. As soon as the timeout is hit, Mocha gives up on the test that timed out.
So you have to set your own timeouts.
Was facing a similar problem & also didn't find a way to do that in mochajs.
However, if you want to run some code on test timeout, you could structure your code like this:
describe('something', function() {
const TEST_TIMEOUT = 10000
this.timeout(TEST_TIMEOUT)
beforeEach('set timeout callback', function() {
setTimeout(function() {console.log('hi')}, TEST_TIMEOUT)
})
})
A few caveats:
1- I'm not sure if it'd affect a test's result. But if it were to, I'd setTimeout with TEST_TIMEOUT minus some time to make sure my assertions run before the mochajs timeout fires.
2- On the contrary, if the goal of the callback is not to do some assertions but to run some code (e.g., cleanup or logging), I'd set the timeout to be TEST_TIMEOUT plus some time.
Starting with mocha v4 (or with --no-exit flag in previous versions), as long as you have a timeout (or other handlers/the process didn't terminate), mocha will not force exit as it used to, so you can be comfortable your code will run.
I've been scouring the web trying to find a straight answer to this. Does anyone know the default timeout lengths for ajax request by browser? Also by version if it's changed?
According to the specs, the timeout value defaults to zero, which means there is no timeout. However, you can set a timeout value on the XHR.timeout property; the value is in milliseconds.
Sources:
http://www.w3.org/TR/2011/WD-XMLHttpRequest2-20110816/#the-timeout-attribute
http://msdn.microsoft.com/en-us/library/cc304105(v=vs.85).aspx
I don't think browsers have a timeout for AJAX, there is only synchronous or asynchronous requests; synchronous - first freezes the JavaScript execution until the request returns,
asynchronous - does not freeze JavaScript execution, it simply takes the request out of the execution flow, and if you have a callback function it will execute the the function in parallel with the running scripts (similar to a thread)
**sync flow:**
running JS script
|
ajax
(wait for response)
|
execute callback
|
running JS script
**async flow:**
running JS script
|
ajax --------------------
| |
running JS script execute callback
I did a modest amount of testing. To test I loaded my website, stopped the local server and then attempted an AJAX request. I set the timeout to something low like 1000ms until I could ensure I had minimal code (you must put the xhr.timeout after open and before send).
Once I got it working my initial goal was to determine the appropriate amount of time to allow however I was surprised how quickly the timeout would be outright ignored by browsers. My goal reformed in to trying to determine what the maximum timeout could be before error handling was no longer viable.That means past these fairly short spans of time your timeout handler script will not work at all. What I found was pretty pathetic.
Chrome 60: 995ms, 996ms will throw a dirty evil error in to the console.
Firefox 52 ESR: ~3000ms, position of mouse or other issue may cause no response around or just under three seconds.
So...
xhr.open(method,url,true);
xhr.timeout = 995;//REALLY short
xhr.send(null);
xhr.ontimeout = function ()
{
//Code will only execute if at or below *effective* timeouts list above.
//Good spot to make a second attempt.
}
So if your timeout is set higher than 995ms Chrome will ignore your code and puke on your nice clean empty console that you worked hard to keep clean. Firefox is not much better and there are unreliable requests that just timeout for well beyond any patience I have and in doing so ignore the ontimeout handler.
Browser does have a timeout value, behavior depends upon browser chrome has timeout value of 5 minutes and after 5 minutes it does resend ajax call