I have API call when loading a chat app, 30 calls/group (it's to load last 30 messages on each group). Let's say in a case, I test a user which has 2 groups only. So I expect to see 60 API calls for this.
I tried with following code.
it('Call 30 group messages APIs for every favorite group', () => {
cy.server()
cy.route(awsUrl + '/**').as('apiMessageContent')
for (let i = 0; i < 60; i++) {
cy.wait('#apiMessageContent', { timeout: 30000 }).then(res => {
expect(res.status).not.to.be.null
})
}
})
But the result cypress randomly only can capture 28-30 API calls, and other route waits after that are failing. In fact, in cypress left panel I can see the 60 XHR are all listed. What is the solution for this?
cy.wait can accept an array of Aliases, so you might be able to wait for #apiMessageContent in this way, rather than looping and waiting 60 times. Although it's unclear why your solution does not work.
cy.wait(Array(60).fill('#apiMessageContent'), { timeout: 30000 }).then((xhrs) => {
xhrs.forEach((res) => {
expect(res.status).not.to.be.null
})
})
Related
We recently installed datadogRUM in our application and now so many DD events kick off in my cypress test that they cause a timeout and failure
I have tried cy.intercept in multiple ways:
cy.intercept('POST', 'https://rum.browser-intake-datadoghq.com/*', {
statusCode: 202,
body: {
},
}).as('datadogRUM');
cy.intercept('POST', 'https://rum-http-intake.logs.datadoghq.com/*', {});
cy.intercept(/\.*datadog.*$/, (req) => {
console.log('DATADOG INTERCEPTED');
req.reply("console.log('datadog intercept');");
});
cy.intercept({
method: 'POST',
url: '/\.*datadog.*$/'
}, req => {
req.destroy();
});
cy.intercept('POST', 'https://rum-http-intake.logs.datadoghq.com/*', { forceNetworkError: true });
just to start. I feel like I've tried every possible variation. I also created a cypress.json file in my /cypress folder
{
"blockHosts": "*datadoghq.com/*"
}
I get hundreds of calls back in my network tab to https://rum.browser-intake-datadoghq.com/api/v2/rum with the preview of console.log('datadog intercept') as I've intercepted them. They all display the solid blue line as if they are being intercepted and blocked. When I set the intercept to an alias I see the alias in my cypress runner window. But there are no 503s or 404s anywhere. The page still fills up with events, cypress gets overloaded, and my test times out.
I even tried copying the data-dog-rum.ts from the src/utils folder to cypress/utils and either commenting out everything or setting the sampleRate to 0, no dice.
EDIT: I am able to get the test passing by adding
Cypress.on('uncaught:exception', () => {
// returning false here prevents Cypress from
// failing the test
return false;
});
to my support/index.js but now whether I add a cy.intercept in my test makes absolutely no difference. The page still fills up with datadog requests regardless, and whether they come back as 200/pending/cancelled, they still delay a single it block in a spec to where it takes 60 seconds to run instead of approx 10 seconds
You can use javascript to perform the stub inside the routeHandler
cy.intercept('*', (req) => { // look at everything
if (req.url.includes('datadoghq')) { // add more conditions if needed
req.reply({}) // prevent request reaching the server
}
})
blockhosts should work with
Pass only the host
{
"blockHosts": "*datadoghq.com"
}
From response A (/list.json) my app receives a list of items. Based on the output of A, my app makes another set of requests B for individual items (/one.txt, /two.txt, ...).
Now in my test I want to make sure that all responses B return HTTP 200.
Waiting (cy.wait) for response A is fine. However, waiting for responses B is more difficult, because I have to start waiting just upon receiving response A where I learn about responses B.
I tried 2 options:
start waiting inside of cy.wait of response A - code,
start waiting outside of cy.wait of response A - code
Neither of those work. With option 1 I get
`cy.wait()` timed out waiting `5000ms` for the 1st request to the route: `one.txt`. No request ever occurred
And with option 2 I get a pass, even though /two.txt doesn't exist. Looks like cy.wait for responses B is added after the responses were received
Since all requests are triggered off the visit, and are dynamic, you need a single intercept that handles all requests.
To me that means adding some javascript and dynamic aliases.
// avoid status code 304, disable browser cache
Cypress.automation('remote:debugger:protocol', {
command: 'Network.clearBrowserCache'
})
describe('spec', () => {
it('test', () => {
let items = [];
cy.intercept('GET', '*', (req) => {
const slug = req.url.substring(req.url.lastIndexOf('/') + 1)
if (slug === 'list.json') {
req.alias = 'list'
}
if (items.includes(slug)) {
req.alias = 'item'
}
req.continue((res) => {
if (slug === 'list.json')) {
items = res.body;
}
})
})
cy.visit('https://demo-cypress.netlify.app');
cy.wait('#list') // wait for list
.then(() => { // now items is populated
for (let item of items) { // really just need the count
cy.wait('#item').then(interception => { // wait n-times
expect(interception.response.statusCode).to.eq(200);
})
}
})
})
})
In my app after login in from a clean state, there are a series of sync queries that are being fired to make sure that the local data is updated. There is a loading screen while this is happening. I just need to cypress to wait for all these calls to finish before performing the test.
cy.intercept() is identifying the call, but cy.wait() only waits for the first one to be finished.
Is there a way to create the alias dynamilcally or have the application wait for the spinner to disapper?
describe('Navigation', function () {
beforeEach(function () {
// Programmatically login via Amazon Cognito API
cy.intercept('POST', '**/graphql').as('dataStore');
cy.loginByCognitoApi(Cypress.env('cognito_username'), Cypress.env('cognito_password'));
cy.wait(['#dataStore']);
});
it('shows logged in', function () {
cy.get('[data-test=logo]').should('be.visible');
});
You can repeat wait on a single intercept, so count up the number of orange dataStore tags (looks like 11) and wait that amount of times
cy.intercept('POST', '**/graphql').as('dataStore');
cy.loginByCognitoApi(Cypress.env('cognito_username'), Cypress.env('cognito_password'));
Cypress._.times(11, () => {
cy.wait('#dataStore')
})
Or it might be 10 - looking at the route defn. In any case, experiment. The app should be consistent in the calls it makes.
I had a similar case. What I do is store an array of objects in a different file and each object represents a specific test scenario. That way you can iterate through your test cases an assign an alias dynamically.
So you could do something like this:
beforeEach(function () {
yourArray.forEach((testcase) => {
cy.intercept('POST', '**/graphql').as(`${testcase.testname}datastore`);
cy.loginByCognitoApi(Cypress.env('cognito_username'),
Cypress.env('cognito_password'));
cy.wait(`#${testcase.testname}datastore`);
}
});
If the number of requests aren't consistent, something I've done is the following (I've since put this in a command to use in multiple places):
cy.intercept('POST', '**/graphql').as('dataStore');
cy.loginByCognitoApi(Cypress.env('cognito_username'),Cypress.env('cognito_password'));
cy.get('#dataStore.all').then(xhrs => cy.wait(Array(xhrs.length).fill('#dataStore')));
Doing a wait on the alias with "all" returns all of the calls made to aliased route that Cypress has seen since the alias was made.
#user16695029 is a great solution.
If you run into the issue of API calls not being predictable (kicked off by a timer async etc), then keeping track of API call count might be useful:
at the start of your test code
let responseCounter = 0;
cy.intercept({ method: 'POST', url: '/save', middleware: true }, req => {
req.on('response', (res) => {
responseCounter++;
})
}).as('save')
then later
let expectedSaveCount = 12;
Cypress._.times(expectedSaveCount - responseCounter, () => {
cy.wait('#save')
})
cy.get('#save.all').should('have.length', expectedSaveCount)
Is it possible to wait until an element is visible?
cy.get('[data-test=submitIsVisible]').should('be.visible');
This should error if the submit button is not visible. I want to wait until the submit button is visible.
The primary use case is visual testing, i.e. taking a screenshot of the page.
You can wait for the element to be visible like so:
// Give this element 10 seconds to appear
cy.get('[data-test=submitIsVisible]', { timeout: 10000 }).should('be.visible');
According to Cypress's Documentation:
DOM based commands will automatically retry and wait for their corresponding elements to exist before failing.
Cypress offers you many robust ways to query the DOM, all wrapped with retry-and-timeout logic.
Another way to wait for an element’s presence in the DOM is through timeouts. Cypress commands have a default timeout of 4 seconds, however, most Cypress commands have customizable timeout options. Timeouts can be configured globally or on a per-command basis. Check the customizable timeout options list here.
In some cases, your DOM element will not be actionable. Cypress gives you a powerful {force:true} option you can pass to most action commands.
Caveat:
As Anthony Cregan pointed out, the .should('be.visible') assertion checks whether an element is visible on the page, not necessarily in the viewport. This means that this assertion will return true even if the element is not within the visible area of the screen when the test is run.
Further recommended readings:
Retry-ability.
Interacting with elements.
Updated for Cypress v12
If you want to see exactly how Cypress waits for something to become visible, follow this example.
Using this code, you can check out how the delay and the timeout can affect the passing or failing of the .should('be.visible') assertion.
Steps
Add a simple page to a VSCode project containing Cypress v12.1.0
Call it index.html
<html>
<body>
<h2>API fetched data</h2>
<span>will become visible here</span>
</body>
<script>
fetch('https://jsonplaceholder.typicode.com/posts/1')
.then(response => response.json())
.then(data => document.querySelector('span').innerText = data.title )
</script>
</html>
Right-click index.html and choose "Open with Live Server" to activate the page.
Add this test to see how Cypress waits for the API data
describe('test the waiting of API data', () => {
const timings = [
{ delay: 0, timeout: 4000 }, // default, passes
{ delay: 2000, timeout: 4000 }, // passes
{ delay: 4000, timeout: 4000 }, // flaky
{ delay: 5000, timeout: 4000 }, // fails
{ delay: 5000, timeout: 10000 }, // passes
]
timings.forEach(timing => {
const {delay, timeout} = timing;
it(`delayed API by ${delay} ms, command timout is ${timeout} ms`, () => {
cy.intercept('https://jsonplaceholder.typicode.com/posts/1', (req) => {
req.continue((res) => res.setDelay(delay))
})
cy.visit('http://127.0.0.1:5500/index.html')
cy.contains('sunt aut facere', {timeout})
.should('be.visible')
})
})
})
Result
This shows that the longer the delay in receiving the data, the bigger the timeout needed on the visibility assertion.
you can also do it by passing below script into your cypress.config.js files
e2e: {
defaultCommandTimeout: 25000,
}
pass defaultCommandTimeout as per your requirement.
Try element.should('have.length.greaterThan', 0).and('be.visible')
Here's my setup:
A Python 3.6 lambda function, which I want to keep pre-warmed at a certain concurrency level (say, 10). The lambda's initialization is painful enough that I don't want to inflict this cost on visitors at random. I call these lambdas "workers"
A Node lambda function which runs every 5 minutes to try to pre-warm 10 instances. It uses the Event invocation type for 9 of them, and RequestResponse for 1. There's only either one or zero of this lambda running at any one time. I call this a "warmer".
I followed the guidelines at [https://www.jeremydaly.com/lambda-warmer-optimize-aws-lambda-function-cold-starts/], namely:
Don’t ping more often than every 5 minutes
Invoke the function directly (i.e. don’t use API Gateway to invoke it)
Pass in a test payload that can be identified as such
Create handler logic that replies accordingly without running the whole function
Here's a problem: this works great for several minutes. Then, as I watch the logs, I start to get timeouts from my worker lambda invocations. The timeouts quickly take over all the invocations that the warmer is trying to launch.
Now, no worker lambdas are prewarmed any more. But the warmer keeps on trying, on a Cloudwatch event cron schedule, suffering 100% timeouts. Finally, Lambda stops trying to launch my worker lambdas at all. It feels like some aspect of Lambda's getting its state scrambled. The only way to recover is to re-deploy the lambda. That buys me another hour with pre-warmed lambdas working.
Questions:
How do I get visibility into why my worker lambdas start timing out, and then become completely non-responsive?
What is the definition of a "Concurrent Execution"? On the main Lambda dashboard it shows me this chart of them. Yet, it seems to have more than twice as many Concurrent Executions as I'm requesting.
Here's the warmup lambda code (Node):
// warmer
"use strict";
/** Generated by Serverless WarmUP Plugin at ${new Date().toISOString()} */
const aws = require("aws-sdk");
aws.config.region = "${this.options.region}";
const lambda = new aws.Lambda({httpOptions: {timeout: 60000}});
const functionNames = ${JSON.stringify(functionNames)};
const delay = ms => new Promise(res => setTimeout(res, ms))
const concurrency = 10;
module.exports.warmUp = async (event, context, callback) => {
console.log("Warm Up Start");
const invokes = await Promise.all(functionNames.map(async (functionName) => {
let invocations = [];
try {
for(let i=1;i <= concurrency;i++){
let params = {
FunctionName: functionName,
InvocationType: (i===concurrency)?'RequestResponse': 'Event',
LogType: 'None',
Qualifier: process.env.SERVERLESS_ALIAS || "$LATEST",
Payload: JSON.stringify({
source: 'serverless-plugin-warmup',
'__WARMER_INVOCATION__': i,
'__WARMER_CONCURRENCY__': concurrency,
'__WARMER_REQUESTED__': new Date().toISOString(),
})
};
invocations.push(lambda.invoke(params).promise())
}
return await delay(75).then(Promise.all(invocations.map(p => p.catch(e => e)))
.then(results => console.log('results', results))
.catch(e => {
console.log(e);
return e;
}
))
} catch (e) {
console.log(\`Warm Up Invoke Error: \${functionName}\`, e);
return false;
}
}));
console.log(\`Warm Up Finished\`);
}
And here's the worker lambda (Python):
source = event.get('source')
if source == 'serverless-plugin-warmup':
time.sleep(0.05)
print(event)
return lambda_gateway_response(200, {"status": "lambda warmup"})
It was the warmer (Node) lambda going haywire, even though all the logs pointed at the worker (Python) lambdas. After setting context.callbackWaitsForEmptyEventLoop = false, the problem disappeared.