Execute event at an exact time in cypress - cypress

I want my cypress test script to click a button as soon as the current time is past 20:00 UTC
I have a cron job on my server which triggers the e2e flow a few minutes before 20:00
So far so good, the test is being executed, I have a loop checking the time and waiting until 20:00 is reached, then hit the button and continue the test
for (let i = 0; i < 300; i++) {
currentHour = (new Date()).getUTCHours();
if(currentHour == 20)
{
cy.get(......).click();
break;
}
cy.wait(500);
}
But the way I understand cypress is built, new Date() always stays the same because all javascript is executed right away, so it contains the timestamp of test initialization...
I've been trying some stuff with cy.task, you can return a new date there but you can't use await to wait for that correct date retrieval since it doesn't return an acutal promise; more something from cypress itself
What else can I try? It seems an easy test but so hard to get exactly what I want

To make Cypress stay on the same date, you need to use cy.clock().
This command can be used to set the system to an exact date and time, but not allow the time to move forward until you issue the cy.tick() command.
It's hard to see what you are expecting to test, but this is how your test might look
cy.clock(new Date(2023, 2, 14, 19, 59, 30))
cy.visit('/')
// test something before the deadline
cy.tick(30_000)
cy.get(......).click()
// test something after the deadline

I couln't use clock as the time I am targetting is defined on server; the application only allows certain actions in a very small timeframe.
Found a solution though with the very use full Cypress wait until plugin
I created a task to get the actual date
cypress.config.js
const { defineConfig } = require("cypress");
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
// implement node event listeners here
on('task', {
getCurrentHour() {
const date = new Date();
console.log(date + ' - ' + 'Checking current hour: ' + date.getUTCHours())
return date.getUTCHours();
}
})
},
},
});
spec.cy.ts
cy.waitUntil(() => cy.task("getCurrentHour").then(hour => hour === targetUTCHour), {timeout: 180000, // waits up to 2000 ms, default to 5000
interval: 1000});
// Execute code here
);

Related

Cypress - Unable to run multiple tests

I am very new to cypress automation and have been following though some examples and have ran into an issue that does not appear to be addressed in any video I have seen, where multiple tests in the same 'describe' do not run as expected.
If I create the following code & run it, it all works perfectly:-
describe('My First Test', () => {
it('Open Google', () => {
cy.visit('https://google.co.uk')
cy.get('#L2AGLb > .QS5gu').click()
cy.get('.gLFyf').type('Automation Step by Step{Enter}')
})
})
I have then attempted to split up the test into individual tests, as follows:-
describe('My First Test', () => {
it('Open Google', () => {
cy.visit('https://google.co.uk')
})
it('Confirm warning', () => {
cy.get('#L2AGLb > .QS5gu').click()
cy.get('.gLFyf').type('Automation Step by Step{Enter}')
})
it('Confirm warning', () => {
cy.get('.gLFyf').type('Automation Step by Step{Enter}')
})
})
The problem now is that after opening Chrome and then going into the next test, which should allow me to type the text, a 'default blank page' is displayed and the tests then fail.
What am I missing here to be able to run these three tests fully?
Code in VS Code
Error after opening Chrome & attempting to type in box
As above really, I was hoping to be able to run all three simple tests together.
EDIT - I rolled back my version of Cypress to 10.10.0 and it works perfectly, so no idea what has changed on the latest version released yesterday.
Try it with Test Isolation turned to false.
Best Practice: Tests should always be able to be run independently from one another and still pass
This was added in Cypress 12.0.0.
But if you want to play without it,
Test Isolation Disabled
testIsolation
beforeEach test
true
- clears page by visiting about:blank
- clears cookies in all domains
- local storage in all domains
- session storage in all domains
false
does not alter the current browser context
cypress.config.js
const { defineConfig } = require('cypress')
module.exports = defineConfig({
e2e: {
testIsolation: false,
},
})
You should visit your page for every test. You can put the cy.visit in a beforeEach function.

Cypress Test timeout on then() not respecting timeout option

I've been trying to figure out when the .then() function in Cypress is not respecting the timeout option. The then() function is asserting a few times and ultimately fails because my call is still waiting for results. What I was hoping for is setting it to 90000 so that it just spends more time asserting.
commonPage
.getSearchInput()
.clear({ force: true })
.type('My Search Term', { force: true })
.type('{enter}')
.then({ timeout: 90000, }, () => {
accountRoles.getLabel().should('have.contain.text', 'My Search Term');
});
getLabel(): Cypress.Chainable<JQuery<HTMLElement>> {
return cy.get('div[col-id="label"]');
}
If you want to increase the timeout for .should('contain.text', 'My Search Term'), putting it on the .then() wrapping is incorrect.
The .should() will retry only the .getLabel(), so whatever is in there needs the timeout option.
One of the down-sides of using page objects in Cypress.
For example, this is the usual pattern
cy.get('div[col-id="label"]', {timeout:90000})
.should('contain.text', 'My Search Term')
This will repeat cy.get('div[col-id="label"]' until .should('contain.text', 'My Search Term') becomes true, or times out.
If your app is sending a request/call, intercept that and wait on it so you don't have to continuously play a guessing game with a timeout. Also, 90 seconds is a lot of time to complete for what looks to be search suggestions.
cy.intercept('GET','/your-call').as('yourCall')
commonPage
.getSearchInput()
.clear({ force: true })
.type('My Search Term', { force: true })
.type('{enter}')
cy.wait('#yourCall')
accountRoles.getLabel().should('have.contain.text', 'My Search Term')

Retries in Cypress and Before hooks

Morning all. I have a slightly unusual design to my tests. A typical example might be
describe('1', () => {
describe('2', () => {
before()
describe('3', () => {
it('1')
// ...
it('n')
});
});
});
If there is a failure in one of the individual tests (it 1..n), I want to re-run ALL of those tests, and run the "before" code first too - ie from "describe 2". If I use a before hook then re-runs don't trigger that again. If I change to a beforeEach, then it gets called before each and every "it" block, which I don't want.
Effectively, each it block is a test check, describe 3 is a test step, describe 2 a test spec, and describe 1 a test "group"
Can anyone suggest a way I can re-run a test spec (describe 2) when one test check fails, including re-running the before code for that spec?
(I know this is probably anti-pattern etc, but....)
You can externalise the before() callback function, and use the test:after:run event to trigger it on a retry.
I haven't tested this extensively, but the gist is
const beforeCallback = () => {...}
before(beforeCallback)
Cypress.on('test:after:run', (result) => {
if (result.currentRetry < result.retries && result.state === 'failed') {
beforeCallback()
}
})
it('fails', {retries:3}, () => expect(false).to.eq(true)) // failing test to check it out

Load/stress test in a SPA with Hasura Cloud Graphql as a backend and subscriptions

I'm trying to do a performance test on a
SPA with a Frontend in React, deployed with Netlify
As a backend we're using Hasura Cloud Graphql (std version) https://hasura.io/, where everything from the client goes directly through Hasura to the DB.
DB is in Postgress housed in Heroku (Std 0 tier).
We're hoping to be able to have around 800 users simultaneous.
The problem is that i'm loss about how to do it or if i'm doing it correctly, seeing how most of our stuff are "subscriptions/mutations" that I had to transform into queries. I tried doing those test with k6 and Jmeter but i'm not sure if i'm doing them properly.
k6 test
At first, i did a quick search and collected around 10 subscriptions that are commonly used. Then i tried to create a performance test with k6 https://k6.io/docs/using-k6/http-requests/ but i wasn't able to create a working subscription test so i just transform each subscription into a query and perform a http.post with this setup:
export const options = {
stages: [
{ duration: '30s', target: 75 },
{ duration: '120s', target: 75 },
{ duration: '60s', target: 50 },
{ duration: '30s', target: 30 },
{ duration: '10s', target: 0 }
]
};
export default function () {
var res = http.post(prod,
JSON.stringify({
query: listaQueries.GetDesafiosCursosByKey(
keys.desafioCursoKey
)}), params);
sleep(1)
}
I did this for every query and ran each test individually. Unfortunately, the numbers i got were bad, and somehow our test environment was getting better times than production. (The only difference afaik is that we're using Hasura Cloud for production).
I tried to implement websocket, but i couldn't getthem work and configure them to do a stress/load test.
K6 result
Jmeter test
After that, i tried something similar with Jmeter, but again i couldn't figure how to set up a subscription test (after i while, i read in a blog that jmeter doesn't support it
https://qainsights.com/deep-dive-into-graphql-in-jmeter/ ) so i simply transformed all subscriptions into a query and tried to do the same, but the numbers I was getting were different and much higher than k6.
Jmeter query Config 1
Jmeter query config 2
Jmeter thread config
Questions
I'm not sure if i'm doing it correctly, if transforming every subscription into a query and perform a http request is a correct approach for it. (At least I know that those queries return the data correctly).
Should i just increase the number of VUS/threads until i get a constant timeout to simulate a stress test? There were some test that are causing a graphql error on the website Graphql error, and others were having a
""WARN[0059] Request Failed error="Post \"https://xxxxxxx-xxxxx.herokuapp.com/v1/graphql\": EOF""
in the k6 console.
Or should i just give up with k6/jmeter and try to search for another tool to perfom those test?
Thanks you in advance, and sorry for my English and explanation, but i'm a complete newbie at this.
I'm not sure if i'm doing it correctly, if transforming every
subscription into a query and perform a http request is a correct
approach for it. (At least I know that those queries return the data
correctly).
Ideally you would be using WebSocket as that is what actual clients will most likely be using.
For code samples, check out the answer here.
Here's a more complete example utilizing a main.js entry script with modularized Subscription code in subscriptions\bikes.brands.js. It also uses the Httpx library to set a global request header:
// main.js
import { Httpx } from 'https://jslib.k6.io/httpx/0.0.5/index.js';
import { getBikeBrandsByIdSub } from './subscriptions/bikes-brands.js';
const session = new Httpx({
baseURL: `http://54.227.75.222:8080`
});
const wsUri = 'wss://54.227.75.222:8080/v1/graphql';
const pauseMin = 2;
const pauseMax = 6;
export const options = {};
export default function () {
session.addHeader('Content-Type', 'application/json');
getBikeBrandsByIdSub(1);
}
// subscriptions/bikes-brands.js
import ws from 'k6/ws';
/* using string concatenation */
export function getBikeBrandsByIdSub(id) {
const query = `
subscription getBikeBrandsByIdSub {
bikes_brands(where: {id: {_eq: ${id}}}) {
id
brand
notes
updated_at
created_at
}
}
`;
const subscribePayload = {
id: "1",
payload: {
extensions: {},
operationName: "query",
query: query,
variables: {},
},
type: "start",
}
const initPayload = {
payload: {
headers: {
"content-type": "application/json",
},
lazy: true,
},
type: "connection_init",
};
console.debug(JSON.stringify(subscribePayload));
// start a WS connection
const res = ws.connect(wsUri, initPayload, function(socket) {
socket.on('open', function() {
console.debug('WS connection established!');
// send the connection_init:
socket.send(JSON.stringify(initPayload));
// send the chat subscription:
socket.send(JSON.stringify(subscribePayload));
});
socket.on('message', function(message) {
let messageObj;
try {
messageObj = JSON.parse(message);
}
catch (err) {
console.warn('Unable to parse WS message as JSON: ' + message);
}
if (messageObj.type === 'data') {
console.log(`${messageObj.type} message received by VU ${__VU}: ${Object.keys(messageObj.payload.data)[0]}`);
}
console.log(`WS message received by VU ${__VU}:\n` + message);
});
});
}
Should i just increase the number of VUS/threads until i get a
constant timeout to simulate a stress test?
Timeouts and errors that only happen under load are signals that you may be hitting a bottleneck somewhere. Do you only see the EOFs under load? These are basically the server sending back incomplete responses/closing connections early which shouldn't happen under normal circumstances.
My expectation is that your test should be replicating the real user activity as close as possible. I doubt that real users will be sending requests to GraphQL directly and well-behaved load test must replicate the real life application usage as close as possible.
So I believe you should move to HTTP protocol level and mimic the network footprint of the real browser instead of trying to come up with individual GraphQL queries.
With regards to JMeter and k6 differences it might be the case that k6 produces higher throughput given the same hardware and running requests at maximum speed as it evidenced by kind of benchmark in the Open Source Load Testing Tools 2021 article, however given you're trying to simulate real users using real browsers accessing your applications and the real users don't hammer the application non-stop, they need some time to "think" between operations you should be getting the same number of requests for both load testing tools, if JMeter doesn't give you the load you want to conduct make sure to follow JMeter Best Practices and/or consider running it in distributed mode .

Nightwatch demo test

on the website, the nightwatch.js demo script is not valid anymore, Google changed the search engine result page html/css, so the earlier element access does not work. Tried some new ways using elements and xpath but I did not find a solution yet.
module.exports = {
'Demo test Google' : function (client) {
client
.url('http://www.google.com')
.waitForElementVisible('body', 1000)
.assert.title('Google')
.assert.visible('input[type=text]')
.setValue('input[type=text]', 'rembrandt van rijn')
.waitForElementVisible('button[name=btnG]', 1000)
.click('button[name=btnG]')
.pause(1000)
.assert.containsText('ol#rso li:first-child',
'Rembrandt - Wikipedia')
.end();
}
};
Any idea what to change that runs the test fully and returns correct results?
This one worked for me.
module.exports = {
'Demo test Google' : function (client) {
client
.url('http://www.google.com')
.waitForElementVisible('body', 1000)
.assert.title(`Google`)
.setValue('input[type=text]', ['rembrandt van rijn', client.keys.ENTER])
.waitForElementVisible('input[name=btnK]', 1000)
.click('input[name=btnK]')
.pause(1000)
.assert.containsText('div.srg > div:first-child',
'Rembrandt - Wikipedia')
.end();
}
};

Resources