How to run the single test with different data set in parallel by using cypress on single machine - parallel-processing

I just have the below Test.json file in the fixture folder:
[
{
"searchKeyword":"cypress"
},
{
"searchKeyword":"QA automation"
},
{
"searchKeyword":"stackoverflow"
}
]
The above file contains three different dataset.
I just have the below spec file and It contains one It (Test case) and It will run multiple times based on the above input.
Test.spec.js file:
describe("Run the test parallel based on the input data",() =>{
const baseUrl = "https://www.google.com/";
before("Login to consumer account", () => {
cy.fixture('Test').then(function (data) {
this.data = data;
})
});
it("Search the keyword", function () {
this.data.forEach((testData) =>{
cy.visit(baseUrl);
cy.xpath("//input[#name='q']").type(testData.searchKeyword);
cy.xpath("//input[#value='Google Search']").click();
cy.get("//ul/li[2]").should("be.visible");
});
});
});
The above code is working as expected. But I just want to run the above single test parallelly by using different dataset.
Example: Three browser instance open and it should pick three different data from the Test.json file and run the single test which is available in the Test.spec.js file.
I just need logic to implement for one of my project, But I'm not able to share the code which is more complex that's reason just create some dummy test data and test script to achieve my logic.
Can someone please share yours thoughts to achieve this.

One way to run multiple instances of Cypress in parallel is via the Module API, which is basically using a Node script to start up the multiple instances.
Node script
// run-parallel.js
const cypress = require('cypress')
const fixtures = require('./cypress/fixtures/Test.json')
fixture.forEach(fixture => {
cypress.run({
env: {
fixture
},
})
})
Test
describe("Run the test for given env data",() =>{
const testData = Cypress.env('fixture')
...
it("Search the keyword", function () {
cy.visit(baseUrl);
cy.xpath("//input[#name='q']").type(testData.searchKeyword);
...
});
});
Awaiting results
cypress.run() returns a promise, so you can collate the results as follows
Videos and screenshots are troublesome, since it tries to save all under the same name, but you can specify a folder for each fixture
const promises = fixtures.map(fixture => {
return cypress.run({
config: {
video: true,
videosFolder: `cypress/videos/${fixture.searchKeyword}`,
screenshotsFolder: `cypress/screenshots/${fixture.searchKeyword}`,
},
env: {
fixture
},
spec: './cypress/integration/dummy.spec.js',
})
})
Promise.all(promises).then((results) => {
console.log(results.map(result => `${result.config.env.fixture.searchKeyword}: ${result.status}`));
});

Related

Postman how to export api response from collection runner with iteration to a file using node script

I am completely new to writing node scripts in Postman.
My requirement:
I have an api to get user details. I want to iterate for n number of users. I created a Runner collection and it executes. But i want to write each request response to a file.
Can anyone help me how to do this?
I watched some youtube video https://www.youtube.com/watch?v=cCRmry10874 for this. But my case is i have runner collection with data file.
When i exported the collection, i dont get the different values from data file.
const newman = require('newman');
newman.run({
collection: require('./collection.json'),
reporters: 'cli'
}, (err) => {
if(err) { throw err; }
console.log('collection run complete');
});
const fs = require('fs');
fs.writeFile('response.txt', 'Some text', (error) => {
if(error) {
console.error(error);
}
})
Thanks
Can you try it?
newman.run(
{
collection: require("./collection.json"),
reporters: "cli",
iterationData: "./data.json",
},
(err, summary) => {
if (err) {
throw err;
}
const results = summary.run.executions;
results.forEach((result) => {
fs.appendFileSync("response.txt", result.response.text());
});
}
);
If you are not limited to using a textfile, I would suggest using htmlextra.
It provides an HTML webpage with your runs and response body.

Cypress: Using same fixture file in multiple test

I need to pass the url and other variable in multiple tests[it-function]. For 1st test code run successfully but for 2nd test it showing error. Is there any workaround or solution? My code is as follows
describe('Document Upload', function()
{
before(function () {
cy.fixture('Credential').then(function (testdata) {
this.testdata = testdata
})
})
//1st test
it('Login as manager',function()
{
const login = new loginPage()
cy.visit(this.testdata.baseUrl);
login.getUserName().type(this.testdata.userDocumentM)
login.getPassword().type(this.testdata.passwordDocumentM)
login.getLoginButton().click()
//Logout
login.getUser().click()
login.getLogout().click()
})
//2nd test
it('Create Documents',function()
{
const login = new loginPage()
cy.visit(this.testdata.baseUrl);
login.getUserName().type(this.testdata.userDocumentM)
})
})
The error is
I have tried with above and also using before function again but same error
before(function () {
cy.fixture('Credential').then(function (testdata) {
this.testdata = testdata
})
})
//2nd test
it('Create Documents',function()
{
const login = new loginPage()
cy.visit(this.testdata.baseUrl);
login.getUserName().type(this.testdata.userDocumentM)
})
Starting with Cypress version 12 Test Isolation was introduced. This now means the Mocha context (aka this) is completely cleaned between tests.
Mocha context
It used to be (undocumented) that the Mocha context could be used to preserve variables across tests, for example
before(function () {
cy.fixture("example.json").as('testdata') // preserved on "this"
})
it("check testdata 1", function () {
expect(this.testdata).to.not.be.undefined // passes
})
it("check testdata 2", function () {
expect(this.testdata).to.not.be.undefined // fails in Cypress v12
})
but now that does not work.
The use of Mocha context is a bit arbitrary anyway, and requires explicit use of function-style functions which is easy to forget, particularly in places like array method callbacks Array.forEach(() => {}).
You can still use the Cypress context to store data
before(function () {
cy.fixture("example").then(function (testdata) {
Cypress.testdata = testdata;
})
})
it("check testdata 1", function () {
expect(Cypress.testdata).to.not.be.undefined // passes
})
it("check testdata 2", function () {
expect(Cypress.testdata).to.not.be.undefined // passes
})
Note this is also undocumented and may change in the future.
Caching methods
Technically, the way to do this is to set the alias with beforeEach().
The cy.fixture() command caches it's value, so you do not get the read overhead for each test (see Fixture returns outdated/false data #4716)
There is also cy.session() for more complicated scenarios, which would be officially supported.
beforeEach(() => {
cy.session('testdata', () => {
cy.fixture("example").then(testdata => {
sessionStorage.setItem("testdata", testdata)
})
})
})
it("check testdata 1", function () {
expect(sessionStorage.getItem("testdata")).to.not.be.undefined
})
it("check testdata 2", function () {
expect(sessionStorage.getItem("testdata")).to.not.be.undefined
})
Lastly, cypress-data-session which fills a gap
From the docs
Feature
cy.session
cy.dataSession
Command is
official ✅
community 🎁
Can cache
the browser session state
anything
Stability
experimental !!! not in v12
production
Cache across specs
yes
yes
Access to the cached data
no ???
yes
Custom validation
no
yes
Custom restore
no
yes
Dependent caching
no
yes
Static utility methods
limited
all
GUI integration
yes
no
Should you use it?
maybe
yes
Cypress version support
newer versions
all
Cypress.env()
This is another way that is officially supported,
before(() => {
cy.fixture("example").then(testdata => {
Cypress.env("testdata", testdata)
})
})
it("log my fixture 1", function () {
expect(Cypress.env("testdata")).to.not.be.undefined // passes
})
it("log my fixture 2", function () {
expect(Cypress.env("testdata")).to.not.be.undefined // passes
})
but there are still certain tests scenarios that reset the browser completely where this may not work.

Pre-scan web page for dynamic tests

Looking for a definitive answer to the question posed by #JeffTanner here about generating dynamic tests. From that question and the Cypress samples, it's clear that we need to know the number of tests required before generating them.
Problem
We have a web page containing a table of Healthcare analytic data that is refreshed many times during the day. Each refresh the team must check the data, and to divvy up the work we run each row as a separate test. But the number of rows varies every time which means I must count the rows and update the system on each run. Looking for a way to programmatically get the row count.
The HTML is a table of <tbody><tr></tr></tbody>, so the following is enough to get the count but I can't run it in a beforeEach(), the error thrown is "No tests found"
let rowCount;
beforeEach(() => {
cy.visit('/analytics')
cy.get('tbody tr').then($els => rowCount = $els.length)
})
Cypress._.times(rowCount => {
it('process row', () => {
...
})
})
The before:run event fires before the tests start, you can scan the web page there.
Set the event listener in setupNodeEvents(). Cypress commands won't run here, but you can use equivalent Node commands.
const { defineConfig } = require("cypress");
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
on('before:run', async (details) => {
try {
const fetch = require('node-fetch');
const fs = require('fs-extra');
const jsdom = require("jsdom");
const { JSDOM } = jsdom;
const response = await fetch(config.env.prescan); // default below
const body = await response.text(); // or pass in command line
const dom = new JSDOM(body);
const rows = dom.window.document.body.querySelectorAll('tr') // query
// save results
fs.writeJson('./cypress/fixtures/analytics-rows.json', {rows:rows.length})
} catch (error) {
console.log('error:', error)
}
})
},
},
env: {
prefetch: 'url-for-analytics-page'
}
})
Test
import {rows} from './cypress/fixtures/analytics-rows.json' // read row count
Cypress._.times(rows, (row) => {
it(`tests row ${row}`, () => {
...
})
}
You can add a script scan-for-rows.js to the project scripts folder, like this
const rp = require('request-promise');
const $ = require('cheerio');
const fs = require('fs-extra');
rp('my-url')
.then(function(html) {
const rowCount = $('big > a', html).length
fs.writeJson('row-count.json', {rowCount})
})
.catch(function(err){
//handle error
});
Then in package.json call a pre-test script every time a new version of the web page appears.
One possibility is to run the above Cypress test in a pretest script which will always run before your main test script.
// package.json
{
...
"scripts": {
"pretest": "npx cypress run --spec cypress/e2e/pre-scan.cy.js",
"test": "npx cypress run --spec cypress/e2e/main-test.cy.js",
}
}
// pre-scan.cy.js
it('scans for table row count', () => {
cy.visit('/analytics');
cy.get('tbody tr').then($els => {
const rowCount = $els.length;
cy.writeFile('cypress/fixtures/rowcount.json', rowCount);
});
});
Here's a way to get the row count in the spec file without using extra packages, plugins, test hooks, or npm scripts.
Basically, you can create a separate module that makes a synchronous HTTP request using the XMLHTTPRequest class to the /analytics endpoint and use the browser's DOMParser class to find the return the number of <tr> tags.
// scripts/get-row-count.js
export function getRowCount() {
let request = new XMLHttpRequest();
// Set async to false because Cypress will not wait for async functions to finish before looking for it() statements
request.open('GET', '/analytics', false);
request.send(null);
const document = new DOMParser().parseFromString(request.response, 'text/html');
const trTags = Array.from(document.getElementsByTagName('tr'));
return trTags.length;
};
Then in the spec file, import the new function and now you can get an updated row count whenever you need it.
import { getRowCount } from '../scripts/get-row-count';
Cypress._.times(getRowCount() => {
it('process row', () => {
...
})
})
The reason for XMLHTTPRequest instead of fetch is because it allows synchronous requests to be made. Synchronous requests are needed because Cypress won't wait for async requests to come back before parsing for it() blocks.
With this, you always have the most up to date row count without it going stale.

Do jest tests wait for previous async tests to resolve?

I have read that jest tests in the same tests file execute sequentially. I have also read that when writing tests that involve callbacks a done parameter should be used.
But when using promises using the async/await syntax that I am using in my code below, can I count on the tests to but run and resolve in sequential order?
import Client from '../Client';
import { Project } from '../Client/types/client-response';
let client: Client;
beforeAll(async () => {
jest.setTimeout(10000);
client = new Client({ host: 'ws://127.0.0.1', port: 8080 , logger: () => {}});
await client.connect();
})
describe('Create, save and open project', () => {
let project: Project;
let filename: string;
beforeAll(async () => {
// Close project
let project = await client.getActiveProject();
if (project) {
let success = await client.projectClose(project.id, true);
expect(success).toBe(true);
}
})
test('createProject', async () => {
project = await client.createProject();
expect(project.id).toBeTruthy();
});
test('projectSave', async () => {
filename = await client.projectSave(project.id, 'jesttest.otii', true);
expect(filename.endsWith('jesttest.otii')).toBe(true);
});
test('projectClose', async () => {
let success = await client.projectClose(project.id);
expect(success).toBe(true);
});
test('projectOpen', async () => {
project = await client.openProject(filename);
expect(filename.endsWith('jesttest.otii')).toBe(true);
});
})
afterAll(async () => {
await client.disconnect();
})
From the docs:
...by default Jest runs all the tests serially in the order they were encountered in the collection phase, waiting for each to finish and be tidied up before moving on.
So while Jest may run test files in parallel, by default it runs the tests within a file serially.
That behavior can be verified by the following test:
describe('test order', () => {
let count;
beforeAll(() => {
count = 0;
})
test('1', async () => {
await new Promise((resolve) => {
setTimeout(() => {
count++;
expect(count).toBe(1); // SUCCESS
resolve();
}, 1000);
});
});
test('2', async () => {
await new Promise((resolve) => {
setTimeout(() => {
count++;
expect(count).toBe(2); // SUCCESS
resolve();
}, 500);
});
});
test('3', () => {
count++;
expect(count).toBe(3); // SUCCESS
});
});
For sure it depends on test runnner configured. Say for Jasmine2 it seems impossible to run tests concurrently:
Because of the single-threadedness of javascript, it isn't really possible to run your tests in parallel in a single browser window
But looking into docs' config section:
--maxConcurrency=
Prevents Jest from executing more than the specified amount of tests at the same time. Only affects tests that use test.concurrent.
--maxWorkers=|
Alias: -w. Specifies the maximum number of workers the worker-pool will spawn for running tests. This defaults to the number of the cores available on your machine. It may be useful to adjust this in resource limited environments like CIs but the default should be adequate for most use-cases.
For environments with variable CPUs available, you can use percentage based configuration: --maxWorkers=50%
Also looking at description for jest-runner-concurrent:
Jest's default runner uses a new child_process (also known as a worker) for each test file. Although the max number of workers is configurable, running a lot of them is slow and consumes tons of memory and CPU.
So it looks like you can configure amount of test files running in parallel(maxWorkers) as well as concurrent test cases in scope of single worker(maxConcurrency). If you use jest as test runner. And this affects only test.concurrent() tests.
For some reason I was unable to find anything on test.concurrent() at their main docs site.
Anyway you can check against your environment by yourselves:
describe('checking concurrent execution', () => {
let a = 5;
it('deferred change', (done) => {
setTimeout(() => {
a = 11;
expect(a).toEqual(11);
done();
}, 1000);
});
it('fails if running in concurrency', () => {
expect(a).toEqual(11);
});
})
Sure, above I used Jasmine's syntax(describe, it) so you may need to replace that with other calls.

Pass data from one step to the next synchronously

Running Cypress 3.1.1 with cypress-cucumber-preprocessor 1.5.1. I need to pass some static data from one step to another (in the same scenario/test). I can do this using an alias, like this:
cy.wrap(someString).as('myString'), but then I have to access it asynchronously:
cy.get('#myString').then(myString => ...)
This is rather cumbersome, particularly when I have to pass multiple values, requiring multiple wrapped closures, for no apparent benefit. (Currently I'm working around this by aliasing an object, but I shouldn't need to do this.)
How can I pass primitive values from one step to another synchronously?
I thought I might be able to simply set this.myString='' to set the value on the Mocha shared context object, but in that case, the property exists but is set to undefined when accessed in later steps.
Even creating my own context variable with let outside of the step definition does not work. Is this simply a limitation of Cypress and/or the cypress-cucumber-preprocessor?
I managed to get it working the following way:
Add 2 tasks to the /plugins/index.js
const testStore = {}
module.exports = (on, config) => {
on('task', {
pushValue({ name, value }) {
console.log(name, value)
testStore[name] = value
console.log(testStore)
return true
},
})
on('task', {
getValue(name) {
return testStore[name]
},
})
Then you can add a variable in any test and reach it in any other place:
it('test', ()=>{
cy.task('pushValue', { name: 'orderNumber', value: orderNumber })
})
it('test 2', ()=>{
cy.task('getValue', 'orderNumber').then((order) => {
cy.visit(`/bookings/${order}`)
})
})
Here is a slightly more complicated (and not fully tested) method. A custom command can be added to save values to a global object.
In the Cypress test runner, all the tests seem to run sequentially, but you may have to be careful if using CI and parallel execution.
In /support/commands.js
export const testStore = {}
Cypress.Commands.add('saveAs', { prevSubject: true }, (value, propName) => {
console.log('saveAs', value, propName)
testStore[propName] = value;
return value;
})
In myTest.spec.js
import { testStore } from '../support/commands.js'
...
it('should have a title', () => {
cy.title()
.saveAs('title') // save for next test
.should('contain', 'myTitle) // this test's expectation
});
it('should test something else', () => {
cy.get('.myElement').contains(testStore.title);
});

Resources