Pre-scan web page for dynamic tests - cypress

Question

Looking for a definitive answer to the question posed by #JeffTanner here about generating dynamic tests. From that question and the Cypress samples, it's clear that we need to know the number of tests required before generating them.
Problem
We have a web page containing a table of Healthcare analytic data that is refreshed many times during the day. Each refresh the team must check the data, and to divvy up the work we run each row as a separate test. But the number of rows varies every time which means I must count the rows and update the system on each run. Looking for a way to programmatically get the row count.
The HTML is a table of <tbody><tr></tr></tbody>, so the following is enough to get the count but I can't run it in a beforeEach(), the error thrown is "No tests found"
let rowCount;
beforeEach(() => {
cy.visit('/analytics')
cy.get('tbody tr').then($els => rowCount = $els.length)
})
Cypress._.times(rowCount => {
it('process row', () => {
...
})
})

Answer 1

The before:run event fires before the tests start, you can scan the web page there.
Set the event listener in setupNodeEvents(). Cypress commands won't run here, but you can use equivalent Node commands.
const { defineConfig } = require("cypress");
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
on('before:run', async (details) => {
try {
const fetch = require('node-fetch');
const fs = require('fs-extra');
const jsdom = require("jsdom");
const { JSDOM } = jsdom;
const response = await fetch(config.env.prescan); // default below
const body = await response.text(); // or pass in command line
const dom = new JSDOM(body);
const rows = dom.window.document.body.querySelectorAll('tr') // query
// save results
fs.writeJson('./cypress/fixtures/analytics-rows.json', {rows:rows.length})
} catch (error) {
console.log('error:', error)
}
})
},
},
env: {
prefetch: 'url-for-analytics-page'
}
})
Test
import {rows} from './cypress/fixtures/analytics-rows.json' // read row count
Cypress._.times(rows, (row) => {
it(`tests row ${row}`, () => {
...
})
}

Answer 2

You can add a script scan-for-rows.js to the project scripts folder, like this
const rp = require('request-promise');
const $ = require('cheerio');
const fs = require('fs-extra');
rp('my-url')
.then(function(html) {
const rowCount = $('big > a', html).length
fs.writeJson('row-count.json', {rowCount})
})
.catch(function(err){
//handle error
});
Then in package.json call a pre-test script every time a new version of the web page appears.

Answer 3

One possibility is to run the above Cypress test in a pretest script which will always run before your main test script.
// package.json
{
...
"scripts": {
"pretest": "npx cypress run --spec cypress/e2e/pre-scan.cy.js",
"test": "npx cypress run --spec cypress/e2e/main-test.cy.js",
}
}
// pre-scan.cy.js
it('scans for table row count', () => {
cy.visit('/analytics');
cy.get('tbody tr').then($els => {
const rowCount = $els.length;
cy.writeFile('cypress/fixtures/rowcount.json', rowCount);
});
});

Answer 4

Here's a way to get the row count in the spec file without using extra packages, plugins, test hooks, or npm scripts.
Basically, you can create a separate module that makes a synchronous HTTP request using the XMLHTTPRequest class to the /analytics endpoint and use the browser's DOMParser class to find the return the number of <tr> tags.
// scripts/get-row-count.js
export function getRowCount() {
let request = new XMLHttpRequest();
// Set async to false because Cypress will not wait for async functions to finish before looking for it() statements
request.open('GET', '/analytics', false);
request.send(null);
const document = new DOMParser().parseFromString(request.response, 'text/html');
const trTags = Array.from(document.getElementsByTagName('tr'));
return trTags.length;
};
Then in the spec file, import the new function and now you can get an updated row count whenever you need it.
import { getRowCount } from '../scripts/get-row-count';
Cypress._.times(getRowCount() => {
it('process row', () => {
...
})
})
The reason for XMLHTTPRequest instead of fetch is because it allows synchronous requests to be made. Synchronous requests are needed because Cypress won't wait for async requests to come back before parsing for it() blocks.
With this, you always have the most up to date row count without it going stale.