I'm wanting to test our tests that we have written. For example, if a developer creates a test with the type command, for example:
cy.get('input').type('Hello, World')
I want to test if this type() has the option of log:false. If there is no option of log:false then fail the test.
Code should look like this:
cy.get('input').type('Hello, World', { log: false })
Has anyone using Cypress accomplished this? Running tests to test our tests.
Not quite what you asked, but
/cypress/support/index.js
Cypress.Commands.overwrite('type', (originalFn, text, options) => {
// enforce no-log policy
return originalFn(text, { ...options, log: false })
})
Or if you like to finger-wag
Cypress.Commands.overwrite('type', (originalFn, text, options) => {
if (!options.log || options.log) {
throw " 👎 No logging allowed on .type()"
}
return originalFn(text, options)
})
Related
When running Cypress headlessly, I can see console.log output from the frontend code under test by using the DEBUG environment variable, like:
DEBUG='cypress:launcher' npx cypress run --browser chrome
However, I haven't found any similar way to see the output of cy.log from the Cypress test code when running headlessly. Even with DEBUG='cypress:*' I don't see them - they only seem to be visible in the interactive interface. It feels like there must be some way to see the cy.log output headlessly - can someone help with that?
The first step is to add a new task in your Cypress config file so that you can run console.log from Node:
import { defineConfig } from "cypress";
export default defineConfig({
e2e: {
setupNodeEvents(on, config) {
on("task", {
log(args) {
console.log(...args);
return null;
}
});
},
},
});
Then, you can override cy.log so that it calls this task whenever you run the command in headless mode, and console.log when you're running in headed mode. You can do this by adding the following to your commands file:
Cypress.Commands.overwrite("log", function(log, ...args) {
if (Cypress.browser.isHeadless) {
return cy.task("log", args, { log: false }).then(() => {
return log(...args);
});
} else {
console.log(...args);
return log(...args);
}
});
How do I conditionally skip a test if the URL contains "xyz"?
some tests that run in the QA environment "abc" should not be run in Production "xyz" environment.
I've not been able to find a good example of conditionally checking for environment to trigger a test. The baseURL needs to be checked dynamically and the test skipped preferably in the beforeEach.
running cypress 6.2.0
beforeEach(() => {
login.loginByUser('TomJones');
cy.visit(`${environment.getBaseUrl()}${route}`);
});
it('test page', function () {
if environment.getBaseUrl().contains("xyz")
then *skip test*
else
cy.intercept('GET', '**/some-api/v1/test*').as('Test'););
cy.get('#submitButton').click();
})
Potential Solution (tested and tried successfully):
I used a combination of filtering (grouping) and folder structures via CLI
I set folders /integrations/smokeTest/QA and /integrations/smokeTest/Prod/
1.QA Test Run:
npm run *cy:filter:qa* --spec "cypresss/integration/smokeTests/QA/*-spec.ts"
2.Run All (both QA and PROD tests)
npm run cypress:open --spec "cypresss/integration/smokeTests/*/*-spec.ts"
3. Prod Test Run:
npm run cy:filter:prod --spec "cypresss/integration/smokeTests/PROD*/*-spec.ts"
Normally I wouldn't write a custom command just to exercise one Cypress command, but in this case it's useful to obtain the global test context this.
Using the function form of callback with the custom command allows access to this, then you are free to use arrow functions on the test themselves.
Cypress.Commands.add('skipWhen', function (expression) {
if (expression) {
this.skip()
}
})
it('test skipping with arrow function', () => {
cy.skipWhen(Cypress.config('baseUrl').includes('localhost'));
// NOTE: a "naked" expect() will not be skipped
// if you call your custom command within the test
// Wrap it in a .then() to make sure it executes on the command queue
cy.then(() => {
expect('this.stackOverflow.answer').to.eq('a.better.example')
})
})
I would add a helper method which you can call from any Mocha.Context (at the time of writing, any it, describe, or context block).
// commands.ts
declare global {
// eslint-disable-next-line #typescript-eslint/no-namespace
namespace Cypress {
interface Chainable {
/**
* Custom command which will skip a test or context based on a boolean expression.
*
* You can call this command from anywhere, just make sure to pass in the the it, describe, or context block you wish to skip.
*
* #example cy.skipIf(yourCondition, this);
*/
skipIf(expression: boolean, context: Mocha.Context): void;
}
}
}
Cypress.Commands.add(
'skipIf',
(expression: boolean, context: Mocha.Context) => {
if (expression) context.skip.bind(context)();
}
);
And from your spec:
describe('Events', () => {
const url = `${environment.getBaseUrl()}${route}`;
before(function () {
cy.visit(url);
cy.skipIf(url.includes('xyz'), this);
});
context('Nested context', () => {
it('test', function () {
cy.skipIf(url.includes('abc'), this);
expect(this.stackOverflow.answer).to.be('accepted');
});
});
});
Now you have a reusable custom command that you can call from anywhere to conditionally skip tests based on any expression that evaluates to a boolean. Careful of classic JS equality and definition gotchas (read more about equality in JS here).
I am trying to write a function in wdio.conf.js that executes when a test passes.
At the end of the test, it shows all of the tests passing, however it never hits the console.log("testpassed") code shown here:
afterTest: function (test) {
if (test.passed === true) {
console.log("testpassed")
}
},
If I console.log - 'test' - it prints [object Object].
However, if I console log test.passed it prints undefined.
At the end of the test it shows all tests as passed.
What am I doing wrong?
Further investigation: These are the only keys returned in the 'test' array:
type,title,fn,body,async,sync,_timeout,_slow,_retries,timedOut,_currentRetry,pending,file,parent,ctx,_events,_eventsCount,callback,timer
So there doesn't seem to be a key for test.passed
afterTest: function (test, context, { passed }) {
if(passed){
console.log("THE TEST PASSED");
}
},
I am trying to pull out a smoke suite from my regression suite written using the Jasmine framework (wdio-jasmine-framework).
Is it possible to just add a tag on specific testcases in Jasmine?
If I remember correctly from my Jasmine/Mocha days, there were several ways to achieve this. I'll detail a few, but I'm sure there might be some others too. Use the one that's best for you.
1. Use the it.skip() statement inside a conditional operator expression to define the state of a test-case (e.g: in the case of a smokeRun, skip the non-smoke tests using: (smokeRun ? it.skip : it)('not a smoke test', () => { // > do smth here < });).
Here is an extended example:
// Reading the smokeRun state from a system variable:
const smokeRun = (process.env.SMOKE ? true : false);
describe('checkboxes testsuite', function () {
// > this IS a smoke test! < //
it('#smoketest: checkboxes page should open successfully', () => {
CheckboxPage.open();
// I am a mock test...
// I do absolutely nothing!
});
// > this IS NOT a smoke test! < //
(smokeRun ? it.skip : it)('checkbox 2 should be enabled', () => {
CheckboxPage.open();
expect(CheckboxPage.firstCheckbox.isSelected()).toEqual(false);
expect(CheckboxPage.lastCheckbox.isSelected()).toEqual(true);
});
// > this IS NOT a smoke test! < //
(smokeRun ? it.skip : it)('checkbox 1 should be enabled after clicking on it', () => {
CheckboxPage.open();
expect(CheckboxPage.firstCheckbox.isSelected()).toEqual(false);
CheckboxPage.firstCheckbox.click();
expect(CheckboxPage.firstCheckbox.isSelected()).toEqual(true);
});
});
2. Use it.only() to achieve mainly the same effect, the difference being the test-case refactor workload. I'll summarize these ideas as:
if you have more smoke tests than non-smoke tests, use the it.skip() approach;
if you have more non-smoke tests than smoke tests, use the it.only() approach;
You can read more about pending-tests here.
3. Use the runtime skip (.skip()) in conjunction with some nested describe statements.
It should look something like this:
// Reading the smokeRun state from a system variable:
const smokeRun = (process.env.SMOKE ? true : false);
describe('checkboxes testsuite', function () {
// > this IS a smoke test! < //
it('#smoketest: checkboxes page should open successfully', function () {
CheckboxPage.open();
// I am a mock test...
// I do absolutely nothing!
});
describe('non-smoke tests go here', function () {
before(function() {
if (smokeRun) {
this.skip();
}
});
// > this IS NOT a smoke test! < //
it('checkbox 2 should be enabled', function () {
CheckboxPage.open();
expect(CheckboxPage.firstCheckbox.isSelected()).toEqual(false);
expect(CheckboxPage.lastCheckbox.isSelected()).toEqual(true);
});
// > this IS NOT a smoke test! < //
it('checkbox 1 should be enabled after clicking on it', function () {
CheckboxPage.open();
expect(CheckboxPage.firstCheckbox.isSelected()).toEqual(false);
CheckboxPage.firstCheckbox.click();
expect(CheckboxPage.firstCheckbox.isSelected()).toEqual(true);
});
});
});
!Note: These are working examples! I tested them using WebdriverIO's recommended Jasmine Boilerplace project.
!Obs: There multiple ways to filter Jasmine tests, unfortunately only at a test-file(testsuite) level (e.g: using grep piped statements, or the built-in WDIO specs & exclude attributes).
Automated the e2e tests in CI enabled framework with protractor and jasmine in VSTS. Used the jasmine custom reporter to load the e2e test results under sauce labs tab in VSTS build definition. But, it is not loading all the tests. It is displaying only the last e2e test ran in the build. Console log is getting printed for all the e2e tests. Please see the code below.
let sauceLabsReporter: jasmine.CustomReporter = {
specDone: (result: jasmine.CustomReporterResult): void => {
Util.log('*** sauceLabsReporter: result.fullName:', result.fullName);
Util.log('*** sauceLabsReporter: result.status:', result.status);
Util.log('*** sauceLabsReporter: result.testCaseId:', result.testCaseId);
if (result.testCaseId) {
result.fullName = `(Testcase ID: ${result.testCaseId}): ${result.fullName}`;
Util.log('*** sauceLabsReporter: UPDATED result.fullName:', result.fullName);
}
Util.updateSauceLabsJobTitle(result.fullName);
if (result.status) {
Util.updateSauceLabsTestState(result.status);
}
}
};
export function updateSauceLabsJobTitle(title: string): promise.Promise<void> {
let fullTitle: string = `${title} | (${getHostname()})`;
return browser.executeScript(`sauce:job-name=${fullTitle}`)
.then(() => browser.getSession())
.then((session: Session) => {
if (isThisVSTSBuildAgent()) {
//The VSTS Sauce Labs add-on gets information by parsing the console log.
//tslint:disable-next-line:no-console
console.log(`SauceOnDemandSessionID=${session.getId()} job-name=${fullTitle}`);
}
});
}
export function updateSauceLabsTestState(state: string): promise.Promise<{}> {
return browser.executeScript(`sauce:job-result=${state}`);
}
I work on the same project as Padma. It had to do with restartBrowserBetweenTests being set to false which conflates all tests into a single job. After setting it to true, each test became its own job.