Check if an error has been written to the console - cypress

I'm trying to find a way to check if an error has been written to the console when running a cypress unit test.
I know how to log something to the console
cy.log('log this to the console');
but not how to check if an error has been written to it.
any suggestions how to read errors from the (browser) console log?
note: probably not the "smart" way to test but sometimes my js libraries which I use would "complain" and write the errors to the browser log. this is to simplify testing.

This does exactly what I needed of catching any error in the console and do an assertion of the logs count. Just add the following in cypress/support/index.js
Cypress.on('window:before:load', (win) => {
cy.spy(win.console, 'error');
cy.spy(win.console, 'warn');
});
afterEach(() => {
cy.window().then((win) => {
expect(win.console.error).to.have.callCount(0);
expect(win.console.warn).to.have.callCount(0);
});
});

There have been some updates since the previous answers.
Because the window is re-created with each cy.visit, Cypress recommends stubbing as a part of the cy.visit command.
cy.visit('/', {
onBeforeLoad(win) {
cy.stub(win.console, 'log').as('consoleLog')
cy.stub(win.console, 'error').as('consoleError')
}
})
//...
cy.get('#consoleLog').should('be.calledWith', 'Hello World!')
cy.get('#consoleError').should('be.calledOnce')
For more details see the official FAQ for stubbing out the console: https://docs.cypress.io/faq/questions/using-cypress-faq.html#How-do-I-spy-on-console-log
And the recipe repository: https://github.com/cypress-io/cypress-example-recipes/tree/master/examples/stubbing-spying__console

Edit: the following does not directly log to terminal when in headless mode, but it nonetheless fails the test on AUT's console.error and displays the error message indirectly, even in the headless terminal, which may be what you want.
I'm not sure exactly what you mean, but let's go through all the places where an output can be logged in cypress, and how to handle several cases.
First, an overview:
To log into the command log, you use:
// from inside your test
cy.log('foo');
To log into devTools console:
// from inside your test
console.log('bar');
To log into terminal, you need to log from within the Cypress' node process:
// from within e.g. your plugin/index.js file
console.log('baz');
How to log AUT's errors to Terminal, Command Log, and fail the test
(note, AUT here stands for Application under test, meaning your application).
I'm also using ansicolor package to make the error red-colored in the terminal, which is optional.
// plugins/index.js
const ansi = require(`ansicolor`);
module.exports = ( on ) => {
on(`task`, {
error ( message ) {
// write the error in red color
console.error( ansi.red(message) );
// play `beep` sound for extra purchase
process.stdout.write(`\u0007`);
return null;
}
});
};
Note: using internal cy.now() command to work around Cypress' tendency to throw Cypress detected that you returned a promise when it (IMO) shouldn't.
(adapted from https://github.com/cypress-io/cypress/issues/300#issuecomment-438176246)
// support/index.js or your test file
Cypress.on(`window:before:load`, win => {
cy.stub( win.console, `error`, msg => {
// log to Terminal
cy.now(`task`, `error`, msg );
// log to Command Log & fail the test
throw new Error( msg );
});
});

Currently there is no straightforward way to do what you are asking but there have been some good discussions on how best to get this information. I copied one solution here but if you follow the github link you can see other solutions proposed.
This snippet was taken from the github issue found here: https://github.com/cypress-io/cypress/issues/300
Just FYI the one easy solution is just to spy on console functions.
cy.window().then((win) => { cy.spy(win.console, "log") })
That will print a command log every time that function is called, and
you could also then assert what has been logged.
Another option depending on why you want to assert that something went wrong is to print the error out under the tests in headless mode. The VP of engineering created an NPM package that does this for you.
Cypress-failed-log

The most easiest way if you simply want to ensure that no error is in the console (which is the most usecase I assume).
# npm
npm install cypress-fail-on-console-error --save-dev
# yarn
yarn add cypress-fail-on-console-error -D
And then add to your support/index.ts file:
import failOnConsoleError from "cypress-fail-on-console-error"
failOnConsoleError()
Now your cypress tests are failing just in time when a console error is printed.

This is the working solution I currently use to check for console errors.
let windowConsoleError;
Cypress.on('window:before:load', (win) => {
windowConsoleError = cy.spy(win.console, 'error');
})
afterEach(() => {
expect(windowConsoleError).to.not.be.called;
})

Related

How to programmatically stop a Cypress test

I have a Cypress test that has only one testing case (using v10.9 with the test GUI):
describe("Basic test", () => {
it.only("First test", () => {
cy.visit("http://localhost:9999");
cy.pause();
//if(cy.get("#importantBox").contains(...) {
//}
//else
{
Cypress.runner.stop();
console.log("Stopping...");
}
console.log("Visiting...");
cy.visit("http://localhost:9999/page1");
If a certain element doesn't exist in the page, I don't want the test to continue, so I try to stop it.
Unfortunately, I can see this in the console:
Stopping...
Visiting...
And the test keeps going without the necessary data...
So, can I somehow stop it without using huge if statements?
Stopping the test is relatively easy, the harder part is the condition check.
Cypress runner is built on the Mocha framework, which has a .skip() method. If you issue it in your else clause and inside the Cypress queue, the test will stop.
Two ways to access skip():
Using a function() callback gives access to this which is the Mocha context
it('stops on skip', function() {
...
cy.then(() => this.skip()) // stop here
})
Use cy.state() internal command (may be removed at some point)
it('stops on skip', () => {
...
cy.then(() => cy.state('test').skip()) // stop here
})
You should be aware that all Cypress commands and queries run on an internal queue which is asynchronous to javascript code in the test like console.log("Visiting..."), so you won't get any useful indication from that line.
To use synchronous javascript on the queue, wrap it in a cy.then() command as shown above with the skip() method, so to console log do
cy.then(() => console.log("Visiting..."))

Referencing Add Command in Webdriverio

I'm using webdriverio (version 6.14.13) and mocha to run my native app tests via Browserstack.
I've created a command script in my command folder, called closeCmp.command.js, as the code was being reused in many tests.
browser.addCommand('cmpDismissal', async function() {
cmpDismissal = $('~ACCEPT AND CLOSE');
if (await this.cmpDismissal.isExisting()) {
this.cmpDismissal.click();
this.cmpDismissal.waitForExist({ reverse: true });
}
},true);
But my question is, how do I reference this in my test script?
I tried what I thought would work;
describe('testing the app cmp.....', () => {
it('close cmp', async () => {
await browser.commands.cmpDismissal();
but when I run the test, the command script is not being run.
It doesn't return an actual failure message, but when I go into Browserstack I can see that the button has not been clicked, indication that the new command has not been executed.
Am I calling it incorrectly in my test script, or is the actual command script the issue here? Thanks

Setting the browser language in Cypress

Is it possible to tell Cypress to launch Chrome with a certain language (e.g. German) as I have an application which I need to test in multiple languages. I can't see this detailed anywhere in the documentation which suggests it is not possible at present.
I have tried adding the --lang argument when Chrome is launched but this does not seem to have any effect and Chrome still uses English. See the pluginsFile code below.
module.exports = (on, config) => {
on('before:browser:launch', (browser = {}, args) => {
if (browser.name === 'chrome') {
args.push('--lang=de')
return args
}
})
}
I have also tried --lang=de-DE which also did not work.
See full example in https://glebbahmutov.com/blog/cypress-tips-and-tricks/#control-navigatorlanguage but in short
it('shows Klingon greeting', () => {
cy.visit('index.html', {
onBeforeLoad (win) {
// DOES NOT WORK
// Uncaught TypeError: Cannot assign to read only property
// 'language' of object '[object Navigator]'
// win.navigator.language = 'Klingon'
// instead we need to define a property like this
Object.defineProperty(win.navigator, 'language', {
value: 'Klingon'
})
}
})
cy.contains('#greeting', 'nuqneH').should('be.visible')
})
I had a similar problem, when launching cypress the browser would be in my default language (Dutch), while all our tests expect English to be the default. I found a question on a support-forum mentioning the parameter --lang param as well, but it had no effect on my browser's language.
In the end I could work around the problem by changing the LANG environment variable - I am using Linux. In the terminal I typed the following:
export LANG="en_EN.UTF-8"
and then I ran cypress from the same terminal.
You could script this, and for other operating systems such as MacOS and Windows there's probably a similar environment variable.
Besides adding command line options, you can also change browser preferences using Cypress' Browser Launch API (documentation). This allows you to override the Accept-Language header setting like this:
on('before:browser:launch', (browser, launchOptions) => {
if (browser.family === 'chromium' && browser.name !== 'electron') {
launchOptions.preferences.default.intl = { accept_languages: "nl" }
return launchOptions
}
}
Note that the launchOptions.preferences.default object may be empty, so trying to assign to launchOptions.preferences.default.intl.accept_languages directly may fail.
For one of our projects, this was enough to get the site we were testing to appear in the right language. If you need more, there are more language settings you can try altering (see Chrome's source code and look for "intl").
On a side note, it looks like the --lang command line option only works on Windows, according to Chrome's documentation. On Mac, you are required to change your system preferences, and on Linux, you can use the LANGUAGE environment variable.
cy.visit('/', {
onBeforeLoad(win: Cypress.AUTWindow) {
Object.defineProperty(win.navigator, 'language', { value: 'de-DE' });
},
});

Sinon useFakeTimers() creates a timeout in before/afterEach

I'm using Sinon with Mocha to test some expiration date values. I used the same code a few months ago and it worked fine, but somewhere between v1.12.x and v1.17.x, something has changed and I can't seem to find the right path.
let sinon = require('sinon');
describe('USER & AUTHENTICATION ENDPOINTS', function(done) {
beforeEach(function() {
this.clock = sinon.useFakeTimers(new Date().getTime());
return fixtures.load(data);
});
afterEach(function() {
this.clock.restore();
return fixtures.clear(data);
});
context('POST /users', function() { ... }
});
I've tried with and without the new Date().getTime() argument.
I've tried passing in and explicitly calling done().
I've tried removing my fixture load/clear processes.
The end result is always the same:
Error: timeout of 5000ms exceeded. Ensure the done() callback is being called in this test.
Has something changed that I just haven't noticed in the documentation? Do I have some kind of error in there that I can't see?
Any thoughts would be appreciated.
UPDATE
So a little more info here. This clearly has something to do with my code, but I'm at a loss.
If I comment every actual test, the tests run and give me a green "0 passing".
If I run an actual test, even one that just this:
context('POST /users', function() {
it('should create a new user', function(done) {
done();
})
});
I'm right back to the timeout. What am I missing?
Mystery solved. It appears to be a conflict between Sinon and versions of Knex > 0.7.6.
Seems to be because pool2 relies on behavior of setTimeout. Using sinon.useFakeTimers(...) replaces several methods including setTimeout with synchronous versions which breaks it. Can fix by replacing with: clock = sinon.useFakeTimers(Number(date), 'Date');
My original code was written in a world where Knex v0.7.6 was the latest version. Now that it's not everything failed even though the code itself was the same. I used the fix mentioned and things look to be fine.
You are passing done to your describe callback in line 2:
describe('USER & AUTHENTICATION ENDPOINTS', function(done) {
Mocha expects you to invoke it... To get rid of the timeout error, just remove the done parameter from the callback.

Is there a way to continue test scenario execution after step failure in a previous scenario?

Whenever there is a step failure while running on a remote server, I would like to capture the failed step and then continue running the remaining scenarios. The captured step would then be included in a file for reporting purposes. Is this a possibility? All replies I've seen elsewhere just say you should fix the test before moving on. I agree, but I only want the tests to stop when running locally, not remotely.
➜ customer git:(pat104) ✗ cucumber.js -f progress (pat104⚡)
...F-----Failed scenario: View and select first contact from contact history
...F-Failed scenario: View and select a contact from multiple contacts in history
..................................................F---Failed scenario: Navigating to profile with url and enrollmentId
...................................................F-Failed scenario: Successful MDN Search with 1 result returned. Tech Selects and continues
.............FFailed scenario: Successful MDN with multiple results
Turns out, one of the step-definitions was using .waitForExist incorrectly. The test was written:
this.browser
.waitForExist('#someElement', 1000, callback)
Callback isn't a parameter for .waitForExist, rewrote to:
.waitForExist('#someElement',1000).then(function (exists) {
assert.equal(exists, true, callback);
})
This is the default behavior, isn't it? Example command
cucumber.js -f json > cucumberOutput.json
Well, that you need to manage in your test itself using callback.fail(e) like below. You can use library like grunt-cucumberjs to add these errors to nice HTML reports.
this.Then(/^the save to wallet button reflects the offer is saved$/, function (callback) {
merchantPage(this.nemo).doStuff().then(function () {
callback();
}, function (e) {
callback.fail(e); //add this to report
});
});
Or you could use Hooks and check whether a scenario is failed and report (take screenshot or add logging etc.)
this.After(function (scenario, callback) {
var driver = this.nemo.driver;
if(scenario.isFailed()){
driver.takeScreenshot().then(function (buffer) {
scenario.attach(new Buffer(buffer, 'base64').toString('binary'), 'image/png');
});
}
driver.quit().then(function () {
callback();
});
});

Resources