Whenever there is a step failure while running on a remote server, I would like to capture the failed step and then continue running the remaining scenarios. The captured step would then be included in a file for reporting purposes. Is this a possibility? All replies I've seen elsewhere just say you should fix the test before moving on. I agree, but I only want the tests to stop when running locally, not remotely.
➜ customer git:(pat104) ✗ cucumber.js -f progress (pat104⚡)
...F-----Failed scenario: View and select first contact from contact history
...F-Failed scenario: View and select a contact from multiple contacts in history
..................................................F---Failed scenario: Navigating to profile with url and enrollmentId
...................................................F-Failed scenario: Successful MDN Search with 1 result returned. Tech Selects and continues
.............FFailed scenario: Successful MDN with multiple results
Turns out, one of the step-definitions was using .waitForExist incorrectly. The test was written:
this.browser
.waitForExist('#someElement', 1000, callback)
Callback isn't a parameter for .waitForExist, rewrote to:
.waitForExist('#someElement',1000).then(function (exists) {
assert.equal(exists, true, callback);
})
This is the default behavior, isn't it? Example command
cucumber.js -f json > cucumberOutput.json
Well, that you need to manage in your test itself using callback.fail(e) like below. You can use library like grunt-cucumberjs to add these errors to nice HTML reports.
this.Then(/^the save to wallet button reflects the offer is saved$/, function (callback) {
merchantPage(this.nemo).doStuff().then(function () {
callback();
}, function (e) {
callback.fail(e); //add this to report
});
});
Or you could use Hooks and check whether a scenario is failed and report (take screenshot or add logging etc.)
this.After(function (scenario, callback) {
var driver = this.nemo.driver;
if(scenario.isFailed()){
driver.takeScreenshot().then(function (buffer) {
scenario.attach(new Buffer(buffer, 'base64').toString('binary'), 'image/png');
});
}
driver.quit().then(function () {
callback();
});
});
Related
I'm setting up a CI-chain and decided to use Cypress for the UI testing. I need to get the result for each individual testcase in my suite. Preferably from within Node in for example a afterEach statement.
Has anyone done this before? Is there any built-in support for this?
I do not want to parse the end result for testcases preferably.
It was possible by using Mocha's this.currentState in conjunction with Cypress plugins.
This is how I solved it:
cypress/plugins/index.js
on("task", {
testFinished(event) {
console.log(event.title, event.result);
return null;
}
});
in my testsuite
afterEach(function() {
cy.task("testFinished", { title: this.currentTest.title, result: this.currentTest.state });
});
The console.log in plugins can now easily be switched for a POST request to wherever you want to store the results.
I'm trying to find a way to check if an error has been written to the console when running a cypress unit test.
I know how to log something to the console
cy.log('log this to the console');
but not how to check if an error has been written to it.
any suggestions how to read errors from the (browser) console log?
note: probably not the "smart" way to test but sometimes my js libraries which I use would "complain" and write the errors to the browser log. this is to simplify testing.
This does exactly what I needed of catching any error in the console and do an assertion of the logs count. Just add the following in cypress/support/index.js
Cypress.on('window:before:load', (win) => {
cy.spy(win.console, 'error');
cy.spy(win.console, 'warn');
});
afterEach(() => {
cy.window().then((win) => {
expect(win.console.error).to.have.callCount(0);
expect(win.console.warn).to.have.callCount(0);
});
});
There have been some updates since the previous answers.
Because the window is re-created with each cy.visit, Cypress recommends stubbing as a part of the cy.visit command.
cy.visit('/', {
onBeforeLoad(win) {
cy.stub(win.console, 'log').as('consoleLog')
cy.stub(win.console, 'error').as('consoleError')
}
})
//...
cy.get('#consoleLog').should('be.calledWith', 'Hello World!')
cy.get('#consoleError').should('be.calledOnce')
For more details see the official FAQ for stubbing out the console: https://docs.cypress.io/faq/questions/using-cypress-faq.html#How-do-I-spy-on-console-log
And the recipe repository: https://github.com/cypress-io/cypress-example-recipes/tree/master/examples/stubbing-spying__console
Edit: the following does not directly log to terminal when in headless mode, but it nonetheless fails the test on AUT's console.error and displays the error message indirectly, even in the headless terminal, which may be what you want.
I'm not sure exactly what you mean, but let's go through all the places where an output can be logged in cypress, and how to handle several cases.
First, an overview:
To log into the command log, you use:
// from inside your test
cy.log('foo');
To log into devTools console:
// from inside your test
console.log('bar');
To log into terminal, you need to log from within the Cypress' node process:
// from within e.g. your plugin/index.js file
console.log('baz');
How to log AUT's errors to Terminal, Command Log, and fail the test
(note, AUT here stands for Application under test, meaning your application).
I'm also using ansicolor package to make the error red-colored in the terminal, which is optional.
// plugins/index.js
const ansi = require(`ansicolor`);
module.exports = ( on ) => {
on(`task`, {
error ( message ) {
// write the error in red color
console.error( ansi.red(message) );
// play `beep` sound for extra purchase
process.stdout.write(`\u0007`);
return null;
}
});
};
Note: using internal cy.now() command to work around Cypress' tendency to throw Cypress detected that you returned a promise when it (IMO) shouldn't.
(adapted from https://github.com/cypress-io/cypress/issues/300#issuecomment-438176246)
// support/index.js or your test file
Cypress.on(`window:before:load`, win => {
cy.stub( win.console, `error`, msg => {
// log to Terminal
cy.now(`task`, `error`, msg );
// log to Command Log & fail the test
throw new Error( msg );
});
});
Currently there is no straightforward way to do what you are asking but there have been some good discussions on how best to get this information. I copied one solution here but if you follow the github link you can see other solutions proposed.
This snippet was taken from the github issue found here: https://github.com/cypress-io/cypress/issues/300
Just FYI the one easy solution is just to spy on console functions.
cy.window().then((win) => { cy.spy(win.console, "log") })
That will print a command log every time that function is called, and
you could also then assert what has been logged.
Another option depending on why you want to assert that something went wrong is to print the error out under the tests in headless mode. The VP of engineering created an NPM package that does this for you.
Cypress-failed-log
The most easiest way if you simply want to ensure that no error is in the console (which is the most usecase I assume).
# npm
npm install cypress-fail-on-console-error --save-dev
# yarn
yarn add cypress-fail-on-console-error -D
And then add to your support/index.ts file:
import failOnConsoleError from "cypress-fail-on-console-error"
failOnConsoleError()
Now your cypress tests are failing just in time when a console error is printed.
This is the working solution I currently use to check for console errors.
let windowConsoleError;
Cypress.on('window:before:load', (win) => {
windowConsoleError = cy.spy(win.console, 'error');
})
afterEach(() => {
expect(windowConsoleError).to.not.be.called;
})
I'm using Sinon with Mocha to test some expiration date values. I used the same code a few months ago and it worked fine, but somewhere between v1.12.x and v1.17.x, something has changed and I can't seem to find the right path.
let sinon = require('sinon');
describe('USER & AUTHENTICATION ENDPOINTS', function(done) {
beforeEach(function() {
this.clock = sinon.useFakeTimers(new Date().getTime());
return fixtures.load(data);
});
afterEach(function() {
this.clock.restore();
return fixtures.clear(data);
});
context('POST /users', function() { ... }
});
I've tried with and without the new Date().getTime() argument.
I've tried passing in and explicitly calling done().
I've tried removing my fixture load/clear processes.
The end result is always the same:
Error: timeout of 5000ms exceeded. Ensure the done() callback is being called in this test.
Has something changed that I just haven't noticed in the documentation? Do I have some kind of error in there that I can't see?
Any thoughts would be appreciated.
UPDATE
So a little more info here. This clearly has something to do with my code, but I'm at a loss.
If I comment every actual test, the tests run and give me a green "0 passing".
If I run an actual test, even one that just this:
context('POST /users', function() {
it('should create a new user', function(done) {
done();
})
});
I'm right back to the timeout. What am I missing?
Mystery solved. It appears to be a conflict between Sinon and versions of Knex > 0.7.6.
Seems to be because pool2 relies on behavior of setTimeout. Using sinon.useFakeTimers(...) replaces several methods including setTimeout with synchronous versions which breaks it. Can fix by replacing with: clock = sinon.useFakeTimers(Number(date), 'Date');
My original code was written in a world where Knex v0.7.6 was the latest version. Now that it's not everything failed even though the code itself was the same. I used the fix mentioned and things look to be fine.
You are passing done to your describe callback in line 2:
describe('USER & AUTHENTICATION ENDPOINTS', function(done) {
Mocha expects you to invoke it... To get rid of the timeout error, just remove the done parameter from the callback.
New to Meteor, and I love it so far. I use vzaar.com as video hosting platform, and they have a node.js package/API which I added to my Meteor project with meteorhacks:npm. Everything in the API works great, but when I upload a video, I need to fetch the video ID from the API when successfully uploaded.
Problem:
I need to save the video id returned from the vzaar API after uploading, but since it happens in the future, my code does not wait on the result and just gives me "undefined". Is it possible to make the Meteor.method wait for the response?
Here is my method so far:
Meteor.methods({
vzaar: function (videopath) {
api.uploadAndProcessVideo(videopath, function (statusCode, data) {
console.log("Video ID: " + data.id);
return data.id;
}, {
title: "my video",
profile: 3
});
console.log(videoid);
}
});
And this is how the Meteor.call looks like right now:
Meteor.call("vzaar", "/Uploads/" + fileInfo.name, function (err, message) {
console.log(message);
});
When I call this method, I immediately get undefined in browser console and meteor console, and after some seconds, I get the video ID in the meteor console.
Solution
I finally solved the problem, after days of trial and error. I learned about Fibers (here and here), and learned more about the core event loop of Node.js. The problem were that this call answered in the future, so my code always returned undefined because it ran before the api had answered.
I first tried the Meteor.wrapAsync, which I thought were going to work, as it is actually the Future fiber. But I ended up using the raw NPM module of Future instead. See this working code:
var Future = Npm.require('fibers/future');
Meteor.methods({
vzaar: function (videopath) {
var fut = new Future();
api.uploadAndProcessVideo(videopath, function (statusCode, data) {
// Return video id
fut.return (data.id);
}, {
// Video options
title: "hello world",
profile: 3
});
// The delayed return
return fut.wait();
}
});
Remember to install the npm module correctly with meteorhacks:npm first.
I learned about how to use the Future fiber in this case via this stackoverflow answer.
I hope this can be useful for others, as it was really easy to implement.
I have a simple ExtJs (3.4) Grid with a Writer. When the user makes some changes the store is saved to the server as follows:
store.on('save', afterSave(resp));
All is fine. However, I want to get a response as to wheather the record has been saved successfully, failed or an update conflict happed. How to best do this?
Are you using Ext.data.proxy.Ajax to load your stores? If so, you can use the reader property to evaluate and handle the server responses.
Another option would be to make AJAX called directly and handle the responses from there as well
I used exception listener to parse the data as suggested here. But, is this the right way to do this.
Ext.data.DataProxy.addListener('exception', function(proxy, type, action,
options, res) {
if (type == 'response') {
var success = Ext.util.JSON.decode(res.responseText).success;
if (success) {
console.log('UPDATE OK');
} else {
console.log('UPDATE FAILED');
}
}
});