Is it possible to set a message (mentioning reason) why a particular test is skipped, so that it could be used in reporters.
describe('xxx', function() {
checkToSkip(1)("test1", function() {\*test goes here*\});
checkToSkip(4)("test2", function() {\*test goes here*\});
});
function checkToSkip(now) {
return now > 3 ? it : xit;
//return now > 3 ? it : it.skip;
}
Here 'test1' will be skipped as 'checkToSkip' returns 'xit' (or it.skip). Is it possible to pass a message to reporter mentioning the reason for the test skip? something like below (or any other possible way).
checkToSkip(4)("test2", function() { \\ test goes here}, "My Skip message!!!!" );
Note: Im using mocha in webdriverIO.
Thanks.
I'd just slightly modify the checkToSkip function and used Pending Tests instead of skip().
function checkToSkip(now, title, testCallable) {
if (now > 3) {
it(title, testCallable);
} else {
it(title+"#My Skip message!#");
}
}
Then use it like:
describe('xxx', function() {
checkToSkip(1, "test1", function() {\*test goes here*\});
});
It's possible to modify the test title itself inside the test before skipping:
it('My cool test', async function() {
if (this.response.status !== 201) {
this._runnable.title += ' - Skipped with reason: wrong response code'
this.skip()
}
expect(this.response.data).to.have.property('mycoolproperty')
})
Also possible to formalize the check and move it to the external function:
function skipIf(that, condition, reason){
if (condition) {
that._runnable.title += ` - Skipped with reason: ${reason}`
that.skip()
}
}
So the test will look like that:
it('My cool test', async function() {
skipIf(this, this.response.status !==201, 'Wrong response status')
expect(this.response.data).to.have.property('mycoolproperty')
})
Related
I want to skip and allow tests in the before each hook as follows
beforeEach(() =>{
if(Cypress.mocha.getRunner().suite.ctx.currentTest.title === `Skip this`){
// skip the first test case only but run the second one [How?]
}
});
it(`Skip this`, () => {
});
it(`Don't skip this`, () => {
});
In the place of [How?] I tried using the following:
cy.skipOn(true) from the cypress skip-test plugin but apparently it skips the beforeEach hook not the test itself.
this.skip() but apparently this is not a valid function. Also, if I changed the beforeEach from an arrow function expression, the skip function works but it skips the whole suite and not just the desired test case.
Any ideas?
Change the function type from arrow function to regular function, then you can use the built-in Mocha skip() method.
beforeEach(function() {
if (condition) {
this.skip()
}
})
Your code sample will look like this:
beforeEach(function() { // NOTE regular function
if (Cypress.mocha.getRunner().suite.ctx.currentTest.title === 'Skip this') {
this.skip()
}
});
it(`Skip this`, () => {
});
it(`Don't skip this`, () => {
});
Or use the Mocha context you already use for test title
beforeEach(() => { // NOTE arrow function is allowed
const ctx = Cypress.mocha.getRunner().suite.ctx
if (ctx.currentTest.title === 'Skip this') {
ctx.skip()
}
});
afterEach()
If you have an afterEach() hook, the this.skip() call does not stop it running for the skipped test.
You should check the condition inside that hook also,
afterEach(function() {
if (condition) return;
... // code that should not run for skipped tests.
})
I try to use aftereach() but it will execute either the testcase is passed or failed. I need to know the context of that should i use for example if condition or what?
For example:
describe('TestSuite', function(){
it('THIS TEST CASE PASSED', function(){
})
it('THIS TEST CASE FAILED', function(){
})
})
I need to make it like this. If the a testcase is passed do the actions
...
...
...
and if the testcase failed do the actions
...
...
...
You didn't give much info about your code, so I will suggest a solution that might not be suited for you. I would use a variable that gets set to "fail" if an assertion fails in the previous test and use that to determine the action in the 2nd test, e.g.
describe('check if first test passed or failed then do A or B', () => {
let result
it('test 1', () => {
cy.request({url: Cypress.env('url')}).its('status').then((status) => {
if (status === 500) {
result = 'fail'
}
})
})
it('test 2', () => {
if (result === 'fail') {
cy.log('Previous test failed, so I did Action A')
Code Action A
}
else
{
cy.log('Previous test passed, so I did Action B')
Code Action B
}
})
})
I have recently installed the jasmine-spec-reporter package in order to produce more verbose and helpful logging during execution of test suites.
I want to be able to log expected and actual values for failed test cases and was wondering if I needed to explicitly have a statement in the form of expect(someCondition).toEqual(true); in order for me to see these values.
For example, I have a function like the following:
it('should log in', function(done) {
var customerNameElement;
customerNameElement = element.all(by.xpath('some_xpath')).first();
core.infoMessage(browser, JSON.stringify(params, undefined, 4));
login.login(browser, undefined, undefined, getWaitTime())
.then(function() {
return browser.wait(protractor.ExpectedConditions.and(
function() { return core.isUnobscured(browser, customerNameElement);
}, protractor.ExpectedConditions.elementToBeClickable(customerNameElement)), getWaitTime());
})
.catch(fail)
.then(done);
});
I can still log the failure to the console but not in the form that I'd like. Does jasmine-spec-reporter handle this or do I have to add the statement from above in each test case?
Also, does the fail keyword in the .catch() have any properties I can use to my advantage? It comes from:
// Type definitions for Jasmine 2.5.2 // Project: http://jasmine.github.io/
Thanks
you can try adding the below in protractor_config file:
let SpecReporter = require('jasmine-spec-reporter').SpecReporter;
exports.config = {
// your config here ...
onPrepare: function () {
jasmine.getEnv().addReporter(new SpecReporter({
spec: {
displayStacktrace: true
}
}));
}
}
Also, add the print function in the jasmineNodeOptssection
jasmineNodeOpts: {
...
print: function() {}
}
I have a suite which includes multiple specs. Each spec uses code on some libraries that return a rejected promise upon failure.
I can easily catch those rejected promises inside my spec. What I'm wondering about is that if I can make Protractor exit the whole suite inside that catch function, because the next specs inside the same suite are dependent on the success of the previous specs.
Pretend I have a suite called testEverything which has these specs openApp,signIn,checkUser,logout. If openApp fails, all next specs will fail due to dependency.
Consider this code for openApp:
var myLib = require('./myLib.js');
describe('App', function() {
it('should get opened', function(done) {
myLib.openApp()
.then(function() {
console.log('Successfully opened app');
})
.catch(function(error) {
console.log('Failed opening app');
if ( error.critical ) {
// Prevent next specs from running or simply quit test
}
})
.finally(function() {
done();
});
});
});
How would I exit the whole test?
There is a module for npm called protractor-fail-fast. Install the module npm install protractor-fail-fast. Here's an example from their site where you would place this code into your conf file:
var failFast = require('protractor-fail-fast');
exports.config = {
plugins: [{
package: 'protractor-fail-fast'
}],
onPrepare: function() {
jasmine.getEnv().addReporter(failFast.init());
},
afterLaunch: function() {
failFast.clean(); // Cleans up the "fail file" (see below)
}
}
Their url is here.
I have managed to come up with a workaround. Now the actual code that I used is way more complex, but the idea is the same.
I added a global variable inside protractor's config file called bail. Consider the following code at the top of config file:
(function () {
global.bail = false;
})();
exports.config: { ...
The above code uses an IIFE (Immediately Invoked Function Expression) which defines bail variable on protractor's global object (which would be available throughout the whole test).
I also have written asynchronous wrappers for the Jasmine matchers I need, which would execute an expect expression followed by a comparison, and return a promise (using Q module). Example:
var q = require('q');
function check(actual) {
return {
sameAs: function(expected) {
var deferred = q.defer();
var expectation = {};
expect(actual).toBe(expected);
expectation.result = (actual === expected);
if ( expectation.result ) {
deferred.resolve(expectation);
}
else {
deferred.reject(expectation);
}
return deferred.promise;
}
};
}
module.exports = check;
Then at the end of each spec, I set the bail value based on the spec's progress, which would be determined by the promise of those asynchronous matchers. Consider the following as first spec:
var check = require('myAsyncWrappers'); // Whatever the path is
describe('Test', function() {
it('should bail on next spec if expectation fails', function(done) {
var myValue = 123;
check(myValue).sameAs('123')
.then(function(expectation) {
console.log('Expectation was met'); // Won't happen
})
.catch(function(expectation) {
console.log('Expectation was not met'); // Will happen
bail = true; // The global variable
})
.finally(function() {
done();
});
});
});
Finally, on the beginning of next specs, I check for bail and return if necessary:
describe('Test', function() {
it('should be skipped due to bail being true', function(done) {
if ( bail ) {
console.log('Skipping spec due to previous failure');
done();
return;
}
// The rest of spec
});
});
Now I want to mention that there's one module out there called protractor-fail-fast which bails on the whole test whenever an expectation fails.
But in my case, I needed to set that bail global variable depending on which type of expectation has been failed. I ended up writing a library (really small) that distinguishes failures as critical and non-critical and then using that, specs would be stopped only if a critical failure has occurred.
Here are 2 samples of the same test. The only difference is that first one uses a promise in beforeAll block to assign a value to the variable while the second one assigns the value directly.
I raised a similar question Running spec after promise has been resolved with one of the comments pointing to this issue https://github.com/jasmine/jasmine/issues/412 which says that this is not supported in Jasmine. Has somebody figured out any workaround?
This fails with TypeError: Cannot read property 'forEach' of undefined
describe('Async car test', function () {
var cars;
beforeAll(function (done) {
// getCars() is a promise which resolves to ['audi', 'bmw']
getCars().then(function (data) {
cars = data;
console.log(cars) // ['audi', 'bmw']
done();
});
});
cars.forEach(function (car) {
it('car ' + car, function () {
expect(car).toBe(car);
});
});
});
This works fine
describe('Car test', function () {
var cars = ['audi', 'bmw'];
cars.forEach(function (car) {
it('car ' + car, function () {
expect(car).toBe(car);
});
});
});
Posting it as an answer, because I can't see things properly in comments.
I'm actually generating tests in my spec as well, and I'm using https://www.npmjs.com/package/jasmine-data-provider , I think you probably cannot generate it directly from resolved promise. And wrapping in another it doesn't work for you. This should work:
var using = require('jasmine-data-provider');
using(cars.forEach, function (car) {
it(car + ' should be' + car, function () {
expect(car).toBe(car);
});
});
This is not an issue with jasmine, it is an issue with your code.
beforeAll does not block subsequent code below the statement. it blocks code that is defined in it('should ...', (done)=>{...});
it('should have cars', (done) => {
cars.forEach(function (car) {
expect(car).toBe(car);
});
});
Since Jasmine does not support adding tests at runtime, the trick is to request the asynchronous data before starting Jasmine, and then using the retrieved data during runtime instead. This can be achieved with a singleton and programmatically starting Jasmine.
See here for a working example.
// car-collection.js
class CarCollection {
static load() {
return this.request()
then((data) => this.cars = data);
}
static request() {
// in practice this function would do something cooler
return Promise.resolve(['audi', 'bmw']);
}
}
modules.export = CarCollection;
Since CarCollection has methods that are static they will be shared across imports and this.cars will persist.
// launcher.js
const Jasmine = require('jasmine');
const CarCollection = require('./car-collection');
CarCollection.load()
.then(() => {
console.log(`car count is ${CarCollection.cars.length}`); // prints: car count is 2
const jasmine = new Jasmine();
jasmine.loadConfigFile(...); // path to jasmine.json
jasmine.execute();
});
An important step here is configure jasmine to know where to look for the test files. Either by loading a config or passing specifics into the execute function.
// car.spec.js
const CarCollection = require('./car-collection');
describe('test', function () {
CarCollection.cars.forEach((car) => {
it('test' + car, () => {
expect(car).toBe(car);
});
});
});
Now run node ./launcher.js and the tests should run.