I am currently writing tests protractor and I was wondering if there is some possibility to cancel test execution as soon as something in the beforeEach fails (and return some useful message like "precondition failed: could not login user").
I.e. I have some helper methods in the beforeEach that login the user and then do some setup.
beforeEach:
1) login user
2) set some user properties
Obviously it does not make any sense to execute the 2nd step if the first one fails (actually its quite harmful as the user gets locked which is not nice). I tried to add an "expect" as part of the 1st step, but the 2nd step was still executed -> fresh out of ideas.
Strictly answering your question and without external dependencies:
beforeEach(function() {
// 1) login user
expect(1).toBe(1);
// This works on Jasmine 1.3.1
if (this.results_.failedCount > 0) {
// Hack: Quit by filtering upcoming tests
this.env.specFilter = function(spec) {
return false;
};
} else {
// 2) set some user properties
expect(2).toBe(2);
}
});
it('does your thing (always runs, even on prior failure)', function() {
// Below conditional only necessary in this first it() block
if (this.results_.failedCount === 0) {
expect(3).toBe(3);
}
});
it('does more things (does not run on prior failure)', function() {
expect(4).toBe(4);
});
So if 1 fails, 2,3,4,N won't run as you expect.
There is also jasmine-bail-fast but I'm not sure how it will behave in your before each scenario.
jasmine.Env.prototype.bailFast = function() {
var env = this;
env.afterEach(function() {
if (!this.results().passed()) {
env.specFilter = function(spec) {
return false;
};
}
});
};
then just call:
jasmine.getEnv().bailFast();
(credit goes to hurrymaplelad who wrote an npm that does just that, however you don't need to use it)
jasmine-bail-fast does exactly what you did overriding the specFilter function, but does it on afterEach. So it will only fail after the first "it" is run. It won't help solving this specific case.
with jasmine2 we can set throwOnExpectationFailure to true.
For example in protractor config:
//protractor.conf.js
exports.config = {
//...
onPrepare: () => {
jasmine.getEnv().throwOnExpectationFailure(true);
}
};
Related
I wrote the following to select values of a dropdown and assert selected value to make sure it is selected, one by one.
When("User should see example Cover option with values {string}", (exampleCovers) => {
const exampleCover = exampleCovers.split("|");
exampleCover.forEach(chooseCoverLimit);
function chooseCoverLimit(item) {
acoSection.popups(`aco_popup`).as('popupACO').then(function () {
cy.get('#popupACO').contains('text');
acoSection.dropDowns(`example_cover`).as('coverLimit').select(item, { force: true }).then(function () {
cy.get('#coverLimit').find(':selected').should('have.text', item)
})
})
}
});
This works locally on Cypress Test Runner, as well as through headless from the CLI.
Cypress results on local machine CLI
But when I run on Jenkins, some tests get failed.
Cypress results on Cypress Cloud
I get the following CypressError on Cypress Cloud.
Timed out retrying after 10000ms: cy.find() failed because the page updated as a result of this command, but you tried to continue the command chain. The subject is no longer attached to the DOM, and Cypress cannot requery the page after commands such as cy.find().
Common situations why this happens:
Your JS framework re-rendered asynchronously
Your app code reacted to an event firing and removed the element
You can typically solve this by breaking up a chain. For example, rewrite:
cy.get('button').click().should('have.class', 'active')
to
cy.get('button').as('btn').click()
cy.get('#btn').should('have.class', 'active')
https://on.cypress.io/element-has-detached-from-dom
As per the suggestion, I tried breaking cy.get('#coverLimit').find(':selected').should('have.text', item) into two but it still didn't solve the problem. The same code works for one environment but not another.
I've made following changes to make it work.
When("User should see example Cover option with values {string}", (exampleCovers) => {
const exampleCover = exampleCovers.split("|");
exampleCover.forEach(chooseCoverLimit);
function chooseCoverLimit(item) {
acoSection.popups(`aco_popup`).as('popupACO').then(function () {
cy.get('#popupACO').contains('text')
acoSection.dropDowns(`example_cover`).select(item, { force: true }).then(function () {
acoSection.dropDowns(`example_cover`).find(':selected').should('have.text', item).wait(1000)
})
})
}
});
I've got the following test using cypress:
// myTest.spec.ts
console.log("the test is starting");
describe("My Test Describe", () => {
const testEmail = makeRandomEmail();
console.log("test email", testEmail);
it("should set up the profile", () => {
// setupProfile just makes some requests and returns a promise
cy.wrap(setupProfile(testEmail), {
timeout: 15000,
});
});
it("should test the thing", () => {
// makeAppUrl just returns a string
cy.visit(makeAppURL());
/* test stuff happens here which relies on the generated testEmail */
});
});
This works fine when I run against my dev env (which has no port in the url since it's on 443).
However, I'm running into a weird scenario where, when I run the tests against my local server (on port 3000), the following happens:
it logs "the test is starting" and "test email generatedTestEmail" in the browser console
it runs the setupProfile fine and that test passes.
Then, it reloads the entire test seemingly and relogs what's in (1) (with a new generated email), but (2) is still shown as passing.
It tries to run my it("should test the thing") block which fails because now I have a new user test email.
When I switch out only my host to point at my dev env instead of local, it works fine and doesn't reload as described in (3).
Has anyone run into something like this before? Could it be related to the fact that I have the port in the URL?
The issue was with the baseUrl configuration. This was pointed to the dev env, and the script to run against local did not override this config as described here: https://docs.cypress.io/guides/references/configuration#Command-Line
This solution works for me:
You need to put real baseUrl in configuration file. See example below.
Common problems
baseUrl is not set
Make sure you do not accidentally place the baseUrl or another top-level config variable into the env block. The following configuration is incorrect and WILL NOT WORK:
//DOES NOT WORK
{
"env": {
"baseUrl": "http://localhost:3030",
"FOO": "bar"
}
}
Solution: place the baseUrl property at the top level, outside the env object.
//THE CORRECT WAY
{
"baseUrl": "https://.....",
"env": {
"FOO": "bar"
}
}
Please see here for more details
https://docs.cypress.io/guides/references/configuration#Common-problems
I would try and make use of mocha's before() functionality, to maintain the same data across all tests in the describe() block.
// myTest.spec.ts
describe("My Test Describe", () => {
let testEmail;
before(() => {
console.log("the test is starting");
testEmail = makeRandomEmail();
console.log("test email", testEmail);
});
it("should set up the profile", () => {
cy.wrap(setupProfile(testEmail), {
timeout: 15000,
});
});
it("should test the thing", () => {
cy.visit(makeAppURL());
/* test stuff happens here which relies on the generated testEmail */
});
});
I am trying to use the global hook afterEach to close the browser after each test, but once the first test completes it does not perform the global afterEach. Here is an example of my global.js, any help would be amazing!
module.exports = {
afterEach: function (browser, done) {
browser.end(function () {
done();
})
},
}
documentation says it'll run after each test suite, not after each test, so that might be what you're experiencing :)
it('should for something', function check(done) {
browser.sleep(2000);
$('.csTx').isPresent().then(function(result) {
if(result) {
done();
} else {
xPage.clickBack();
check(done);
}
})
}, 30000);
Can someone explain how done() works and what this is for. I googled it but cannot find any information that would be easy enough for me to understand. I am automating with protractor and jasmine. please consider the above code.
You need to use done if your test creates a parallel TaskQueue in your test's Control Flow (read more about promises and control flow).
For example:
describe('Control Flow', function() {
function logFromPromise(text) {
var deferred = protractor.promise.defer();
deferred.then(function() {
console.log(text);
});
deferred.fulfill();
return deferred;
}
it('multiple control flows', function() {
setTimeout(function() {
logFromPromise('1');
});
logFromPromise('0');
});
}
Calling setTime creates a parallel Task Queue in the control:
ControlFlow
| TaskQueue
| | Task<Run fit("multiple control flows") in control flow>
| | | TaskQueue
| | | | Task <logFromPromise('0');>
| TaskQueue
| | Task <setTimeout>
Protractor thinks the test is "done" after 0 is printed. In this example, 1 will probably be printed after the test is completed. To make protractor wait for Task <setTimeout>, you need to call the done function:
it('multiple control flows', function(done) {
setTimeout(function() {
logFromPromise('1').then(function() {
done();
});
});
logFromPromise('0');
});
If you can, let protractor handle when the test is "done". Having parallel TaskQueues can lead to unexpected race conditions in your test.
Here is a sample describe that you can run and see what happens. I have to mention that I don't use Protractor so there might exist some additional considerations to be made concerning its specific capabilities.
describe('Done functionality', function(){
var echoInOneSecond = function(value){
console.log('creating promise for ', value);
return new Promise(function(resolve, reject){
console.log('resolving with ', value);
resolve(value);
});
};
it('#1 this will untruly PASS', function(){
var p = echoInOneSecond('value #1');
p.then(function(value){
console.log('#1 expecting...and value is ', value);
expect(value).toBe('value #1');
});
});
it('#2 this will NOT FAIL', function(){
var p = echoInOneSecond('value #2');
p.then(function(value){
console.log('#2 expecting... and value is ', value);
expect(value).not.toBe('value #2');
});
});
it('3 = will truly FAIl', function(done){
var p = echoInOneSecond('value #3');
p.then(function(value){
console.log('#3 expecting... and value is ', value);
expect(value).not.toBe('value #3');
done();
});
});
it('4 = this will truly PASS', function(done){
var p = echoInOneSecond('value #4');
p.then(function(value){
console.log('#4 expecting... and value is ', value);
expect(value).toBe('value #4');
done();
});
});
});
when running the test you will note the sequence: first promises #1, #2, #3 will be created and resolved one by one. Please note that expectation for #1 and #2 will not be run yet because promises are resolved asynchronously.
Then, since #3 test uses done, after #3 promise is created, functions for thens of all previous promises are evaluated: you will see '#1 expecting...' and '#2 expecting...', but jasmine won't care about that because tests #1 and #2 are already finished and everything concerning them done. Only after those #3 expectation is made and it will truly fail because jasmine does take care of everything that happens before done() is made.
And then you can watch #4 test normal flow -- creating promise, resolving, expectation, everything considered by jasmine so expectation will truly pass.
I haven't used Protractor. For Jasmine, my understanding is that done makes Jasmine wait but not in the traditional sense of timeout. It is not like a timer which is always run. I think done acts as a checkpoint in Jasmine. When Jasmine sees that a spec uses done, it knows that it cannot proceed to the next step (say run next spec or mark this spec as finished i.e. declare verdict of the current spec) unless the code leg containing done has been run.
For example, jasmine passes this spec even though it should fail as it doesn't wait for setTimeout to be called.
fit('lets check done',()=>{
let i=0;
setTimeout(function(){
console.log("in timeout");
expect(i).toBeTruthy();//the spec should fail as i is 0 but Jasmine passes it!
},1000);
//jasmine reaches this point and see there is no expectation so it passes the spec. It doesn't wait for the async setTimeout code to run
});
But if my intention is that Jasmine waits for the the async code in setTimeout, then I use done in the async code
fit('lets check done',(done)=>{
let i=0;
setTimeout(function(){
console.log("in timeout");
expect(i).toBeTruthy();//with done, the spec now correctly fails with reason Expected 0 to be truthy.
done();//this should make jasmine wait for this code leg to be called before declaring the verdict of this spec
},1000);
});
Note that done should be called where I want to check the assertions.
fit('lets check done',(done)=>{
let i=0;
setTimeout(function(){
console.log("in timeout");
expect(i).toBeTruthy();//done not used at the right place, so spec will incorrectly ypass again!.
//done should have been called here as I am asserting in this code leg.
},1000);
done();//using done here is not right as this code leg will be hit inn normal execution of it.
});
In summary, think of done as telling Jasmine - "I am done now" or "I'll be done when this code hits"
I would like to make pause each test.
I've create this function:
afterEach:function(browser){
browser.pause(2000);
},
But when I run tests, I will get error:
TypeError: browser.pause is not a function
Why ? In tests browser.pause is function.
The answer, provided by beatfactor, on the linked GitHub issue, is
When using afterEach you need to use the done callback argument always if you want to use the browser object. That is for backwards compatibility. So you need to do either:
afterEach(browser, done) {
// ...
done();
}
I've written on GitHub as issue and I've got a solution: https://github.com/nightwatchjs/nightwatch/issues/921
RESOLVE:
use
afterEach(done) {
// ...
done();
}
instead of
afterEach(browser, done) {
// ...
done();
}
When using afterEach, if you want access to the browser object then you need to add the done callback argument after browser to your function.
So your function should look like this:
afterEach(browser, done) {
// ...
done();
}
Note that you have to call done() to signify the completion of the test.
If you only have one argument in afterEach then it's the done argument.
afterEach(done) {
// ...
done();
}
In other words, if you write
afterEach(browser) {
// ...
}
browser is actually the done callback. You've named it browser but that's not what it is.
My dear friend you have to do "something" and then pause!
(i.e. assert the url and then pause!)
Does it works?
afterEach:function(browser){
browser
.assert.urlEquals('http://www.google.com')
.pause(2000)
},