Cypress seems to restart the test after cy.visit - cypress

I've got the following test using cypress:
// myTest.spec.ts
console.log("the test is starting");
describe("My Test Describe", () => {
const testEmail = makeRandomEmail();
console.log("test email", testEmail);
it("should set up the profile", () => {
// setupProfile just makes some requests and returns a promise
cy.wrap(setupProfile(testEmail), {
timeout: 15000,
});
});
it("should test the thing", () => {
// makeAppUrl just returns a string
cy.visit(makeAppURL());
/* test stuff happens here which relies on the generated testEmail */
});
});
This works fine when I run against my dev env (which has no port in the url since it's on 443).
However, I'm running into a weird scenario where, when I run the tests against my local server (on port 3000), the following happens:
it logs "the test is starting" and "test email generatedTestEmail" in the browser console
it runs the setupProfile fine and that test passes.
Then, it reloads the entire test seemingly and relogs what's in (1) (with a new generated email), but (2) is still shown as passing.
It tries to run my it("should test the thing") block which fails because now I have a new user test email.
When I switch out only my host to point at my dev env instead of local, it works fine and doesn't reload as described in (3).
Has anyone run into something like this before? Could it be related to the fact that I have the port in the URL?

The issue was with the baseUrl configuration. This was pointed to the dev env, and the script to run against local did not override this config as described here: https://docs.cypress.io/guides/references/configuration#Command-Line

This solution works for me:
You need to put real baseUrl in configuration file. See example below.
Common problems
baseUrl is not set
Make sure you do not accidentally place the baseUrl or another top-level config variable into the env block. The following configuration is incorrect and WILL NOT WORK:
//DOES NOT WORK
{
"env": {
"baseUrl": "http://localhost:3030",
"FOO": "bar"
}
}
Solution: place the baseUrl property at the top level, outside the env object.
//THE CORRECT WAY
{
"baseUrl": "https://.....",
"env": {
"FOO": "bar"
}
}
Please see here for more details
https://docs.cypress.io/guides/references/configuration#Common-problems

I would try and make use of mocha's before() functionality, to maintain the same data across all tests in the describe() block.
// myTest.spec.ts
describe("My Test Describe", () => {
let testEmail;
before(() => {
console.log("the test is starting");
testEmail = makeRandomEmail();
console.log("test email", testEmail);
});
it("should set up the profile", () => {
cy.wrap(setupProfile(testEmail), {
timeout: 15000,
});
});
it("should test the thing", () => {
cy.visit(makeAppURL());
/* test stuff happens here which relies on the generated testEmail */
});
});

Related

How to get cypress to block datadog requests

We recently installed datadogRUM in our application and now so many DD events kick off in my cypress test that they cause a timeout and failure
I have tried cy.intercept in multiple ways:
cy.intercept('POST', 'https://rum.browser-intake-datadoghq.com/*', {
statusCode: 202,
body: {
},
}).as('datadogRUM');
cy.intercept('POST', 'https://rum-http-intake.logs.datadoghq.com/*', {});
cy.intercept(/\.*datadog.*$/, (req) => {
console.log('DATADOG INTERCEPTED');
req.reply("console.log('datadog intercept');");
});
cy.intercept({
method: 'POST',
url: '/\.*datadog.*$/'
}, req => {
req.destroy();
});
cy.intercept('POST', 'https://rum-http-intake.logs.datadoghq.com/*', { forceNetworkError: true });
just to start. I feel like I've tried every possible variation. I also created a cypress.json file in my /cypress folder
{
"blockHosts": "*datadoghq.com/*"
}
I get hundreds of calls back in my network tab to https://rum.browser-intake-datadoghq.com/api/v2/rum with the preview of console.log('datadog intercept') as I've intercepted them. They all display the solid blue line as if they are being intercepted and blocked. When I set the intercept to an alias I see the alias in my cypress runner window. But there are no 503s or 404s anywhere. The page still fills up with events, cypress gets overloaded, and my test times out.
I even tried copying the data-dog-rum.ts from the src/utils folder to cypress/utils and either commenting out everything or setting the sampleRate to 0, no dice.
EDIT: I am able to get the test passing by adding
Cypress.on('uncaught:exception', () => {
// returning false here prevents Cypress from
// failing the test
return false;
});
to my support/index.js but now whether I add a cy.intercept in my test makes absolutely no difference. The page still fills up with datadog requests regardless, and whether they come back as 200/pending/cancelled, they still delay a single it block in a spec to where it takes 60 seconds to run instead of approx 10 seconds
You can use javascript to perform the stub inside the routeHandler
cy.intercept('*', (req) => { // look at everything
if (req.url.includes('datadoghq')) { // add more conditions if needed
req.reply({}) // prevent request reaching the server
}
})
blockhosts should work with
Pass only the host
{
"blockHosts": "*datadoghq.com"
}

Cypress: browser authentication not working inside a custom command

I have a website which needs browser authentication on all page visits. If I use the below code on all page visits it works fine:
describe('WW2-3461 validate patches for contrib modules and core', () => {
it('should visit my base URL', () => {
cy.visit(Cypress.env('base_url'), {
// Add basic auth headers
auth: {
username: Cypress.env('browserAuthentication').username,
password: Cypress.env('browserAuthentication').password
},
failOnStatusCode: false
})
})
})
I am passing the base_url as an environment through CLI:
npx cypress run --spec file.spec.js --env base_url=$base_url
I don't want to use browserauthentication on all page visits, so I created a custom command like below (in integration/project/utility.js file):
Cypress.Commands.add('addBrowserAuthentication', (url) => {
cy.visit(url, {
// Add basic auth headers
auth: {
username: Cypress.env('browserAuthentication').username,
password: Cypress.env('browserAuthentication').password
},
failOnStatusCode: false
})
})
However if I call this command inside a spec file, like below:
import './utility.spec'
describe('WW2-3461 validate patches for contrib modules and core', () => {
beforeEach(() => {
cy.addBrowserAuthentication(Cypress.env('cypress_host'))
})
it('should visit my base URL', () => {
cy.visit(Cypress.env('cypress_host'), {
})
})
the browser authentication does not seem to work (tried to call it inside before() hook or directly inside the test scenario) as it returns
> 401: Unauthorized
This was considered a failure because the status code was not `2xx`.
This http request was redirected '1' time to:
- 302: https://mysite.local
If you do not want status codes to cause failures pass the option: `failOnStatusCode: false`
I can't seem to work around this issue. Tried using cy.request() but getting the same issue. Has someone faced this issue?
You don't need to pass env variable as a parameter, you can directly access it inside the custom command.
Cypress.Commands.add('addBrowserAuthentication', () => {
cy.visit(Cypress.env('base_url'), {
// Add basic auth headers
auth: {
username: Cypress.env('browserAuthentication').username,
password: Cypress.env('browserAuthentication').password,
},
failOnStatusCode: false,
})
})
And in your test directly use:
cy.addBrowserAuthentication()
Also I am not sure if its required to visit the website two times. So you can remove cy.visit(Cypress.env('cypress_host') if you are already using cy.addBrowserAuthentication().
Since this works
cy.visit(Cypress.env('base_url'), {
// Add basic auth headers
auth: {
username: Cypress.env('browserAuthentication').username,
password: Cypress.env('browserAuthentication').password
},
failOnStatusCode: false
})
but this doesn't
cy.addBrowserAuthentication(Cypress.env('cypress_host'))
it looks like Cypress.env('cypress_host') is not the same as Cypress.env('base_url').
Everything else looks good in your command, so I'd try with
cy.addBrowserAuthentication(Cypress.env('base_url'))
When you cy.visit(Cypress.env('cypress_host') without headers inside the test, the auth headers from addBrowserAuthentication are lost. I suggest removing that cy.visit().

Nightwatch.js Global afterEach not working

I am trying to use the global hook afterEach to close the browser after each test, but once the first test completes it does not perform the global afterEach. Here is an example of my global.js, any help would be amazing!
module.exports = {
afterEach: function (browser, done) {
browser.end(function () {
done();
})
},
}
documentation says it'll run after each test suite, not after each test, so that might be what you're experiencing :)

Problems with $httpBackend.verifyNoOutstandingExpectation()

I have recently started writting unit tests using Karma + Karma-jasmine but I am having problems with the following tests:
describe("WEBSERVICE:", function () {
var webservice,
$httpBackend,
authRequestHandler,
webserviceURL = "http://localhost:8006/";
beforeEach(inject(function (Webservice, $injector) {
webservice = Webservice;
$httpBackend = $injector.get("$httpBackend");
authRequestHandler = $httpBackend
.when("GET", webserviceURL + "users/login")
.respond(200, "ok");
}));
afterEach(function() {
$httpBackend.verifyNoOutstandingExpectation();
$httpBackend.verifyNoOutstandingRequest();
});
it("should EXISTS", function () {
expect(webservice).toBeDefined();
});
it("should throw a WebserviceError if we are not logged in" , function () {
expect(function () {
webservice.item("negs", "RPT");
}).toThrow(webserviceAuthenticationError);
});
it("should NOT HAVE credentials when instantiated", function () {
expect(webservice.hasCredentials()).toBeFalsy();
});
it("should log in when valid credentials are given", function () {
$httpBackend.expectGET("users/login");
webservice.withCredentials("sam", "password");
});
});
It appears to be the following which creates the problem since all tests pass when I remove it:
afterEach(function() {
$httpBackend.verifyNoOutstandingExpectation();
$httpBackend.verifyNoOutstandingRequest();
});
I was just wondering if anyone could help me with this.
Thanks a lot.
The reason you having problems is with
$httpBackend.verifyNoOutstandingExpectation();
is due to your last test
it("should log in when valid credentials are given", function () {
$httpBackend.expectGET("users/login");
webservice.withCredentials("sam", "password");
});
having unsatisfied requests which you can see in this jsfiddle
Error: Unsatisfied requests: GET users/login
If you comment out
$httpBackend.verifyNoOutstandingExpectation()
your first three tests pass but the last one is amber as there is no expectations, see this fiddle.
WEBSERVICE:
should EXISTS
should throw a WebserviceError if we are not logged in
should NOT HAVE credentials when instantiated
SPEC HAS NO EXPECTATIONS should log in when valid credentials are given
In the AngularJS documentation it says
verifyNoOutstandingExpectation();
Verifies that all of the requests defined via the expect api were made. If any of the requests were not made, verifyNoOutstandingExpectation throws an exception.
You will need to restructure that test so that
webservice.withCredentials("sam", "password");
makes a request through $httpBackend

Jasmine/Protractor: stop test on failure in beforeEach

I am currently writing tests protractor and I was wondering if there is some possibility to cancel test execution as soon as something in the beforeEach fails (and return some useful message like "precondition failed: could not login user").
I.e. I have some helper methods in the beforeEach that login the user and then do some setup.
beforeEach:
1) login user
2) set some user properties
Obviously it does not make any sense to execute the 2nd step if the first one fails (actually its quite harmful as the user gets locked which is not nice). I tried to add an "expect" as part of the 1st step, but the 2nd step was still executed -> fresh out of ideas.
Strictly answering your question and without external dependencies:
beforeEach(function() {
// 1) login user
expect(1).toBe(1);
// This works on Jasmine 1.3.1
if (this.results_.failedCount > 0) {
// Hack: Quit by filtering upcoming tests
this.env.specFilter = function(spec) {
return false;
};
} else {
// 2) set some user properties
expect(2).toBe(2);
}
});
it('does your thing (always runs, even on prior failure)', function() {
// Below conditional only necessary in this first it() block
if (this.results_.failedCount === 0) {
expect(3).toBe(3);
}
});
it('does more things (does not run on prior failure)', function() {
expect(4).toBe(4);
});
So if 1 fails, 2,3,4,N won't run as you expect.
There is also jasmine-bail-fast but I'm not sure how it will behave in your before each scenario.
jasmine.Env.prototype.bailFast = function() {
var env = this;
env.afterEach(function() {
if (!this.results().passed()) {
env.specFilter = function(spec) {
return false;
};
}
});
};
then just call:
jasmine.getEnv().bailFast();
(credit goes to hurrymaplelad who wrote an npm that does just that, however you don't need to use it)
jasmine-bail-fast does exactly what you did overriding the specFilter function, but does it on afterEach. So it will only fail after the first "it" is run. It won't help solving this specific case.
with jasmine2 we can set throwOnExpectationFailure to true.
For example in protractor config:
//protractor.conf.js
exports.config = {
//...
onPrepare: () => {
jasmine.getEnv().throwOnExpectationFailure(true);
}
};

Resources