When you configure cypress to use reportportal, you can add this code to a test
it('a test', () => { cy.setTestCaseId('12123'); cy.setTestDescription('desc'); cy.setTestAttributes([{ key: 'keyName', value: 'VALUE' }]); });
It should be possible to do this with the suites as well, but I can't get that working. I tried putting it in a before section with no luck, also verified if there are are functions to use dedicated to suites but there doesn't seem to be.
Does someone know how you can do this?
Related
A lot of this is wrapped in commands, but I've left that part out to make the problem more feasible.
Consider these two tests:
# Test1: Test login for user
- Step1: Logs in manually (go to login-URL, fill out credentials and click 'Log in').
- Step2: Save auth-cookies as fixtures.
# Test2: Test something is dashboard for user.
- Step1: Set auth-cookies (generated in Test1)
- Step2: Visits https:://example.org/dashboard and ensures the user can see the dashboard.
If they run as written as listed above, then everything is fine.
But if Test2 runs before Test1, then Test2 will fail, since Test1 hasn't to generated the cookies yet.
So Test1 is kind of a prerequisite for Test2.
But Test1 doesn't need to run every time Test2 runs - only if the auth-cookies aren't generated.
I wish I could define my Test2 to be like this instead:
Test2: Test something is dashboard for user.
- Step1: Run ensureAuthCookiesExists-command
- Step2: If the AuthCookies.json-fixture doesn't exist, then run Test1
- Step3: Sets auth-cookies (generated in Test1)
- Step4: Visits https:://example.org/dashboard and ensures the user can see the dashboard.
Solution attempt 1: Control by order
For a long time I've done this using this answer: How to control order of tests. And then having my tests defines like this:
{
"baseUrl": "http://localhost:5000",
"testFiles": [
"preparations/*.js",
"feature-1/check-header.spec.js",
"feature-2/check-buttons.spec.js",
"feature-3/check-images.spec.js",
"feature-4/check-404-page.spec.js",
//...
]
}
But that is annoying, since it means that I keep having to add to add new features to that list, which get's annoying.
And this only solves the problem if I want to run all the tests. If I want to run preparations.spec.js and thereafter: feature-2/check-buttons.spec.js. Then I can't do that easily.
Solution attempt 2: Naming tests smartly
I also tried simply naming them appropriately, like explain here: naming your tests in Cypress.
But that pollutes the naming of the tests, making it more cluttered. And it faces the same issues as solution attempt 1 (that I can't easily run two specific tests after one another).
Solution attempt 3: Making a command for it
I considered making a command that tests for it. Here is some pseudo-code:
beforeEach(() => {
if( preparationsHasntCompleted() ){
runPreparations();
}
}
This seems smart, but it would add extra runtime to all my tests.
This may not suit your testing aims, but the new cy.session() can assure cookie is set regardless of test processing order.
Use it in support in beforeEach() to run before every test.
The first test that runs (either test1 or test2) will perform the request, subsequent tests will use cached values (not repeating the requests).
// cypress/support/e2e.js -- v10 support file
beforeEach(() => {
cy.session('init', () => {
// request and set cookies
})
})
// cypress/e2e/test1.spec.cy.js
it('first test', () => {
// the beforeEach() for 1st test will fire the request
...
})
// cypress/e2e/test2.spec.cy.js
it('second test', () => {
// the beforeEach() for 2nd test will set same values as before from cache
// and not resend the request
})
Upside:
performing login once per run (ref runtime concerns)
performing tests in any order
using the same token for all tests in session (if that's important)
Downside:
if obtaining auth cookies manually (vi UI), effectively moving the login test to a beforeEach()
Example logging in via request
Rather than obtaining the auth cookie via UI, it may be possible to get it via cy.request().
Example from the docs,
cy.session([username, password], () => {
cy.request({
method: 'POST',
url: '/login',
body: { username, password },
}).then(({ body }) => {
cy.setCookie('authToken', body.token)
})
})
It is generally not recommended to write tests that depend on each other as stated in the best practices. You can never be sure that they run correctly. In your case if the first test fails all the other ones will fail, even if the component/functionality is functioning propperly. Especially if you test more than just the pure login procedure, e.g. design.
As #Fody said before you can ensure being logged in in the beforeEach() hook.
I would do it like this:
Use one describe to test the login via UI.
Use another describe where you put the login (via REST) in the before() and the following command Cypress.Cookies.preserveOnce('<nameOfTheCookie>'); in the beforeEach() to not delete the test for the following it()s
I recently started doing automation tests for iOS using Appium and Jasmine.
it('should add product to cart', () => {
driver.pause(1000);
$("~ProductDetailCTAView.addToCartButton").click();
driver.pause(3000);
});
it('should see elements when clicked on the cart button', () => {
$("~UIBarButtonItem.shopBarButton").click();
$("~CartController.CartCell.0");
driver.pause(3000);
});
What I want to do is, call these methods if addToCartButton is enabled. I tried putting them in an if condition but it says I can't put 'it' methods inside if-else conditions. I looked for asynchronous testing in Jasmine on internet but got confused.
You really can't put an 'IF' on Jasmine unit testing 'it' ,
If you need to test different paths of your code, you should write an entire 'it' to it.
I started using Protractor and the first thing I've tried to do is to use Mocha and Chai instead of Jasmine. Although now I'm not sure if that was a good idea.
first I needed to make Chai accessible from all the spec files, without having to import everytime, I found it's possible to do in protractor.conf file:
onPrepare: ->
global.chai = require 'chai'
chai.use require 'chai-string'
chai.use require 'chai-as-promised'
global.expect = chai.expect
now in a spec like this:
it "when clicked should sort ",->
headerColumns.get(0).click()
firstCellText = $$(".first-cell").getText()
secondCellText = $$(".second-cell").getText()
# this won't work
expect(firstCellText).eventually.be.above(secondCellText)
to make it work I could do:
# now this works
$$(".second-cell").getText().then (secondCellText)->
expect(firstCellText).eventually.be.above(secondCellText)
but that's ugly, and I don't want to wrap stuff inside of .then all the time. I'm thinking there should be a better way(?)
I was having the same problem. The issue for me was to add a longer timeout to mocha through the Protractor config.js.
This works because Protractor tests take considerably longer relative to other modules like supertest since an actual browser is being interacted with.
I added
mochaOpts: {
timeout: 5000
}
to my protractor config and the tests passed.
Either you need to mention timeout for the whole test suite using "this.timeout(1000);" just below the describe block, or you can change it for individual test cases by explicitly defining timeout for each "it" block.
example for describe block:
describe("test-suite",function () {
this.timeout(5000);
it("test-case",function () {
//test case code goes here
});
});
example for it block:
it("test-case",function () {
this.timeout("20000");
});
I found this question while I was trying to do the same thing in TypeScript, but the approach in the protractor.conf.js is identical.
Making chai available globally
It appears to achieve this you're right that it needs to be done in the on prepare. a fairly terse example is below. As I understand it this is needed because chai is of course not apart of mocha but some additional candy that we can use with mocha. Unlike jasmine where everything is bundled into the framework.
protractor.conf.js fragment
onPrepare: function() {
var chai = require('chai'); // chai
var chaiAsPromised = require("chai-as-promised"); // deal with promises from protractor
chai.use(chaiAsPromised); // add promise candy to the candy of chai
global.chai = chai; // expose chai globally
}
example spec
Once you've got chai and chai-as-promised setup globally you need to add a "little" boiler plate to your spec to expose expect which comes from chai.
example.e2e-spec.ts fragment
const expect = global['chai'].expect; // obviously TypeScript
it('will display its title', () => {
pageObject.navigateTo();
const title = element(by.css('app-root h1')).getText();
expect(title).to.eventually.contain('An awesome title');
});
Avoiding then
I can't tell what your $$ references are, but if you're using the protractor components browser and by etc. then things clean up a little bit.
The call const title = element(by.css('app-root h1')).getText(); seems to return a promise which jasmine seems to deal with out of the box while mocha+chai doesn't. That's where chai-as-promised comes in.
using our additional syntax candy expect(title).to.eventually.contain('An awesome title'); resolves the promise quite neatly and we've avoided all those then calls, but we do need the eventually.
I've provided you a link to a working TypeScript example that might help demonstrate.
I need to get the job ID / session ID from a protractor run into a file so I can create links to screenshots / videos at Saucelabs . Is there a correct way to do this?
One approach I'm looking at is to get the session ID from the browser object then pass to a custom reporter that writes it to a file:
// protractor.conf.js
onPrepare: function () {
var sessionIdP = q.defer();
browser.getSession().then(function(session) {
sessionIdP.resolve(session.getId());
});
jasmine.getEnv().addReporter(new SessionIdWriter({
sessionId: sessionIdP
});
}
Should work but can this be done more cleanly?
I'm aware that Saucelabs offers a REST api that can return the latest job ID, but this presents a race condition with other users of the account. Besides the ID is known locally so a call shouldn't be needed.
I think what you are looking for are build: 'some build number' and name: 'my awesome webpage' properties in the capabilities section of your config file. these parameters will be pass through to your SL account and show up in the test run table
more info available https://docs.saucelabs.com/reference/test-configuration/#job-annotation
I'm writing some page-object driven tests using Protractor and Astrolabe.
Jasmine is being used to implement describe/it style specs.
Adding custom matchers won't work using this.addMatchers (TypeError: Object #<Object> has no method 'toContainLowered'), so I used this guide to implement them.
It seems to be working, until I look closely at the output of my test run:
$> grunt test:func
Running "test:func" (test) task
Running "shell:protractor" (shell) task
Using the selenium server at http://localhost:4444/wd/hub
..
Finished in 6.727 seconds
2 tests, 1 assertion, 0 failures
Here is my code:
var loginPage = require('./../pages/loginPage');
describe('Login page', function () {
var ptor = loginPage.driver;
beforeEach(function () {
jasmine.Matchers.prototype.toContainLowered = function (expected) {
return this.actual.toLowerCase().indexOf(expected) > -1;
};
loginPage.go();
ptor.waitForAngular();
});
it('should display login page', function () {
expect(loginPage.currentUrl).toEqual(ptor.baseUrl);
});
it('should display an error when the username or password is incorrect', function() {
loginPage.login('bad', 'credentials');
ptor.waitForAngular();
expect(loginPage.lblError.getText()).toContainLowered('invalid username and/or password');
// expect(loginPage.lblError.getText()).toContain('Invalid Username and/or Password');
});
});
If I uncomment the last line and remove the toContainLowered matcher, I get the proper output:
2 tests, 2 assertions, 0 failures
I'm having a really difficult time debugging this promise-based code, and any efforts to put a console.log(this.actual.toLowerCase().indexOf(expected) > -1); will print false, which is confusing.
I even tried replacing the entire function definition with just return false;. Which still does not do anything. Finally, I tried passing no argument to the matcher, which should have thrown an Invalid Argument Error or something.
How do I define my own matchers in Jasmine when using Protractor/Astrolabe tests?
I've had similar problems with matchers, in particular with the .not matchers, which all seem to not work. I hypothesise that protractor is extending the Jasmine matchers to deal with the promises, and that that extension hasn't been applied to the .not, or to the custom matchers.
In my case, I wanted a .not.toMatch, and so I just wrote a convoluted regex that gave me what I wanted, with the not embedded in the regex.
I note that your matcher is called "toContainLowered", so perhaps you're looking for lowercase, and therefore you could instead do this with a regex by using .toMatch?
The issue I raised on this on the protractor github is here: https://github.com/angular/protractor/issues/266
I also see, in this code file: https://github.com/angular/protractor/blob/master/jasminewd/spec/adapterSpec.js, that the last commit is marked as "patched matcher should understand not". That might either fix the custom matchers for you, or provide an indication of what needs to be done to fix that custom matcher.
EDIT: now looking further into that issue thread, I see you've already been there. Which makes my answer somewhat superfluous. :-)