Using Jest, a testing library for JS, it is possible to have a "snapshot" as followed:
test('foo', () => {
expect(42).toMatchSnapshot("my_snapshot");
})
Basically, on first run this saves the tested value into a file. And on later runs, it compares the passed value with what was into the file. So that if the passed value differ from the value inside that file, the test fail.
This is quite useful because it allows to create tests easily.
Is there any way to do this using the testing framework provided by Flutter?
It is possible for widgets only, using testWidgets:
testWidgets('golden', (tester) async {
await tester.pumpWidget(Container(
color: Colors.red,
));
await expectLater(
find.byType(Container), matchesGoldenFile("red_container.png"));
});
First, you have to pump the widget you want to test (here a red container).
Then you can use matchesGoldenFile combined with expectLater. This will take a screen capture of the widget and compare it to the previously saved capture.
On the first run or when you want to update your goldens, you'll have to pass a flag to flutter test:
flutter test --update-goldens
Related
A lot of this is wrapped in commands, but I've left that part out to make the problem more feasible.
Consider these two tests:
# Test1: Test login for user
- Step1: Logs in manually (go to login-URL, fill out credentials and click 'Log in').
- Step2: Save auth-cookies as fixtures.
# Test2: Test something is dashboard for user.
- Step1: Set auth-cookies (generated in Test1)
- Step2: Visits https:://example.org/dashboard and ensures the user can see the dashboard.
If they run as written as listed above, then everything is fine.
But if Test2 runs before Test1, then Test2 will fail, since Test1 hasn't to generated the cookies yet.
So Test1 is kind of a prerequisite for Test2.
But Test1 doesn't need to run every time Test2 runs - only if the auth-cookies aren't generated.
I wish I could define my Test2 to be like this instead:
Test2: Test something is dashboard for user.
- Step1: Run ensureAuthCookiesExists-command
- Step2: If the AuthCookies.json-fixture doesn't exist, then run Test1
- Step3: Sets auth-cookies (generated in Test1)
- Step4: Visits https:://example.org/dashboard and ensures the user can see the dashboard.
Solution attempt 1: Control by order
For a long time I've done this using this answer: How to control order of tests. And then having my tests defines like this:
{
"baseUrl": "http://localhost:5000",
"testFiles": [
"preparations/*.js",
"feature-1/check-header.spec.js",
"feature-2/check-buttons.spec.js",
"feature-3/check-images.spec.js",
"feature-4/check-404-page.spec.js",
//...
]
}
But that is annoying, since it means that I keep having to add to add new features to that list, which get's annoying.
And this only solves the problem if I want to run all the tests. If I want to run preparations.spec.js and thereafter: feature-2/check-buttons.spec.js. Then I can't do that easily.
Solution attempt 2: Naming tests smartly
I also tried simply naming them appropriately, like explain here: naming your tests in Cypress.
But that pollutes the naming of the tests, making it more cluttered. And it faces the same issues as solution attempt 1 (that I can't easily run two specific tests after one another).
Solution attempt 3: Making a command for it
I considered making a command that tests for it. Here is some pseudo-code:
beforeEach(() => {
if( preparationsHasntCompleted() ){
runPreparations();
}
}
This seems smart, but it would add extra runtime to all my tests.
This may not suit your testing aims, but the new cy.session() can assure cookie is set regardless of test processing order.
Use it in support in beforeEach() to run before every test.
The first test that runs (either test1 or test2) will perform the request, subsequent tests will use cached values (not repeating the requests).
// cypress/support/e2e.js -- v10 support file
beforeEach(() => {
cy.session('init', () => {
// request and set cookies
})
})
// cypress/e2e/test1.spec.cy.js
it('first test', () => {
// the beforeEach() for 1st test will fire the request
...
})
// cypress/e2e/test2.spec.cy.js
it('second test', () => {
// the beforeEach() for 2nd test will set same values as before from cache
// and not resend the request
})
Upside:
performing login once per run (ref runtime concerns)
performing tests in any order
using the same token for all tests in session (if that's important)
Downside:
if obtaining auth cookies manually (vi UI), effectively moving the login test to a beforeEach()
Example logging in via request
Rather than obtaining the auth cookie via UI, it may be possible to get it via cy.request().
Example from the docs,
cy.session([username, password], () => {
cy.request({
method: 'POST',
url: '/login',
body: { username, password },
}).then(({ body }) => {
cy.setCookie('authToken', body.token)
})
})
It is generally not recommended to write tests that depend on each other as stated in the best practices. You can never be sure that they run correctly. In your case if the first test fails all the other ones will fail, even if the component/functionality is functioning propperly. Especially if you test more than just the pure login procedure, e.g. design.
As #Fody said before you can ensure being logged in in the beforeEach() hook.
I would do it like this:
Use one describe to test the login via UI.
Use another describe where you put the login (via REST) in the before() and the following command Cypress.Cookies.preserveOnce('<nameOfTheCookie>'); in the beforeEach() to not delete the test for the following it()s
New cypress user here, I am aware that cypress does not handle variables like how testcafe and others do due to the asyn nature of it. Using the example given and what I could find I have this as an example:
cy.get('selector').invoke('text').as('text_needed')
cy.get('#text_needed')
const txtneeded = this.text_needed
cy.log(txtneeded)
This looks at a given selector, takes what it finds and uses it as text and set it as a variable usable at any time in the test and outputs it to the log. The plan is to use that text in a search filter in another page to find the item it references.
The problem is that it fails with Cannot read properties of undefined (reading 'text_needed')
Is this because the content of the selector is not assigned to text properly, the outer html is <a data-v-78d50a00="" data-v-3d3629a7="" href="#">PO90944</a> The PO90944 is what I want to capture.
Your help would be appreciated!
You cannot save an alias and access it via this.* in the same execution context (callback) because it's a synchronous operation and your alias is not yet resolved at this time.
This is a correct way to go:
cy.get('selector').invoke('text').as('text_needed')
cy.get('#text_needed').then(txtneeded => {
cy.log(txtneeded)
})
First, make sure to define it as traditional function, not as an arrow function as this context doesn't work as you'd expect there, more info here.
Next, typically in a single test you should use .then() callback to perform additional actions on elements, and use aliases when you need to share context between hooks or different tests, so please consider the following:
// using aliases together with this within the single test won't work
cy.get(<selector>).invoke('text').as('text_needed')
cy.get('#text_needed').should('contain', 'PO90944') // works fine
cy.log(this.text_needed) // undefined
// this will work as expected
cy.get(<selector>).invoke('text').then(($el) => {
cy.wrap($el).should('contain', 'PO90944'); // works fine
cy.log($el) // works fine
});
Setting alias in beforeEach hook for example, would let you access this.text_needed in your tests without problems.
Everything nicely explained here.
Edit based on comments:
it('Some test', function() {
cy.visit('www.example.com');
cy.get('h1').invoke('text').as('someVar');
});
it('Some other test', function() {
cy.visit('www.example.com');
cy.log('I expect "Example Domain" here: ' + this.someVar);
});
And here's the output from cypress runner:
The Cypress documentation suggests that commands are the right way to reuse fragments of code, e.g.
Cypress.Commands.add("logout", () => {
cy.get("[data-cy=profile-picture]").click();
cy.contains("Logout").click();
});
cy.logout();
For simple cases like this, why would I use a command over a plain JS function (and all the nice IDE assistance that comes with it). What are the drawbacks of rewriting the above snippet as
export function logout(){
cy.get("[data-cy=profile-picture]").click();
cy.contains("Logout").click();
}
// and now somewhere in a test
logout();
Based on my experience with Cypress (one year project and several hundred test cases), I can say that a plan JS function is great for grouping cy commands.
From my point of view, a custom cy command may be really useful only if it is incorporated into the chain processing (utilizes the subject parameter or returns a Chainable to be used further in the chain). Otherwise, a plain JS function is preferable due to it simplicity and full IDE support (unless you're using an additional plugin).
If you for any reason need to do something inside the cypress loop, you can always wrap you code by cy.then() in a plain JS function:
function myFunction() {
cy.then(() => {
console.log(("I'm inside the Cypress event loop"))
})
}
Commands are for behavior that is needed across all tests. For example, cy.setup or cy.login. Otherwise, use functions.
See official docs: https://docs.cypress.io/api/cypress-api/custom-commands#1-Don-t-make-everything-a-custom-command
I'm trying to share values between my before and beforeEach hooks using aliases. It currently works if my value is a string but when the value is an object, the alias is only defined in the first test, every test after that this.user is undefined in my beforeEach hook. How can I share a value which is an object between tests?
This is my code:
before(function() {
const email = `test+${uuidv4()}#example.com`;
cy
.register(email)
.its("body.data.user")
.as("user");
});
beforeEach(function() {
console.log("this.user", this.user); // This is undefined in every test except the first
});
The alias is undefined in every test except the first because aliases are cleared down after each test.
Aliased variables are accessed via cy.get('#user') syntax. Some commands are inherently asynchronous, so using a wrapper to access the variable ensures it is resolved before being used.
See documentation Variables and Aliases and get.
There does not seem to be a way to explicitly preserve an alias, as the is with cookies
Cypress.Cookies.preserveOnce(names...)
but this recipe for preserving fixtures shows a way to preserve global variables by reinstating them in a beforeEach()
let city
let country
before(() => {
// load fixtures just once, need to store in
// closure variables because Mocha context is cleared
// before each test
cy.fixture('city').then((c) => {
city = c
})
cy.fixture('country').then((c) => {
country = c
})
})
beforeEach(() => {
// we can put data back into the empty Mocha context before each test
// by the time this callback executes, "before" hook has finished
cy.wrap(city).as('city')
cy.wrap(country).as('country')
})
If you want to access a global user value, you might try something like
let user;
before(function() {
const email = `test+${uuidv4()}#example.com`;
cy
.register(email)
.its("body.data.user")
.then(result => user = result);
});
beforeEach(function() {
console.log("global user", user);
cy.wrap(user).as('user'); // set as alias
});
it('first', () => {
cy.get('#user').then(val => {
console.log('first', val) // user alias is valid
})
})
it('second', () => {
cy.get('#user').then(val => {
console.log('second', val) // user alias is valid
})
})
Replace
console.log("global user", this.user);
with
cy.log(this.user);
and it should work as expected.
The reason for this is the asynchronous nature of cypress commands. Think of it as a two-step process: All the cypress commands are not doing what you think, when they run. They just build up a chain of commands. This chain is executed as the test later on.
This is obviously not the case for other commands like console.log(). This command is executed when preparing the test.
This is explained in great detail in the cypress documentation:
But I felt it very hard to get my head around this. You have to get used to it.
One rule of thumb: Almost every command in your test should be a cypress command.
So just use cy.log instead of console.log
If you must use console.log you can do it like this:
cy.visit("/).then(() => console.log(this.user))
this way the console.log is chained. Or if you do not have a subject to chain off, build your own custom command like this:
Cypress.Commands.add("console", (message) => console.log(message))
cy.console(this.user)
Another mistake with using this in cypress is using arrow functions. If you do, you don't have access to the this you are expecting. See Avoiding the use of this in the cypress docs.
TL;DR: If you want an aliased user object available in each of your tests, you must define it in a beforeEach hook not a before hook.
Cypress performs a lot of cleanup between tests and this includes clearing all aliases. According to the Sharing Contexts section of Variables and Aliases: "Aliases and properties are automatically cleaned up after each test." The result you are seeing (your alias is cleaned after the first test and subsequently undefined) is thus expected behavior.
I cannot determine what register does in the original post, but it seems your intention is to save the overhead of performing API calls repeatedly in a beforeEach hook. It is definitely easiest to put everything you want in the beforeEach hook and ignore the overhead (also, pure API calls with no UI interaction will not incur much penalty).
If you really need to avoid repetition, this should not be accomplished through regular variables due to potential timing problems with Cypress' custom chainables. This is an anti-pattern they publish. The best way to do this would be:
Create a fixture file with static user data that you will use to conduct the test. (Remove the uuidv4.)
For the set of tests that need your user data, call register in a before hook using the fixture data. This will create the data in the system under test.
Use a beforeEach hook to load the fixture data and alias it for each of your tests. Now, the static data you need is accessible with no API calls and it is guaranteed to be in the system properly thanks to the before hook.
Run your tests using the alias.
Clean up the data in an after hook (since your user no longer has a random email, you need to add this step).
If you need to do the above for the whole test suite, put your before and after hooks in the support file to make them global.
Using protractor and jasmine(wd) we want to check that a table on the web page contains expected values. We get fetch the table from the page using a CSS selector:
var table = element(by.css('table#forderungenTable')).all(by.tagName('tr'));
We then set our expectations:
table.then(function(forderungen){
...
forderungen[2].all(by.tagName('td')).then(function(columns){
expect(columns[1].getText()).toEqual('1');
expect(columns[5].getText()).toEqual('CHF 277.00');
});
});
Is it possible to change this code so that we don't have to pass functions to then, in the same way that using jasminewd means that we don't have to do this? See this page, which states:
Protractor uses jasminewd, which wraps around jasmine's expect so that you can write:
expect(el.getText()).toBe('Hello, World!')
Instead of:
el.getText().then(function(text) {
expect(text).toBe('Hello, World!');
});
I know that I could write my own functions in a way similar to which jasminewd does it, but I want know if there is a better way to construct such expectations using constructs already available in protractor or jasminewd.
You can actually call getText() on an ElementArrayFinder:
var texts = element(by.css('table#forderungenTable')).all(by.tagName('tr')).get(2).all(by.tagName('td'));
expect(texts).toEqual(["text1", "text2", "text3"]);