I try to add another test to my existing .opa.qunit.js file which requires a complete restart of my app.
What I tried was to call "iTeardownMyAppFrame" in my test and then again "iStartMyAppInAFrame" to ensure a clean setup.
At first the iFrame is shown but closed immediatly and after some time the test just times out. Both methods below just call "iTeardownMyAppFrame" and "iStartMyAppInAFrame" nothing else.
opaTest("FirstTest", function(Given, When, Then) {
Given.iStartTheSampleApp();
//Testlogic
});
opaTest("TestWithCleanState", function(Given, When, Then) {
Given.iShutdownTheApp();
//Until here everything above works fine
Given.iStartTheSampleApp();
//Testlogic
});
//EOF
There is no error on the console, just some two messages repeating every second:
sap-ui-core.js:15219 2015-03-11 10:05:37 Opa check was undefined -
sap-ui-core.js:15219 2015-03-11 10:05:37 Opa is executing the check: function () {
if (!bFrameLoaded) {
return;
}
return checkForUI5ScriptLoaded();
} -
What's the intended functionality of "iTeardownMyAppFrame"?
Should it only be used to teardown the whole test at the end of all tests?
Or can it also be used to reset the app to ensure a clean state at the beginning of the test? If this is the case how should it work?
Thanks
teardonw removes the iframe and in the next test you have to bring it up again.
This way you can write separated tests that can be run standalone.
An example is here:
Opa sample with 2 isolated tests
If you press the rerun button on test2 it will execute standalone with no dependency on test1
BR,
Tobias
Related
I am attempting to use the Cypress (at 7.6.0) retries feature as per https://docs.cypress.io/guides/guides/test-retries#How-It-Works
but for some reason it seems to be not working, in that a test that is guaranteed to always fail:
describe('Deliberate fail', function() {
it('Make an assertion that will fail', function() {
expect(true).to.be.false;
});
});
When run from the command line with the config retries set to 1,
npx cypress run --config retries=1 --env server_url=http://localhost:3011 -s cypress/integration/tmp/deliberate_fail.js
it seems to pass, with the only hint that something is being retried being the text "Attempt 1 of 2" and the fact that a screenshot has been made:
The stats on the run look also to be illogical:
1 test
0 passing
0 failing
1 skipped (but does not appear as skipped in summary)
Exactly the same behavior when putting the "retries" option in cypress.json, whether as a single number or options for runMode or openMode.
And in "open" mode, the test does not retry but just fails.
I am guessing that I'm doing something face-palmingly wrong, but what?
I think your problem is that you are not testing anything. Cypress will retry operations that involve the DOM, because the DOM takes time to render. Retrying is more efficient than a straight wait, because it might happen quicker.
So I reckon because you are just comparing 2 literal values, true and false, Cypress says, "Hey, there is nothing to retry here, these two values are never going to change, I'm outta here!"
I was going to say, if you set up a similar test with a DOM element, it might behave as you are expecting, but in fact it will also stop after the first attempt, because when it finds the DOM element, it will stop retrying. The purpose of the retry is to allow the element to be instantiated rather than retrying because the value might be different.
I will admit that I could be wrong in this, but I have definitely convinced myself - what do you think?
Found the cause .. to fix the problem that Cypress does not abort a spec file when one of the test steps (the it() steps) fails, I had the workaround for this very old issue https://github.com/cypress-io/cypress/issues/518 implemented
//
// Prevent it just running on if a step fails
// https://github.com/cypress-io/cypress/issues/518
//
afterEach(function() {
if (this.currentTest.state === 'failed') {
Cypress.runner.stop()
}
});
This means that a describe() will stop on fail, but does not play well with the retry apparently.
My real wished-for use case is to retry at the describe() level, but that may ne expensive as the above issue just got resolved but the solution is only available to those on the Business plan at $300/mo. https://github.com/cypress-io/cypress/issues/518#issuecomment-809514077
Using Rspec and Capybara after recently adding in a debounce to most of my pages the test now fail randomly.
Now locally these are passing fine but on Semaphore 2.0 I am getting the random failings on shorter tests.
We use WebMock to stub the request in remoteFetch() and it seems that this is removed on shorter tests. As this is called afterwards, the stub doesn't exist and the test fails
function debouncedFetch(ids) {
store.idsToFetch.push(ids);
$timeout.cancel(store.fetchTimeoutFn);
store.fetchTimeoutFn = $timeout(() => { remoteFetch(store.idsToFetch); }, 200);
}
I have tried putting the debounce/timeout to 0 still with no joy.
Is there a way to check if tests/rootscope have finished or destroyed or something and not run the remoteFetch function.
Or get the test to wait for this function to run
Assuming you're using the default Capybara configuration where Capybara manages the running of the app under test it will wait for all network connections to be closed during the test reset in an after block. Since you're cleaning up your WebMock in an after block it's possible it's occurring before the Capybara registered block. To fix that you can change the order they're defined in or defined your WebMock cleanup with append_after rather than after so it's guaranteed to run after the Capybara session reset.
It turns out that the after(each:) { WebMock.reset! } as part of the gem is called before Capybara.reset_sessions!
This causes a race condition in the code. the way around this is to change the order and make sure in your spec_helper that require 'webmock/rspec' is called before require 'rspec/rails'
This ensures the order of the hooks are setup in the right order.
hope this helps someone else
I have some async form validation code that I'd like to put under test using Cypress. The code is pretty simple -
on user input, enter async validation UI state (or stay in that state if there are previous validation requests that haven't been responded to)
send a request to the server
receive a response
if there are no pending requests, leave async validation UI state
Step 1 is the part I want to test. Right now, this means checking if some element has been assigned some class -- but the state changes can happen very fast, and most of the time (not always!) Cypress times out waiting for something that has ALREADY happened (in other words, step 4 has already occurred by the time we get around to seeing if step 1 happened).
So the failing test looks like:
cy.get("#some-input").type("...");
cy.get("#some-target-element").should("have.class", "class-to-check-for");
Usually, by the time Cypress gets to the second line, step 4 has already ran and the test fails. Is there a common pattern I should know about to solve this? I would naturally prefer not to have change the code under test.
Edit 1:
I'm not certain that I've 100% solved the "race" condition here, but if I use the underlying native elements (discarding the jQuery abstraction), I haven't had a failure yet.
So, changing:
cy.get("#some-input").type("...")
to:
cy.get("#some-input").then(jQueryObj => {
let nativeElement = jQueryObj[0];
nativeElement.value = "...";
nativeElement.dispatchEvent(new Event("input")); // make sure the app knows this element changed
});
And then running Cypress' checks for what classes have / haven't been added has been effective.
You can stub the server request that happens during form validation - and slow it down, see delay parameter https://docs.cypress.io/api/commands/route.html#Use-delays-for-responses
While the request is delayed, your app's validation UI is showing, you can validate it and then once the request finishes, check if the UI goes away.
I would like to write some customised logs into files with cypress. Such as 'current test case name', 'user's id used for this test', 'fail or pass', etc.
I googled awhile, found cy.writeFile meet my needs. But it seems most people would recommend cy.task for logging.
So is cy.writeFile for logging a good idea? If not, what's the reason?
Thanks!
BTW, here's the codes, very simple:
function logger (log) {
cy.writeFile('logs/combined.log', log + '\n', { flag: 'a+' })
}
module.exports = {
logger
}
The cy.task() command is used generally when you do not want the tests themselves to interrupt the logging or if you want to interact with the node process itself. Whereas the cy.writeFile() has to be called between each test and cannot interact with the node process. You can add things in your plugin files so that continous logs can be produced regardless of the test being ran and concatenate those things into the same file.
// in plugins/index.js file
on('task', {
readJson: (filename) => {
// reads the file relative to current working directory
return fsExtra.readJson(path.join(process.cwd(), filename)
}
})
https://docs.cypress.io/api/commands/task.html#Command
cy.task() provides an escape hatch for running arbitrary Node code, so
you can take actions necessary for your tests outside of the scope of
Cypress. This is great for:
Seeding your test database. Storing state in Node that you want
persisted between spec files. Performing parallel tasks, like making
multiple http requests outside of Cypress. Running an external
process. In the task plugin event, the command will fail if undefined
is returned. This helps catch typos or cases where the task event is
not handled.
If you do not need to return a value, explicitly return null to signal
that the given event has been handled.
I have a test case in Protractor which loads the home page and then clicks a button that redirects to another page. In that other page I want to grab the value of an element.
describe('todo list', function() {
it('should find the contact phone number from the home page', function() {
browser.get('http://homepage...');
element(by.id('re_direct_to_contact_page')).click();
var number = element(by.id('phonenumber')).getText();
expect(number).toEqual('412-....-...');
});
});
However although it loads the page it does not check the element value and the test case fails.
How can Protractor load another page to check a value?
NOTE: - I made up this example but in my real test case I am sending data from one page to another so I cannot load the page I want directly.
EDIT: STACK TRACE
protractor conf.js Using the selenium server
at http://localhost:4444/wd/hub [launcher] Running 1 instances of
WebDriver
Error: Error while waiting for Protractor to sync with the page:
"[ng:test] http://errors.angularjs.or/1.4.0/ng/test"
StackTrace: undefined
1 test, 1 assertion, 1 failure
It's possible that you might be facing a timing issue. Try re-structuring your test from this:
var number = element(by.id('phonenumber')).getText();
expect(number).toEqual('412-....-...');
To this:
expect(element(by.id('phonenumber')).getText()).toEqual('412-....-...');
While Jasmine and Protractor try to resolve every promise before moving on, sometimes it doesn't happen as quickly as you would like. Putting the entire promise chain into the expect helps to enforce the resolution of all of them before going through the matcher.