Our team has been testing our application with Selenium as it's heavily JavaScript driven we've always had issues with tests occasionally failing. As the number of tests has increased the probability of at least one two tests failing in a complete run has become a certainty.
What we recently figured out is that we probably have a race condition where selenium will click links before the initialization JavaScript has had a chance to attach event handlers to the element that is being clicked. Of course at this point the effects we're looking for don't happen and we get a failing test.
For the time being we've added a slight delay before clicks to give the initialization JavaScript code time to finish, this is obviously a bit hackish, adds time to overall test execution, and doesn't guarantee tests won't still fail so we're looking for a better solution.
The best idea we've come up with so far is to inject a hidden element into the DOM that Selenium can wait for, before firing the click event to know that it's ready. This will be a lot of extra overhead in terms of developer time when we're working our asynchronous events, removing and adding the element. Also it adds extra stuff to our pages that really isn't necessary for the application.
Does anyone have any better strategies? What have you done to effectively solve this problem?
We moved to Selenium 2 (WebDriver) and are using Page Objects pattern with PageFactory/AjaxElementLocatorFactory - an example of this is here
I did exactly like you : add some delay and wait for some elements to be present on the page. And I'm perfectly fine with it. Maybe switching to Webdriver / selenium 2.0 would help though. Test execution can be trimmed down if you work with an in-memory database or sharing the same selenium/selenium server between tests, or even with parallelization (easy with TestNG for instance).
Have you tried the waitForElementPresent command, and then make it click ?
To eliminate race conditions use Selenium's runScript(String initCondition) combined with waitForCondition(String jsConditional, String timeout) methods.
For example, if the AJAX functionality you want to test causes a new element to be added to the dom you can use something like the following.
String jsPoll = "";
jsPoll += "selenium.browserbot.getCurrentWindow()";
jsPoll += ".document.getElementById('DOMID')";
selenium.waitForCondition(jsPoll, "30000");
The condition will evaluate true when the element is added and the method will continue. If your AJAX function swaps elements (ie: one div for another similarly identified div), you can initialize your conditional with something like the following.
String jsInit = "";
jsInit += "!selenium.browserbot.getCurrentWindow()";
jsInit += ".document.getElementById('DOMID').setAttribute('SELENIUMTEST','1')";
String jsPoll = "";
selenium.runScript(jsInit);
jsPoll += "selenium.browserbot.getCurrentWindow()";
jsPoll += ".document.getElementById('DOMID').getAttribute('SELENIUMTEST') != 1";
selenium.waitForCondition(jsPoll, "30000");
The condition evaluates true when the element is swapped out by the AJAX function.
Related
I have some async form validation code that I'd like to put under test using Cypress. The code is pretty simple -
on user input, enter async validation UI state (or stay in that state if there are previous validation requests that haven't been responded to)
send a request to the server
receive a response
if there are no pending requests, leave async validation UI state
Step 1 is the part I want to test. Right now, this means checking if some element has been assigned some class -- but the state changes can happen very fast, and most of the time (not always!) Cypress times out waiting for something that has ALREADY happened (in other words, step 4 has already occurred by the time we get around to seeing if step 1 happened).
So the failing test looks like:
cy.get("#some-input").type("...");
cy.get("#some-target-element").should("have.class", "class-to-check-for");
Usually, by the time Cypress gets to the second line, step 4 has already ran and the test fails. Is there a common pattern I should know about to solve this? I would naturally prefer not to have change the code under test.
Edit 1:
I'm not certain that I've 100% solved the "race" condition here, but if I use the underlying native elements (discarding the jQuery abstraction), I haven't had a failure yet.
So, changing:
cy.get("#some-input").type("...")
to:
cy.get("#some-input").then(jQueryObj => {
let nativeElement = jQueryObj[0];
nativeElement.value = "...";
nativeElement.dispatchEvent(new Event("input")); // make sure the app knows this element changed
});
And then running Cypress' checks for what classes have / haven't been added has been effective.
You can stub the server request that happens during form validation - and slow it down, see delay parameter https://docs.cypress.io/api/commands/route.html#Use-delays-for-responses
While the request is delayed, your app's validation UI is showing, you can validate it and then once the request finishes, check if the UI goes away.
I'm currently using Calabash framework to automate functional testing for a native Android and IOS application. During my time studying it, I stumbled upon this example project from Xamarin that uses page objects design pattern which I find to be much better to organize the code in a Selenium fashion.
I have made a few adjustments to the original project, adding a file called page_utils.rb in the support directory of the calabash project structure. This file has this method:
def change_page(next_page)
sleep 2
puts "current page is #{current_page_name} changing to #{next_page}"
#current_page = page(next_page).await(PAGE_TRANSITION_PARAMETERS)
sleep 1
capture_screenshot
#current_page.assert_info_present
end
So in my custom steps implementation, when I want to change the page, I trigger the event that changes the page in the UI and update the reference for Calabash calling this method, in example:
#current_page.click_to_home_page
change_page(HomePage)
PAGE_TRANSITION_PARAMETERS is a hash with parameters such as timeout:
PAGE_TRANSITION_PARAMETERS = {
timeout: 10,
screenshot_on_error: true
}
Just so happens to be that whenever I have a timeout waiting for any element in any screen during a test run, I get a generic error message such as:
Timeout waiting for elements: * id:'btn_ok' (Calabash::Android::WaitHelpers::WaitError)
./features/support/utils/page_utils.rb:14:in `change_page'
./features/step_definitions/login_steps.rb:49:in `/^I enter my valid credentials$/'
features/04_support_and_settings.feature:9:in `And I enter my valid credentials'
btn_ok is the id defined for the trait of the first screen in my application, I don't understand why this keeps popping up even in steps ahead of that screen, masking the real problem.
Can anyone help getting rid of this annoyance? Makes really hard debugging test failures, specially on the test cloud.
welcome to Calabash!
As you might be aware, you'll get a Timeout waiting for elements: exception when you attempt to query/wait for an element which can't be found on the screen. When you call page.await(opts), it is actually calling wait_for_elements_exist([trait], opts), which means in your case that after 10 seconds of waiting, the view with id btn_ok can't be found on the screen.
What is assert_info_present ? Does it call wait_for_element_exists or something similar? More importantly, what method is actually being called in page_utils.rb:14 ?
And does your app actually return to the home screen when you invoke click_to_home_page ?
Unfortunately it's difficult to diagnose the issue without some more info, but I'll throw out a few suggestions:
My first guess without seeing your application or your step definitions is that #current_page.click_to_home_page is taking longer than 10 seconds to actually bring the home page back. If that's the case, simply try increasing the timeout (or remove it altogether, since the default is 30 seconds. See source).
My second guess is that the element with id btn_ok is not actually visible on screen when your app returns to the home screen. If that's the case, you could try changing the trait definition from * id:'btn_ok' to all * id:'btn_ok' (the all operator will include views that aren't actually visible on screen). Again, I have no idea what your app looks like so it's hard to say.
My third guess is it's something related to assert_info_present, but it's hard to say without seeing the step defs.
On an unrelated note, I apologize if our sample code is a bit outdated, but at the time of writing we generally don't encourage the use of #current_page to keep track of a page. Calabash was written in a more or less stateless manner and we generally encourage step definitions to avoid using state wherever possible.
Hope this helps! Best of luck.
I am new to automated web testing and I am currently migrating from an old Selenium RC implementation to Selenium 2 in Ruby. Is there a way to halt the execution of commands until the page gets loaded, similar to "wait_for_page_to_load" in Selenium RC?
I fixed a lot of issues I was having in that department adding this line after starting my driver
driver.manage.timeouts.implicit_wait = 20
This basically makes every failed driver call you make retry for maximum 20 seconds before throwing an exception, which is usually enough time for your AJAX to finish.
Try using Javascript to inform you!
I created a couple methods that checks via our javascript libraries and waits to see if the page has finished loading the DOM and that all ajax requests are complete. Here's a sample snippet. The javascript you will need to use is just going to depend on your library.
Selenium::WebDriver::Wait.new(:timeout => 30).until { #driver.execute_script("[use javascript to return true once loaded, false if not]"}
I then wrapped these methods in a clickAndWait method that clicks the element and calls the waitForDomLoad and waitForAjaxComplete. Just for good measure, the very next command after a clickAndWait is usually a waitForVisble element command to ensure that we are on the right page.
# Click element and wait for page elements, ajax to complete, and then run whatever else
def clickElementAndWait(type, selector)
#url = #driver.current_url
clickElement(type, selector)
# If the page changed to a different URL, wait for DOM to complete loading
if #driver.current_url != #url
waitForDomLoad
end
waitForAjaxComplete
if block_given?
yield
end
end
If you are using capybara, whenever you are testing for page.should have_content("foo"), capybara will not fail instantly if the page doesn't have the content (yet) but will wait for a while to see if an ajax call will change that.
So basically: after you click, you want to check right away for have_content("some content that is a consequence of that click").
How can I make concurrent ajax calls.When I try to do that its happening one after one in sequential manner.
I have made two ajax calls to two different jsps to get response.In first jsp for first ajax call
I used following code.
int i = 5000;
while (i == 0)
{
out.println(i);
i--;
}
For second ajax call i used following code to get response.
int i = 5;
while (i == 0)
{
out.println(i);
i--;
}
Response from second call suppose to come first.But response from second call comes after first one is executed.How can I make them concurrent.
That depends on the browser the client is using - I've come across this problem before - older browsers don't thread things very well.
Best advice I can give is to use JQuery... I think that sorted out my problems when I came across that behaviour, though not for all browsers.
Use asynchronous calls instead of synchronous.
But do be careful in structuring your code to ensure, one section doesn't rely on values set by the other as it's not guaranteed what code is being executed at any given time.
Both QWebFrame and QWebPage have void loadFinished(bool ok) signal which can be used to detect when a web page is completely loaded. The problem is when a web page has some content loaded asynchronously (ajax). How to know when the page is completely loaded in this case?
I haven't actually done this, but I think you may be able to achieve your solution using QNetworkAccessManager.
You can get the QNetworkAccessManager from your QWebPage using the networkAccessManager() function. QNetworkAccessManager has a signal finished ( QNetworkReply * reply ) which is fired whenever a file is requested by the QWebPage instance.
The finished signal gives you a QNetworkReply instance, from which you can get a copy of the original request made, in order to identify the request.
So, create a slot to attach to the finished signal, use the passed-in QNetworkReply's methods to figure out which file has just finished downloading and if it's your Ajax request, do whatever processing you need to do.
My only caveat is that I've never done this before, so I'm not 100% sure that it would work.
Another alternative might be to use QWebFrame's methods to insert objects into the page's object model and also insert some JavaScript which then notifies your object when the Ajax request is complete. This is a slightly hackier way of doing it, but should definitely work.
EDIT:
The second option seems better to me. The workflow is as follows:
Attach a slot to the QWebFrame::javascriptWindowObjectCleared() signal. At this point, call QWebFrame::evaluateJavascript() to add code similar to the following:
window.onload = function() { // page has fully loaded }
Put whatever code you need in that function. You might want to add a QObject to the page via QWebFrame::addToJavaScriptWindowObject() and then call a function on that object. This code will only execute when the page is fully loaded.
Hopefully this answers the question!
To check the load of specific element you can use a QTimer. Something like this in python:
#pyqtSlot()
def on_webView_loadFinished(self):
self.tObject = QTimer()
self.tObject.setInterval(1000)
self.tObject.setSingleShot(True)
self.tObject.timeout.connect(self.on_tObject_timeout)
self.tObject.start()
#pyqtSlot()
def on_tObject_timeout(self):
dElement = self.webView.page().currentFrame().documentElement()
element = dElement.findFirst("css selector")
if element.isNull():
self.tObject.start()
else:
print "Page loaded"
When your initial html/images/etc finishes loading, that's it. It is completely loaded. This fact doesn't change if you then decide to use some javascript to get some extra data, page views or whatever after the fact.
That said, what I suspect you want to do here is expose a QtScript object/interface to your view that you can invoke from your page's script, effectively providing a "callback" into your C++ once you've decided (from the page script) that you've have "completely loaded".
Hope this helps give you a direction to try...
The OP thought it was due to delayed AJAX requests but there also could be another reason that also explains why a very short time delay fixes the problem. There is a bug that causes the described behaviour:
https://bugreports.qt-project.org/browse/QTBUG-37377
To work around this problem the loadingFinished() signal must be connected using queued connection.