I have such code:
until #driver.find_element(:id, "ctl00_cp_lblC").displayed?
puts "invalid page solution"
enter_page
end
and i need to do some method, until page will have some element with some id, now it throw's error, that selenium couldn't locate element with this id. What i do wrong, and how to solve it?
Also maybe it is easier to do with watir?
As mentioned, by RobbieWareham, find_element will throw an exception if the element does not exist. You want to:
Rescue the exception of the element not being found and call your enter_page method
Wrap all of that in a wait.until, so that it does not run indefinitely
This would look like:
wait = Selenium::WebDriver::Wait.new(:timeout => 5) # seconds
begin
element = wait.until do
begin
#driver.find_element(:id, "ctl00_cp_lblC")
rescue
enter_page
end
end
ensure
#driver.quit
end
In Watir, it would be easier. It would simply be:
browser.element(:id => "ctl00_cp_lblC").wait_until_present
Or if you need to do stuff when the element is not present:
browser.wait_until(2) do
present = browser.element(:id => "ctl00_cp_lblC").present?
enter_page unless present
present
end
Webdriver throws an error when an element is not found. Displayed? only shows whether an object is hidden or not. As you have seen, when the object is not even in the html, a NoSuchElementFound exception occurs.
Apologies for the java but you should get the idea;
Boolean elementFound = false;
do {
getDriver().manage().timeouts().implicitlyWait(500, TimeUnit.MILLISECONDS);
try {
elementFound = getDriver().findElement(By.id("ct100_cp_lblC")).isDisplayed();
}
catch (NoSuchElementException e){
}
getDriver().navigate().refresh();
} while (!elementFound);
getDriver().manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
You would need extra code to stop an infinite loop but you shoudl get the idea.
This is a big difference between the Watir & WebDriver APIs, but I suppose it is whatever you are used to.
Related
I'm using Ruby and Selenium to get some data from a page. I want to define variable with driver.find_element, but element is not currently visible on page.
next = driver.find_element(:class, 'right')
It returns Selenium::WebDriver::Error::NoSuchElementError
It works fine when element is present.
Any solutions?
Thank you!
Selenium works by executing Javascript commands. By using find_element it will search for the element on the DOM. If it cannot find it you will get the error you are getting. After all if an element is not on the DOM it cannot be found.
The real question is why do you want too find an element that is not currently present on the DOM? You can't do anything with somehing that doesn't exist.
All I could think of is that the element becomes present after the DOM has been loaded due to Javascript not being fully executed yet. If that is the case you can use a WebDriver::Wait to try and find the element for a certain amount of time.
A small example:
wait = Selenium::WebDriver::Wait.new(:timeout => 10) # seconds
begin
element = wait.until { driver.find_element(:id => "some-dynamic-element") }
ensure
driver.quit
end
Edit to include try-catch example:
begin
next = driver.find_element(:class, 'right')
# Code for when element is found here
rescue NoSuchElementError
# Code for when element is not found here
end
Using Ruby, we can wait for particular element by doing following:
wait = Selenium::WebDriver::Wait.new(:timeout => 10)
wait.until { driver.find_element(:class, 'gritter-item') }
but if I want particular element to disappear from DOM, I write method like:
def disappear_element
begin
driver.find_element(:class, 'gritter-item')
rescue Selenium::WebDriver::Error::NoSuchElementError
true
else
false
end
end
and called it like:
wait.until { disappear_element }
This way I could achieve absence of element. Is there any better way in Ruby to achieve the same?
You can write disappear_element as follow (using find_elements instead of find_element):
def disappear_element
driver.find_elements(:class, 'gritter-item').size == 0
end
def scrape!(url)
Anemone.crawl(url) do |anemone|
anemone.on_pages_like %[/events/detail/.*] do |page|
show = {
headliner: page.doc.at_css('h1.summary').text,
openers: page.doc.at_css('.details h2').text
}
puts show
end
end
end
Writing a scraper in Anemone, which uses Nokogiri under the hood..
Sometime the selector .details h2'returns nothing because its not in the HTML, and calling text on it throws an exception.
I'd like to avoid if/elses all over the place...
if page.doc.at_css('.details h2').empty?
openers: page.doc.at_css('.details h2').text
end
Is there any more eloquent way of handling errors produced by inconsistant mark up? For instance CoffeeScript has the existentional operator person.name?.first(). If the HTML has the element, great make the object and call text on it. If not, move on and dont add it to the hash.
You just need do:
anemone.on_pages_like %[/events/detail/.*] do |page|
if not page.nil?
...#your code
end
end
I'm trying to write simple test. My problem is, that i want to wait until the page is loaded completly. At the moment i'm waiting until some elements are presen, but that is not really what i want. Is it possible to make something like this:
driver = Selenium::WebDriver.for :chrome
driver.navigate.to url
driver.wait_for_page_to_load "30000"
With Java isn't problem, but how to make it with ruby?
This is how the Selenium docs () suggest:
require 'rubygems'
require 'selenium-webdriver'
driver = Selenium::WebDriver.for :firefox
driver.get "http://google.com"
element = driver.find_element :name => "q"
element.send_keys "Cheese!"
element.submit
puts "Page title is #{driver.title}"
wait = Selenium::WebDriver::Wait.new(:timeout => 10)
wait.until { driver.title.downcase.start_with? "cheese!" }
puts "Page title is #{driver.title}"
driver.quit
If that is not an option you can try the suggestion from this SO post though it would require some Javascript on top of the Ruby/Rails.
It seems that wait.until is being/has been phased out. The new suggested process it to look for the page to have an element you know will be there:
expect(page).to have_selector '#main_div_id'
As far as I understand webdriver, you dont need to wait for page loads because WebDriver has a blocking API but you can sure set a page load timeout.
driver.manage.timeouts.page_load = 10 # seconds
So in Ruby, whenever you use get to open a URL, the ruby script proceeds ONLY when the page completely loads.
So in your case you would simply do :-
driver.get url
That's not needed with WebDriver anymore.
WebElement click() and Actions click() both "wait for page load" if needed automatically.
You can use imclicit and explicit (in this order) wait instead (described at seleniumhq) if you need to wait for some ajax content for instance.
There have been instances where either AJAX or CSS changes caused my tests to fail at times. I added these methods to my static driver instance so that I can have the test wait for certain conditions if needed. (c#)
TimedWait in the WaitForCssChange Method is basically just a Threading.Thread.Sleep This is not the most beautiful way I guess, but it works well for my needs.
For Ajax wait:
public static void WaitForAjax()
{
var wait = new WebDriverWait(Driver.Instance, TimeSpan.FromSeconds(25));
wait.Until(d => (bool)(d as IJavaScriptExecutor).ExecuteScript("return jQuery.active == 0"));
}
For CSS Changes:
public static void WaitForCssChange(IWebElement element, string value)
{
int counter = 0;
while (true)
{
if(element.GetAttribute("style").Contains(value) || counter > 50)
{
break;
}
TimedWait(20);
counter++;
}
}
I am trying to write a crawler that crawls all links from loaded page and logs all request and response headers along with response body in some file say XML or txt. I am opening all links from first loaded page in new browser window so I wont get this error:
Element not found in the cache - perhaps the page has changed since it was looked up
I want to know what could be the alternate way to make requests and receive response from all links and then locate input elements and submit buttons form all opened windows.
I am able to do above to some extent except when opened window has common site searh box like one on this http://www.testfire.net in the upper right corner.
What I want to do is I want to omit such common boxes so that I can fill other inputs with values using i.send_keys "value" method of webdriver and dont get this error
ERROR: Element not found in the cache - perhaps the page has changed since it was looked up.
What is the way to detect and distinguish input tags from each opened window so that value does not get filled repeatably in common input tags that appear on most pages of website.
My code is following:
require 'rubygems'
require 'selenium-webdriver'
require 'timeout'
class Clicker
def open_new_window(url)
#driver = Selenium::WebDriver.for :firefox
#url = #driver.get " http://test.acunetix.com "
#link = Array.new(#driver.find_elements(:tag_name, "a"))
#windows = Array.new(#driver.window_handles())
#link.each do |a|
a = #driver.execute_script("var d=document,a=d.createElement('a');a.target='_blank';a.href=arguments[0];a.innerHTML='.';d.body.appendChild(a);return a", a)
a.click
end
i = #driver.window_handles
i[0..i.length].each do |handle|
#driver.switch_to().window(handle)
puts #driver.current_url()
inputs = Array.new(#driver.find_elements(:tag_name, 'input'))
forms = Array.new(#driver.find_elements(:tag_name, 'form'))
inputs.each do |i|
begin
i.send_keys "value"
puts i.class
i.submit
rescue Timeout::Error => exc
puts "ERROR: #{exc.message}"
rescue Errno::ETIMEDOUT => exc
puts "ERROR: #{exc.message}"
rescue Exception => exc
puts "ERROR: #{exc.message}"
end
end
forms.each do |j|
begin
j.send_keys "value"
j.submit
rescue Timeout::Error => exc
puts "ERROR: #{exc.message}"
rescue Errno::ETIMEDOUT => exc
puts "ERROR: #{exc.message}"
rescue Exception => exc
puts "ERROR: #{exc.message}"
end
end
end
#Switch back to the original window
#driver.switch_to().window(i[0])
end
end
ol = Clicker.new
url = ""
ol.open_new_window(url)
Guide me how can I get all requeat and response headers with response body using Selenium Webdriver or using http.set_debug_output of ruby's net/http ?
Selenium is not one of the best options to use to attempt to build a "web-crawler". It can be too flakey at times, especially when it comes across unexpected scenarios. Selenium WebDriver is a great tool for automating and testing expectancies and user interactions.
Instead, good old fashioned curl would probably be a better option for web-crawling. Also, I am pretty sure there are some ruby gems that might help you web-crawl, just Google search it!
But To answer the actual question if you were to use Selenium WebDriver:
I'd work out a filtering algorithm where you can add the HTML of an element that you interact with to an variable array. Then, when you go on to the next window/tab/link, it would check against the variable array and skip the element if it finds a matching HTML value.
Unfortunately, SWD does not support getting request headers and responses with its API. The common work-around is to use a third party proxy to intercept the requests.
============
Now I'd like to address a few issues with your code.
I'd suggest before iterating over the links, add a #default_current_window = #driver.window_handle. This will allow you to always return back to the correct window at the end of your script when you call #driver.switch_to.window(#default_current_window).
In your #links iterator, instead of iterating over all the possible windows that could be displayed, use #driver.switch_to.window(#driver.window_handles.last). This will switch to the most recently displayed new window (and it only needs to happen once per link click!).
You can DRY up your inputs and form code by doing something like this:
inputs = []
inputs << #driver.find_elements(:tag_name => "input")
inputs << #driver.find_elements(:tag_name => "form")
inputs.flatten
inputs.each do |i|
begin
i.send_keys "value"
i.submit
rescue e
puts "ERROR: #{e.message}"
end
end
Please note how I just added all of the elements you wanted SWD to find into a single array variable that you iterate over. Then, when something bad happens, a single rescue is needed (I assume you don't want to automatically quit from there, which is why you just want to print the message to the screen).
Learning to DRY up your code and use external gems will help you achieve a lot of what you are trying to do, and at a faster pace.