I'm using Ruby and Selenium to get some data from a page. I want to define variable with driver.find_element, but element is not currently visible on page.
next = driver.find_element(:class, 'right')
It returns Selenium::WebDriver::Error::NoSuchElementError
It works fine when element is present.
Any solutions?
Thank you!
Selenium works by executing Javascript commands. By using find_element it will search for the element on the DOM. If it cannot find it you will get the error you are getting. After all if an element is not on the DOM it cannot be found.
The real question is why do you want too find an element that is not currently present on the DOM? You can't do anything with somehing that doesn't exist.
All I could think of is that the element becomes present after the DOM has been loaded due to Javascript not being fully executed yet. If that is the case you can use a WebDriver::Wait to try and find the element for a certain amount of time.
A small example:
wait = Selenium::WebDriver::Wait.new(:timeout => 10) # seconds
begin
element = wait.until { driver.find_element(:id => "some-dynamic-element") }
ensure
driver.quit
end
Edit to include try-catch example:
begin
next = driver.find_element(:class, 'right')
# Code for when element is found here
rescue NoSuchElementError
# Code for when element is not found here
end
Related
I am trying to parse a website using Selenium/Capybara. Right now it looks like this:
session = Capybara::Session.new(:selenium)
session.visit "https://somesite.com/page1"
element = session.all( :css, '.table .row a' ).each do |el|
el.click
# get some element's data
session.evaluate_script('window.history.back()')
end
# repeat
Problem is, when I'm trying to get the data from the second page I am on, Capybara tells me that Either the element is no longer attached to the DOM or the page has been refreshed. which absolutely makes sense, however I'm struggling to find a way to create a new DOM variable and parse it in the documentation.
Same thing happens if I navigate back trying to repeat the actions and click on the second link in a row. I assume I need to re-create the session or is there a better way?
To work like you're trying you're going to need to keep a counter and find the elements each time through your loop - something along the lines of
counter = 0
while (el = session.all( :css, '.table .row a', minimum: 1 )[counter]) do
el.click
# get some element's data
counter += 1
session.go_back
end
or if the links are just standard you could gather the hrefs and then just visit them
element = session.all( :css, '.table .row a', minimum:1 ).map {|a| a['href']} do |url|
session.visit(url)
# get some element's data
end
I've got an array of certain <a> elements like this:
array = browser.as(:class => 'foo')
I try to go to the links like this:
$i = 0
while $i < num do
browser.goto array[$i].href
.
.
.
$i += 1
end
Which works the for the first loop, but not for the second. Why is this happening? If I do
puts array[1].href
puts array[2].href
puts array[-1].href
before
browser.goto array[$i].href
it shows all the links on the first loop in terminal window.
Someone who knows this better than I do will need to verify / clarify. My understanding is that elements are given references when the page is loaded. On loading a new page you are building all new references, even if it looks like the same link on each page.
You are storing a reference to that element and then asking it to go ahead and pull the href attribute from it. It works the first time, as the element still exists. Once the new page has loaded it no longer exists. The request to pull the attribute fails.
If all you care about are the hrefs there's a shorter, more Ruby way of doing it.
array = []
browser.as(:class => 'foo').each { |link|
array << link.href
}
array.each { |link|
browser.goto link
}
By cramming the links into the array we don't have to worry about stale references.
Yes, unlike with individual elements that store the locator and will re-look them up on page reloads, an element collection stores only the Selenium Element which can't be re-looked up.
Another solution for when you need this pattern and more than just an href value is not to create a collection and to use individual elements with an index:
num.times do |i|
link = browser.link(class: 'foo', index: i)
browser.goto link.href
end
In short i have a custom step definition to scroll down a page of search results until a text string is found.
It's scrolling down and finding the text string
But the while loop is not ending, despite it correctly scrolling down and stopping when the text string is found.
This is obviously causing me no end of but hurt.
Then /^I scroll until I see the "This is a test" text$/ do
q = query("android.widget.TextView text:'This is a test'")
while q.empty?
scroll("android.view.View id:'search_listView'", :down)
q = query("android.widget.TextView text:'This is a test'")
end
end
is the ruby that I've been using.
Which version of Calabash do you use?
I have written similiar step definition and it works perfectly for me (Calabash 0.5.1, Ruby 2.1.2).
Here is the code (I think it's more universal and simple):
Then /^I scroll down until I see '(.*?)' text$/ do |text|
while element_does_not_exist("TextView marked:'#{text}'")
scroll_down
end
end
Both element_does_not_exist() and scroll_down are Calabash Ruby API's functions.
Instead scroll_down you can try to use your function to scroll specified ScrollView.
(edit: Sorry, I didn't look at comments ;))
Try This...
def scroll_to_text(text)
element = "android.widget.TextView text:'#{text}'"
if !element_exists(element)
wait_poll(:until_exists => element, :timeout => 60, :screenshot_on_error => true) do
scroll("android.view.View id:'search_listView'", :down)
end
end
end
This method will give you a screenshot and raise an error if it didn't find the text scrolling after 60 seconds. you can modify to use as you wanted. (I just get the code from your post so if something is wrong at the first time try modifying this).
I am using a combination of cucumber and pageobject to test my web application. Sometimes, the script tries to click an element even before the page that contains the element starts loading. (I confirmed this by capturing the screenshots of failing scenarios)
This inconsistency is not wide-spread and it happens repeatedly only for a few elements. Instead of directly accessing those elements, if I do example_element.when_visible.click, the test suite always passes.
As of now, I click a link using link_name (generated by pageobject module on calling link(:name, identifier: {index: 0}, &block)
I would like to not edit the above mentioned snippet, but act as if i called link_name_element.when_visible.click. The reason is, the test suite is pretty large and it would be tedious to change all the occurences and I also believe that the functionality is already present and somehow I don't see it anywhere. Can anybody help me out?!
This seems solution seems quite hacky and may not be considering some edge cases. However, I will share it since there are no other answers yet.
You can add the following monkey patch assuming that you are using watir-webdriver. This would be added after you require page-object.
require 'watir-webdriver'
require 'page-object'
module PageObject
module Platforms
module WatirWebDriver
class PageObject
def find_watir_element(the_call, type, identifier, tag_name=nil)
identifier, frame_identifiers, wait = parse_identifiers(identifier, type, tag_name)
the_call, identifier = move_element_to_css_selector(the_call, identifier)
if wait
element = #browser.instance_eval "#{nested_frames(frame_identifiers)}#{the_call}.when_present"
else
element = #browser.instance_eval "#{nested_frames(frame_identifiers)}#{the_call}"
end
switch_to_default_content(frame_identifiers)
type.new(element, :platform => :watir_webdriver)
end
def process_watir_call(the_call, type, identifier, value=nil, tag_name=nil)
identifier, frame_identifiers, wait = parse_identifiers(identifier, type, tag_name)
the_call, identifier = move_element_to_css_selector(the_call, identifier)
if wait
modified_call = the_call.dup.insert(the_call.rindex('.'), '.when_present')
value = #browser.instance_eval "#{nested_frames(frame_identifiers)}#{modified_call}"
else
value = #browser.instance_eval "#{nested_frames(frame_identifiers)}#{the_call}"
end
switch_to_default_content(frame_identifiers)
value
end
def parse_identifiers(identifier, element, tag_name=nil)
wait = identifier.has_key?(:wait) ? false : true
identifier.delete(:wait)
frame_identifiers = identifier.delete(:frame)
identifier = add_tagname_if_needed identifier, tag_name if tag_name
identifier = element.watir_identifier_for identifier
return identifier, frame_identifiers, wait
end
end
end
end
end
Basically, the intent of this patch is that the Watir when_present method is always called. For example, your page object call will get translated to Watir as browser.link.when_present.click. In theory, it should get called for any method called on a page object element.
Unfortunately, there is a catch. There are some situations where you probably do not want to wait for the element to become present. For example, when doing page.link_element.when_not_visible, you would not want to wait for the element to appear before checking that it does not appear. In these cases, you can force the standard behaviour of not waiting by including :wait => false in the element locator:
page.link_element(:wait => false).when_not_visible
I am trying to write a crawler that crawls all links from loaded page and logs all request and response headers along with response body in some file say XML or txt. I am opening all links from first loaded page in new browser window so I wont get this error:
Element not found in the cache - perhaps the page has changed since it was looked up
I want to know what could be the alternate way to make requests and receive response from all links and then locate input elements and submit buttons form all opened windows.
I am able to do above to some extent except when opened window has common site searh box like one on this http://www.testfire.net in the upper right corner.
What I want to do is I want to omit such common boxes so that I can fill other inputs with values using i.send_keys "value" method of webdriver and dont get this error
ERROR: Element not found in the cache - perhaps the page has changed since it was looked up.
What is the way to detect and distinguish input tags from each opened window so that value does not get filled repeatably in common input tags that appear on most pages of website.
My code is following:
require 'rubygems'
require 'selenium-webdriver'
require 'timeout'
class Clicker
def open_new_window(url)
#driver = Selenium::WebDriver.for :firefox
#url = #driver.get " http://test.acunetix.com "
#link = Array.new(#driver.find_elements(:tag_name, "a"))
#windows = Array.new(#driver.window_handles())
#link.each do |a|
a = #driver.execute_script("var d=document,a=d.createElement('a');a.target='_blank';a.href=arguments[0];a.innerHTML='.';d.body.appendChild(a);return a", a)
a.click
end
i = #driver.window_handles
i[0..i.length].each do |handle|
#driver.switch_to().window(handle)
puts #driver.current_url()
inputs = Array.new(#driver.find_elements(:tag_name, 'input'))
forms = Array.new(#driver.find_elements(:tag_name, 'form'))
inputs.each do |i|
begin
i.send_keys "value"
puts i.class
i.submit
rescue Timeout::Error => exc
puts "ERROR: #{exc.message}"
rescue Errno::ETIMEDOUT => exc
puts "ERROR: #{exc.message}"
rescue Exception => exc
puts "ERROR: #{exc.message}"
end
end
forms.each do |j|
begin
j.send_keys "value"
j.submit
rescue Timeout::Error => exc
puts "ERROR: #{exc.message}"
rescue Errno::ETIMEDOUT => exc
puts "ERROR: #{exc.message}"
rescue Exception => exc
puts "ERROR: #{exc.message}"
end
end
end
#Switch back to the original window
#driver.switch_to().window(i[0])
end
end
ol = Clicker.new
url = ""
ol.open_new_window(url)
Guide me how can I get all requeat and response headers with response body using Selenium Webdriver or using http.set_debug_output of ruby's net/http ?
Selenium is not one of the best options to use to attempt to build a "web-crawler". It can be too flakey at times, especially when it comes across unexpected scenarios. Selenium WebDriver is a great tool for automating and testing expectancies and user interactions.
Instead, good old fashioned curl would probably be a better option for web-crawling. Also, I am pretty sure there are some ruby gems that might help you web-crawl, just Google search it!
But To answer the actual question if you were to use Selenium WebDriver:
I'd work out a filtering algorithm where you can add the HTML of an element that you interact with to an variable array. Then, when you go on to the next window/tab/link, it would check against the variable array and skip the element if it finds a matching HTML value.
Unfortunately, SWD does not support getting request headers and responses with its API. The common work-around is to use a third party proxy to intercept the requests.
============
Now I'd like to address a few issues with your code.
I'd suggest before iterating over the links, add a #default_current_window = #driver.window_handle. This will allow you to always return back to the correct window at the end of your script when you call #driver.switch_to.window(#default_current_window).
In your #links iterator, instead of iterating over all the possible windows that could be displayed, use #driver.switch_to.window(#driver.window_handles.last). This will switch to the most recently displayed new window (and it only needs to happen once per link click!).
You can DRY up your inputs and form code by doing something like this:
inputs = []
inputs << #driver.find_elements(:tag_name => "input")
inputs << #driver.find_elements(:tag_name => "form")
inputs.flatten
inputs.each do |i|
begin
i.send_keys "value"
i.submit
rescue e
puts "ERROR: #{e.message}"
end
end
Please note how I just added all of the elements you wanted SWD to find into a single array variable that you iterate over. Then, when something bad happens, a single rescue is needed (I assume you don't want to automatically quit from there, which is why you just want to print the message to the screen).
Learning to DRY up your code and use external gems will help you achieve a lot of what you are trying to do, and at a faster pace.