Can you show me the way to use selenium in ruby to scroll down to bottom of page? I read this
element = driver.find_element(:xpath, "//div[#class='footer small']")
element.location_once_scrolled_into_view
but in this link, i can't find any element. Can you show me the way without element like that found. Thank you!
When I looked at the page, I did not see a div with class footer. This might be why you cannot find the element.
For me, the last visible element appears to be the wallpapers - ie div with class pic. You can get the last picture and scroll to it using the following. Note that we find all of the pictures and then take the last one in the collection.
last_picture = driver.find_elements(:css, 'div.pic').last
last_picture.location_once_scrolled_into_view
After you scroll to the last wallpaper, you will want to wait for the page to finish loading. For example, the following will wait until the image count increases:
require 'selenium-webdriver'
driver = Selenium::WebDriver.for :firefox
driver.navigate.to 'http://www.mobileswall.com/'
# Check how many elements are there initially
puts driver.find_elements(:css, 'div.pic').length
#=> 30
# Scroll to the last image
driver.find_elements(:css, 'div.pic').last.location_once_scrolled_into_view
# Wait for the additional images to load
current_count = driver.find_elements(:css, 'div.pic').length
until current_count < driver.find_elements(:css, 'div.pic').length
sleep(1)
end
# Check how many elements are there now
puts driver.find_elements(:css, 'div.pic').length
#=> 59
Related
I am trying to parse a website using Selenium/Capybara. Right now it looks like this:
session = Capybara::Session.new(:selenium)
session.visit "https://somesite.com/page1"
element = session.all( :css, '.table .row a' ).each do |el|
el.click
# get some element's data
session.evaluate_script('window.history.back()')
end
# repeat
Problem is, when I'm trying to get the data from the second page I am on, Capybara tells me that Either the element is no longer attached to the DOM or the page has been refreshed. which absolutely makes sense, however I'm struggling to find a way to create a new DOM variable and parse it in the documentation.
Same thing happens if I navigate back trying to repeat the actions and click on the second link in a row. I assume I need to re-create the session or is there a better way?
To work like you're trying you're going to need to keep a counter and find the elements each time through your loop - something along the lines of
counter = 0
while (el = session.all( :css, '.table .row a', minimum: 1 )[counter]) do
el.click
# get some element's data
counter += 1
session.go_back
end
or if the links are just standard you could gather the hrefs and then just visit them
element = session.all( :css, '.table .row a', minimum:1 ).map {|a| a['href']} do |url|
session.visit(url)
# get some element's data
end
I'm using Ruby and Selenium to get some data from a page. I want to define variable with driver.find_element, but element is not currently visible on page.
next = driver.find_element(:class, 'right')
It returns Selenium::WebDriver::Error::NoSuchElementError
It works fine when element is present.
Any solutions?
Thank you!
Selenium works by executing Javascript commands. By using find_element it will search for the element on the DOM. If it cannot find it you will get the error you are getting. After all if an element is not on the DOM it cannot be found.
The real question is why do you want too find an element that is not currently present on the DOM? You can't do anything with somehing that doesn't exist.
All I could think of is that the element becomes present after the DOM has been loaded due to Javascript not being fully executed yet. If that is the case you can use a WebDriver::Wait to try and find the element for a certain amount of time.
A small example:
wait = Selenium::WebDriver::Wait.new(:timeout => 10) # seconds
begin
element = wait.until { driver.find_element(:id => "some-dynamic-element") }
ensure
driver.quit
end
Edit to include try-catch example:
begin
next = driver.find_element(:class, 'right')
# Code for when element is found here
rescue NoSuchElementError
# Code for when element is not found here
end
I have a Ruby application using Selenium Webdriver and Nokogiri. I want to choose a class, and then for each div corresponding to that class, I want to perform an action based on the contents of the div.
For example, I'm parsing the following page:
https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=puppies
It's a page of search results, and I'm looking for the first result with the word "Adoption" in the description. So the bot should look for divs with className: "result", for each one check if its .description div contains the word "adoption", and if it does, click on the .link div. In other words, if the .description does not include that word, then the bot moves on to the next .result.
This is what I have so far, which just clicks on the first result:
require "selenium-webdriver"
require "nokogiri"
driver = Selenium::WebDriver.for :chrome
driver.navigate.to "https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=puppies"
driver.find_element(:class, "link").click
You can get list of elements that contains "adopt" and "Adopt" by XPath using contains() then use union operator (|) to union results from "adopt" and "Adopt". See code below:
driver = Selenium::WebDriver.for :chrome
driver.navigate.to "https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=puppies"
sleep 5
items = driver.find_elements(:xpath,"//div[#class='g']/div[contains(.,'Adopt')]/h3/a|//div[#class='g']/div[contains(.,'adopt')]/h3/a")
for element in items
linkText = element.text
print linkText
element.click
end
The pattern to handle each iteration will be determined by the type of action executed on each item. If the action is a click, then you can't list all the links to click on each of them since the first click will load a new page, making the elements list obsolete.
So If you wish to click on each link, then one way is to use an XPath containing the position of the link for each iteration:
# iteration 1
driver.find_element(:xpath, "(//h3[#class='r']/a)[1]").click # click first link
# iteration 2
driver.find_element(:xpath, "(//h3[#class='r']/a)[2]").click # click second link
Here is an example that clicks on each link from a result page:
require 'selenium-webdriver'
driver = Selenium::WebDriver.for :chrome
wait = Selenium::WebDriver::Wait.new(timeout: 10000)
driver.navigate.to "https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=puppies"
# define the xpath
search_word = "Puppies"
xpath = ("(//h3[#class='r']/a[contains(.,'%s')]" % search_word) + ")[%s]"
# iterate each result by inserting the position in the XPath
i = 0
while true do
# wait for the results to be loaded
wait.until {driver.find_elements(:xpath, "(//h3[#class='r']/a)[1]").any?}
# get the next link
link = driver.find_elements(:xpath, xpath % [i+=1]).first
break if !link
# click the link
link.click
# wait for a new page
wait.until {driver.find_elements(:xpath, "(//h3[#class='r']/a)[1]").empty?}
# handle the new page
puts "Page #{i}: " + driver.title
# return to the main page
driver.navigate.back
end
puts "The end!"
I don't code in ruby, but one way you could do it in python is:
driver.find_elements
notice how elements is plural, I would grab all the links and put them into an array like.
href = driver.find_elements_by_xpath("//div[#class='rc]/h3/a").getAttribute("href");
Then get all of the descriptions the same way. Do a for loop for every element of description, if the description has the word "Adoption" in it navigate to that website.
for example:
if description[6] has the word adoption find the string href[6] and navigate to href[6].
I hope that makes sense!
I am trying to automate an online survey on a website but I get this error each time:
Selenium::WebDriver::Error::UnknownError: unknown error: Element is not clickable at
point (561, 864). Other element would receive the click: a id="habla_oplink_a"
class="habla_oplink_a_normal hbl_pal_header_font_size hbl_pal_title_fg "
What I need to understand is how I can scroll to a certain point of the page so that my script can resume filling out the survey on the page.
This is my code that manages to fill out a portion of the survey but fails when it reaches a row which is not in view inside the browser (a row that requires the user to scroll down to):
buttons = browser.elements(:class => "assessment-choice")
buttons.each do |button|
button.click
end
I would also like to be able to change my code so that it only selects a specific option but the HTML on the page is not very friendly.
This is the webpage I am looking at: https://staging2.clearfit.com/assessment/assessment/95867fb272df436352a0bd5fbdd
The HTML of one of the options on the survey:
<a id="answers_79_0" class="assessment-choice" onmouseover="answerOver(this)" onmouseout="answerOut(this)" onclick="setAssessmentAnswer(this, 3, '0', '79', '#answers_49839163')">Strongly<br>Agree</a>
Using execute_script
To scroll to an element, you will need to execute javascript:
browser.execute_script('arguments[0].scrollIntoView();', button)
This can be seen to be working in the following script. Without the line to scroll, a chat tab overlays one of the buttons causing an exception.
require 'watir-webdriver'
browser = Watir::Browser.new :chrome
browser.goto 'https://staging2.clearfit.com/assessment/assessment/95867fb272df436352a0bd5fbdd'
buttons = browser.elements(:class => "assessment-choice")
buttons.each do |button|
browser.execute_script('arguments[0].scrollIntoView();', button)
button.click
end
Using the watir-scroll gem
Note that you can install the watir-scroll gem to make the scrolling line nicer. The gem allows the line to simply be:
browser.scroll.to button
The script would then look like:
require 'watir-webdriver'
require 'watir-scroll'
browser = Watir::Browser.new :chrome
browser.goto 'https://staging2.clearfit.com/assessment/assessment/95867fb272df436352a0bd5fbdd'
buttons = browser.elements(:class => "assessment-choice")
buttons.each do |button|
browser.scroll.to button
button.click
end
Firstly, this should be unnecessary. According to the spec, all element interactions require implicit scrolling to the element. If something does prevent this from happening, though, you can use this Selenium method instead of a javascript implementation:
buttons = browser.elements(:class => "assessment-choice")
buttons.each do |button|
button.wd.location_once_scrolled_into_view
button.click
end
public
def scroll_to(param)
args = case param
when :top, :start
'window.scrollTo(0, 0);'
when :center
'window.scrollTo(window.outerWidth / 2, window.outerHeight / 2);'
when :bottom, :end
'window.scrollTo(0, document.body.scrollHeight);'
when Array
['window.scrollTo(arguments[0], arguments[1]);', Integer(param[0]), Integer(param[1])]
else
raise ArgumentError, "Don't know how to scroll to: #{param}!"
end
#browser.execute_script(*args)
end
public
# This method pulls the object on the page you want to interact with, then it 'jumps to it'.
def jump_to(param)
# Leveraging the scroll_to(param) logic, this grabs the cooridnates,
# and then makes them an array that is able to be located and moved to.
# This is helpful when pages are getting too long and you need to click a button
# or interact with the browser, but the page 'Cannot locate element'.
location = param.wd.location
location = location.to_a
$helper.scroll_to(location)
end
Then you just call jump_to(element) and it "Jumps" to it.
This is how I got around it- not sure if that is a normal way. The problem is it goes to point (0,0); working on a version that moves to it to center screen.
I'm trying to use the DOM inspector to find the ID of the "More" button at the bottom of this page that reveals more results.
I'm trying to do something like this example:
require 'watir-webdriver'
b = Watir::Browser.new
b.goto 'svpply.com/editors_pick'
#count products
puts b.elements(:xpath => '//li[#data-class="Product"]').count
#=> 30
#Now click button
show_all = b.button(:id => "btn_all")
show_all.click
sleep 4
#count products again
puts b.elements(:xpath => '//li[#data-class="Product"]').count
#=>60
However, I'm unclear on how to search for that particular id within the DOM structure. Can someone also explain the difference between an attribute, element, id, and node?
To use the DOM Inspector for the More button:
Open the DOM Inspector (Ctrl+Shift+I)
Click the [More] button that you want to inspect
On the right side of the DOM Inspector bar, click the [HTML] button
This should show the HTML for the page, which will include the details of the [More] control. You'll notice that the element is actually a DIV not a button. As well that the ID is in the form "_more".
--> This should show the HTML for the page, which will show the details of the [More] control. You'll notice that the element is actually a DIV not a button. As well that the ID is in the form "_more".
So to do your example with the Quora page, you would do something like:
require 'watir-webdriver'
class QuoraPage
def initialize(browser)
#browser = browser
end
def goto()
#browser.goto 'http://www.quora.com/Startups/best_questions'
wait_questions_loaded
end
def click_more()
#browser.div(:id, /_more/).click
wait_questions_loaded
end
def questions_count()
#browser.links(:class, 'question_link').count{ |x| x.visible? }
end
def wait_questions_loaded()
begin
questions_start_count = questions_count()
sleep(2)
end while questions_start_count != questions_count()
end
end
page = QuoraPage.new(Watir::Browser.new :chrome)
page.goto
puts page.questions_count
page.click_more
puts page.questions_count
Note that I had to put the sleeps in otherwise webdriver hangs like anonygoose mentioned. I tried different wait_untils, but did not manage to find something that worked (other than sleep which is not very robust).
Regarding your question about nodes, elements, etc. I think you are best to look at http://www.w3schools.com/dom/default.asp.
To press the button on svpply you can use simply
b.button(:text => "Show All").click
Counting all the products that appear on the page could potentially be done with
b.lis(:class => "grab large").count
This is all for the svpply site. I can't get quora to automate at all, it just stalls my watir-webdriver indefinitely.
You'll also want to wait before you have watir count the products. This can be done with:
b.wait_until{b.lis(:class => "grab large").count > 30}