I have the following problem with my Selenium in Ruby. It generates the error, that the element is no longer attached to the DOM. I found some solutions to wait, but I wasn`t able to figure out if I can wait for an element which has no ID. Can i wait for an element if I only have the className?
require 'selenium-webdriver'
#require Firefox installation !!
browser = Selenium::WebDriver.for :firefox
browser.get <URL>
wait = Selenium::WebDriver::Wait.new(:timeout => 20)
js_code = "return document.getElementsByClassName('Cell ')"
rawdata = Array.new
puts rawdata.size
elements = browser.execute_script(js_code)
elements.each{|e| rawdata.push(e.text) }
puts rawdata.size
arrSize = rawdata.length
puts rawdata.at(5) + " " + rawdata.at(4) + " " + rawdata.at(9) + " " + rawdata.at(6)
This answers your question but not necessarily resolves your exception. If it doesn't, you might want to post HTML snippets and stacktrace.
Here is how to use WebDriverWait in Ruby:
# create wait like you have already done
wait = Selenium::WebDriver::Wait.new(:timeout => 20)
# wait until something, you can use any locators you want, not just ids
# don't inject JavaScript directly, unless you have to
wait.until { driver.find_element(:class => "dojoxGridCell") }
# do stuff to your raw data
Related
I'm doing a web scraping with a dynamic website that has a "Load more" button. Though I solve the load more problems by using a while loop. It has another challenge when I try to scrape the data it just keeps multiplying. So the first batch of data is 24 data when I scrape the second batch it also scrapes the first batch so it scrape 48 data with only 24 new data being added and soon.
heres my code.
require "selenium-webdriver"
driver = Selenium::WebDriver.for :chrome
url ="https://www.example.com/categories/car-parts"
driver.navigate.to "#{url}"
wait = Selenium::WebDriver::Wait.new(:timeout => 20)
while driver.page_source.include? "Load more"
load_more = wait.until {
load_more_element = driver.find_element(css: ".styles__loadMore___yYAF4")
}
sleep 3
load_more.click()
puts "load_more"
sleep 3
seller_url = wait.until {
element = driver.find_elements(:css, ".desktop__itemOneFourth___2t71A .styles__link___9msaS:nth-child(1)")
}
seller_url.each do |line|
seller_uri = line.attribute("href")
seller_hand = seller_uri[/https:\/\/www.example.com(.*\/([.\w+]+))/i]
seller_handle = seller_hand.gsub("https://www.example.com/", "")
seller = Seller.new
seller.seller_url = seller_uri
seller.seller_handle = seller_handle
seller.save
puts seller_handle
end
puts seller_url.size
sleep 3
What I want is that i continues to load but i want to scrape the last loaded batch minus all the previous batch.
You know how many records are loaded each time you hit the load more button so you can easily access only new records in the seller_url array:
items_per_page = 24
while driver.page_source.include? "Load more"
# ...
seller_url = wait.until {
element = driver.find_elements(:css, ".desktop__itemOneFourth___2t71A .styles__link___9msaS:nth-child(1)")
}
seller_url.last(items_per_page).each do |line|
# do stuff
end
pages_loaded += 1
end
No matter what I try to do, the browser tries to run the test too fast before it has a chance to find the element that I am looking for. If I put in a simple "sleep 2", it has a chance for the drop down menu to drop and load and successfully find the element. But I am wanting to learn to use the Selenium Wait command. I have tried numerous combinations of the below and looked all over the web for documentation or perhaps examples. I have found plenty of firefox and people say that some of the things below worked perfectly in firefox, but for me and my project team mates, we can not get any of the waits, implicit or explicit, to pause long enough for it to detect the element. The element does not exist until the drop down menu is fully dropped, then it can detect it. It doesn't take 2 seconds but it seems that none of my wait commands will actually make it wait. Like I said, I have tried numerous different things and almost all of them are below. If anyone can help guide me, i would appreciate it. Here is some of the code I have tried:
def setup
#driver = Selenium::WebDriver.for :chrome
#driver.get "https://website.herokuapp.com/"
#wait = Selenium::WebDriver::Wait.new(:timeout => 10) # seconds
# #driver = Selenium::WebDriver.for:chrome
# #driver.manage.timeouts.implicit_wait = 30
# #wait = Selenium::WebDriver::Wait.new(:timeout => 15)
# #wait = Selenium::WebDriver::Wait.new(:timeout => 10)
# #driver.manage.timeouts.implicit_wait = 10
# #wait = Selenium::WebDriver::Wait.new(timeout: 10)
#driver.manage.window.maximize()
# #driver.navigate.to("https://website.herokuapp.com/")
end
def test_user_name_is_present
login()
#driver.find_element(:class, "navbar-toggle").click()
# user = #driver.find_element(:class, "dropdown-toggle").text
# #wait.until{#driver.find_element(:class, "dropdown-toggle")}
#driver.find_element(:class, "dropdown-toggle")
#wait.until { #driver.find_element(:class => "dropdown-toggle") }
user = #driver.find_element(:class, "dropdown-toggle").text
assert_equal(true, user.include?('HEATHER'), "no user")
end
I'm more familiar with JavaScript or Java bindings.
But what about:
def test_user_name_is_present
login()
#driver.find_element(:class, "navbar-toggle").click()
#wait.until { #driver.find_element(:class => "dropdown-toggle").displayed? }
user = #driver.find_element(:class, "dropdown-toggle").text
assert_equal(true, user.include?('HEATHER'), "no user")
end
Our team uses this, just add to your test_helper.rb if using rails
def wait_for_ajax(sleep_sec = 4)
assert page.has_no_content?(:css, 'body.loading')
sleep sleep_sec if sleep_sec
end
I am trying to get a screenshot at every step with the current date and time, but I am getting the error
Error: test_login(Login_page): Argument Error: wrong number of arguments (1 for 0)
The code is
def setup
#driver = Selenium::WebDriver.for :chrome
#driver.manage.window.maximize
#driver.navigate.to "https://www.findmedecor.com"
wait = Selenium::WebDriver::Wait.new(:timeout => 10)
screenshot()
end
def test_login
#driver.find_element(:class,'open-overlay').click
screenshot(DateTime.now)
wait = Selenium::WebDriver::Wait.new(:timeout => 10)
login_email = wait.until {
element = #driver.find_element(:name, "login_email")
element if element.displayed?
}
login_email.send_keys("suwarna.wade#rohagroup.com")
puts "Test Passed: login pop up found" if login_email.displayed?
screenshot(DateTime.now)
#driver.find_element(:id,'pass').send_keys('123456')
#driver.find_element(:id,'btn_login').click
puts "Logged in successfully"
puts "Time of test = ", DateTime.now
screenshot(DateTime.now)
end
$i = DateTime.now
def screenshot
#driver.save_screenshot("screenshot #{'$i'}.png")
$i= +1
end
end
The problem is that Time.now returns a format like '2016-09-28 04:45:40 +0000' which is not a valid filename on Windows. You can just reformat the date/time to something valid like
Time.now.strftime('%Y-%m-%d_%H.%M.%S')
which outputs 2016-09-27_23.33.59 and then put that in your filename.
http://ruby-doc.org/core-2.2.0/Time.html#method-i-strftime
My goal is to dynamically get website content created by Javascript. I have the following code:
browser = Selenium::WebDriver.for :firefox
browser.get "https://gls-group.eu/AT/de/paket-verfolgen?match=00000000000"
wait = Selenium::WebDriver::Wait.new(:timeout => 20)
js_code = "return document.getElementsByTagName('div')"
elements = browser.execute_script(js_code)
puts elements
browser.close
The output is:
#<Selenium::WebDriver::Element:0x4e4c920>
#<Selenium::WebDriver::Element:0x4e4c770>
#<Selenium::WebDriver::Element:0x4e4c230>
#<Selenium::WebDriver::Element:0x4e55650>
#<Selenium::WebDriver::Element:0x4e55848>
#<Selenium::WebDriver::Element:0x4e57e58>
#<Selenium::WebDriver::Element:0x4e57c00>
#<Selenium::WebDriver::Element:0x4e57a08>
and so on. How do I get the divs?
browser.execute_script(js_code) gives all the html elements as you asked as instances of Selenium::WebDriver::Element class. Write as below using method Selenium::WebDriver::Element#text to get the content of those div elements :
require 'selenium-webdriver'
browser = Selenium::WebDriver.for :firefox
browser.get "https://gls-group.eu/AT/de/paket-verfolgen?match=00000000000"
wait = Selenium::WebDriver::Wait.new(:timeout => 20)
js_code = "return document.getElementsByTagName('div')"
elements = browser.execute_script(js_code)
elements.each{|e| puts e.text }
Please follow the code below:
driver.get "https://example.com/"
driver.find_element(:class, "button").submit
driver.navigate.refresh
wait = Selenium::WebDriver::Wait.new(:timeout => 10) # seconds
element = wait.until { driver.find_element(:name => "username") }
I wrote the code keeping in my mind that till the page which contains element : username comes, continue the previous page to refresh. But it seems my code not meeting that requirement. Thus script throwing error as below "
Error
C:/Ruby193/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.27.2/lib/selenium/webdr
iver/common/wait.rb:57:in `until': timed out after 10 seconds (Unable to locate
element: {"method":"name","selector":"username"})} (Selenium::WebDriver::Error::
TimeOutError)
Any good idea to meet my requirement,please?
Thanks,
I have not come across a built-in way to do this in selenium-webdriver, so I would do the following:
#Submit your first page
driver.get "https://example.com/"
driver.find_element(:class, "button").submit
#Refresh page until your element appears
end_time = Time.now + 10 #seconds
begin
element = driver.find_element(:name => "username")
rescue Selenium::WebDriver::Error::NoSuchElementError
if Time.now < end_time
driver.navigate.refresh
retry
end
end
Basically this is attempting to find the element. If it is not found, catches the exception, refreshes the page and retries again. This is repeated until the time limit hsa been reached.