No matter what I try to do, the browser tries to run the test too fast before it has a chance to find the element that I am looking for. If I put in a simple "sleep 2", it has a chance for the drop down menu to drop and load and successfully find the element. But I am wanting to learn to use the Selenium Wait command. I have tried numerous combinations of the below and looked all over the web for documentation or perhaps examples. I have found plenty of firefox and people say that some of the things below worked perfectly in firefox, but for me and my project team mates, we can not get any of the waits, implicit or explicit, to pause long enough for it to detect the element. The element does not exist until the drop down menu is fully dropped, then it can detect it. It doesn't take 2 seconds but it seems that none of my wait commands will actually make it wait. Like I said, I have tried numerous different things and almost all of them are below. If anyone can help guide me, i would appreciate it. Here is some of the code I have tried:
def setup
#driver = Selenium::WebDriver.for :chrome
#driver.get "https://website.herokuapp.com/"
#wait = Selenium::WebDriver::Wait.new(:timeout => 10) # seconds
# #driver = Selenium::WebDriver.for:chrome
# #driver.manage.timeouts.implicit_wait = 30
# #wait = Selenium::WebDriver::Wait.new(:timeout => 15)
# #wait = Selenium::WebDriver::Wait.new(:timeout => 10)
# #driver.manage.timeouts.implicit_wait = 10
# #wait = Selenium::WebDriver::Wait.new(timeout: 10)
#driver.manage.window.maximize()
# #driver.navigate.to("https://website.herokuapp.com/")
end
def test_user_name_is_present
login()
#driver.find_element(:class, "navbar-toggle").click()
# user = #driver.find_element(:class, "dropdown-toggle").text
# #wait.until{#driver.find_element(:class, "dropdown-toggle")}
#driver.find_element(:class, "dropdown-toggle")
#wait.until { #driver.find_element(:class => "dropdown-toggle") }
user = #driver.find_element(:class, "dropdown-toggle").text
assert_equal(true, user.include?('HEATHER'), "no user")
end
I'm more familiar with JavaScript or Java bindings.
But what about:
def test_user_name_is_present
login()
#driver.find_element(:class, "navbar-toggle").click()
#wait.until { #driver.find_element(:class => "dropdown-toggle").displayed? }
user = #driver.find_element(:class, "dropdown-toggle").text
assert_equal(true, user.include?('HEATHER'), "no user")
end
Our team uses this, just add to your test_helper.rb if using rails
def wait_for_ajax(sleep_sec = 4)
assert page.has_no_content?(:css, 'body.loading')
sleep sleep_sec if sleep_sec
end
I got Ruby to travel to a web site, iterate through a list of campaigns and scrape the pages for specific data. The problem I have now is getting it from the structure Nokogiri gives me, and outputting it into a readable form.
campaign_list = Array.new
campaign_list.push(1042360, 1042386, 1042365, 992307)
browser = Watir::Browser.new :chrome
browser.goto '<redacted>'
browser.text_field(:id => 'email').set '<redacted>'
browser.text_field(:id => 'password').set '<redacted>'
browser.send_keys :enter
file = File.new('hourlysales.csv', 'w')
data = {}
campaign_list.each do |campaign|
browser.goto "<redacted>"
if browser.text.include? "Application Error"
puts "Error loading page, I recommend restarting script"
# Possibly automatic restart of script
else
hourly_data = Nokogiri::HTML.parse(browser.html).text
# file.write data
puts hourly_data
end
This is the output I get:
{"views":[[17,145],[18,165],[19,99],[20,71],[21,31],[22,26],[23,10],[0,15],[1,1], [2,18],[3,19],[4,35],[5,47],[6,44],[7,67],[8,179],[9,141],[10,112],[11,95],[12,46],[13,82],[14,79],[15,70],[16,103]],"orders":[[17,10],[18,9],[19,5],[20,1],[21,1],[22,0],[23,0],[0,1],[1,0],[2,1],[3,0],[4,1],[5,2],[6,1],[7,5],[8,11],[9,6],[10,5],[11,3],[12,1],[13,2],[14,4],[15,6],[16,7]],"conversion_rates":[0.06870229007633588,0.05442176870748299,0.050505050505050504,0.014084507042253521,0.03225806451612903,0.0,0.0,0.06666666666666667,0.0,0.05555555555555555,0.0,0.02857142857142857,0.0425531914893617,0.022727272727272728,0.07462686567164178,0.06134969325153374,0.0425531914893617,0.044642857142857144,0.031578947368421054,0.021739130434782608,0.024390243902439025,0.05063291139240506,0.08571428571428572,0.06741573033707865]}
The arrays stand for { views [[hour, # of views], [hour, # of views], etc. }. Same with orders. I don't need conversion rates.
I also need to add the values up for each key, so after doing this for 5 pages, I have one key for each hour of the day, and the total number of views for that hour. I tried a couple each loops, but couldn't make any progress.
I appreciate any help you guys can give me.
It looks like the output (which from your code I assume is the content of hourly_data) is JSON. In that case, it's easy to parse and add up the numbers. Something like this:
require "json" # at the top of your script
# ...
def sum_hours_values(data, hours_values=nil)
# Start with an empty hash that automatically initializes missing keys to `0`
hours_values ||= Hash.new {|hsh,hour| hsh[hour] = 0 }
# Iterate through the [hour, value] arrays, adding `value` to the running
# count for that `hour`, and return `hours_values`
data.each_with_object(hours_values) do |(hour, value), hsh|
hsh[hour] += value
end
end
# ... Watir/Nokogiri stuff here...
# Initialize these so they persist outside the loop
hours_views, orders_views = nil
campaign_list.each do |campaign|
browser.goto "<redacted>"
if browser.text.include? "Application Error"
# ...
else
# ...
hourly_data_parsed = JSON.parse(hourly_data)
hours_views = sum_hours_values(hourly_data_parsed["views"], hours_views)
hours_orders = sum_hours_values(hourly_data_parsed["orders"], orders_views)
end
end
puts "Views by hour:"
puts hours_views.sort.map {|hour_views| "%2i\t%4i" % hour_views }
puts "Orders by hour:"
puts hours_orders.sort.map {|hour_orders| "%2i\t%4i" % hour_orders }
P.S. There's a really nice recursive version of sum_hours_values I didn't include since the iterative version is clearer to most Ruby programmers. If you're into recursion I leave it as an exercise for you. ;)
So basically I asked a question earlier and got an answer which solved that question but I now realise I need more help as I have spent the last few hours trying to fix this but keep getting "nil:Nilclass: error. Basically I need to go through every show listed on this site (so through each letter, and through each page this letter has) and get the following:
1. The shows title (this part I have done)
2. then either copy the page url for each show and add "/episodes/" to the end of it, or click the show and then the episode tab and copy that url.
This is what I have so far:
require 'mechanize'
shows = Array.new
agent = Mechanize.new
agent.get 'http://www.tv.com/shows/sort/a_z/'
agent.page.search('//div[#class="alphabet"]//li[not(contains(#class, "selected"))]/a').each do |letter_link|
agent.get letter_link[:href]
agent.page.search('//li[#class="show"]/a').each { |show_link| shows << show_link.text }
while next_page_link = agent.page.at('//div[#class="_pagination"]//a[#class="next"]') do
agent.get next_page_link[:href]
agent.page.search('//li[#class="show"]/a').each { |show_link| shows << show_link.text }
end
end
require 'pp'
pp shows
So an end result would like something like the following:
Title: Game of Thrones
URL: http://www.tv.com/shows/game-of-thrones/episodes/
I have tried everything (even writing it from scratch) but just can't seem to add the extra parts so I was hoping someone here my be able to help me do so. Thanks
How about this
require 'mechanize'
shows = {}
base_uri = "http://www.tv.com/"
agent = Mechanize.new
agent.get 'http://www.tv.com/shows/sort/a_z/'
agent.page.search('//div[#class="alphabet"]//li[not(contains(#class, "selected"))]/a').each do |letter_link|
agent.get letter_link[:href]
letter = letter_link.text.upcase
shows[letter] = agent.page.search('//li[#class="show"]/a').map{ |show_link| {show_link.text => base_uri << show_link[:href].to_s << 'episodes/'} }
while next_page_link = agent.page.at('//div[#class="_pagination"]//a[#class="next"]') do
agent.get next_page_link[:href]
shows[letter] << agent.page.search('//li[#class="show"]/a').map{ |show_link| {show_link.text => base_uri << show_link[:href].to_s << 'episodes/'} }
end
shows[letter].flatten!
end
puts shows
This will create the following structure Hash[letter] => Array[{ShowName => LinktoEpisodes}]
Example:
{"A"=>[{"A Show"=>"http://www.tv.com/shows/a-show/episodes/"},{"Another Show"=>"http://www.tv.com/shows/another-show/episodes/"},...],"B"=>[B SHOWS....],....}
Hope this helps.
I am trying to click all links on stackoveflow horizontal menu (Questions, Tags, Users, Badges, Unanswered). I have this code but this clicks on first link (this link is Questions), then prints 1, and after that raises error. What could be problem with this?
require 'watir-webdriver'
class Stackoverflow
def click_all_nav_links
b = Watir::Browser.new
b.goto "http://stackoverflow.com"
counter = 0
b.div(:id => 'hmenus').div(:class => 'nav mainnavs').ul.lis.each do |li|
li.a.click
puts counter += 1
end
end
end
stackoverflow = Stackoverflow.new
stackoverflow.click_all_nav_links
Error message is:
https://gist.github.com/3242300
The StaleElementReferenceError often occurs when storing elements and then trying to access them after going to another page. In this case, the reference to the lis becomes stale after you click the links and navigate to a new page.
You have to store off attributes or the index of the lis first. This will allow you to get a fresh reference to each li after clicking a link.
Try this:
class Stackoverflow
def click_all_nav_links
b = Watir::Browser.new
b.goto "http://stackoverflow.com"
#Store the text of each locate so that it can be located later
tabs = b.div(:id => 'hmenus').div(:class => 'nav mainnavs').ul.lis.collect{ |x| x.text }
#Iterate through the tabs, using a fresh reference each time
tabs.each do |x|
b.div(:id => 'hmenus').div(:class => 'nav mainnavs').ul.li(:text, x).a.click
end
end
end
stackoverflow = Stackoverflow.new
stackoverflow.click_all_nav_links
I want to use shoes to be able to download the body of a website that a user enters in an edit_line. How can I make this work? Can anyone help explain why the below code does not work? It just pops up a new window, and does not download the site from the text entered....
Here's my code thus far:
Shoes.app do
stack (:left => 175, :top => 200) do
para "Enter a url:"
flow do
#url = edit_line
button "OK" do
window do
stack do
title "Searching site", :size => 16
#status = para "One moment..."
# Search site for query and print body
download #url.text do |site|
#status.text = "Body: " + site.response.body.inspect
end
end
end
end
end
end
end
Nevermind - I figured it out. I just didn't pop to a new window and it downloads and prints the body fine. : )