I'm opening a set of URLs with a WebDriver in ruby – kind of a slideshow with 3 seconds intervals between "slides" (pages). Person looking at that happening might click Space, and I need that page URL saved to another file. How could I handle those interruptions – catch the event of Space pressed?
require "watir-webdriver"
urls = [...list of URLs here...]
saved = []
b = Watir::Browser.new
urls.each do |url|
b.goto url
sleep(3)
# ...what should I put here to handle Space pressed?
if space_pressed
saved << b.url
end
end
It looks like your problem might be solvable with STDIN.getch.
If you create a file with the following script and then run it in a command prompt (eg "ruby script.rb"), the script will:
Navigate to the url.
Ask if the url should be captured.
If the user does not input anything in 10 seconds, it will proceed onto the next url. I changed it from 3 since it was too fast. You can change the time back to 3 seconds in the line Timeout::timeout(10).
If the user did input something, it will save the url if the input was a space. Otherwise it will ignore it and move on.
Script:
require "watir-webdriver"
require 'io/console'
require 'timeout'
urls = ['www.google.ca', 'www.yahoo.ca', 'www.gmail.com']
saved = []
b = Watir::Browser.new
urls.each do |url|
b.goto url
# Give user 10 seconds to provide input
puts "Capture url '#{url}'?"
$stdout.flush
input = Timeout::timeout(10) {
input = STDIN.getch
} rescue ''
# If the user's input is a space, save the url
if input == ' '
saved << b.url
end
end
p saved
A couple of notes:
Note that the inputs need to be to the command prompt rather than the browser.
If the user presses a key before the allotted timeout, the script will immediately proceed to the next url.
Related
The script is working fine for me.
Now, Iam downloading 500 files at a time.
I want to download the files by specifying some range like (10-30) files at one time and (30-60) at another time so on using ruby watir.
These is my code:
require 'watir'
require 'rubygems'
begin
chromedriver_path = File.join(File.absolute_path(File.dirname(__FILE__)),"browser","chromedriver.exe")
Selenium::WebDriver::Chrome.driver_path = chromedriver_path
browser = Watir::Browser.new:chrome
browser.goto "" //url to login
sleep 3
browser.text_field(:name=>"").set "" #e_id
sleep 3
browser.text_field(:name=>"").set "" #pwd
browser.button(:value=>"Login").click #submit
browser.div(:id=>"DivMenu").click
#sleep 3
browser.span(:class =>"down").click
sleep 3
browser.execute_script("document.getElementById('hlGenerateStatusReports').click();")
sleep 3
browser.execute_script("document.getElementById('Report').click();")
sleep 3
optncount = browser.select_list(:id => 'head_ddlClient').options.count
puts optncount
i = 0
while i <= optncount do
puts "Inside the loop i = "+i.to_s
i +=1
browser.select_list(:id => 'ddlClient').option(:index => i).select
sleep 3
browser.button(:value=>"Generate Report").click #submit
sleep 10
end
browser.goto " " //url to logout
rescue Exception => e
puts e.message
puts e.backtrace.inspect
end
I also have download scripts for images etc.
The small part of the one I use here to demonstrate the technique is for 500px.com.
I keep all downloaded files in a textfile and check if allready downloaded against this file. That way you can break off at any moment and resume later.
Of course you could break off if downloaded reached a limit.
I don't publish the whole of the script, just what matters regarding your question.
def download url
filename = "#{url[-32..-1]}.jpg"
if get(url, filename, SAVE_FOLDER)
File.open(PROGRESS_FILE,'a+'){|f|f.puts filename}
end
end
PROGRESS_FILE = './500px.txt'
downloaded = 0
....
response = http.get(path, headers)
json = JSON.parse(response.body)["data"]
processed = File.read(PROGRESS_FILE)
json.each do |item|
url = item['images'].last['url']
signature = url[-32..-1]
filename = "#{signature}.jpg"
# check if the filenames is in the textfile and so was downloaded allready
unless processed[filename]
download url
downloaded += 1
end
end
The file has more than 500 million lines so far and works fast enough (the downloading takes much longer). If I hit a limit I can easily put the lines in a simple database like Sqlite.
I have it where my script signs in and goes to a browser url, yet when it signs out of the current web page it just sits there and won't restart the loop. How can I get the loop to realize its done and to restart?
x = 0
while x <= 5
File.open("yahoo_accounts.txt") do |email|
email.each do |item|
email, password = item.chomp.split(',')
emails << email
passwords << password
emails.zip(passwords) { |name, pass|
browser = Watir::Browser.new :ff
browser.goto "url"
#logs in and does what its suppose to do with the name and pass
}
end
x += 1
next
end
end
When the script is done it just sits at the webpage...I'm trying to get it to go to the beginning again...
You would think it would take each name,pass and go back to the beginning url.
Thanks for your help.
It looks like you may not be calling browser.close appropriately. In my quick mock-up testing, I definitely get weird behaviour if I don't do that. You're also using non-idiomatic Ruby looping. Try this:
5.times do
File.open("yahoo_accounts.txt") do |email|
email.each do |item|
email, password = item.chomp.split(',')
emails << email
passwords << password
emails.zip(passwords) do |name, pass|
browser = Watir::Browser.new :ff
browser.goto "url"
#logs in and does what its suppose to do with the name and pass
browser.close
end
end
end
end
EDIT:
Alternatively, if you want the same exact Watir::Browser instance to be doing all the work, initialize and close outside of your main loop. Right now, you're spawning a new Browser instance with every iteration of emails.zip, times every iteration of email.each, times the 5 iterations of your while loop. This is just ungainly, and may be screwing up your expected results. So just doing:
browser = Watir::Browser.new :ff
5.times do
... loop code ...
end
browser.close
Will at least make whatever's happening under the hood clearer.
I am using webdriver with FF38, but leaving the browser open window after my script is done. I find that dialogs no longer open in that window, if I continue after the testing.
The script is meant to automate forms input rather than doing it by hand, but the website does use dialog boxes to express choices -- (for example, deleting data that the script has just entered, so that I can rerun the script without overwriting information)
Is there a way to disconnect the webdriver dialog handling after I'm done?
I'm feeling a little foolish, but my searches haven't born fruit, so I may be using the wrong words in my search, given my newness to ruby and webdriver.
Example would be this:
require "watir-webdriver"
l_Browser = Watir::Browser.new :firefox
l_Browser.goto "http://www.w3schools.com/js/tryit.asp?filename=tryjs_alert"
# Click the button that opens the dialog
l_Browser.div(:class => "container").div(:class => "iframecontainer"). \
div(:class => "iframewrapper").iframe(:id => "iframeResult"). \
button(:onclick => "myFunction()").click
The result is that a popup will appear, but no popups will appear further attempts to click on the button once the script is done.
This includes even if no popup is triggered during the script (ie:, last line commented out)... Once the script is finished running, no popups appear in a watir webdriver opened window. (They will open if I click on the button while the script is running, but not after)
Based on the answer below, I am using:
begin
b = Watir::Browser.new :firefox
File.open('d:\\MARK.TXT', 'w') {|f| f.write(YAML.dump(b)) }
# Load MessageBox and wait here
b = YAML.load(File.read('d:\\MARK.TXT'))
ensure
if !b.nil?
b.close()
end
end
... but it currently allows for errors that can be ignored... I just don't know how wise it is to ignore them in the long run:
D:/Ruby193/lib/ruby/gems/1.9.1/gems/childprocess-0.5.6/lib/childprocess/windows/handle.rb:50:in `exit_code': The handle is invalid. (6) (ChildProcess::Error)
from D:/Ruby193/lib/ruby/gems/1.9.1/gems/childprocess-0.5.6/lib/childprocess/windows/process.rb:41:in `exited?'
from D:/Ruby193/lib/ruby/gems/1.9.1/gems/childprocess-0.5.6/lib/childprocess/abstract_process.rb:147:in `poll_for_exit'
from D:/Ruby193/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.46.2/lib/selenium/webdriver/firefox/binary.rb:59:in `quit'
from D:/Ruby193/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.46.2/lib/selenium/webdriver/firefox/launcher.rb:62:in `quit'
from D:/Ruby193/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.46.2/lib/selenium/webdriver/firefox/bridge.rb:75:in `quit'
from D:/Ruby193/lib/ruby/gems/1.9.1/gems/selenium-webdriver-2.46.2/lib/selenium/webdriver/common/driver.rb:165:in `quit'
from D:/Ruby193/lib/ruby/gems/1.9.1/gems/watir-webdriver-0.7.0/lib/watir-webdriver/browser.rb:136:in `close'
from D:/Users/risendevil/Documents/Aptana Studio 3 Workspace/Ruby Test/Default.rb:19:in `<main>'
Versions:
Firefox 38.0.5
selenium (0.2.11)
selenium-webdriver (2.46.2, 2.45.0)
watir-webdriver (0.7.0)
I learned something new answering your question: Turning an object into text is called serialization. Turning text into an object is called deserialization.
And here's a gist of you want to do, specifically.
The important part is
my_object = SomeObject.new
my_object.some_method # => returns your expected result
File.open('path/to/some.file', 'w') {|f| f.write(YAML.dump(my_object)) }
# Do whatever you want
my_object_reloaded = YAML.load(File.read('path/to/some.file'))
my_object_reloaded.some_method # => returns your expected result
You could even do this directly to your browser:
b = Watir::Browser.new
b.goto 'http://google.com' # => goes to Google
File.open('path/to/some.file', 'w') {|f| f.write(YAML.dump(b)) }
b = nil
# Do whatever you want, wait as long as you want.
# (Disclaimer: There are probably major limitations to 'as long as you want'.)
b = YAML.load(File.read('path/to/some.file'))
b.goto 'http://yahoo.com' # => goes to Yahoo
require "watir-webdriver"
l_Browser = Watir::Browser.new :firefox
l_Browser.goto "http://www.w3schools.com/js/tryit.asp?filename=tryjs_alert"
l_Browser.iframe(:id => 'iframeResult').button(:xpath => "//button[text()='Try it']").when_present.click # click on "Try it" button
l_Browser.alert.close # closing popup
I'm trying to make a loop so that during each loop it will take the name and password variables from the file and enter where called.
array = []
File.open("file_users.txt") do |login|
login.each do |item|
name, password = item.chomp.split(',')
array << "#{name}" "#{password}"
browser.goto "https://website.com"
browser.text_field(:id => "user_name").set "#{name}"
browser.text_field(:id => "user_password").set "#{password}"
browser.button(:id => "login").click
sleep(5)
browser.close
end
end
I think the main issue is trying to make the loop call the next set of email and password after using the previous ones.
*edited:
The result I'm trying to get is to pull text from a file, then give it a "name" and "password" value, then have it be entered into the text field on the browser when called...
for example, the text file looks like:
jerryname
jerrypassword
careyname
careypassword
britneyname
britneypassword
The result I want is:
#=> loop 1
puts jerrynamme
puts jerrypassword
#=> logs in
#=> waits, then closes browser
#=> loop 2
puts careyname
puts careypassword
#=> logs in
#=> waits, then closes browser... and so on.
The result I get is the browser opening and the name first being entered then the code just stops....the browser doesn't close, it just remains still.
You say:
the text file looks like:
jerryname jerrypassword careyname careypassword britneyname britneypassword
If the file_users.txt file is as you describe, one line with a list of names and password pairs separated by spaces, the line
File.open("file_users.txt") do |login|
will return the entire contents of the file in first login value. The code
name, password = item.chomp.split(',')
will assign the entire contents of the file to name and set password to nil
You need to build a new "file_users.txt" file in the following form (note the commas and line breaks):
jerryname,jerrypassword
careyname,careypassword
britneyname,britneypassword
Then your code will be closer to working.
PS, this line doesn't seem to be used for anything and can be removed.
array << "#{name}" "#{password}"
I'm trying to automate the process of searching for alternative telephone numbers using SayNoTo0870 . Every time one searches for an alternate number or name it brings up the '/companysearch.php' page.
Clearly this page has no reference, and in my mind you can't just link to this page.
What I'm hoping to do is use the code below, to automate the opening of a browser, searching of a name/number, stripping out the HTML and then providing the top 5 results. I've got the automation part down, but clearly when trying to save the webpage using Hpricot it only brings up the 'Sorry nothing can be found page' because I can't link directly to the search result page.
Here is my code thus far:
(I've removed comments to shorten it)
require 'rubygems'
require 'watir'
require 'hpricot'
require 'open-uri'
class OH870
def searchName(name)
browser = Watir::Browser.new
browser.goto 'http://www.saynoto0870.com/search.php'
browser.text_field(:name => 'search_name').set name
browser.button(:name => 'submit').click
end
def searchNumber(number)
browser = Watir::Browser.new
browser.goto 'http://www.saynoto0870.com/search.php'
browser.text_field(:name => 'number').set number
browser.button(:name => 'submit').click
end
def loadNew(website)
doc = Hpricot(open(website))
puts(doc)
end
def strip_tags
stripped = website.gsub( %r{</?[^>]+?>}, '' )
puts stripped
end
end # class
class Main < OH870
puts "What is the name of the place you want?"
website = 'http://www.saynoto0870.com/companysearch.php'
question = gets.chomp
whichNumber = OH870.new
whichNumber.searchName(question)
#result = OH870.new
#withoutTags = website.strip_tags
#result.loadNew(withoutTags)
end
Now I'm not sure whether there's a way of "asking watir to follow through to the companysearch.php page and dump the results without having to pass this page as a variable.
I wonder if anyone has any suggestions here?
With WATIR, minus the extraneous libraries, here's all it takes to accomplish what you've described (using the 'name' test case only). I've pulled it out of the function format since you already know how to do that, and this will be a clearer test case path.
require 'watir'
#browser = Watir::Browser.new :firefox #open a browser called #browser
#browser.goto "http://(your search page here)" #go to the search page
#browser.text_field(:name => 'name').value = "Awesome" #fill in the 'name' field
#browser.button(:name => 'submit').click #submit the form
If all goes well, we should now be looking at the search results. WATIR already knows it's on a new page - we don't have to specify a URL. In the case that the results are in a frame, we do need to access that frame before we can view its content. Let's pretend they're in a DIV element with an ID of "search_results":
results = #browser.div(:id => "search_results").text
resultsFrame = #browser.frame(:index => 1) #in the case of a frame
results = resultsFrame.div(id => "search_results).text
As you can see, you do not need to save the entire page to parse the results. They could be in table cells, they could be in a different div per line, or a new frame. All are easily accessible with WATIR to be stored in a variable, array, or immediately written to the console or log file.
#results = Array.new #create an Array to store our results
#browser.divs.each do |div| #for each div element on the page
if div.id == "search_results" #if the div ID equals "search_results"
#results << div.text #add it to our array named #results
end
end
Now, if you just wanted the top 5 there are many ways to access them.
#results[0] #first element
#results[0..4] #first 5 elements
I'd also suggest you look into a few programming principles like DRY (Don't Repeat Yourself). In your function definitions where you see that they share code, like opening the browser and visiting the same URL - you can consolidate those:
def search(how, what)
#browser = Watir::Browser.new :firefox
#browser.goto "(that search url again)"
#browser.text_field(:name => how).value = what
etc...
end
search("name", "Hilton")
search("number", "555555")
Since we know that the two available text_field names are "name" and "number", and those make good logical sense as a 'how', we can parameterize them and use a single function for both the Search by Name and Search by Number test cases. This is more efficient, as long as the test cases remain similar enough to be shared.