Until recently, I was using mechanize to get a web page, then do some parsing with nokogiri. But because some content was loaded with Ajax after I have started using Watir instead. My code looks like this:
def get_page(url)
browser = Watir::Browser.start url
sleep 1
page = Nokogiri::HTML.parse(browser.html)
browser.close
return page
end
It works fine but since I am getting a lot of pages the browser.start will open a tons of windows. I found the close as you see, but is there a way to just not show the browser window at all?
Related
Using Ruby and Watir, can I save a web page the same way as doing a right mouse-click and " save page with name "?
I need to save the current web page from a script.
Yes, you can do it with watir. Just open a page and save the browser.html to any destination you want:
b = Watir::Browser.new :phantomjs # I am using phantomjs for scripted browsing
b.goto 'http://google.com'
File.open('/tmp/google', 'w') {|f| f.write b.html }
I don't know about watir, but I know the way to do it using Selenium Web Driver is using the page source method.
Check out the docs for that here:
http://selenium.googlecode.com/git/docs/api/rb/Selenium/WebDriver/Driver.html#page_source-instance_method
using this, you should get the whole source.
You can now save the source, by just creating a new file. I haven't tried this but you can check it out.
driver = Selenium::WebDriver.for(:firefox)
driver.get(URL_of_page_to_save)
file = File.new(filename, "w")
file.puts(driver.page_source)
file.close
Not sure if this saves all elements of the page.
Hope this helped a bit!
Following is the code i am using
def self.yes_publish
sleep 5
driver.find_element(:id, 'dialogConfirmChanges-publishButton').displayed?
WAIT.until { driver.find_element(:id, 'dialogConfirmChanges-publishButton') }.click
puts driver.find_element(:id, 'embed-left-center-part').displayed?
end
But i am unable to click on it. This id works fine in irb.
I get an error modal dialog present, as webdriver is unable to locate element it closes to window after a particular timeout.
This popup is to publish changes made on a page.
xpath = .//*[#id='dialogConfirmChanges-publishButton']
You have to use the switch_to method to deal with the pop-up. Look at the documentation of JavaScript dialogs :
You can use webdriver to handle Javascript alert(), prompt() and confirm() dialogs. The API for all three is the same.
Note: At this time alert handling is only available in Firefox and IE (or in those browsers through the remote server), and only alerts that are generated post onload can be captured.
require "selenium-webdriver"
driver = Selenium::WebDriver.for :firefox
driver.navigate.to "http://mysite.com/page_with_alert.html"
driver.find_element(:name, 'element_with_alert_javascript').click
a = driver.switch_to.alert
if a.text == 'A value you are looking for'
a.dismiss
else
a.accept
end
EDIT
As per the comment box HTML given by you..I think below should work :
driver.find_element(:xpath,"//div[#class='ui-dialog-buttonset']/button[#id='dialogConfirmChanges-publishButton']").click
I want to click on browser "stop loading this page" icon, when timeout error occurs,what should i do for it? Some thing like this:
browser.stop # this is wrong.
I searched forever and the best I could come up with was:
b = Watir::Browser.new
b.driver.manage.timeouts.implicit_wait = 3
This worked great for me with firefox.
The below link may help you :
Stop loading page watir-webdriver
Is there have method to stop page loading in watir-webdriver
Just try this logic when you want to stop the loading,
#browser.send_keys :escape
It is not the best way of doing what you want but still the above approach worked out for me.
If you use Watir-Webdriver then you can try this for IE
#browser.execute_script "document.execCommand('Stop')"
I have a script that visits fcc.gov, then clicks a link which triggers a download:
require "mechanize"
docket_number = "12-268" #"96-128"
url = "http://apps.fcc.gov/ecfs/comment_search/execute?proceeding=#{docket_number}"
agent = Mechanize.new
agent.pluggable_parser.default = Mechanize::DirectorySaver.save_to 'downloads'
agent.get(url) do |page|
link = page.link_with(:text => "Export to Excel file")
xls = agent.click(link)
end
This works fine when docket_number is "12-268". But when you change it to "96-128", Mechanize downloads the html of the page instead of the desired spreadsheet.
The urls for both pages are:
http://apps.fcc.gov/ecfs/comment_search/execute?proceeding=12-268 (works)
http://apps.fcc.gov/ecfs/comment_search/execute?proceeding=96-128 (this is where I need help)
As you can see, if you visit each page in a browser (I'm using Chrome) and click "Export to Excel file", a spreadsheet file is downloaded and there is not problem. "96-128" has many more rows, so when you click on the Export link, it takes you to a new page that refreshes every 10 seconds or so until the file begins downloading. How can I get around this and why is there this inconsistency?
Clicking Export on 96-128 takes you to a page that refreshes using this kind of a tag (I've never heard of it before):
<meta http-equiv="refresh" content="5;url=/ecfs/comment_search/export?exportType=xls"/>
By default, Mechanize will not follow these refreshes. To get around that, change a setting on agent:
agent.follow_meta_refresh = true
Source: https://stackoverflow.com/a/2166480/94154
The proceeding 12-268 has 48 entries, 96-128 has 4046.
When I click at 'Export to Excel File' on the latter, there sometimes is a page saying:
Finished processing 933 of 4046 records.
Click if this page does not reload automatically.
I guess mechanize is seeing this, too.
Experiencing a problem when running watir-web driver to test Ajax responses from our site. The site is designed to contain a list of items. When a new item is added the site will submit the entry via Ajax and wait for a response to tell the site to display this new entry. We are finding these Ajax responses are coming back blank whilst running the page via watir in both chrome and Firefox. However when trying the scenario out side of watir the Ajax responses contain data.
My question is are we over looking something when setting up our browsers?
Additional info using latest versions of gems but testing on Ubuntu 12.04.
here is the code to set-up the two browsers:
download_directory = "#{Dir.pwd}/Results" #download location (FF only)
profileFF = Selenium::WebDriver::Firefox::Profile.new
profileFF['browser.download.folderList'] = 2 # custom location
profileFF['browser.download.dir'] = download_directory
profileFF['browser.helperApps.neverAsk.saveToDisk'] = "application/msword" #auto save rulw
c = Watir::Browser.new :chrome
f = Watir::Browser.new :firefox ,:profile => profileFF