I'm using mechanize to fill out a form, but I want to review it on the webpage before submission. The goal is to open a browser with the pre-filled form.
require 'mechanize'
mechanize = Mechanize.new
page = mechanize.get('https://www.linuxtoday.com/contribute.html')
form = page.form_with :name => 'upload'
form.sub_name = "mbb"
form.email = "mbb#mbb.com"
form.author_name = "Mr Potatohead"
form.title = "Mr Potatohead writes Ruby"
form.link = "https://potato.potato"
form.body = "More text"
`open #{page.uri}`
Calling out to the operating system to open the site is, of course, empty form. I don't see a page.open or similar method available. Is there a way to achieve this (using mechanize or other gems)?
That won't work because setting form fields doesn't even update the DOM.
If you want to review the form data you can inspect form.request_data
As others have mentioned in the comments try selenium, you'll need chrome or firefox driver installed, here's example with chrome to get you started:
require 'selenium-webdriver'
require 'pry' # optional
driver = Selenium::WebDriver.for :chrome
driver.navigate.to 'https://www.linuxtoday.com/contribute.html'
form = driver.find_element(id: 'upload')
form.find_element(id: 'sub_name').send_keys 'mbb'
form.find_element(id: 'email').send_keys 'mbb#mbb.com'
binding.pry # or sleep 60
driver.quit
For more instructions see documentation
I'm using Nokogiri and Selenium Webdriver to parse the full contents of a class on a web page:
require "selenium-webdriver"
require "nokogiri"
browser = Selenium::WebDriver.for :chrome
browser.get "https://jsfiddle.net"
doc = Nokogiri::HTML.parse(browser.page_source)
puts doc.xpath(".//*[#class='aiButton']").inner_text
There are elements with class aiButton, containing text like "Run", "Save", "Update", etc. This is the result in my terminal:
> RunSaveUpdateForkTidyCollaborateSign in
Is there a way I can print the contents but include a delimiter, like a space or comma or something, between the elements? My preferred output would be:
> Run Save Update Fork Tidy Collaborate Sign in
You can use join like so:
doc.css('.aiButton').map(&:text).join(' ')
I'm trying to insert the html generated from phantom js into a mechanize object so that I can easily search it. I've tried the following to no avail...
b = Watir::Browser.new :phantomjs
url = "www.google.com"
b.goto url
agent = Mechanize.new
#Following is not executed at same time...
#Error 1: lots of errors
page = agent.get(b.html)
#Error 2: `parse': wrong number of arguments (1 for 3) (ArgumentError)
page = agent.parse(b.html)
#Error 3 last ditch effort: undefined method `agent'
page = agent(b.html)
As I think it through I'm beginning to wonder if I can mechanize an existing html object... I initially got onto it via: http://shane.in/2014/01/headless-web-scraping/ & http://watirmelon.com/2013/02/05/watir-webdriver-with-ghostdriver-on-osx-headless-browser-testing/
I was in the same situation. I write a lot of code with Mechanize so that I do not want to move to nokogiri when using watir. Below code is how I did.
require 'watir'
require 'mechanize'
b = Watir::Browser.new
b.goto(url)
html = b.html
a = Mechanize.new
page = Mechanize::Page.new(nil, {'content-type'=>'text/html'}, html, nil, a)
You could use page to search for elements.
require 'watir'
require 'nokogiri'
b = Watir::Browser.new :phantomjs
url = "http://google.com"
b.goto url
p Nokogiri::HTML(b.html)
You are probably better off just using Nokogiri for this [that is, if you only need to search for some data in source].
My goal is to dynamically get website content created by Javascript. I have the following code:
browser = Selenium::WebDriver.for :firefox
browser.get "https://gls-group.eu/AT/de/paket-verfolgen?match=00000000000"
wait = Selenium::WebDriver::Wait.new(:timeout => 20)
js_code = "return document.getElementsByTagName('div')"
elements = browser.execute_script(js_code)
puts elements
browser.close
The output is:
#<Selenium::WebDriver::Element:0x4e4c920>
#<Selenium::WebDriver::Element:0x4e4c770>
#<Selenium::WebDriver::Element:0x4e4c230>
#<Selenium::WebDriver::Element:0x4e55650>
#<Selenium::WebDriver::Element:0x4e55848>
#<Selenium::WebDriver::Element:0x4e57e58>
#<Selenium::WebDriver::Element:0x4e57c00>
#<Selenium::WebDriver::Element:0x4e57a08>
and so on. How do I get the divs?
browser.execute_script(js_code) gives all the html elements as you asked as instances of Selenium::WebDriver::Element class. Write as below using method Selenium::WebDriver::Element#text to get the content of those div elements :
require 'selenium-webdriver'
browser = Selenium::WebDriver.for :firefox
browser.get "https://gls-group.eu/AT/de/paket-verfolgen?match=00000000000"
wait = Selenium::WebDriver::Wait.new(:timeout => 20)
js_code = "return document.getElementsByTagName('div')"
elements = browser.execute_script(js_code)
elements.each{|e| puts e.text }
I am trying to get familiar with the new ruby selenium-webdriver as it appears more intuitive mostly than the previous version of selenium and the ruby driver that went with it. Also, i had trouble getting the old selenium to work with ruby 1.9.1 in windows so I thought i'd look for an alternative.
So far i've done this with my script:
require "selenium-webdriver"
driver = Selenium::WebDriver.for :firefox
driver.get "https://example.com"
element = driver.find_element(:name, 'username')
element.send_keys "mwolfe"
element = driver.find_element(:name, 'password')
element.send_keys "mypass"
driver.find_element(:id, "sign-in-button").click
driver.find_element(:id,"menu-link-my_profile_professional_info").click
driver.find_element(:id,"add_education_btn").click
country_select = driver.find_element(:name, "address_country")
So basically I'm logging into my site and trying to add an education entry to my user profile.. I have a reference to a select box with options (in the country_select variable) and now i want to select an option with a given value.. I don't see how to do this in the new client.. The only thing I can think of doing is looping through all the options until I find the one I want, and then call execute_script:
http://selenium.googlecode.com/svn/trunk/docs/api/rb/Selenium/WebDriver/Driver.html#execute_script-class_method
method to set the selectedIndex.
Is there any other way to do this?
In the java api for selenium 2.0/webdriver here: http://seleniumhq.org/docs/09_webdriver.html
there is an example of doing this
Select select = new Select(driver.findElement(By.xpath("//select")));
select.deselectAll();
select.selectByVisibleText("Edam");
It doesn't appear that the ruby version has this feature though unless I'm missing something.
Any help would be appreciated.
Full disclosure here: I have absolutely no working knowledge of Ruby.
However, I'm pretty good with Selenium so I think I can help. I think what you're looking for is the select method. If ruby is anything like the other drivers you can use the select method to tell one of the options it is selected.
In pseudocode/java terms it would look something like this
WebElement select = driver.findElement(By.name("select"));
List<WebElement> options = select.findElements(By.tagName("option"));
for(WebElement option : options){
if(option.getText().equals("Name you want")) {
option.click();
break;
}
}
The Select object you have above is actually in a special Support package. It only exists for Java and .Net at the moment (Jan 2011)
Please note that none of the above will work anymore. Element#select and Element#toggle have been deprecated. You need to do something like:
my_select.click
my_select.find_elements( :tag_name => "option" ).find do |option|
option.text == value
end.click
I don't know what version Selenium this came about, but it looks like there is the Select class that pnewhook mentioned in Selenium 2.20
http://selenium.googlecode.com/svn-history/r15117/trunk/docs/api/rb/Selenium/WebDriver/Support/Select.html
option = Selenium::WebDriver::Support::Select.new(#driver.find_element(:xpath => "//select"))
option.select_by(:text, "Edam")
pnewhook got it but I'd like to post the ruby version here so everyone can see it:
require "selenium-webdriver"
driver = Selenium::WebDriver.for :firefox
driver.manage.timeouts.implicit_wait = 10
driver.get "https://example.com"
country_select = driver.find_element(:id=> "address_country")
options = country_select.find_elements(:tag_name=>"option")
options.each do |el|
if (el.attributes("value") == "USA")
el.click()
break
end
end
You can use XPath to avoid looping:
String nameYouWant = "Name you want";
WebElement select = driver.findElement(By.id(id));
WebElement option =
select.findElement(By.xpath("//option[contains(text(),'" + nameYouWant + "')]"));
option.click();
or
WebElement option =
select.findElement(By.xpath("//option[text()='" + nameYouWant + "']"));
require "selenium-webdriver"
webdriver = Selenium::WebDriver.for :firefox
driver.navigate.to url
dropdown = webdriver.find_element(:id,dropdownid)
return nil if dropdown.nil?
selected = dropdown.find_elements(:tag_name,"option").detect { |option| option.attribute('text').eql? value}
if selected.nil? then
puts "Can't find value in dropdown list"
else
selected.click
end
In my case this is only one working sample.
For the latest version of Webdriver (RC3) you should use "click()" instead of setSelected(). Also option.getText().equals("Name you want") should be used instead of option.getText()=="Name you want" in JAVA:
<!-- language: lang-java -->
WebElement select = driver.findElement(By.name("select"));
List<WebElement> options = select.findElements(By.tagName("option"));
for(WebElement option : options){
if(option.getText().equals("Name you want"){
option.click();
break;
}
}
Ruby Code with Example:
require "selenium-webdriver"
driver = Selenium::WebDriver.for :ie
driver.navigate.to "http://google.com"
a=driver.find_element(:link,'Advanced search')
a.click
a=driver.find_element(:name,'num')
options=a.find_elements(:tag_name=>"option")
options.each do |g|
if g.text == "20 results"
g.click
break
end
end
#SELECT FROM DROPDOWN IN RUBY USING SELENIUM WEBDRIVER
#AUTHOR:AYAN DATE:14 June 2012
require "rubygems"
require "selenium-webdriver"
begin
#driver = Selenium::WebDriver.for :firefox
#base_url = "http://www.yoururl.com"
#driver.manage.timeouts.implicit_wait = 30
#driver.get "http://www.yoursite.com"
#select Urugway as Country
Selenium::WebDriver::Support::Select.new(#driver.find_element(:id, "country")).select_by(:text, "Uruguay")
rescue Exception => e
puts e
#driver.quit
end
The easiest way I found was:
select_elem.find_element(:css, "option[value='some_value']").click