I was following this tutorial for screen scraping with Ruby and Watir.
I tried to write a simple script to return text from Wikipedia:
require "selenium-webdriver"
browser = Selenium::WebDriver.for :chrome
browser.get "https://wikipedia.org"
require "nokogiri"
puts doc.xpath(".//*[#id='langsearch-input']/p").inner_text
But when I run the script, I get this error in my terminal:
$ ruby app/views/layouts/scraper.rb
app/views/layouts/scraper.rb:7:in `<main>': undefined local variable or method `doc' for main:Object (NameError)
I have nokogiri 1.6.7.2, watir-webdriver 0.9.1, and watir 4.0.2 installed.
What am I doing wrong?
You are missing a line to that converts the browser HTML into a Nokogiri document. In other words, you have not defined what doc is.
require "selenium-webdriver"
browser = Selenium::WebDriver.for :chrome
browser.get "https://wikipedia.org"
require "nokogiri"
doc = Nokogiri::HTML.parse(browser.page_source)
puts doc.xpath(".//*[#id='langsearch-input']/p").inner_text
#=> ""
Note that while this will address the exception, the inner_text will return an empty string - ie "". The element with id "langsearch-input" is an input field, which dos not have a child p element or a text node.
Also note that you are not actually using Watir at all. To use Watir, it would look like:
require 'watir-webdriver'
browser = Watir::Browser.new :chrome
browser.goto "https://wikipedia.org"
require 'nokogiri'
doc = Nokogiri::HTML.parse(browser.html)
puts doc.xpath(".//*[#id='langsearch-input']/p").inner_text
#=> ""
However, unless you are doing a lot of parsing of a single large HTML chunk, using Watir without Nokogiri might be easier:
require 'watir-webdriver'
browser = Watir::Browser.new :chrome
browser.goto "https://wikipedia.org"
puts browser.text_field(id: 'langsearch-input').value
Related
With the function below, I can receive a print of the last execution of the test, however I want to learn how to receive a print at every step executed by the automation.
How can this be done?
env.rb
# encoding: utf-8
require 'watir'
require 'rspec'
hooks.rb
# coding: utf-8
require 'json'
require 'magic_encoding'
require 'win32console'
require 'watir'
require 'rspec'
browser = Watir::Browser.new
browser.driver.manage.window.maximize
Before do
#browser
#browser = browser
end
After do |_scenario|
browser.screenshot.save 'screenshot.png'
embed 'screenshot.png', 'image/png'
end
login.rb
given("que estou na tela de login") do
#browser.goto "url"
#I want a screenshot of this step
end
There is an AfterStep hook if you want to do an action after each step - eg:
AfterStep do
browser.screenshot.save 'screenshot.png'
end
I am getting the following error when I execute the below code.
Please let me know the way to execute the method 'startup' in Eclipse.
Error Message:
': undefined local variable or method ***startup***' for PageObjects:Class (NameError) from C:/Technical/RubyTraining/PageObjects.rb:1:in'
Code:
class PageObjects
require 'selenium-webdriver'
require 'page-object'
def startup
#browser = Selenium::WebDriver.for :firefox
#browser.manage.window.maximize
end
startup
end
From the code you have provided, you don't need the class PageObjects (If you have more code, please post them all at once). So you might simply try this:
require 'selenium-webdriver'
# require 'page-object' # You are not using page-object either.
def startup
#browser = Selenium::WebDriver.for :firefox
#browser.manage.window.maximize
end
startup
However, if you want to use this class, you need to call this instance method setup outside your class after initializing the class. Like this:
class PageObjects
require 'selenium-webdriver'
require 'page-object'
def startup
#browser = Selenium::WebDriver.for :firefox
#browser.manage.window.maximize
end
end
PageObjects.new.startup
It appears I'm retracing the steps taken in the SO post: Capybara, Poltergeist and Phantomjs and giving an empty response in body. (Mark this as a duplicate if you want, but I'm including a minimal standalone test case and version numbers.)
questions
Am I doing anything obviously wrong? Is there another minimal test I can run that might help isolate the problem?
file: pgtest.rb
require 'rubygems'
require 'capybara'
require 'capybara/dsl'
require 'capybara/poltergeist'
module PGTest
include Capybara::DSL
extend self
def test
Capybara.register_driver :poltergeist do |app|
Capybara::Poltergeist::Driver.new(app)
end
Capybara.current_driver = :poltergeist
session = Capybara::Session.new(:poltergeist)
visit "http://www.google.com"
sleep 5
puts session.html
end
end
PGTest.test
When invoked as follows, it prints an empty page:
$ ruby pgtest.rb
<html><head></head><body></body></html>
environment
OS X 10.8.5 (12F45)
ruby 2.0.0p247 (2013-06-27) [x86_64-darwin12.4.0]
phantomjs 1.9.2
capybara (2.1.0)
poltergeist (1.4.1)
absolving phantomjs
It's worth noting that I can use phantomjs extract the html from www.google.com:
file: pjs_dump.js
var page = require('webpage').create();
page.open("http://www.google.com", function () {
var html = page.evaluate(function () {
return document.documentElement.outerHTML;
});
console.log(html);
phantom.exit();
});
When I run 'phantomjs pjs_dump.js', it prints the html from www.google.com, so phantomjs appears to be working properly.
This does the trick for me:
require 'rubygems'
require 'capybara'
require 'capybara/dsl'
require 'capybara/poltergeist'
Capybara.run_server = false
Capybara.current_driver = :poltergeist
class PGTest
include Capybara::DSL
def test
visit "http://www.google.com"
puts page.body
end
end
PGTest.new.test
I had similar issue when testing rails application with rspec, capybara and poltergeist. Spec failed when trying to find element when the body was empty.
It turned out, that the problem was in my rails code. It caused an error, the page did not render, and capybara received an empty page. And I had nothing in stack trace.
I get this error on running my feature file.
Unable to pick a platform for the provided browser (RuntimeError)
Help required, please.
Here is the code;
class GooglePage
include PageObject
def self.visitor
visit("http://www.google.com")
end
end
env.rb
require 'selenium-webdriver'
require 'page-object'
require 'rubygems'
require 'page-object/page_factory'
World (PageObject::PageFactory)
#browser = Selenium::WebDriver.for :firefox
Step-Definitions
require_relative 'GooglePage'
Given(/^I am on the Google home page$/) do
visit(GooglePage)
# visit('http://www.google.com')
on(GooglePage).visitor
end
This won't work:
visit(GooglePage)
because you haven't called page_url in GooglePage (ln 4 below)
class GooglePage
include PageObject
page_url "http://www.google.com"
def self.visitor
visit("http://www.google.com")
end
end
Move the line #browser = Selenium::WebDriver.for :firefox to the Before method in hooks.rb
Before do
#browser = Selenium::WebDriver.for :firefox
end
After do
#browser.close
end
what if you make your rake file pass on tags and BROWSER and URL,e.g.
rake my_task BROWSER=chrome URL=http://google.com.au
and hooks will check if the BROWSER is chrome, then will use the specific Webdriver for chrome. But how to pass the URL in the PageObject?
Currently from hooks I have the ff:
when "chrome" then
caps = Selenium::WebDriver::Remote::Capabilities.chrome
caps.version = "40.0.2214.115 m"
caps.native_events = false
caps.javascript_enabled= true
# This is for increasing the default timeout to 180
client = Selenium::WebDriver::Remote::Http::Default.new
client.timeout = 180
browser = Selenium::WebDriver.for :chrome
browser.manage.window.maximize
end
Before do
browser.cookies.clear
#browser = browser
end
After do
unless ENV["BROWSER"].eql? "ie"
browser.close
sleep 2
end
end
Then on my HomePage pageobject I have the ff:
class HomePage
include PageObject
page_url("#{ENV['URL']}")
#opens the url defined in ENV['URL']
def goto_homepage
visit(HomePage)
end
end
Using Watir, the URL got opened, so trying this out in Selenium and it won't work.
Can any one have more descriptive example of Rspec? I used the the example from site https://gist.github.com/1053934 works very fine. But i want to learn more like go to site X (eg. Google) -> enter data in text field (eg. Ruby)
Check for the HTML is Ruby available in the list?
Can any one have script that i can run on Rspec and that will generate report on that pass fail status of case.
I appreciate all the responses in advance.
Here's a complete example, courtesy of Watir docs.
require "rubygems"
require "rspec"
describe "google.com" do
let(:browser) { #browser ||= Watir::Browser.new :firefox }
before { browser.goto "http://google.com" }
after { browser.close }
it "should search for watir" do
browser.text_field(:name => "q").set "watir"
browser.button.click
browser.div(:id => "resultStats").wait_until_present
browser.title.should == "watir - Google Search"
end
end