I'm trying to write Rspec tests for my Mechanize agent.
My agent is supposed to go to a website, log into the form, then scrape some data off the website. I also downloaded FakeWeb to stub the HTTP requests, and make my tests faster.
Here is my account_spec.spec file:
require 'spec_helper'
describe Account do
before(:each) { #account = Account.new('bob', '1234') }
describe '#login' do
before(:each) do
home_page = File.read('spec/html/home_page.html')
login_page = File.read('spec/html/login_page.html')
FakeWeb.register_uri(:get,
"https://www.example.com/",
body: home_page,
status: ["200", "Success"],
content_type: "text/html")
FakeWeb.register_uri(:get,
"https://www.example.com/account/login",
body: login_page,
status: ["200", "Success"],
content_type: "text/html")
#web_crawler = Mechanize.new
#home_page = #web_crawler.get("https://www.example.com/")
#login_page = #web_crawler.get("https://www.example.com/account/login")
end # -- before :each
it 'finds the login form' do
login_form = #login_page.form_with(:class => "form login")
puts login_form.class # ==> nil:NilClass
end
end # -- #login
end # -- Account
However, when I comment out the FakeWeb uri for example/account/login (it then accesses the real server), it actually returns the correct form. Basically, if I am searching for the form in my locally saved HTML file, Mechanize can not find it, but if I check the actual website, it does find it. I would like to know if there is a way around this, and why this happens.
Any help would be greatly appreciated.
Related
I am using the HTTParty gem to make a call to the GitHub API to access a list of user's repos.
It is a very simple application using Sinatra that displays a user's favourite programming language based on the most common language that appears in their repos.
I am a bit stuck on how I can write an RSpec expectation that mocks out the actual API call and instead just checks that json data is being returned.
I have a mock .json file but not sure how to use it in my test.
Any ideas?
github_api.rb
require 'httparty'
class GithubApi
attr_reader :username, :data, :languages
def initialize(username)
#username = username
#response = HTTParty.get("https://api.github.com/users/#{#username}/repos")
#data = JSON.parse(#response.body)
end
end
github_api_spec.rb
require './app/models/github_api'
require 'spec_helper'
describe GithubApi do
let(:github_api) { GithubApi.new('mock_user') }
it "receives a json response" do
end
end
Rest of the files for clarity:
results.rb
require 'httparty'
require_relative 'github_api'
class Results
def initialize(github_api = Github.new(username))
#github_api = github_api
#languages = []
end
def get_languages
#github_api.data.each do |repo|
#languages << repo["language"]
end
end
def favourite_language
get_languages
#languages.group_by(&:itself).values.max_by(&:size).first
end
end
application_controller.rb
require './config/environment'
require 'sinatra/base'
require './app/models/github_api'
class ApplicationController < Sinatra::Base
configure do
enable :sessions
set :session_secret, "#3x!ilt£"
set :views, 'app/views'
end
get "/" do
erb :index
end
post "/user" do
#github = GithubApi.new(params[:username])
#results = Results.new(#github)
#language = #results.favourite_language
session[:language] = #language
session[:username] = params[:username]
redirect '/results'
end
get "/results" do
#language = session[:language]
#username = session[:username]
erb :results
end
run! if app_file == $0
end
There are multiple ways you could approach this problem.
You could, as #anil suggested, use a library like webmock to mock the underlying HTTP call. You could also do something similar with VCR (https://github.com/vcr/vcr) which records the results of an actual call to the HTTP endpoint and plays back that response on subsequent requests.
But, given your question, I don't see why you couldn't just use an Rspec double. I'll show you how below. But, first, it would be a bit easier to test the code if it were not all in the constructor.
github_api.rb
require 'httparty'
class GithubApi
attr_reader :username
def initialize(username)
#username = username
end
def favorite_language
# method to calculate which language is used most by username
end
def languages
# method to grab languages from repos
end
def repos
repos ||= do
response = HTTParty.get("https://api.github.com/users/#{username}/repos")
JSON.parse(response.body)
end
end
end
Note that you do not need to reference the #username variable in the url because you have an attr_reader.
github_api_spec.rb
require './app/models/github_api'
require 'spec_helper'
describe GithubApi do
subject(:api) { described_class.new(username) }
let(:username) { 'username' }
describe '#repos' do
let(:github_url) { "https://api.github.com/users/#{username}/repos" }
let(:github_response) { instance_double(HTTParty::Response, body: github_response_body) }
let(:github_response_body) { 'response_body' }
before do
allow(HTTParty).to receive(:get).and_return(github_response)
allow(JSON).to receive(:parse)
api.repos
end
it 'fetches the repos from Github api' do
expect(HTTParty).to have_received(:get).with(github_url)
end
it 'parses the Github response' do
expect(JSON).to have_received(:parse).with(github_response_body)
end
end
end
Note that there is no need to actually load or parse any real JSON. What we're testing here is that we made the correct HTTP call and that we called JSON.parse on the response. Once you start testing the languages method you'd need to actually load and parse your test file, like this:
let(:parsed_response) { JSON.parse(File.read('path/to/test/file.json')) }
You can mock those API calls using https://github.com/bblimke/webmock and send back mock.json using webmock. This post, https://robots.thoughtbot.com/how-to-stub-external-services-in-tests walks you through the setup of webmock with RSpec (the tests in the post mock GitHub API call too)
I have some experience of Selenium in Python and Cucumber/Watir/RSpec in Ruby, and can write scripts that execute successfully, but they aren't using classes, so I am trying to learn more about classes and splitting the scripts up in to pageobejcts.
I found this example to learn from: http://watir.com/guides/page-objects/ so copied the script and made some minor edits as you'll see below.
I'm using SublimeText 3.x with Ruby 2.4.x on Win10, so you know what tools I'm using.
I put the whole script in to a single .rb file (the only differences are that I replaced the URL and the elements to enter the username and password) and tried to execute it and get the following error:
C:/selenium/ruby/lotw/lotwlogin.rb:3:in `<main>': uninitialized constant Site (NameError).
I added the top line (required 'watir') line and it made no difference to the error encountered.
So I have in lotwlogin.rb essentilly the structure and syntax of the original script with custom elements. However, the core structure is reporting an error and I don't know what to do about it.
Here is my script:
require 'watir'
site = Site.new(Watir::Browser.new :chrome) # was :firefox but that no longer works since FF63
login_page = site.login_page.open
user_page = login_page.login_as "testuser", "testpassword" # dummy user and password for now
user_page.should be_logged_in
class BrowserContainer
def initialize(browser)
#browser = browser
end
end
class Site < BrowserContainer
def login_page
#login_page = LoginPage.new(#browser)
end
def user_page
#user_page = UserPage.new(#browser)
end
def close
#browser.close
end
end
class LoginPage < BrowserContainer
URL = "https://lotw.arrl.org/lotw/login"
def open
#browser.goto URL
##browser.window.maximize
self # no idea what this is for
end
def login_as(user, pass)
user_field.set user
password_field.set pass
login_button.click
next_page = UserPage.new(#browser)
Watir::Wait.until { next_page.loaded? }
next_page
end
private
def user_field
#browser.text_field(:name => "login")
end
def password_field
#browser.text_field(:name => "password")
end
def login_button
#browser.button(:value => "Log On")
end
end # LoginPage
class UserPage < BrowserContainer
def logged_in?
logged_in_element.exists?
end
def loaded?
#browser.h3 == "Welcome to Your Logbook of the World User Account Home Page"
end
private
def logged_in_element
#browser.div(:text => "Log off")
end
end # UserPage
Any assistance how to not get the Site error would be appreciated.
Thanks
Mike
You define class Site only a few lines below. But at that point, it's not yet known.
Move this logic to after all class definitions:
site = Site.new(Watir::Browser.new :chrome) # was :firefox but that no longer works since FF63
login_page = site.login_page.open
user_page = login_page.login_as "testuser", "testpassword" # dummy user and password for now
user_page.should be_logged_in
I want to implement Facebook login for web apps. All I need is the basic public information of a user for the account creation steps.
This is what I have done:
Created a basic Facebook app with nearly no custom permissions.
Used the APP_ID and APP_SECRET in Koala to get access_token.
Everything worked perfectly, I am able to login/logout.
Just that the only information I am able to get back when I do: graph.get_object('me') is the logged in user's name and an id (It doesn't look like the default Facebook id).
Surprised whether something changed in the new API, I tested the gem in the console using the access_token from graph explorer (where all permissions are enabled by default). And I get all data using the same method call.
When I review what all the app gets while signing up; I see that the user's basic information, profile pic and other public data will be accessible to the app.
Any idea why this is so? It seems I am missing something obvious. The code is available in Github. But this is pretty much everything to it:
require 'bundler'
Bundler.require :default
Dotenv.load '.env'
require_relative './app/constants.rb'
module Banana
class App < Sinatra::Base
use Rack::Session::Cookie, secret: COOKIE_SECRET
set :public_folder, File.dirname(__FILE__) + '/bower_components'
get '/' do
if logged_in?
haml :welcome_in, layout: :layout
else
haml :log_in, layout: :layout
end
end
get '/log_out' do
session['oauth'] = nil
session['access_token'] = nil
redirect '/'
end
get '/log_in' do
session['oauth'] = Koala::Facebook::OAuth.new(APP_ID, APP_SECRET, "#{request.base_url}/call_back")
redirect session['oauth'].url_for_oauth_code()
end
get '/call_back' do
begin
session['access_token'] = session['oauth'].get_access_token(params[:code])
rescue
redirect '/?error=user_denied'
end
redirect '/'
end
get '/test' do
if logged_in?
p graph.get_object("rakeshbs")
"e"
else
redirect '/'
end
end
def logged_in?
!session['access_token'].nil?
end
def toggle_access
logged_in? ? '/log_out' : '/log_in'
end
def graph
#graph ||= Koala::Facebook::API.new(session['access_token'])
end
def errored?
!params["error"].nil?
end
def user
p graph.get_connections(:me, :photos) # This is just nil
#user ||= OpenStruct.new(
name: graph.get_object("me")["name"], # All I get here is just a hash with the name and an id!
photo: 'http://semantic-ui.com/images/avatar/small/elliot.jpg'
)
end
end
end
You should add fields parameter.
Something like this:
graph.get_object('me', { fields: 'id,first_name,last_name,gender,birthday,photos,email' })
I'm developing a DSL for building API wrappers, named Hendrix. I am having problems with the testing of the DSL. As it is a API wrapper, it needs to interact with external services. I am not sure how to approach this in terms of testing. I'm using RSpec and tried configuring VCR with WebMock, but no luck. How am I supposed to test this particular scenario if I don't have direct access to what request is being made?
This is my spec_helper.rb:
$VERBOSE = nil
require 'simplecov'
require 'coveralls'
SimpleCov.formatter = SimpleCov::Formatter::MultiFormatter[
SimpleCov::Formatter::HTMLFormatter,
Coveralls::SimpleCov::Formatter
]
SimpleCov.start { add_filter '/spec/' }
lib = File.expand_path('../lib', __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
require 'hendrix'
require 'vcr'
VCR.configure do |c|
c.cassette_library_dir = 'spec/cassettes'
c.hook_into :webmock
end
RSpec.configure do |config|
config.treat_symbols_as_metadata_keys_with_true_values = true
config.run_all_when_everything_filtered = true
config.filter_run :focus
config.order = 'random'
config.extend VCR::RSpec::Macros
end
The project is in its early stages (working towards version 0.1.0 at the moment). The syntax of the DSL is as follows:
require 'hendrix'
Hendrix.build 'Jimi' do
base 'https://api.github.com'
client :issues do
action :issue, '/repos/:owner/:repo/issues/:number'
end
end
Jimi.issue('rafalchmiel', 'hendrix', 1)
# => {"url"=>"https://api.github.com/repos/rafalchmiel/hendrix/issues/1",
# "labels_url"=> ...
Jimi.issue('rafalchmiel', 'hendrix', 1).title
# => "Implement parameters in actions"
In most specs, I'm testing what the methods from the master module (in this case Jimi.issue etc) return and whether it is in a Hashie::Mash format. How would I test this? I don't know where to start.
For integration tests, I usually stub the endpoint with webmock directly, without trying to record an actual request. This means you can control the response and the expectation in the same place. You can place expectations on whether your library parses the response correctly and you can write tests that verify that the request has been made correctly. Go through each of the features of your gem to get a list of features. Here's an example:
require "webmock/rspec"
describe "parsing results" do
let(:url) { "http://..." }
it "parses results into nice methods" do
stub_request(:get, url)
.to_return(
body: { title: "implement" }.to_json,
headers: { content_type: "application/json" },
)
perform_request
expect(response.title).to eq "implement"
end
it "sends the user agent header correctly" do
stub_request(:get, url)
perform_request
expect(a_request(:get, url).with(
headers: { user_agent: "hendrix" }
)).to have_been_made.once
end
it "interpolates values in URLs"
it "supports DELETE requests"
it "supports HTTP Basic"
def perform_request
# ...
end
end
Try not to record real requests: it's hard to control the right circumstances with real web servers, especially if you're not the one who wrote the specs. Especially when you write a general purpose library like this. VCR is nice if you want to access one particular server and your code really depends on that one server.
Also don't check on types. I see that quite a lot in your gem right now. Nobody cares if you return a Hashie::Mash object. As my first spec shows, you just want to be able to access the attributes cleanly.
I have a request spec which passes up until the point where I need to check to see if content is present on the page, which I am using page.should have_content to do. The content is actually a message which appears on successful form submission, which is rendered from a messages partial. The test fails even though If I test through the browser, functionality works as expected and the content appears as it should. I'm also using FactoryGirl to generate the users to use for the form submission.
Here's the error I get after running the spec with the --format d option:
UserSignup
shows a thank you message on successful form submission (FAILED - 1)
Failures:
1) UserSignup shows a thank you message on successful form submission
Failure/Error: page.should have_content("Thank you. You will be notified of our launch at #{user.email}.")
expected #has_content?("Thank you. You will be notified of our launch at quinn.purdy#casperzboncak.org.") to return true, got false
# ./spec/requests/user_signup_spec.rb:21:in `block (2 levels) in <top (required)>'
user_signup_spec.rb:
require 'spec_helper'
describe "UserSignup" do
it "shows a thank you message on successful form submission" do
user = FactoryGirl.create(:user)
visit sign_up_path
fill_in "user_fullname", with: user.fullname
fill_in "user_email", with: user.email
click_button "Sign up"
current_path.should eq(sign_up_path)
page.should have_content("Thank you. You will be notified of our launch at #{user.email}.")
end
end
users_controller.rb:
class UsersController < ApplicationController
def new
#user = User.new
end
def create
#user = User.new(secure_params)
if #user.valid?
#user.subscribe
flash[:notice] = "Thank you. You will be notified of our launch at #{#user.email}."
redirect_to sign_up_path
else
render :new
end
end
private
def secure_params
params.require(:user).permit(:fullname, :email)
end
end
I'm wondering if it could be because I render the messages partial from the application layout, but when it gets outputted in the users view, the message appears inside the body of the source, but outside the main class.
So I seem to have got the tests passing by adding the line , :js => true do beside 'it' and using the selenium web driver. There's got to be a way to do it without selenium I'm thinking, because you have to sit and wait while it actually runs it in a browser which is the downside.
Maybe I'm going about it the wrong way, and I should actually be checking for the partial in a view spec (currently it was just part of the feature test).