I am trying to use Watir to get the source code of Facebook after I authenticate using Watir. It gives this specific error.
/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/protocol.rb:158:in `rescue in rbuf_fill': Net::ReadTimeout (Net::ReadTimeout)
I believe that because there are too many AJAX requests in the homepage, webdriver detects it as the page is not fully loaded. So after I logged in, I did this:
p "starts"
Watir::Wait.until {
browser.div(:'class' => '_586i').exists?
}
p "finishes"
But after it prints "starts" then it gives a timeout error, and doesn't get the source code of the website.
I've been getting this error for some websites quite a lot after I try to, eg, browser.button.click that is redirecting to another page heavily loaded with Ajax. I found this:
browser.execute_script('document.getElementsByTag('button')[0].click()')
sleep 10
with adjusted sleep (or, much better, .wait_until_present) helps.
You can force the browser to wait until all ajax calls has been loaded with
sleep(1) until browser.execute_script("return jQuery.active") == 0
Related
We use Capybara along with Chrome Headless for integration testing. I'd like to write a linter, that checks some metrics on the HTML structure everytime chrome navigates to another page. Then I'd raise an error when something is against our linter.
We have some tests without javascript, and monkey patching rack-test works so far:
Capybara::RackTest::Browser.class_eval do
alias_method :process_orig, :process
def process *args
response = process_orig *args
# do some linting
response
end
end
But I haven't found a way inside Capybara and/or Chrome Headless where I could intercept a response, and check the body of the page.
Is it possible to trigger a hook, when a page changes? Or is there some kind of API Chrome provides where I could get the body of every request? Or would a proxy be a feasible solution?
This isn't possible directly with Capybara, since it doesn't actually know about page transitions/requests that happen in the browser unless they are specifically user initiated with visit.
A potential way to do what you want would be using a programmable proxy like puffing-billy to handle every request to the app under test. If you use puffing-billy you'll want to look at the pass_request feature - https://github.com/oesmith/puffing-billy#in-your-tests-capybarawatir - to forward on the initial request and then do whatever you want with the response.
I'd not tangle capybara tests with HTML linting. It may seem smart at this moment, as you get a list of URL's to check for free with each test, but:
You'd probably lint each URL few times because some tests go through it
You might get failures because HTML is not perfect, even though the feature you're testing is actually ok.
You probably have something like sitemap.xml or other sources of all available URLs. I'd use it to make a separate check, which will be simple: request the URL, lint the response. Rinse and repeat.
If still not convinced, try using page.html and do sth like
expect(page.html).to pass_linter
https://github.com/teamcapybara/capybara#debugging
You can then add it as an around hook for every type: :feature spec if you want.
EDIT: here's another workaround to have every visited path. Just parse the server log file (like this cat log/devlopment.log | grep path) to get a full list if visited paths:
method=POST path=/users/login format=html controller=SessionsController action=create
status=302 duration=256.82 view=0.00 db=52.29 location=http://0.0.0.0:3000/platform/admin/dashboard params={"utf8"=>"✓", "authenticity_token"=>"ubGnWKOq8gbUE5C/aK375QQn5DpjHarUYxHtBLglGe6Lr9Ie3O5XPq90k5gr/SZbIPoDiiasvY0mGlwhzD/MsQ==", "user"=>{"email"=>"alex-3d51048235c9d1a8#toptal.io", "password"=>"[FILTERED]", "remember_me"=>"0"}, "commit"=>"Log in"} uid=983 remote_ip=127.0.0.1 x_forwarded_for= x_request_id=
method=GET path=/admin/dashboard format=html controller=XXX action=show status=200 duration=3285.54 view=1051.32 db=2016.87 params={} uid=983 remote_ip=127.0.0.1 x_forwarded_for= x_request_id=
and use it for linting.
Here sample code:
require 'nokogiri'
require 'open-uri'
begin
doc = Nokogiri::HTML(open(url))
rescue
puts "Fehler ist aufgetretten..."
end
Some parts of the page are loaded asynchronous and i'm missing some values, which are loaded later. Is there any way to open the url, to wait 10 seconds and after that to assign it to the variable doc? Any solutions/ideas with bash/lynx/wget are welcome too :)
Unfortunately, waiting 10 seconds isn't going to work, because neither open-uri nor Nokogiri will execute the javascript that loads content asynchronously. You'll need to use a browser driver like Watir or Watir-webdriver. If JRuby is an option, you can use Celerity which is a browser emulator that supports some javascript (using the Watir API), and will likely perform the task you need.
I have this:
<a class="top_level_active" href="javascript:Submit('menu_home')">Account Summary</a>
I want to click that link but I get an error when using link_to.
I've tried:
bot.click(page.link_with(:href => /menu_home/))
bot.click(page.link_with(:class => 'top_level_active'))
bot.click(page.link_with(:href => /Account Summary/))
The error I get is:
NoMethodError: undefined method `[]' for nil:NilClass
That's a javascript link. Mechanize will not be able to click it, since it does not evaluate javascript. Sorry!
Try to find out what happens in your browser when you click that link. Does it create a POST or GET request? What are the parameters that are sent to the server. Once you know that, you can emulate the same action in your Mechanize script. Chrome dev tools / Firebug will help out.
If that doesn't work, try switching to a library that supports javascript evaluation. I've used watir-webdriver to great success, but you could also try out phantomjs, casperjs, pjscrape, or other tools
The first 2 should have worked so try this, print out the hrefs to make sure it's really there:
puts page.links.map(&:href)
Remember that just because you can see it in your browser does not mean it appears in the response. It could have been sent as an ajax update.
Also you can just do this which I think is cleaner:
page.link_with(:href => /menu_home/).click
However I don't think clicking that link will do what you want since it's javascript.
Here's a way to handle it. Assume your page returns this content:
puts page.body
<HTML><SCRIPT LANGUAGE="JavaScript"><!--
top.location="http://www.example.com/pages/myaccount/dashboard.aspx?";
// --></SCRIPT>
<NOSCRIPT>Javascript required.</NOSCRIPT></HTML>
We know it's coming so we know what to check for:
link_search = %r{top.location="([^"]+)"}
js_link = page.body.match(link_search)[1]
page = agent.get(js_link)
I am experimenting with using rspec and watir to do some tdd and have come across a problem I can't seem to get past. I want to have watir click a link (target="_blank") and then get the url of the newly loaded page. Watir clicks the link but when I attempt to get the url I receive the old url not the current. Watir docs seem to indicate that the Browser url method will return the current url. I found a blog post that seems to solve this issue by having Watir execute some javascript to get the current url but this isn't working for me. Is there anyway to get the current url from a link click with Watir?
<!-- the html -->
LinkedIn
#The rspec code
it "should load LinkedIn" do
browser.link(:href => "http://www.linkedin.com").click
browser.url.should == "http://www.linkedin.com"
end
The target will load the link in a new browser window, therefore you need to switch to that window to assert the url:
it "should load LinkedIn" do
browser.link(:href => "http://www.linkedin.com").click
browser.window(:title => /.*LinkedIn.*/).use do
browser.url.should == "http://www.linkedin.com"
end
end
See: http://watirwebdriver.com/browser-popups/ for more examples
I started writing a simple Sinatra app today and I am trying to understand the error reporting but for some reason I can't get it to work correctly.
I know here, http://railsapi.com/doc/sinatra-v1.0/, it talks about working with error reporting/handling but when I run their examples I can't get it to work.
require 'sinatra'
error 400..510 do
'Boom'
end
get '/say/*' do
params[:splat]
end
When I run the app on my computer I get a 404 error code, but the 'Boom' text does not display in the browser, just the browser 404 page. I am sure I am doing something wrong, but just can't figure it out.
I will wager its your browser. On my MacBook Pro:
Chrome "helpfully" displays a "Oops! This link appears to be broken." page.
Safari displays the expected Boom text.
Firefox displays the expected Boom text.
It seems that Sinatra throws Sinatra::NotFound exception (404) to a specific handler. Simply modify the code as follows,
require 'sinatra'
not_found do
'Boom in NOT_FOUND.'
end
error 400..510 do
'Boom'
end
get '/say/*' do
params[:splat]
end
It works well in Chrome and Firefox on Mac OSX.