We use Capybara along with Chrome Headless for integration testing. I'd like to write a linter, that checks some metrics on the HTML structure everytime chrome navigates to another page. Then I'd raise an error when something is against our linter.
We have some tests without javascript, and monkey patching rack-test works so far:
Capybara::RackTest::Browser.class_eval do
alias_method :process_orig, :process
def process *args
response = process_orig *args
# do some linting
response
end
end
But I haven't found a way inside Capybara and/or Chrome Headless where I could intercept a response, and check the body of the page.
Is it possible to trigger a hook, when a page changes? Or is there some kind of API Chrome provides where I could get the body of every request? Or would a proxy be a feasible solution?
This isn't possible directly with Capybara, since it doesn't actually know about page transitions/requests that happen in the browser unless they are specifically user initiated with visit.
A potential way to do what you want would be using a programmable proxy like puffing-billy to handle every request to the app under test. If you use puffing-billy you'll want to look at the pass_request feature - https://github.com/oesmith/puffing-billy#in-your-tests-capybarawatir - to forward on the initial request and then do whatever you want with the response.
I'd not tangle capybara tests with HTML linting. It may seem smart at this moment, as you get a list of URL's to check for free with each test, but:
You'd probably lint each URL few times because some tests go through it
You might get failures because HTML is not perfect, even though the feature you're testing is actually ok.
You probably have something like sitemap.xml or other sources of all available URLs. I'd use it to make a separate check, which will be simple: request the URL, lint the response. Rinse and repeat.
If still not convinced, try using page.html and do sth like
expect(page.html).to pass_linter
https://github.com/teamcapybara/capybara#debugging
You can then add it as an around hook for every type: :feature spec if you want.
EDIT: here's another workaround to have every visited path. Just parse the server log file (like this cat log/devlopment.log | grep path) to get a full list if visited paths:
method=POST path=/users/login format=html controller=SessionsController action=create
status=302 duration=256.82 view=0.00 db=52.29 location=http://0.0.0.0:3000/platform/admin/dashboard params={"utf8"=>"✓", "authenticity_token"=>"ubGnWKOq8gbUE5C/aK375QQn5DpjHarUYxHtBLglGe6Lr9Ie3O5XPq90k5gr/SZbIPoDiiasvY0mGlwhzD/MsQ==", "user"=>{"email"=>"alex-3d51048235c9d1a8#toptal.io", "password"=>"[FILTERED]", "remember_me"=>"0"}, "commit"=>"Log in"} uid=983 remote_ip=127.0.0.1 x_forwarded_for= x_request_id=
method=GET path=/admin/dashboard format=html controller=XXX action=show status=200 duration=3285.54 view=1051.32 db=2016.87 params={} uid=983 remote_ip=127.0.0.1 x_forwarded_for= x_request_id=
and use it for linting.
Related
I need to test a GET request in sendRequest.js:
$.get( '/test/test2', {name: 'Larry', time: '2pm'} );
This sends the request fine and everything works on the JavaScript end, but obviously returns a 404 (route not found) so I added in app.rb:
get '/test/test2' do
logger.info "this is a test log"
end
I sent the request again, and I got the same 404.
This scenario originates from none of my changes in app.rb going into effect. I've deleted entire routes, commented stuff out, etc., nothing I do in app.rb seems to have any effect on the server.
Any ideas?
Have you tried stopping and restarting the server?
By default Sinatra won't know its code has changed, and Ruby will have loaded all the script into memory and won't look at the file.
Stopping the server, by using Cntrl+C then restarting the server will refresh Ruby's idea of what should be done.
We have a feature:
#smoke #acceptance
Scenario: Home page is available
When I visit the home page url
Then I expect no error code
With the final line implemented as:
Then(/^I expect no error code$/) do
expect(page.status_code).to eq 200
end
This sometimes fails. When debugging we've found the following:
the page itself always responds with 200
however, one resource within the page sometimes responds with 204
(which causes the test to fail)
This suggests that page.status_code does not equal the status code of the actual URL requested, but could be set to the status code of any (or presumably the last?) of the resources requested by the page.
Is this the correct explanation, and is it the expected behaviour of page.status_code?
Notes:
I realise that 204 could be construed as success, but this is not the
focus of this issue
status_code returns the result of the last request that triggered the phantomjs onLoadStarted callback - this should be the page, or potentially ajax requests initiated by the page -- it should not be assets being loaded directly as resources on the page (img, javascripts, css, etc). If it is a dependent asset then phantomjs has an issue and should be reported with reproducible example on that project. What type of request is reporting with a 204 response? If it's an ajax request then it is as expected - if not then its a bug in phantomjs. Note: checking response codes when using Capybara really is an anti-pattern, and you should generally stick to testing visible changes on the page.
Practically everything I am looking for is said in the title - I need to access warden user variable in test to check whether authentication worked. Another way is also possible for me, just trying to test authentication nicely :)
should "authenticate" do
post "/login", {:login => "test_login", :password => "password"}, {"HTTP_HOST" => "test.host"}
assert last_response.redirect?
assert_equal last_response.env["warden"].user.login, "test_login"
end
You can't get at the internal request environment in a rack-test-style full-stack test. If you want to verify that you've been logged in as a specific user you'll need to look at something like the redirect URL (if it redirects to a user-identifiable URL) or follow the redirect (easy with rack-test's follow_redirect! helper) and then look for evidence of the User ID in the HTML.
I would have thought that you don't really need to test Warden itself, but you will want to make sure you're providing the correct information to it, and aren't mangling it in the middleware stack.
You might find something like Cucumber handy for doing genuine form-filling and submission.
Finally, Warden has its own test helpers (which definitely work with rack-test) so you can set up a request to be logged in without having to actually run through the logging in request/redirect cycle in each test - https://github.com/hassox/warden/wiki/testing has more details.
I'm trying to click the Settings button on the home page, but when I do I get this page back:
#<WWW::Mechanize::Page
{url
#<URI::HTTP:0x1023c5fc0 URL:http://www.facebook.com/editaccount.php?ref=mb&drop>}
{meta}
{title nil}
{iframes}
{frames}
{links}
{forms}>
which is.. kinda empty! Is there some problems with these iframes and frames stuff maybe?
As roja mentioned, following redirects might be what you need. Here's an example of how to do this:
#agent = Mechanize.new
#agent.redirect_ok = :all
#agent.follow_meta_refresh = :anywhere
Then you can pretty much ignore the fact that there's redirects involved - Mechanize will simply put you on the resulting page.
Facebook redirects me to: https://register.facebook.com/editaccount.php which I assume is the final destination. Assuming that WWW::Mechanize is set up to follow https redirects you should end up there too.
Much of facebook like most modern websites is generated by javascript which I think that WWW::Mechanize is unable to cope with, this could be the source of your problem. I recommend trying to scrape while appending "?_fb_noscript=1" to the url's you visit. This turns off much of facebooks javascript system and should enable a smoother ride for your little bot.
(Do remember this is only an idea and doubtless whatever you do is against facebooks usage policy and this makes you a "baddy." I don't condone such badness and beleve that baddies should be forced to go to bed early etc... ad nauseum)
I am working on a website hosted on microsoft's office live service. It has a contact form enabling visitors to get in touch with the owner. I want to write a Ruby script that sits on a seperate sever and which the form will POST to. It will parse the form data and email the details to a preset address. The script should then redirect the browser to a confirmation page.
I have an ubuntu hardy machine running nginx and postfix. Ruby is installed and we shall see about using Thin and it's Rack functionality to handle the script. Now it's come to writing the script and i've drawn a blank.
It's been a long time and if i remember rightly the process is something like;
read HTTP header
parse parameters
send email
send redirect header
Broadly speaking, the question has been answered. Figuring out how to use the answer was more complicated than expected and I thought worth sharing.
First Steps:
I learnt rather abruptly that nginx doesn't directly support cgi scripts. You have to use some other process to run the script and get nginx to proxy requests over. If I was doing this in php (which in hind sight i think would have been a more natural choice) i could use something like php-fcgi and expect life would be pretty straight forward.
Ruby and fcgi felt pretty daunting. But if we are abandoning the ideal of loading these things at runtime then Rack is probably the most straight forward solution and Thin includes all we need. Learning how to make basic little apps with them has been profoundly beneficial to a relative Rails newcomer like me. The foundations of a Rails app can seem hidden for a long time and Rack has helped me lift the curtain that little bit further.
Nonetheless, following Yehuda's advice and looking up sinatra has been another surprise. I now have a basic sinatra app running in a Thin instance. It communicates with nginx over a unix socket in what i gather is the standard way. Sinatra enables a really elegant way to handle different requests and routes into the app. All you need is a get '/' {} to start handling requests to the virtual host. To add more (in a clean fashion) we just include a routes/script.rb into the main file.
# cgi-bin.rb
# main file loaded as a sinatra app
require 'sinatra'
# load cgi routes
require 'routes/default'
require 'routes/contact'
# 404 behaviour
not_found do
"Sorry, this CGI host does not recognize that request."
end
These route files will call on functionality stored in a separate library of classes:
# routes/contact.rb
# contact controller
require 'lib/contact/contactTarget'
require 'lib/contact/contactPost'
post '/contact/:target/?' do |target|
# the target for the message is taken from the URL
msg = ContactPost.new(request, target)
redirect msg.action, 302
end
The sheer horror of figuring out such a simple thing will stay with me for a while. I was expecting to calmly let nginx know that .rb files were to be executed and to just get on with it. Now that this little sinatra app is up and running, I'll be able to dive straight in if I want to add extra functionality in the future.
Implementation:
The ContactPost class handles the messaging aspect. All it needs to know are the parameters in the request and the target for the email. ContactPost::action kicks everything off and returns an address for the controller to redirect to.
There is a separate ContactTarget class that does some authentication to make sure the specified target accepts messages from the URL given in request.referrer. This is handled in ContactTarget::accept? as we can guess from the ContactPost::action method;
# lib/contact/contactPost.rb
class ContactPost
# ...
def action
return failed unless #target.accept? #request.referer
if send?
successful
else
failed
end
end
# ...
end
ContactPost::successful and ContactPost::failed each return a redirect address by combining paths supplied with the HTML form with the request.referer URI. All the behaviour is thus specified in the HTML form. Future websites that use this script just need to be listed in the user's own ~/cgi/contact.conf and they'll be away. This is because ContactTarget looks in /home/:target/cgi/contact.conf for the details. Maybe oneday this will be inappropriate, but for now it's just fine for my purposes.
The send method is simple enough, it creates an instance of a simple Email class and ships it out. The Email class is pretty much based on the standard usage example given in the Ruby net/smtp documentation;
# lib/email/email.rb
require 'net/smtp'
class Email
def initialize(from_alias, to, reply, subject, body)
#from_alias = from_alias
#from = "cgi_user#host.domain.com"
#to = to
#reply = reply
#subject = subject
#body = body
end
def send
Net::SMTP.start('localhost', 25) do |smtp|
smtp.send_message to_s, #from, #to
end
end
def to_s
<<END_OF_MESSAGE
From: #{#from_alias}
To: #{#to}
Reply-To: #{#from_alias}
Subject: #{#subject}
Date: #{DateTime::now().to_s}
#{#body}
END_OF_MESSAGE
end
end
All I need to do is rack up the application, let nginx know which socket to talk to and we're away.
Thank you everyone for your helpful pointers in the right direction! Long live sinatra!
It's all in the Net module, here's an example:
#net = Net::HTTP.new 'http://www.foo.com', 80
#params = {:name => 'doris', :email => 'doris#foo.com'}
# Create HTTP request
req = Net::HTTP::Post.new( 'script.cgi', {} )
req.set_form_data #params
# Send request
response = #net.start do |http|
http.read_timeout = 5600
http.request req
end
Probably the best way to do this would be to use an existing Ruby library like Sinatra:
require "rubygems"
require "sinatra"
get "/myurl" do
# params hash available here
# send email
end
You'll probably want to use MailFactory to send the actual email, but you definitely don't need to be mucking about with headers or parsing parameters.
CGI class of Ruby can be used for writing CGI scripts. Please check: http://www.ruby-doc.org/stdlib/libdoc/cgi/rdoc/index.html
By the way, there is no need to read the HTTP header. Parsing parametres will be easy using CGI class. Then, send the e-mail and redirect.