I have an RSS feed that I am writing an RSpec test for. I want to test that the XML document has the correct nodes and structure. Unfortunately, I can't find any good examples of how to do this in a clean way. I have only found some half-implemented solutions and outdated blog posts. How can I test the structure of an XML document using RSpec?
Hi I can recommend you to use custom matcher for this.
require 'nokogiri'
RSpec::Matchers.define :have_xml do |xpath, text|
match do |body|
doc = Nokogiri::XML::Document.parse(body)
nodes = doc.xpath(xpath)
nodes.empty?.should be_false
if text
nodes.each do |node|
node.content.should == text
end
end
true
end
failure_message_for_should do |body|
"expected to find xml tag #{xpath} in:\n#{body}"
end
failure_message_for_should_not do |response|
"expected not to find xml tag #{xpath} in:\n#{body}"
end
description do
"have xml tag #{xpath}"
end
end
Full example can be found here
https://gist.github.com/Fivell/8025849
No longer necessary to roll your own. We deal this problem daily, using the equivalent-xml matcher at https://github.com/mbklein/equivalent-xml .
require 'rspec/matchers'
require 'equivalent-xml'
...
expect(node_1).to be_equivalent_to(node_2)
Has options for edge cases like whitespace-preservation.
Your other option is to use a formal XSD template for strict validation.
context 'POST #join' do
it 'does successfully hit join xml route' do
post :join,
format: :xml
response.content_type.should == "application/xml"
response.should be_ok
end
end
This worked for me. I didn't realize I had to pass format: :xml. My join route responds to /join.xml and I was testing that this was successful.
Give Approvals a try, it works with rspec, I have used for testing Json payload, and it is used with Minitest in exercism.io
EDIT
it "returns available traffic information around me" do
post '/search_traffic_around', {location: [-87.688219, 41.941149]}.to_json
output = last_response.body
options = {format: :json, name: 'traffic_around_location'}
Approvals.verify(output,options)
end
the JSON I am verifying against is located in spec/fixtures folder named traffic_around_location.approved.json
Implementation where the above snippet is pulled from is available here
How it works is you supply it an expected Payload, JSON, XML, TXT and HTML this I am sure it supports in spec/fixtures and when you run the test it checks to confirm that the payload received matches the expected(approved) payload the test would pass if it matches else the test fails
Related
I can't find a basic explanation anywhere about how I can test, with Rack::Test, that a Ruby/Sinatra post method successfully saves data to a YAML store/file. (This explains testing get, which I can do(!), but not post; other mentions of testing post methods with rack/test seem irrelevant.) For self-study, I'm building a "to do" app in Ruby/Sinatra and I'm trying to use TDD everything and unit test like a good little boy. A requirement I have is: When a user posts a new task, it is saved in the YML store.
I was thinking of testing this either by seeing if a "Task saved" was shown in the response to the user (which of course isn't directly testing the thing itself...but is something I'd also like to test):
assert last_response.body.include?("Task saved")
or by somehow testing that a test task's description is now in the YML file. I guess I could open up the YML file and look, and then delete it from the YML file, but I'm pretty sure that's not what I'm supposed to do.
I've confirmed post does correctly save to a YML file:
get('/') do |*user_message|
# prepare erb messages
#user_message = session[:message] if session[:message]
#overlong_description = session[:overlong_description] if
session[:overlong_description]
session[:message] = nil # clear message after being used
session[:overlong_description] = nil # ditto
#tasks = store.all
erb :index #, user_message => {:user_message => params[:user_message]}
end
post('/newtask') do
#task = Task.new(store, params)
# decide whether to save & prepare user messages
if #task.complete == true # task is complete!
#task.message << " " + "Task saved!"
session[:message] = #task.message # use session[:message] for user messages
#task.message = ""
store.save(#task)
else
#task.message << " " + "Not saved." # task incomplete
session[:message] = #task.message # use session[:message] for user messages
session[:overlong_description] = #task.overlong_description if
#task.overlong_description
#task.message = ""
#task.overlong_description = nil
end
redirect '/'
end
As you can see, it ends in a redirect...one response I want to test is actually on the slash route, not on the /newtask route.
So of course the test doesn't work:
def test_post_newtask
post('/newtask', params = {"description"=>"Test task 123"})
# Test that "saved" message for user is in returned page
assert last_response.body.include?("Task saved") # boooo
end
Github source here
If you can give me advice on a book (chapter, website, blog, etc.) that goes over this in a way accessible to a relative beginner, I'd be most grateful.
Be gentle...I'm very new to testing (and programming).
Nobody answered my question and, since I have figured out what the answer is, I thought I would share it here.
First of all, I gather that it shouldn't be necessary to check if the data is actually saved to the YAML store; the main thing is to see if the web page returns the correct result (we assume the database is groovy if so).
The test method I wrote above was correct; it was simply missing the single line follow_redirect!. Apparently I didn't realize that I needed to instruct rake/test to follow the redirect.
Part of the problem was that I simply hadn't found the right documentation. This page does give the correct syntax, but doesn't give much detail. This page helped a lot, and this bit covers redirects.
Here's the updated test method:
def test_post_newtask
post "/newtask", params = {"description" => "Write about quick brown foxes",
"categories" => "writing823"}
follow_redirect!
assert last_response.body.include?("Task saved")
assert last_response.body.include?("Write about quick brown foxes")
end
(With thanks to the Columbus Ruby Brigade.)
I have an XML file, and before I process it I need to make sure that a certain element exists and is not blank.
Here is the code I have:
CSV.open("#{csv_dir}/products.csv","w",{:force_quotes => true}) do |out|
out << headers
Dir.glob("#{xml_dir}/*.xml").each do |xml_file|
gdsn_doc = GDSNDoc.new(xml_file)
logger.info("Processing xml file #{xml_file}")
:x
#desc_exists = #gdsn_doc.xpath("//productData/description")
if !#desc_exists.empty?
row = []
headers.each do |col|
row << product[col]
end
out << row
end
end
end
The following code is not working to find the "description" element and to check whether it is blank or not:
#desc_exists = #gdsn_doc.xpath("//productData/description")
if !#desc_exists.empty?
Here is a sample of the XML file:
<productData>
<description>Chocolate biscuits </description>
<productData>
This is how I have defined the class and Nokogiri:
class GDSNDoc
def initialize(xml_file)
#doc = File.open(xml_file) {|f| Nokogiri::XML(f)}
#doc.remove_namespaces!
The code had to be moved up to an earlier stage, where Nokogiri was initialised. It doesn't get runtime errors, but it does let XML files with blank descriptions get through and it shouldn't.
class GDSNDoc
def initialize(xml_file)
#doc = File.open(xml_file) {|f| Nokogiri::XML(f)}
#doc.remove_namespaces!
desc_exists = #doc.xpath("//productData/descriptions")
if !desc_exists.empty?
You are creating your instance like this:
gdsn_doc = GDSNDoc.new(xml_file)
then use it like this:
#desc_exists = #gdsn_doc.xpath("//productData/description")
#gdsn_doc and gdsn_doc are two different things in Ruby - try just using the version without the #:
#desc_exists = gdsn_doc.xpath("//productData/description")
The basic test is to use:
require 'nokogiri'
doc = Nokogiri::XML(<<EOT)
<productData>
<description>Chocolate biscuits </description>
<productData>
EOT
# using XPath selectors...
doc.xpath('//productData/description').to_html # => "<description>Chocolate biscuits </description>"
doc.xpath('//description').to_html # => "<description>Chocolate biscuits </description>"
xpath works fine when the document is parsed correctly.
I get an error "undefined method 'xpath' for nil:NilClass (NoMethodError)
Usually this means you didn't parse the document correctly. In your case it's because you're not using the right variable:
gdsn_doc = GDSNDoc.new(xml_file)
...
#desc_exists = #gdsn_doc.xpath("//productData/description")
Note that gdsn_doc is not the same as #gdsn_doc. The later doesn't appear to have been initialized.
#doc = File.open(xml_file) {|f| Nokogiri::XML(f)}
While that should work, it's idiomatic to write it as:
#doc = Nokogiri::XML(File.read(xml_file))
File.open(...) do ... end is preferred if you're processing inside the block and want Ruby to automatically close the file. That isn't necessary when you're simply reading then passing the content to something else for processing, hence the use of File.read(...) which slurps the file. (Slurping isn't necessary a good practice because it can have scalability problems, but for reasonable sized XML/HTML it's OK because it's easier to use DOM-based parsing than SAX.)
If Nokogiri doesn't raise an exception it was able to parse the content, however that still doesn't mean the content was valid. It's a good idea to check
#doc.errors
to see whether Nokogiri/libXML had to do some fix-ups on the content just to be able to parse it. Fixing the markup can change the DOM from what you expect, making it impossible to find a tag based on your assumptions for the selector. You could use xmllint or one of the XML validators to check, but Nokogiri will still have to be happy.
Nokogiri includes a command-line version nokogiri that accepts a URL to the document you want to parse:
nokogiri http://example.com
It'll open IRB with the content loaded and ready for you to poke at it. It's very convenient when debugging and testing. It's also a decent way to make sure the content actually exists if you're dealing with HTML containing DHTML that loads parts of the page dynamically.
I'm brand new to ruby (first day working with ruby) so please forgive any novice questions and lack of understanding.
I'm trying to validate the responses to http callouts.
For example, let's say the endpoint is the following:
https://applicationname-api-sbox02.herokuapp.com
And, I'm trying to authenticate a user by sending a get request like this:
get_response = RestClient.get( "https://applicationname-api-sbox02.herokuapp.com/api/v1/users",
{
"Content-Type" => "application/json",
"Authorization" => "token 4d012314b7e46008f215cdb7d120cdd7",
"Manufacturer-Token" => "8d0693ccfe65104600e2555d5af34213"
}
)
Now, I want to validate the response and do the following:
- parse the response to ensure that it is valid JSON
- do some validation and verify the JSON has the correct data (verify that id == 4 for example)
- if an error is encountered, raise an exception using the 'raise' method.
In my first feeble attempt I tried the following:
puts get_response.body
if get_response.code == 200
puts "********* Get current user successful"
else
puts "Get current user failed!!"
end
Now, this returned that getting the current user was successful, but how do I actually parse the json, verify the correct id, and raise an exception if an error occurred?
Instead of raising an exception, write a test.
A straightforward approach, using the json parser and unit test framework from the std lib:
require 'minitest/autorun'
require 'rest_client'
require 'json'
class APITest < MiniTest::Unit::TestCase
def setup
response = RestClient.get("https://applicationname-api-sbox02.herokuapp.com/api/v1/users",
{
"Content-Type" => "application/json",
"Authorization" => "token 4d012314b7e46008f215cdb7d120cdd7",
"Manufacturer-Token" => "8d0693ccfe65104600e2555d5af34213"
}
)
#data = JSON.parse response.body
end
def test_id_correct
assert_equal 4, #data['id']
end
end
Execute with ruby $filename
JSON.parse parses a JSON string into a ruby hash
Getting started with minitest
If you are using ruby 1.8, you'll need to install the json gem and either install the minitest gem, or switch to the older testunit API. If you choose the latter, then you'll need to change require 'minitest/autorun' -> require 'test/unit' and MiniTest::Unit::TestCase -> Test::Unit::TestCase
I'm a little late to the party, but I recently co-created an rspec driven framework called Airborne for just this purpose. Check it out: https://github.com/brooklynDev/airborne
here is an example from our specs so you can see how we test json api:
it 'returns charge' do
get "/charges/#{charge.id}", '', headers
expect(response.status).to eq(200)
expect(response).to match_response_schema(:charge)
expect(response).to match_json(<<-JSON)
{
"id":"{id}",
"email": "{email}",
"ip": "127.0.0.1",
"amount": 10500,
"state": "captured",
"captured_amount": 10500,
}
JSON
end
Lets look at it closely
match_response_schema(:charge)
This matcher checks that json we get in response is in general valid. We use json-schema (json schema validator) for it. Guys from Thoughtbot have a detailed guide how to use json schema validator and create own matcher in this blog post.
Understanding JSON Schema is where I got a lot of useful information on how to create schemas for JSON documents.
match_json
This is our own matcher and we have released match_json gem recently. Using it you can test structure and values of your json. Here are two great features of this matcher:
if you don't know exact values, you can use patterns like {id}, {uuid} {date_time}, etc. we have predefined patterns but you can add your own too.
you get clear failure message what is wrong with your json e.g. "5" was not found in " > array":[1,2,3]
Parsing json can be done with the json gem: http://flori.github.com/json/
Parsed json is accessed through key/value just like in javascript. You can easily verify the values and conditionally raise errors.
Raising errors is done like so:
raise "the ID was #{id} instead of 4"
And writing unit tests can be done with Test::Unit - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/test/unit/rdoc/Test/Unit.html
I'm currently testing my mailers with RSpec, but I've started setting up multipart emails as described in the Rails Guides here: http://guides.rubyonrails.org/action_mailer_basics.html#sending-multipart-emails
I have both mailer templates in text and html formats, but it looks like my tests are only checking the HTML portion. Is there a way to check the text template separately?
Is it only checking the HTML view because it's first in the default order?
To supplement, nilmethod's excellent answer, you can clean up your specs by testing both text and html versions using a shared example group:
spec_helper.rb
def get_message_part (mail, content_type)
mail.body.parts.find { |p| p.content_type.match content_type }.body.raw_source
end
shared_examples_for "multipart email" do
it "generates a multipart message (plain text and html)" do
mail.body.parts.length.should eq(2)
mail.body.parts.collect(&:content_type).should == ["text/plain; charset=UTF-8", "text/html; charset=UTF-8"]
end
end
your_email_spec.rb
let(:mail) { YourMailer.action }
shared_examples_for "your email content" do
it "has some content" do
part.should include("the content")
end
end
it_behaves_like "multipart email"
describe "text version" do
it_behaves_like "your email content" do
let(:part) { get_message_part(mail, /plain/) }
end
end
describe "html version" do
it_behaves_like "your email content" do
let(:part) { get_message_part(mail, /html/) }
end
end
This can be tested with regular expressions.
Finding things in the HTML portion (use #should after this to match):
mail.body.parts.find {|p| p.content_type.match /html/}.body.raw_source
Finding things in the plain text portion (use #should after this to match):
mail.body.parts.find {|p| p.content_type.match /plain/}.body.raw_source
Checking that it is, indeed, generating a multipart message:
it "generates a multipart message (plain text and html)" do
mail.body.parts.length.should == 2
mail.body.parts.collect(&:content_type).should == ["text/html; charset=UTF-8", "text/plain; charset=UTF-8"]
end
To make things even simpler, you can use
message.text_part and
message.html_part
to find the respective parts. This works even for structured multipart/alternative messages with attachments. (Tested on Ruby 1.9.3 with Rails 3.0.14.)
These methods employ some kind of heuristic to find the respective message parts, so if your message has multiple text parts (e.g. as Apple Mail creates them) it might fail to do the "right thing".
This would change the above method to
def body_should_match_regex(mail, regex)
if mail.multipart?
["text", "html"].each do |part|
mail.send("#{part}_part").body.raw_source.should match(regex)
end
else
mail.body.raw_source.should match(regex)
end
end
which works for both plaintext (non-multipart) messages and multipart messages and tests all message bodies against a specific regular expression.
Now, any volunteers to make a "real" RSpec matcher out of this? :) Something like
#mail.bodies_should_match /foobar/
would be a lot nicer ...
If your email has attachments the text and html parts will end be placed in a multipart/alternative part. This is noted on under Sending Emails with Attachments in the Rails 3 Guide.
To handle this, I first simplified the get_message_part method above to:
def get_message_part(mail, content_type)
mail.body.parts.find { |p| p.content_type.match content_type }
end
Then in my test:
multipart = get_message_part(email, /multipart/)
html = get_message_part(multipart, /html/)
html_body = html.body.raw_source
assert_match 'some string', html_body
I have done this way, I found it simpler since the content of both emails is gonna be similar except styles and markup.
context 'When there are no devices' do
it 'makes sure both HTML and text version emails are sent' do
expect(mail.body.parts.count).to eq(2)
# You can even make sure the types of the part are `html` and `text`
end
it 'does not list any lockboxes to be removed in both types emails' do
mail.body.parts.each do |part|
expect(part.body).to include('No devices to remove')
end
end
end
I am learning Ruby and I have written the following code to find out how to consume SOAP services:
require 'soap/wsdlDriver'
wsdl="http://www.abundanttech.com/webservices/deadoralive/deadoralive.wsdl"
service=SOAP::WSDLDriverFactory.new(wsdl).create_rpc_driver
weather=service.getTodaysBirthdays('1/26/2010')
The response that I get back is:
#<SOAP::Mapping::Object:0x80ac3714
{http://www.abundanttech.com/webservices/deadoralive} getTodaysBirthdaysResult=#<SOAP::Mapping::Object:0x80ac34a8
{http://www.w3.org/2001/XMLSchema}schema=#<SOAP::Mapping::Object:0x80ac3214
{http://www.w3.org/2001/XMLSchema}element=#<SOAP::Mapping::Object:0x80ac2f6c
{http://www.w3.org/2001/XMLSchema}complexType=#<SOAP::Mapping::Object:0x80ac2cc4
{http://www.w3.org/2001/XMLSchema}choice=#<SOAP::Mapping::Object:0x80ac2a1c
{http://www.w3.org/2001/XMLSchema}element=#<SOAP::Mapping::Object:0x80ac2774
{http://www.w3.org/2001/XMLSchema}complexType=#<SOAP::Mapping::Object:0x80ac24cc
{http://www.w3.org/2001/XMLSchema}sequence=#<SOAP::Mapping::Object:0x80ac2224
{http://www.w3.org/2001/XMLSchema}element=[#<SOAP::Mapping::Object:0x80ac1f7c>,
#<SOAP::Mapping::Object:0x80ac13ec>,
#<SOAP::Mapping::Object:0x80ac0a28>,
#<SOAP::Mapping::Object:0x80ac0078>,
#<SOAP::Mapping::Object:0x80abf6c8>,
#<SOAP::Mapping::Object:0x80abed18>]
>>>>>>> {urn:schemas-microsoft-com:xml-diffgram-v1}diffgram=#<SOAP::Mapping::Object:0x80abe6c4
{}NewDataSet=#<SOAP::Mapping::Object:0x80ac1220
{}Table=[#<SOAP::Mapping::Object:0x80ac75e4
{}FullName="Cully, Zara"
{}BirthDate="01/26/1892"
{}DeathDate="02/28/1979"
{}Age="(87)"
{}KnownFor="The Jeffersons"
{}DeadOrAlive="Dead">,
#<SOAP::Mapping::Object:0x80b778f4
{}FullName="Feiffer, Jules"
{}BirthDate="01/26/1929"
{}DeathDate=#<SOAP::Mapping::Object:0x80c7eaf4>
{}Age="81"
{}KnownFor="Cartoonists"
{}DeadOrAlive="Alive">]>>>>
I am having a great deal of difficulty figuring out how to parse and show the returned information in a nice table, or even just how to loop through the records and have access to each element (ie. FullName,Age,etc). I went through the whole "getTodaysBirthdaysResult.methods - Object.new.methods" and kept working down to try and work out how to access the elements, but then I get to the array and I got lost.
Any help that can be offered would be appreciated.
If you're going to parse the XML anyway, you might as well skip SOAP4r and go with Handsoap. Disclaimer: I'm one of the authors of Handsoap.
An example implementation:
# wsdl: http://www.abundanttech.com/webservices/deadoralive/deadoralive.wsdl
DEADORALIVE_SERVICE_ENDPOINT = {
:uri => 'http://www.abundanttech.com/WebServices/DeadOrAlive/DeadOrAlive.asmx',
:version => 1
}
class DeadoraliveService < Handsoap::Service
endpoint DEADORALIVE_SERVICE_ENDPOINT
def on_create_document(doc)
# register namespaces for the request
doc.alias 'tns', 'http://www.abundanttech.com/webservices/deadoralive'
end
def on_response_document(doc)
# register namespaces for the response
doc.add_namespace 'ns', 'http://www.abundanttech.com/webservices/deadoralive'
end
# public methods
def get_todays_birthdays
soap_action = 'http://www.abundanttech.com/webservices/deadoralive/getTodaysBirthdays'
response = invoke('tns:getTodaysBirthdays', soap_action)
(response/"//NewDataSet/Table").map do |table|
{
:full_name => (table/"FullName").to_s,
:birth_date => Date.strptime((table/"BirthDate").to_s, "%m/%d/%Y"),
:death_date => Date.strptime((table/"DeathDate").to_s, "%m/%d/%Y"),
:age => (table/"Age").to_s.gsub(/^\(([\d]+)\)$/, '\1').to_i,
:known_for => (table/"KnownFor").to_s,
:alive? => (table/"DeadOrAlive").to_s == "Alive"
}
end
end
end
Usage:
DeadoraliveService.get_todays_birthdays
SOAP4R always returns a SOAP::Mapping::Object which is sometimes a bit difficult to work with unless you are just getting the hash values that you can access using hash notation like so
weather['fullName']
However, it does not work when you have an array of hashes. A work around is to get the result in xml format instead of SOAP::Mapping::Object. To do that I will modify your code as
require 'soap/wsdlDriver'
wsdl="http://www.abundanttech.com/webservices/deadoralive/deadoralive.wsdl"
service=SOAP::WSDLDriverFactory.new(wsdl).create_rpc_driver
service.return_response_as_xml = true
weather=service.getTodaysBirthdays('1/26/2010')
Now the above would give you an xml response which you can parse using nokogiri or REXML. Here is the example using REXML
require 'rexml/document'
rexml = REXML::Document.new(weather)
birthdays = nil
rexml.each_recursive {|element| birthdays = element if element.name == 'getTodaysBirthdaysResult'}
birthdays.each_recursive{|element| puts "#{element.name} = #{element.text}" if element.text}
This will print out all elements that have any text.
So once you have created an xml document you can pretty much do anything depending upon the methods the library you choose has ie. REXML or Nokogiri
Well, Here's my suggestion.
The issue is, you have to snag the right part of the result, one that is something you can actually iterator over. Unfortunately, all the inspecting in the world won't help you because it's a huge blob of unreadable text.
What I do is this:
File.open('myresult.yaml', 'w') {|f| f.write(result.to_yaml) }
This will be a much more human readable format. What you are probably looking for is something like this:
--- !ruby/object:SOAP::Mapping::Object
__xmlattr: {}
__xmlele:
- - &id024 !ruby/object:XSD::QName
name: ListAddressBooksResult <-- Hash name, so it's resul["ListAddressBooksResult"]
namespace: http://apiconnector.com
source:
- !ruby/object:SOAP::Mapping::Object
__xmlattr: {}
__xmlele:
- - &id023 !ruby/object:XSD::QName
name: APIAddressBook <-- this bastard is enumerable :) YAY! so it's result["ListAddressBooksResult"]["APIAddressBook"].each
namespace: http://apiconnector.com
source:
- - !ruby/object:SOAP::Mapping::Object
The above is a result from DotMailer's API, which I spent the last hour trying to figure out how to enumerate over the results. The above is the technique I used to figure out what the heck is going on. I think it beats using REXML etc this way, I could do something like this:
result['ListAddressBooksResult']['APIAddressBook'].each {|book| puts book["Name"]}
Well, I hope this helps anyone else who is looking.
/jason