I am trying to parse a SOAP response using Savon. The response is XML but is being returned as one long string. If I use #to_hash the entire XML object is still a string, now stored in
hash[:response][:return]
which means it is still a huge unusable mess.
My code looks like
response = soapClient.request(:get_sites_user_can_access) do
soap.body = { :sessionid => session[:login_response][:login_return],
:eid => user }
end
rep = response.to_hash
pp rep[:get_sites_user_can_access_response][:get_sites_user_can_access_return]
What step am I missing to get useful information out of the response? Note: Unfortunately I can't post the XML response because of the info it contains, but it looks like an entire XML document stored as a string. It's class is Nori::StringWithAttributes
I was able to get the desired results but parsing the Nori string(?) using this documentation. This seems like a less than ideal method, but I realized the last element is an array of hashes. So it's hash, of hashes, with an array of hashes. Anyway, here is what worked for me. Advice on how to make this less ugly and clunky would be appreciated.
response = soapClient.request(:get_sites_user_can_access) do
soap.body = { :sessionid => session[:login_response][:login_return],
:eid => user }
end
rep = response.to_hash[:get_sites_user_can_access_response][:get_sites_user_can_access_return]
hrep = Nori.parse(rep)
hrep[:list][:item].each { |item| pp item[:site_id] }
Related
I am trying to convert a CSV file to JSON using Ruby. I am very, very, green when it comes to working with Ruby (or any language for that matter) so the answers may need to be dumbed down for me. Putting it in JSON seems like the most reasonable solution to me because I understand how to work with JSON when assigning variables equal to the attributes that come in the response. If there is a better way to do it, feel free to teach me.
My CSV is in the following format:
Header1,Header,Header3
ValueX,ValueY,ValueZ
I would like to be able to use the data to say something along the lines of this:
For each ValueX in Row 1 after the headers, check if valueZ is > ValueY. If yes, do this, if no do that. I understand how to do the if statement, just now how to parse out my information into variables/arrays.
Any ideas here?
require 'csv'
require 'json'
rows = []
CSV.foreach('a.csv', headers: true, converters: :all) do |row|
rows << row.to_hash
end
puts rows.to_json
# => [{"Header1":"ValueX","Header":"ValueY","Header3":"ValueZ"}]
Here is a first pointer:
require 'csv'
data = CSV.read('your_file.csv', { :col_sep => ',' }
Now you should have the data in data; you can test in irb.
I don't entirely understand the question:
if z > y
# do this
else
# do that
end
For JSON, you should be able to do JSON.parse().
I am not sure what target format JSON requires, probably a Hash.
You can populate your hash with the dataset from the CVS:
hash = Hash.new
hash[key_goes_here] = value_here
I'd like to skip parsing some json returned by my DB when I just turn it back into json again immediately in the response from my padrino app.
ie I have
get :data, provides: :json do
Record.order(:day).map do |r|
{r.day.to_s => JSON.parse!(r.data)}
end.reduce({}, :merge!).to_json
end
and I'd like something like the following (inspired by String#html_safe):
get :data, provides: :json do
Record.order(:day).map do |r|
{r.day.to_s => r.data.json_literal}
end.reduce({}, :merge!).to_json
end
I know I can move the hash creation to the model with #as_json but that doesn't address the unnecessary performance hit from parsing and re-encoding the json.
Example output:
{
"2010-01-01":{"linux64":12186213,"mac":24131170},
"2010-01-02":{"linux":10650417,"mac":24139611,"win":12210218},
"2010-01-03":{"linux":10628353,"linux64":12184435,"win":12229263}
}
where the object that is the value of each key/value pair is available as a json-string in r.data eg '{"linux":10650417,"mac":24139611,"win":12210218}' which is why i want to avoid parsing r.data and just inline it.
I tried bypassing the JSON parse/dump altogether with the following:
get :data, provides: :json do
"{"+Record.order(:day).map do |r|
"\"#{r.day}\":#{r.data},"
end.reduce(&:+).delete(' ').chop+"}"
end
but performance was even worse than the version with the unnecessary parsing. (Which is weird, I'm not sure if string concatenation is slow or string interpolation...)
It turns out that the best way to do this is actually at the DB layer, as postgres can inline json columns into json objects and aggregate strings far faster than ruby can. So I did that and dropped my latency from 30ms to 8ms. The body of the method is below:
Sequel::Model.db.fetch("select json_object_agg(day,data) as jsobj from records;").first[:jsobj]
References:
Create json with column values as object keys
How to run raw SQL queries with Sequel
Sorry for the newbie question but still very new to Ruby and Mongo.
Not 100% sure what I am missing, but how would I get just the value back
from "url" in the following json result?
[{"url":"www.google.com"}]
So I want something like if I do a
puts url_var
it should just display www.google.com
MongoMapper code to get URl from Mongo
redirect_url = Surl.all(:url_key => "#{urlkey}")
myjson = redirect_url.to_json(:only => [:url])
puts redirect_url
gives me
[{"url":"www.google.com"}]
if I do something like
redirect_url.each do |key,value|
puts key
puts value
end
key is still the full json string and value is empty.
Surely I must be missing something really basic here
Okay, I assume Surl is a model of yours, which has at least these fields url_key and url, so to access the url you don't have to convert the result to json and then fetch to key, but you can access it directly.
Surl.all returns an array (check the mongo mapper docs), if you want a single result use Surl.find_by_url_key(urlkey). Now you can access the url directly by using puts redirect_url.url
redirect_url = Surl.find_by_url_key(urlkey)
puts redirect_url.url # => "www.google.com"
Be sure to read up on http://mongomapper.com/documentation/plugins/dynamic-querying.html and http://mongomapper.com/documentation/plugins/querying.html, to understand the results and differences.
Here is my YELP client using signet but once I get response, I'm not able to convert to ruby hash to inspect response element.
require 'rubygems'
require 'json'
require 'net/http'
client = Signet::OAuth1::Client.new(
:client_credential_key =>
'xxxxxxxxxxxxxxxxxxx',
:client_credential_secret =>
'xxxxxxxxxxxxxxxxxxx',
:token_credential_key =>
'xxxxxxxxxxxxxxxxxxx',
:token_credential_secret =>
'xxxxxxxxxxxxxxxxxxx'
)
response = client.fetch_protected_resource(
:uri => 'http://api.yelp.com/v2/search?term=food&location=san+francisco'
)
# The Rack response format is used here
status, headers, body = response
puts body["businesses"]
Error:
`[]': can't convert String into Integer (TypeError)
Body prints fine in nice JSON format but I can do body["businesses"] for instance
JSON.parse(body).inspect is also not working.
Btw body outputs itself appears as JSON format but JSON.parse(body) doesn't produce hash
puts body
{"region":{"span":{"latitude_delta":0.0,"longitude_delta":0.0},"center":{"latitude":37.660418999999997,"longitude":-121.876508}},"total":853,"businesses":[{"rating":4.0,"mobile_url":"http://m.yelp.com/biz/TT1t4oHeZmqkoiuwgCN4bQ","rating_img_url":"http://media2.ak.yelpcdn.com/static/201012164084228337/img/ico/stars/stars_4.png","review_count":150,"name":"India Garden","rating_img_url_small":"http://media2.ak.yelpcdn.com/static/20101216418129184/img/ico/stars/stars_small_4.png","url":"http://www.yelp.com/biz/india-garden-pleasanton-2","phone":"9254854800","snippet_text":"We went to this place without seeing any reviews while we returning to San Jose from Cache Creek in Brooks. This place looks like a house which was...","image_url":"http://s3-media4.ak.yelpcdn.com/bphoto/8iFj1S9YaU5IdUazwZOG8A/ms.jpg","snippet_image_url":"http://s3-media3.ak.yelpcdn.com/photo/d2TovvsTn2eUw4xqTB4jyw/ms.jpg","display_phone":"+1-925-485-4800","rating_img_url_large":"http://media4.ak.yelpcdn.com/static/20101216169592178/img/ico/stars/stars_large_4.png","id":"india-garden-pleasanton-2","categories":[["Indian","indpak"],["Pakistani","pakistani"]],"location":{"cross_streets":"Main St & Neal St","city":"Pleasanton","display_address":["210 Rose Ave","(b/t Main St & Neal St)","Pleasanton, CA 94566"],"geo_accuracy":8,"postal_code":"94566","country_code":"US","address":["210 Rose Ave"],"coordinate":{"latitude":37.660418999999997,"longitude":-121.876508},"state_code":"CA"}}]}
Actually i'm pretty sure, that body is an Array at this point, since response contains four parts and not three, so the last two parts (an array) are put into the body-object.
Also Array is the only core-object i know, which complains about an [] parameter being not an integer. If it was a string, it would try a regex/contain match.
So to sum up, body is an Array with only one Value containing a String. So to get your Hash (from JSON) you have to real_body = JSON.parse body[0]. Then you should get your hash and
real_body["businesses"] puts your businesses (the output is rather long so i will not be posting it here)
body is a string at this point, not a hash. The [] operator is complaining because the string [] operator only takes an integer, so it's trying to turn your string into an integer and failing.
edit: you can test this by just printing out body.class
I am learning Ruby and I have written the following code to find out how to consume SOAP services:
require 'soap/wsdlDriver'
wsdl="http://www.abundanttech.com/webservices/deadoralive/deadoralive.wsdl"
service=SOAP::WSDLDriverFactory.new(wsdl).create_rpc_driver
weather=service.getTodaysBirthdays('1/26/2010')
The response that I get back is:
#<SOAP::Mapping::Object:0x80ac3714
{http://www.abundanttech.com/webservices/deadoralive} getTodaysBirthdaysResult=#<SOAP::Mapping::Object:0x80ac34a8
{http://www.w3.org/2001/XMLSchema}schema=#<SOAP::Mapping::Object:0x80ac3214
{http://www.w3.org/2001/XMLSchema}element=#<SOAP::Mapping::Object:0x80ac2f6c
{http://www.w3.org/2001/XMLSchema}complexType=#<SOAP::Mapping::Object:0x80ac2cc4
{http://www.w3.org/2001/XMLSchema}choice=#<SOAP::Mapping::Object:0x80ac2a1c
{http://www.w3.org/2001/XMLSchema}element=#<SOAP::Mapping::Object:0x80ac2774
{http://www.w3.org/2001/XMLSchema}complexType=#<SOAP::Mapping::Object:0x80ac24cc
{http://www.w3.org/2001/XMLSchema}sequence=#<SOAP::Mapping::Object:0x80ac2224
{http://www.w3.org/2001/XMLSchema}element=[#<SOAP::Mapping::Object:0x80ac1f7c>,
#<SOAP::Mapping::Object:0x80ac13ec>,
#<SOAP::Mapping::Object:0x80ac0a28>,
#<SOAP::Mapping::Object:0x80ac0078>,
#<SOAP::Mapping::Object:0x80abf6c8>,
#<SOAP::Mapping::Object:0x80abed18>]
>>>>>>> {urn:schemas-microsoft-com:xml-diffgram-v1}diffgram=#<SOAP::Mapping::Object:0x80abe6c4
{}NewDataSet=#<SOAP::Mapping::Object:0x80ac1220
{}Table=[#<SOAP::Mapping::Object:0x80ac75e4
{}FullName="Cully, Zara"
{}BirthDate="01/26/1892"
{}DeathDate="02/28/1979"
{}Age="(87)"
{}KnownFor="The Jeffersons"
{}DeadOrAlive="Dead">,
#<SOAP::Mapping::Object:0x80b778f4
{}FullName="Feiffer, Jules"
{}BirthDate="01/26/1929"
{}DeathDate=#<SOAP::Mapping::Object:0x80c7eaf4>
{}Age="81"
{}KnownFor="Cartoonists"
{}DeadOrAlive="Alive">]>>>>
I am having a great deal of difficulty figuring out how to parse and show the returned information in a nice table, or even just how to loop through the records and have access to each element (ie. FullName,Age,etc). I went through the whole "getTodaysBirthdaysResult.methods - Object.new.methods" and kept working down to try and work out how to access the elements, but then I get to the array and I got lost.
Any help that can be offered would be appreciated.
If you're going to parse the XML anyway, you might as well skip SOAP4r and go with Handsoap. Disclaimer: I'm one of the authors of Handsoap.
An example implementation:
# wsdl: http://www.abundanttech.com/webservices/deadoralive/deadoralive.wsdl
DEADORALIVE_SERVICE_ENDPOINT = {
:uri => 'http://www.abundanttech.com/WebServices/DeadOrAlive/DeadOrAlive.asmx',
:version => 1
}
class DeadoraliveService < Handsoap::Service
endpoint DEADORALIVE_SERVICE_ENDPOINT
def on_create_document(doc)
# register namespaces for the request
doc.alias 'tns', 'http://www.abundanttech.com/webservices/deadoralive'
end
def on_response_document(doc)
# register namespaces for the response
doc.add_namespace 'ns', 'http://www.abundanttech.com/webservices/deadoralive'
end
# public methods
def get_todays_birthdays
soap_action = 'http://www.abundanttech.com/webservices/deadoralive/getTodaysBirthdays'
response = invoke('tns:getTodaysBirthdays', soap_action)
(response/"//NewDataSet/Table").map do |table|
{
:full_name => (table/"FullName").to_s,
:birth_date => Date.strptime((table/"BirthDate").to_s, "%m/%d/%Y"),
:death_date => Date.strptime((table/"DeathDate").to_s, "%m/%d/%Y"),
:age => (table/"Age").to_s.gsub(/^\(([\d]+)\)$/, '\1').to_i,
:known_for => (table/"KnownFor").to_s,
:alive? => (table/"DeadOrAlive").to_s == "Alive"
}
end
end
end
Usage:
DeadoraliveService.get_todays_birthdays
SOAP4R always returns a SOAP::Mapping::Object which is sometimes a bit difficult to work with unless you are just getting the hash values that you can access using hash notation like so
weather['fullName']
However, it does not work when you have an array of hashes. A work around is to get the result in xml format instead of SOAP::Mapping::Object. To do that I will modify your code as
require 'soap/wsdlDriver'
wsdl="http://www.abundanttech.com/webservices/deadoralive/deadoralive.wsdl"
service=SOAP::WSDLDriverFactory.new(wsdl).create_rpc_driver
service.return_response_as_xml = true
weather=service.getTodaysBirthdays('1/26/2010')
Now the above would give you an xml response which you can parse using nokogiri or REXML. Here is the example using REXML
require 'rexml/document'
rexml = REXML::Document.new(weather)
birthdays = nil
rexml.each_recursive {|element| birthdays = element if element.name == 'getTodaysBirthdaysResult'}
birthdays.each_recursive{|element| puts "#{element.name} = #{element.text}" if element.text}
This will print out all elements that have any text.
So once you have created an xml document you can pretty much do anything depending upon the methods the library you choose has ie. REXML or Nokogiri
Well, Here's my suggestion.
The issue is, you have to snag the right part of the result, one that is something you can actually iterator over. Unfortunately, all the inspecting in the world won't help you because it's a huge blob of unreadable text.
What I do is this:
File.open('myresult.yaml', 'w') {|f| f.write(result.to_yaml) }
This will be a much more human readable format. What you are probably looking for is something like this:
--- !ruby/object:SOAP::Mapping::Object
__xmlattr: {}
__xmlele:
- - &id024 !ruby/object:XSD::QName
name: ListAddressBooksResult <-- Hash name, so it's resul["ListAddressBooksResult"]
namespace: http://apiconnector.com
source:
- !ruby/object:SOAP::Mapping::Object
__xmlattr: {}
__xmlele:
- - &id023 !ruby/object:XSD::QName
name: APIAddressBook <-- this bastard is enumerable :) YAY! so it's result["ListAddressBooksResult"]["APIAddressBook"].each
namespace: http://apiconnector.com
source:
- - !ruby/object:SOAP::Mapping::Object
The above is a result from DotMailer's API, which I spent the last hour trying to figure out how to enumerate over the results. The above is the technique I used to figure out what the heck is going on. I think it beats using REXML etc this way, I could do something like this:
result['ListAddressBooksResult']['APIAddressBook'].each {|book| puts book["Name"]}
Well, I hope this helps anyone else who is looking.
/jason