Here is my YELP client using signet but once I get response, I'm not able to convert to ruby hash to inspect response element.
require 'rubygems'
require 'json'
require 'net/http'
client = Signet::OAuth1::Client.new(
:client_credential_key =>
'xxxxxxxxxxxxxxxxxxx',
:client_credential_secret =>
'xxxxxxxxxxxxxxxxxxx',
:token_credential_key =>
'xxxxxxxxxxxxxxxxxxx',
:token_credential_secret =>
'xxxxxxxxxxxxxxxxxxx'
)
response = client.fetch_protected_resource(
:uri => 'http://api.yelp.com/v2/search?term=food&location=san+francisco'
)
# The Rack response format is used here
status, headers, body = response
puts body["businesses"]
Error:
`[]': can't convert String into Integer (TypeError)
Body prints fine in nice JSON format but I can do body["businesses"] for instance
JSON.parse(body).inspect is also not working.
Btw body outputs itself appears as JSON format but JSON.parse(body) doesn't produce hash
puts body
{"region":{"span":{"latitude_delta":0.0,"longitude_delta":0.0},"center":{"latitude":37.660418999999997,"longitude":-121.876508}},"total":853,"businesses":[{"rating":4.0,"mobile_url":"http://m.yelp.com/biz/TT1t4oHeZmqkoiuwgCN4bQ","rating_img_url":"http://media2.ak.yelpcdn.com/static/201012164084228337/img/ico/stars/stars_4.png","review_count":150,"name":"India Garden","rating_img_url_small":"http://media2.ak.yelpcdn.com/static/20101216418129184/img/ico/stars/stars_small_4.png","url":"http://www.yelp.com/biz/india-garden-pleasanton-2","phone":"9254854800","snippet_text":"We went to this place without seeing any reviews while we returning to San Jose from Cache Creek in Brooks. This place looks like a house which was...","image_url":"http://s3-media4.ak.yelpcdn.com/bphoto/8iFj1S9YaU5IdUazwZOG8A/ms.jpg","snippet_image_url":"http://s3-media3.ak.yelpcdn.com/photo/d2TovvsTn2eUw4xqTB4jyw/ms.jpg","display_phone":"+1-925-485-4800","rating_img_url_large":"http://media4.ak.yelpcdn.com/static/20101216169592178/img/ico/stars/stars_large_4.png","id":"india-garden-pleasanton-2","categories":[["Indian","indpak"],["Pakistani","pakistani"]],"location":{"cross_streets":"Main St & Neal St","city":"Pleasanton","display_address":["210 Rose Ave","(b/t Main St & Neal St)","Pleasanton, CA 94566"],"geo_accuracy":8,"postal_code":"94566","country_code":"US","address":["210 Rose Ave"],"coordinate":{"latitude":37.660418999999997,"longitude":-121.876508},"state_code":"CA"}}]}
Actually i'm pretty sure, that body is an Array at this point, since response contains four parts and not three, so the last two parts (an array) are put into the body-object.
Also Array is the only core-object i know, which complains about an [] parameter being not an integer. If it was a string, it would try a regex/contain match.
So to sum up, body is an Array with only one Value containing a String. So to get your Hash (from JSON) you have to real_body = JSON.parse body[0]. Then you should get your hash and
real_body["businesses"] puts your businesses (the output is rather long so i will not be posting it here)
body is a string at this point, not a hash. The [] operator is complaining because the string [] operator only takes an integer, so it's trying to turn your string into an integer and failing.
edit: you can test this by just printing out body.class
Related
I am trying to convert the following JSON to CSV via Ruby, but am having trouble with my code. I am learning as I go, so any help is appreciated.
require 'json'
require 'net/http'
require 'uri'
require 'csv'
uri = 'https://www.mapquestapi.com/search/v2/radius?key=Imjtd%7Clu6t200zn0,bw=o5-layg1&radius=3000&callback=processPOIs&maxMatches=4000&origin=40.7686973%2C-73.9918181&hostedData=mqap.33882_stores_prod%7Copen_status%20=%20?%20OR%20open_status%20=%20?%20OR%20open_status%20=%20?%7CExisting,Coming%20Soon,New%7C'
response = Net::HTTP.get_response(URI.parse(uri))
struct = JSON.parse(response.body.scan(/processPOIs\((.*)\);/).first.first)
CSV.open("output.csv", "w") do |csv|
JSON.parse(struct).read.each do |hash|
csv << hash.values
end
end
The error I receive is:
from c:/RailsInstaller/Ruby2.2.0/lib/ruby/gems/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:in `new'
from c:/RailsInstaller/Ruby2.2.0/lib/ruby/gems/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:in `parse'
from test.rb:14:in `block in <main>'
from c:/RailsInstaller/Ruby2.2.0/lib/ruby/2.2.0/csv.rb:1273:in `open'
from test.rb:13:in `<main>'
I am trying to get all the data off of the following link and put it into a CSV file that I can analyse later. https://www.mapquestapi.com/search/v2/radius?key=Imjtd%7Clu6t200zn0,bw=o5-layg1&radius=3000&callback=processPOIs&maxMatches=4000&origin=40.7686973%2C-73.9918181&hostedData=mqap.33882_stores_prod%7Copen_status%20=%20?%20OR%20open_status%20=%20?%20OR%20open_status%20=%20?%7CExisting,Coming%20Soon,New%7C
You have several problems here, the most significant of which is that you're calling JSON.parse twice. The second time you call it on struct, which was the result of calling JSON.parse the first time. You're basically doing JSON.parse(JSON.parse(string)). Oops.
There's another problem on the line where you call JSON.parse a second time: You call read on the value it returns. As far as I know JSON.parse does not ordinarily return anything that responds to read.
Fixing those two errors, your code looks something like this:
struct = JSON.parse(response.body.scan(/processPOIs\((.*)\);/).first.first)
CSV.open("output.csv", "w") do |csv|
struct.each do |hash|
csv << hash.values
end
end
This ought to work iif struct is an object that responds to each (like an array) and the values yielded by each all respond to values (like a hash). In other words, this code assumes that JSON.parse will return an array of hashes, or something similar. If it doesn't—well, that's beyond the scope of this question.
As an aside, this is not great:
response.body.scan(/processPOIs\((.*)\);/).first.first
The purpose of String#scan is to find every substring in a string that matches a regular expression. But you're only concerned with the first match, so scan is the wrong choice.
An alternative is to use String#match:
matches = response.body.match(/processPOIs\((.*)\)/)
json = matches[1]
struct = JSON.parse(json)
However, that's overkill. Since this is a JSONP response, we know that it will look like this:
processPOIs(...);
...give or take a trailing semicolon or newline. We don't need a regular expression to find the parts inside the parentheses, because we already know where it is: It starts 13 characters from the start (i.e. index 12) and ends two characters before the end ("index" -3). That makes it easy work with String#slice, a.k.a. String#[]:
json = response.body[12..-3]
struct = JSON.parse(json)
Like I said, "give or take a trailing semicolon or newline," so you might need to tweak that ending index depending on what the API returns. And with that, no more ugly .first.first, and it's faster, too.
Thank you everybody for the help. I was able to get everything into a CSV and then just used some VBA to organize it the way I wanted.
require 'json'
require 'net/http'
require 'uri'
require 'csv'
uri = 'https://www.mapquestapi.com/search/v2/radius?key=Imjtd%7Clu6t200zn0,bw=o5-layg1&radius=3000&callback=processPOIs&maxMatches=4000&origin=40.7686973%2C-73.9918181&hostedData=mqap.33882_stores_prod%7Copen_status%20=%20?%20OR%20open_status%20=%20?%20OR%20open_status%20=%20?%7CExisting,Coming%20Soon,New%7C'
response = Net::HTTP.get_response(URI.parse(uri))
matches = response.body.match(/processPOIs\((.*)\)/)
json = response.body[12..-3]
struct = JSON.parse(json)
CSV.open("output.csv", "w") do |csv|
csv << struct['searchResults'].map { |result| result['fields']}
end
I have a JSON string which has been generated by Jbuilder:
json = "{name: 'Peter', email: 'peter#stackoverflow.com'}"
This is currently a string. However I want to combine it into a new hash (ideally in Ruby) before finally outputting it as JSON.
i.e.
output = {result: :success, data: json}
However if I convert this to JSON the json value gets double-encoded such that it's sent as a string:
output.to_json
#=> "{\"result\":\"success\",\"data\":\"{name: 'Peter', email: 'peter#stackoverflow.com'}\"}"
Now I could parse the JSON into a Ruby hash and then re-output it but that seems like a big fat waste of parsing when what I'd really like to do is to say "hey, this node is already JSON, don't re-encode it already!".
Is there any equivalent to the raw() method Rails has in views? i.e.
output = {result: :success, data: raw(json)}
so that the json evaluation of this then becomes:
output.to_json
#=> "{\"result\":\"success\",\"data\": {\"name\":\"Peter\",\"email\":\"peter#stackoverflow.com\"}"
Here’s a way you can do this, it’s a bit of a hack but you might find it useful.
First restating the problem:
# Note the quotes, your example isn't actually valid
json = "{\"name\": \"Peter\", \"email\": \"peter#stackoverflow.com\"}"
output = {result: :success, data: json}
# Without changing anything
puts JSON.generate(output)
This results in the following, where the value of data is a single string:
{"result":"success","data":"{\"name\": \"Peter\", \"email\": \"peter#stackoverflow.com\"}"}
The json gem uses a to_json method that is added to all objects to convert them to json, so the simplest fix would be to replace that method on objects you want to behave differently:
# As before
json = "{\"name\": \"Peter\", \"email\": \"peter#stackoverflow.com\"}"
# Replace to_json on the singleton object
def json.to_json *args
self
end
output = {result: :success, data: json}
# Generate the output (output.to_json gives the same result)
puts JSON.generate(output)
This creates the following, where the data value is now itself a hash, as desired:
{"result":"success","data":{"name": "Peter", "email": "peter#stackoverflow.com"}}
A cleaner way to do this, to avoid manipulating singletons in your code could be to create a subclass of string that has this behaviour:
class JsonSafeString < String
def to_json *args
self
end
end
You can now create a JsonSafeString when you want the contents included directly in a JSON object:
json = "{\"name\": \"Peter\", \"email\": \"peter#stackoverflow.com\"}"
output = {result: :success, data: JsonSafeString.new(json)}
puts JSON.generate(output)
The result is the same as above:
{"result":"success","data":{"name": "Peter", "email": "peter#stackoverflow.com"}}
You could wrap the call to JsonSafeString.new in a method like raw_json if you wanted.
Obviously this leaves the task of ensuring your string is valid to you – the main point of using a library for this is the user doesn’t have to concern themselves with things like whether to use single or double quotes, so you could be vulnerable to generating invalid JSON if you’re not careful. Also this is just a quick hack, there are probably a load of things I haven’t considered. In particular I haven’t taken character encodings into account, so watch out.
This doesn't address your question, but may help you avoid it altogether...
Do you really need to generate your json variable into JSON before adding it to the hash? Jbuilder can generate a hash just as easily as a JSON string, e.g.:
hash = Jbuilder.new do |json|
json.name 'Peter'
json.email 'peter#stackoverflow.com'
end.attributes!
# => {"name"=>"Peter", "email"=>"peter#stackoverflow.com"}
output = {result: :success, data: hash}
eval will put it out as raw code.
eval "{name: 'Peter', email: 'peter#stackoverflow.com'}"
=> {:name=>"Peter", :email=>"peter#stackoverflow.com"}
And the results.
output = {result: :success, data: eval("{name: 'Peter', email: 'peter#stackoverflow.com'}") }
=> {:result=>:success, :data=>{:name=>"Peter", :email=>"peter#stackoverflow.com"}}
And to string
output.to_s
=> "{:result=>:success, :data=>{:name=>\"Peter\", :email=>\"peter#stackoverflow.com\"}}"
And JSON
require 'json'
=> true
output.to_json
=> "{\"result\":\"success\",\"data\":{\"name\":\"Peter\",\"email\":\"peter#stackoverflow.com\"}}"
I have tried:
require 'net/http'
require 'json'
require 'pp'
require 'uri'
url = "http://xyz.com"
resp = Net::HTTP.get_response(URI.parse(url))
buffer = resp.body
result = JSON.parse(buffer)
#result.to_hash
#pp result
puts result
And got the output as:
{"id"=>"ABC", "account_id"=>"123", "first_name"=> "PEUS" }
in JSON format but I only need the value of id to be printed as ABC.
Your incoming string in JSON would look like:
{"id":"ABC","account_id":"123","first_name":"PEUS"}
After parsing with JSON it's the hash:
{"id"=>"ABC", "account_id"=>"123", "first_name"=> "PEUS" }
So, I'd use:
hash = {"id"=>"ABC", "account_id"=>"123", "first_name"=> "PEUS" }
hash['id'] # => "ABC"
Here's a more compact version:
require 'json'
json = '{"id":"ABC","account_id":"123","first_name":"PEUS"}'
hash = JSON[json]
hash['id'] # => "ABC"
Note I'm using JSON[json]. The JSON [] class method is smart enough to sense what the parameter being passed in is. If it's a string it'll parse the string. If it's an Array or Hash it'll serialize it. I find that handy because it allows me to write JSON[...] instead of having to remember whether I'm parsing or using to_json or something. Using it is an example of the first virtue of programmers.
My ExtJS frontend sends such a param hash to my Sinatra backend:
{"_dc"=>"1365959782607", "page"=>"6", "start"=>"250", "limit"=>"50", "sort"=>"[{\"property\":\"port\",\"direction\":\"ASC\"}]"}
How to get the params 'property' and 'direction'?
You could do it the following way:
require 'json'
a = {"_dc"=>"1365959782607", "page"=>"6", "start"=>"250", "limit"=>"50", "sort"=>"[{\"property\":\"port\",\"direction\":\"ASC\"}]"}
sort = JSON.parse a["sort"]
p sort[0]["property"] # "port"
p sort[0]["direction"] # "ASC"
Your question has nothing to do Sinatra, it's a basic question of how to extract values from a hash and deal with JSON:
require 'json'
hash = {"_dc"=>"1365959782607", "page"=>"6", "start"=>"250", "limit"=>"50", "sort"=>"[{\"property\":\"port\",\"direction\":\"ASC\"}]"}
JSON[hash['sort']].first.values_at('property', 'direction')
=> ["port", "ASC"]
Parsing the serialized object using JSON[hash['sort']] returns an array containing a single hash. first will return that hash. At that point it's just the usual methods to get at the values. I used values_at to return them as an array.
Pass JSON[] a string and JSON will try to parse it, expecting a JSON encoded object. Pass JSON[] another object, like an array or hash, and JSON will encode it into its serialized format.
I am trying to parse a SOAP response using Savon. The response is XML but is being returned as one long string. If I use #to_hash the entire XML object is still a string, now stored in
hash[:response][:return]
which means it is still a huge unusable mess.
My code looks like
response = soapClient.request(:get_sites_user_can_access) do
soap.body = { :sessionid => session[:login_response][:login_return],
:eid => user }
end
rep = response.to_hash
pp rep[:get_sites_user_can_access_response][:get_sites_user_can_access_return]
What step am I missing to get useful information out of the response? Note: Unfortunately I can't post the XML response because of the info it contains, but it looks like an entire XML document stored as a string. It's class is Nori::StringWithAttributes
I was able to get the desired results but parsing the Nori string(?) using this documentation. This seems like a less than ideal method, but I realized the last element is an array of hashes. So it's hash, of hashes, with an array of hashes. Anyway, here is what worked for me. Advice on how to make this less ugly and clunky would be appreciated.
response = soapClient.request(:get_sites_user_can_access) do
soap.body = { :sessionid => session[:login_response][:login_return],
:eid => user }
end
rep = response.to_hash[:get_sites_user_can_access_response][:get_sites_user_can_access_return]
hrep = Nori.parse(rep)
hrep[:list][:item].each { |item| pp item[:site_id] }