I have tried:
require 'net/http'
require 'json'
require 'pp'
require 'uri'
url = "http://xyz.com"
resp = Net::HTTP.get_response(URI.parse(url))
buffer = resp.body
result = JSON.parse(buffer)
#result.to_hash
#pp result
puts result
And got the output as:
{"id"=>"ABC", "account_id"=>"123", "first_name"=> "PEUS" }
in JSON format but I only need the value of id to be printed as ABC.
Your incoming string in JSON would look like:
{"id":"ABC","account_id":"123","first_name":"PEUS"}
After parsing with JSON it's the hash:
{"id"=>"ABC", "account_id"=>"123", "first_name"=> "PEUS" }
So, I'd use:
hash = {"id"=>"ABC", "account_id"=>"123", "first_name"=> "PEUS" }
hash['id'] # => "ABC"
Here's a more compact version:
require 'json'
json = '{"id":"ABC","account_id":"123","first_name":"PEUS"}'
hash = JSON[json]
hash['id'] # => "ABC"
Note I'm using JSON[json]. The JSON [] class method is smart enough to sense what the parameter being passed in is. If it's a string it'll parse the string. If it's an Array or Hash it'll serialize it. I find that handy because it allows me to write JSON[...] instead of having to remember whether I'm parsing or using to_json or something. Using it is an example of the first virtue of programmers.
Related
I am trying to convert the following JSON to CSV via Ruby, but am having trouble with my code. I am learning as I go, so any help is appreciated.
require 'json'
require 'net/http'
require 'uri'
require 'csv'
uri = 'https://www.mapquestapi.com/search/v2/radius?key=Imjtd%7Clu6t200zn0,bw=o5-layg1&radius=3000&callback=processPOIs&maxMatches=4000&origin=40.7686973%2C-73.9918181&hostedData=mqap.33882_stores_prod%7Copen_status%20=%20?%20OR%20open_status%20=%20?%20OR%20open_status%20=%20?%7CExisting,Coming%20Soon,New%7C'
response = Net::HTTP.get_response(URI.parse(uri))
struct = JSON.parse(response.body.scan(/processPOIs\((.*)\);/).first.first)
CSV.open("output.csv", "w") do |csv|
JSON.parse(struct).read.each do |hash|
csv << hash.values
end
end
The error I receive is:
from c:/RailsInstaller/Ruby2.2.0/lib/ruby/gems/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:in `new'
from c:/RailsInstaller/Ruby2.2.0/lib/ruby/gems/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:in `parse'
from test.rb:14:in `block in <main>'
from c:/RailsInstaller/Ruby2.2.0/lib/ruby/2.2.0/csv.rb:1273:in `open'
from test.rb:13:in `<main>'
I am trying to get all the data off of the following link and put it into a CSV file that I can analyse later. https://www.mapquestapi.com/search/v2/radius?key=Imjtd%7Clu6t200zn0,bw=o5-layg1&radius=3000&callback=processPOIs&maxMatches=4000&origin=40.7686973%2C-73.9918181&hostedData=mqap.33882_stores_prod%7Copen_status%20=%20?%20OR%20open_status%20=%20?%20OR%20open_status%20=%20?%7CExisting,Coming%20Soon,New%7C
You have several problems here, the most significant of which is that you're calling JSON.parse twice. The second time you call it on struct, which was the result of calling JSON.parse the first time. You're basically doing JSON.parse(JSON.parse(string)). Oops.
There's another problem on the line where you call JSON.parse a second time: You call read on the value it returns. As far as I know JSON.parse does not ordinarily return anything that responds to read.
Fixing those two errors, your code looks something like this:
struct = JSON.parse(response.body.scan(/processPOIs\((.*)\);/).first.first)
CSV.open("output.csv", "w") do |csv|
struct.each do |hash|
csv << hash.values
end
end
This ought to work iif struct is an object that responds to each (like an array) and the values yielded by each all respond to values (like a hash). In other words, this code assumes that JSON.parse will return an array of hashes, or something similar. If it doesn't—well, that's beyond the scope of this question.
As an aside, this is not great:
response.body.scan(/processPOIs\((.*)\);/).first.first
The purpose of String#scan is to find every substring in a string that matches a regular expression. But you're only concerned with the first match, so scan is the wrong choice.
An alternative is to use String#match:
matches = response.body.match(/processPOIs\((.*)\)/)
json = matches[1]
struct = JSON.parse(json)
However, that's overkill. Since this is a JSONP response, we know that it will look like this:
processPOIs(...);
...give or take a trailing semicolon or newline. We don't need a regular expression to find the parts inside the parentheses, because we already know where it is: It starts 13 characters from the start (i.e. index 12) and ends two characters before the end ("index" -3). That makes it easy work with String#slice, a.k.a. String#[]:
json = response.body[12..-3]
struct = JSON.parse(json)
Like I said, "give or take a trailing semicolon or newline," so you might need to tweak that ending index depending on what the API returns. And with that, no more ugly .first.first, and it's faster, too.
Thank you everybody for the help. I was able to get everything into a CSV and then just used some VBA to organize it the way I wanted.
require 'json'
require 'net/http'
require 'uri'
require 'csv'
uri = 'https://www.mapquestapi.com/search/v2/radius?key=Imjtd%7Clu6t200zn0,bw=o5-layg1&radius=3000&callback=processPOIs&maxMatches=4000&origin=40.7686973%2C-73.9918181&hostedData=mqap.33882_stores_prod%7Copen_status%20=%20?%20OR%20open_status%20=%20?%20OR%20open_status%20=%20?%7CExisting,Coming%20Soon,New%7C'
response = Net::HTTP.get_response(URI.parse(uri))
matches = response.body.match(/processPOIs\((.*)\)/)
json = response.body[12..-3]
struct = JSON.parse(json)
CSV.open("output.csv", "w") do |csv|
csv << struct['searchResults'].map { |result| result['fields']}
end
I have a query string that looks as follows:
http://localhost:3000/events?appointment_practices%5B10%5D=Injury&appointment_practices%5B18%5D=Immigration&appointment_practices%5B8%5D=Bankruptcy
appointment_practices is actually a hash I inserted into the query string during a redirect:
appointment_practices = practices.reduce({}) do |acc, practice|
acc[practice.id] = practice.class.name
acc
end
redirect_to events_path(appointment_practices: appointment_practices)
Now I want to parse that query string. When I tried to parse it with decode_www_form, it returns an array with a nil element:
[nil]
This is the code that is giving me the nil element:
#http_refer = #_env['HTTP_REFERER']
begin
uri = URI.parse #http_refer
practices = Hash[URI::decode_www_form(uri.query)].values_at('appointment_practices')
puts "practices: #{practices}"
rescue StandardError
end
I am trying to extract the hash. For example, in appointment_practices%5B10%5D=Injury, the id is 10 and the practice is Injury.
What other options do I have besides regex?
You can use Rack::Utils.parse_nested_query:
require 'uri'
require 'rack'
uri = URI.parse('http://localhost:3000/events?appointment_practices%5B10%5D=Injury&appointment_practices%5B18%5D=Immigration&appointment_practices%5B8%5D=Bankruptcy')
Rack::Utils.parse_nested_query(uri.query)
#=> {"appointment_practices"=>{"10"=>"Injury", "18"=>"Immigration", "8"=>"Bankruptcy"}}
Code:
#!/usr/bin/ruby
require 'rubygems'
require 'open-uri'
require 'json'
def getData
file = open("http://goo.gl/BI6h7a")
#json = JSON.parse(file.read)
end
getData
cveIds = #json['cve_id']
puts cveIds
You can see the JSON response here: http://goo.gl/BI6h7a
Console:
./cve.rb:13:in `[]': can't convert String into Integer (TypeError) from ./cve.rb:13:in `<main>'
I don't know why this is happening. "Convert String into Integer"? WHAT?
The #json gets the content fine, but the cveIds doesn't.
The top element in the json that you're reading is actually an Array, each of its elements is actually a hash, it's like this:
[
{
"cve_id": "CVE-2014-3976"
// other key/value pairs
}
{
"cve_id": "CVE-2014-3975"
// other key/value pairs
}
{
"cve_id": "CVE-2014-3974"
// other key/value pairs
}
// .... more hashes
]
so #json is an array. And if you want to access any of its elements you have to access it with a numeric integer index like, so:
#json[0] # => { "cve_id": "CVE-2014-3976", // other key/value pairs }
I think you are trying to collect the cve_id fields of all these hashes, this can be done as follows:
cveIds = #json.collect { |h| h["cve_id"] }
# The result:
=> ["CVE-2014-3976", "CVE-2014-3975", "CVE-2014-3974", "CVE-2014-3962", "CVE-2014-3961",
"CVE-2014-3878", "CVE-2014-3871", "CVE-2014-3842", "CVE-2014-3806", "CVE-2014-3792",
"CVE-2014-3791", "CVE-2014-3443", "CVE-2014-3247", "CVE-2014-3246", "CVE-2014-3225",
"CVE-2014-3216", "CVE-2014-3139", "CVE-2014-3138", "CVE-2014-3008", "CVE-2014-2996",
"CVE-2014-2994", "CVE-2014-2976", "CVE-2014-2850", "CVE-2014-2847", "CVE-2014-2671",
"CVE-2014-2668", "CVE-2014-2588", "CVE-2014-2587","CVE-2014-2586", "CVE-2014-2579"]
I'm not a ruby developer but what you have there is a list if dictionaries.
My guess in order for you to read cve_id you need to create some kind of a for loop.
for example in python I would write it like this:
for line in my_data:
print line['cve_id']
I guess in ruby it would look like this:
for i in #json do
cveIds = i['cve_id']
puts cveIds
end
cveIds = #json['cve_id']
What are you doing here is equivalent to:
arr = [1, 2, 3, 4]
puts arr["hello"] # using a string here on an indexed based array!
Hence your error message about Ruby trying to convert a String to an int.
Try the following instead
cveIds = #json.first['cve_id'] # equivalent to #json[0]['cve_id']
puts cveIds
In the above code sample, we are getting the first element from the array, which is a hash we can then access cve_id from.
My ExtJS frontend sends such a param hash to my Sinatra backend:
{"_dc"=>"1365959782607", "page"=>"6", "start"=>"250", "limit"=>"50", "sort"=>"[{\"property\":\"port\",\"direction\":\"ASC\"}]"}
How to get the params 'property' and 'direction'?
You could do it the following way:
require 'json'
a = {"_dc"=>"1365959782607", "page"=>"6", "start"=>"250", "limit"=>"50", "sort"=>"[{\"property\":\"port\",\"direction\":\"ASC\"}]"}
sort = JSON.parse a["sort"]
p sort[0]["property"] # "port"
p sort[0]["direction"] # "ASC"
Your question has nothing to do Sinatra, it's a basic question of how to extract values from a hash and deal with JSON:
require 'json'
hash = {"_dc"=>"1365959782607", "page"=>"6", "start"=>"250", "limit"=>"50", "sort"=>"[{\"property\":\"port\",\"direction\":\"ASC\"}]"}
JSON[hash['sort']].first.values_at('property', 'direction')
=> ["port", "ASC"]
Parsing the serialized object using JSON[hash['sort']] returns an array containing a single hash. first will return that hash. At that point it's just the usual methods to get at the values. I used values_at to return them as an array.
Pass JSON[] a string and JSON will try to parse it, expecting a JSON encoded object. Pass JSON[] another object, like an array or hash, and JSON will encode it into its serialized format.
I have a string that is URL encoded:
a = "%5B%22552A8619-6ECA-4A95-A798-C1E2CE75BFFF%22%2C%2264c19b5b2d0257ddb382dbd3660de3fd%22%2C%22share%22%5D"
If I URL decode this string then it will look like:
"[\"552A8619-6ECA-4A95-A798-C1E2CE75BFFF\",\"64c19b5b2d0257ddb382dbd3660de3fd\",\"share\"]"
From this string I want to get this array:
["552A8619-6ECA-4A95-A798-C1E2CE75BFFF","64c19b5b2d0257ddb382dbd3660de3fd","share"]
How to do that without nasty string replacements?
the_given_string.scan(/"(.*?)"/).flatten
The string is an array encoded using JSON:
require 'cgi'
require 'json'
a = "%5B%22552A8619-6ECA-4A95-A798-C1E2CE75BFFF%22%2C%2264c19b5b2d0257ddb382dbd3660de3fd%22%2C%22share%22%5D"
JSON[CGI::unescape(a)]
[
[0] "552A8619-6ECA-4A95-A798-C1E2CE75BFFF",
[1] "64c19b5b2d0257ddb382dbd3660de3fd",
[2] "share"
]
JSON[CGI::unescape(a)].last will return "share", putting you home free.
CGI::escape is used to remove the encoding, which turns it back to a "normal" JSON-encoded array.
JSON[] (AKA JSON.parse) converts it from the JSON notation back to a Ruby array.
You could delete characters and split, or evaluate it:
"[\"A798-C1E2CE75BFFF\",\"643fd\",\"share\"]".delete('\"[]').split(',')
# => ["A798-C1E2CE75BFFF", "643fd", "share"]
eval "[\"A798-C1E2CE75BFFF\",\"643fd\",\"share\"]"
# => ["A798-C1E2CE75BFFF", "643fd", "share"]
You could eval the string:
require 'cgi'
a = "%5B%22552A8619-6ECA-4A95-A798-C1E2CE75BFFF%22%2C%2264c19b5b2d0257ddb382dbd3660de3fd%22%2C%22share%22%5D"
x = eval( CGI.unescape(a))
p x #["552A8619-6ECA-4A95-A798-C1E2CE75BFFF", "64c19b5b2d0257ddb382dbd3660de3fd", "share"]
But eval is evil.
You could use , what you call nasty string replacement:
p CGI.unescape(a).gsub(/\A\["|"\]\Z/,'').split(/","/)
Or you could try JSON:
require 'cgi'
require 'json'
a = "%5B%22552A8619-6ECA-4A95-A798-C1E2CE75BFFF%22%2C%2264c19b5b2d0257ddb382dbd3660de3fd%22%2C%22share%22%5D"
x = JSON.load( CGI.unescape(a))