How do I read JSON received from a REST API? - ruby

I'm trying to read some JSON that I received from a REST API but I'm having some issues.
To get my JSON, I'm using Open::URI. I created my request like this:
require "open-uri"
require "json"
content = open("http://foo.bar/test.json").read
result = JSON.parse(content)
At this point my JSON is supposed to be parsed from a string, and so if I correctly understood, a hash containing my JSON is built assuming the JSON I received has a structure that looks like this:
{
"root":
{
"foos":
{
"1":
{
"title" : "zero",
"number" : 0
},
"2":
{
"title" : "twenty",
"number" : 20
},
...
}
}
}
I would like to iterate over each foos and, for each of them, get the title and the number. I tried this:
content["root"]["foos"].each do |foo| puts foo.title + " " + foo.number end
But, as output, I got:
#<Enumerator:0x007fceb8b33718>
Where is/are my mistake(s)?
Thanks in advance,

Here's an option... Iterate over the keys inside of the foos object.
json = JSON.parse(your_sample_json)
=> {"root"=>{"foos"=>{"1"=>{"title"=>"zero", "number"=>0}, "2"=>{"title"=>"twenty", "number"=>20}}}}
foos = json["root"]["foos"]
=> {"1"=>{"title"=>"zero", "number"=>0}, "2"=>{"title"=>"twenty", "number"=>20}}
foos.keys.each { |key| puts foos[key]["title"] }
zero
twenty
Also, if you have control over the JSON object you're parsing, you could make foos an array instead of a bunch of numbered objects.

I'd do it like this:
require 'json'
require 'pp'
hash = JSON.parse(
'{
"root": {
"foos": {
"1": {
"title": "zero",
"number": 0
},
"2": {
"title": "twenty",
"number": 20
}
}
}
}'
)
results = hash['root']['foos'].map{ |k, v|
[v['title'], v['number']]
}
pp results
After running it outputs an array of arrays:
[["zero", 0], ["twenty", 20]]
map might behave a bit differently than you'd expect with a hash; It assigns each key/value of the hash as an array of two elements. The key is the first element, the value is the second. Because your structure is a hash of hashes of hashes of hashes, when iterating over hash['root']['foos'] the values for keys "1" and "2" are a hash, so you can access their values like you would a hash.
Back to your code:
hash["root"]["foos"].each do |foo|
puts foo.title + " " + foo.number
end
won't work. It doesn't return an enumerator at all, so that part of the question is inaccurate. What your code returns is:
undefined method `title' for ["1", {"title"=>"zero", "number"=>0}]:Array (NoMethodError)

Related

Merge duplicate values in json using ruby

I have the following item.json file
{
"items": [
{
"brand": "LEGO",
"stock": 55,
"full-price": "22.99",
},
{
"brand": "Nano Blocks",
"stock": 12,
"full-price": "49.99",
},
{
"brand": "LEGO",
"stock": 5,
"full-price": "199.99",
}
]
}
There are two items named LEGO and I want to get output for the total number of stock for the individual brand.
In ruby file item.rb i have code like:
require 'json'
path = File.join(File.dirname(__FILE__), '../data/products.json')
file = File.read(path)
products_hash = JSON.parse(file)
products_hash["items"].each do |brand|
puts "Stock no: #{brand["stock"]}"
end
I got output for stock no individually for each brand wherein I need the stock to be summed for two brand name "LEGO" displayed as one.
Anyone has solution for this?
json = File.open(path,'r:utf-8',&:read) # in case the JSON uses UTF-8
items = JSON.parse(json)['items']
stock_by_brand = items
.group_by{ |h| h['brand'] }
.map do |brand,array|
[ brand,
array
.map{ |item| item['stock'] }
.inject(:+) ]
end
.to_h
#=> {"LEGO"=>60, "Nano Blocks"=>12}
It works like this:
Enumerable#group_by takes the array of items and creates a hash mapping the brand name to an array of all item hashes with that brand
Enumerable#map turns each brand/array pair in that hash into an array of the brand (unchanged) followed by:
Enumerable#map on the array of items picks out just the "stock" counts, and then
Enumerable#inject sums them all together
Array#to_h then turns that array of two-value arrays into a hash, mapping the brand to the sum of stock values.
If you want simpler code that's less functional and possibly easier to understand:
stock_by_brand = {} # an empty hash
items.each do |item|
stock_by_brand[ item['brand'] ] ||= 0 # initialize to zero if unset
stock_by_brand[ item['brand'] ] += item['stock']
end
p stock_by_brand #=> {"LEGO"=>60, "Nano Blocks"=>12}
To see what your JSON string looks like, let's create it from your hash, which I've denoted h:
require 'json'
j = JSON.generate(h)
#=> "{\"items\":[{\"brand\":\"LEGO\",\"stock\":55,\"full-price\":\"22.99\"},{\"brand\":\"Nano Blocks\",\"stock\":12,\"full-price\":\"49.99\"},{\"brand\":\"LEGO\",\"stock\":5,\"full-price\":\"199.99\"}]}"
After reading that from a file, into the variable j, we can now parse it to obtain the value of "items":
arr = JSON.parse(j)["items"]
#=> [{"brand"=>"LEGO", "stock"=>55, "full-price"=>"22.99"},
# {"brand"=>"Nano Blocks", "stock"=>12, "full-price"=>"49.99"},
# {"brand"=>"LEGO", "stock"=>5, "full-price"=>"199.99"}]
One way to obtain the desired tallies is to use a counting hash:
arr.each_with_object(Hash.new(0)) {|g,h| h.update(g["brand"]=>h[g["brand"]]+g["stock"])}
#=> {"LEGO"=>60, "Nano Blocks"=>12}
Hash.new(0) creates an empty hash (represented by the block variable h) with with a default value of zero1. That means that h[k] returns zero if the hash does not have a key k.
For the first element of arr (represented by the block variable g) we have:
g["brand"] #=> "LEGO"
g["stock"] #=> 55
Within the block, therefore, the calculation is:
g["brand"] => h[g["brand"]]+g["stock"]
#=> "LEGO" => h["LEGO"] + 55
Initially h has no keys, so h["LEGO"] returns the default value of zero, resulting in { "LEGO"=>55 } being merged into the hash h. As h now has a key "LEGO", h["LEGO"], will not return the default value in subsequent calculations.
Another approach is to use the form of Hash#update (aka merge!) that employs a block to determine the values of keys that are present in both hashes being merged:
arr.each_with_object({}) {|g,h| h.update(g["brand"]=>g["stock"]) {|_,o,n| o+n}}
#=> {"LEGO"=>60, "Nano Blocks"=>12}
1 k=>v is shorthand for { k=>v } when it appears as a method's argument.

Sorting a hash of hashes in ruby

How can I sort this hash of hashes by "clients". I tried using sort_by, but this transforms it into an array of hashes. I am using JSON.parse to create this object from a json file. Thanks!
{
"default_attributes": {
"clients": {
"ABC": {
"db_name": "databaseabc"
},
"HIJ": {
"db_name": "databasehij"
},
"DEF": {
"db_name": "databasedef"
}
}
}
}
Why do you want to sort a hash? There's no advantage to it. Instead, get the keys, sort those, then use the keys to retrieve the data in the order you want.
For instance:
hash = {'z' => 26, 'a' => 1}
sorted_keys = hash.keys.sort # => ["a", "z"]
hash.values_at(*sorted_keys) # => [1, 26]
Using your example hash:
hash = {
"default_attributes": {
"clients": {
"ABC": {
"db_name": "databaseabc"
},
"HIJ": {
"db_name": "databasehij"
},
"DEF": {
"db_name": "databasedef"
}
}
}
}
clients = hash[:default_attributes][:clients]
sorted_keys = clients.keys.sort # => [:ABC, :DEF, :HIJ]
clients.values_at(*sorted_keys)
# => [{:db_name=>"databaseabc"},
# {:db_name=>"databasedef"},
# {:db_name=>"databasehij"}]
Or:
sorted_keys.each do |k|
puts clients[k][:db_name]
end
# >> databaseabc
# >> databasedef
# >> databasehij
Note: From looking at your "hash", it really looks like a JSON string missing the original surrounding { and }. If it is, this question becomes somewhat of an "XY problem". The first question should be "how do I convert a JSON string back to a Ruby object?":
require 'json'
hash = '{
"default_attributes": {
"clients": {
"ABC": {
"db_name": "databaseabc"
},
"HIJ": {
"db_name": "databasehij"
},
"DEF": {
"db_name": "databasedef"
}
}
}
}'
foo = JSON[hash]
# => {"default_attributes"=>
# {"clients"=>
# {"ABC"=>{"db_name"=>"databaseabc"},
# "HIJ"=>{"db_name"=>"databasehij"},
# "DEF"=>{"db_name"=>"databasedef"}}}}
At that point foo would contain a regular hash, and the inconsistent symbol definitions like "default_attributes": and "clients": would make sense because they ARE JSON hash keys, and the resulting parsed object would be a standard Ruby hash definition. And, you'll have to adjust the code above to access the individual nested hash keys.
If you are using Ruby <1.9, hashes are order-undefined. Sorting them makes no sense.
Ruby 1.9+ has ordered hashes; you would use sort_by, then convert your array of hashes back into a hash. Ruby 2.0+ provides Array#to_h for this.
data["default_attributes"]["clients"] = data["default_attributes"]["clients"].sort_by(&:first).to_h
hash = {
default_attributes: {
clients: {
ABC: {
"db_name": "databaseabc"
},
HIJ: {
"db_name": "databasehij"
},
DEF: {
"db_name": "databasedef"
}
}
}
}
If you do not wish to mutate hash, it's easiest to first make a deep copy:
h = Marshal.load(Marshal.dump(hash))
and then sort the relevant part of h:
h[:default_attributes][:clients] =
h[:default_attributes][:clients].sort.to_h
h
#=> {:default_attributes=>
# {:clients=>
# {:ABC=>{:db_name=>"databaseabc"},
# :DEF=>{:db_name=>"databasedef"},
# :HIJ=>{:db_name=>"databasehij"}}}}
Confirm hash was not mutated:
hash
#=> {:default_attributes=>
# {:clients=>
# {:ABC=>{:db_name=>"databaseabc"},
# :HIJ=>{:db_name=>"databasehij"},
# :DEF=>{:db_name=>"databasedef"}}}}
One of our interns came up with a pretty slick gem to perform deep sorts on hashes/arrays:
def deep_sort_by(&block)
Hash[self.map do |key, value|
[if key.respond_to? :deep_sort_by
key.deep_sort_by(&block)
else
key
end,
if value.respond_to? :deep_sort_by
value.deep_sort_by(&block)
else
value
end]
end.sort_by(&block)]
end
You can inject it into all hashes and then just call it like this:
[myMap.deep_sort_by { |obj| obj }][1]
The code would be similar for an array. We published "deepsort" as a gem for others to use. See "Deeply Sort Nested Ruby Arrays And Hashes" for additional details.
Disclaimer: I work for this company.

Sorting an array of hashes by a date field

I have an object with many arrays of hashes, one of which I want to sort by a value in the 'date' key.
#array['info'][0] = {"name"=>"personA", "date"=>"23/09/1980"}
#array['info'][1] = {"name"=>"personB", "date"=>"01/04/1970"}
#array['info'][2] = {"name"=>"personC", "date"=>"03/04/1975"}
I have tried various methods using Date.parse and with collect but an unable to find a good solution.
Edit:
To be clear I want to sort the original array in place
#array['info'].sort_by { |i| Date.parse i['date'] }.collect
How might one solve this elegantly the 'Ruby-ist' way. Thanks
Another way, which doesn't require converting the date strings to date objects, is the following.
Code
def sort_by_date(arr)
arr.sort_by { |h| h["date"].split('/').reverse }
end
If arr is to be sorted in place, use Array#sort_by! rather than Enumerable#sort_by.
Example
arr = [{ "name"=>"personA", "date"=>"23/09/1980" },
{ "name"=>"personB", "date"=>"01/04/1970" },
{ "name"=>"personC", "date"=>"03/04/1975" }]
sort_by_date(arr)
#=> [{ "name"=>"personB", "date"=>"01/04/1970" },
# { "name"=>"personC", "date"=>"03/04/1975" },
# { "name"=>"personA", "date"=>"23/09/1980" }]
Explanation
For arr in the example, sort_by passes the first element of arr into its block and assigns it to the block variable:
h = { "name"=>"personA", "date"=>"23/09/1980" }
then computes:
a = h["date"].split('/')
#=> ["23", "09", "1980"]
and then:
b = a.reverse
#=> ["1980", "09", "23"]
Similarly, we obtain b equal to:
["1970", "04", "01"]
and
["1975", "04", "03"]
for each of the other two elements of arr.
If you look at the docs for Array#<=> you will see that these three arrays are ordered as follows:
["1970", "04", "01"] < ["1975", "04", "03"] < ["1980", "09", "23"]
There is no need to convert the string elements to integers.
Looks fine overall. Although you can drop the collect call since it's not needed and use sort_by! to modify the array in-place (instead of reassigning):
#array['info'].sort_by! { |x| Date.parse x['date'] }

Summing Nested Json

So, we have a json response like:
Link to Formatted Sample Json
{"C":{"1":{"1":{"A":[18],"B":[18],"C":[20],"D":[24],"E":[24],"F":[2],"G":[15],"H":[21],"I":[8]},"2":{"A":[9],"B":[26],"C":[12],"D":[10],"E":[10],"F":[3],"G":[7]},"3":{"A":[6],"B":[4],"C":[5],"D":[3],"E":[4],"F":[13]},"4":{"A":[3],"B":[2],"C":[5],"D":[13],"E":[5],"F":[5],"G":[4],"H":[7]},"5":{"A":[10],"B":[10],"C":[10],"D":[10],"E":[10],"F":[15]},"6":{"A":[10],"B":[7],"C":[5],"D":[4],"E":[7],"F":[10],"G":[4],"H":[18]},"7":{"A":[2],"B":[18],"C":[6],"D":[3],"E":[2],"F":[5],"G":[7],"H":[5],"I":[17]},"8":{"A":[20],"B":[2],"C":[10],"D":[3],"E":[5],"F":[10]},"Review 1":{"A":[30]},"Review 2":{"A":[30]}},"2":{"1":{"A":[2],"B":[3],"C":[10],"D":[10],"E":[10],"F":[15]},"10":{"A":[10],"B":[3],"C":[3],"D":[3],"E":[20]},"11":{"A":[2],"B":[6],"C":[5],"D":[10],"E":[10],"F":[13]},"2":{"A":[5],"B":[5],"C":[5],"D":[6],"E":[6],"F":[12],"G":[6],"H":[8]},"3":{"A":[3],"B":[4],"C":[8],"D":[3],"E":[2],"F":[3],"G":[12]},"4":{"A":[10],"B":[10],"C":[10],"D":[11],"E":[10],"F":[20]},"5":{"A":[8],"B":[4],"C":[8],"D":[5],"E":[14]},"6":{"A":[5],"B":[10],"C":[14],"D":[14]},"7":{"A":[3],"B":[5],"C":[8],"D":[9],"E":[10],"F":[16]},"8":{"A":[2],"B":[2],"C":[4],"D":[2],"E":[3],"F":[6],"G":[8]},"9":{"A":[2],"B":[6],"C":[5],"D":[11]},"_mex":{"1":[9]},"Review 1":{"A":[31]},"Review 2":{"A":[30]},"Review 3":{"A":[30]}},"3":{"1":{"A":[1],"B":[1],"C":[1],"D":[2],"E":[6]},"2":{"A":[2],"B":[4],"C":[7],"D":[8],"E":[8],"F":[9]},"3":{"A":[5],"B":[8],"C":[11]},"4":{"A":[10],"B":[10],"C":[11]},"5":{"A":[2],"B":[4],"C":[5],"D":[1],"E":[3],"F":[8]},"6":{"A":[4],"B":[8],"C":[8],"D":[12],"E":[8],"F":[20]},"7":{"A":[25],"B":[12],"C":[13],"D":[15],"E":[12],"F":[20]},"8":{"A":[5],"B":[3],"C":[3],"D":[7],"E":[1],"F":[1],"G":[1],"H":[1],"I":[1],"J":[3],"K":[17]},"mex2":{"A":[7]},"_mex2":{"A":[7]},"Review 1":{"A":[30]},"Review 2":{"A":[30]}},"4":{"1":{"A":[10],"B":[2],"C":[2],"D":[8],"E":[3],"F":[3]},"2":{"A":[5],"B":[10],"C":[5],"D":[10],"E":[10]},"3":{"A":[6],"B":[4],"C":[3],"D":[11]},"4":{"A":[4],"B":[4],"C":[4],"D":[4],"E":[11],"F":[21]},"5":{"A":[5],"B":[8],"C":[3],"D":[4],"E":[5],"F":[7],"G":[15],"H":[5],"I":[5],"J":[6],"K":[14]},"6":{"A":[2],"B":[4],"C":[3],"D":[2],"E":[2],"F":[3],"G":[4],"H":[4],"I":[4],"J":[4],"K":[7],"L":[34]},"_mex2":{"A":[7]},"Review 1":{"A":[77]}}}}
What I want to do is sum all the numbers contained in the response.
Ive tried iterating through all the nesting but I was only been able to do one section. Using:
#number = 0
json["C"]["1"]["1"].each do |key, val|
val.map do |x|
#number+=x
end
end
#=> 150
Any suggestions how I would do that same for json["C"]["1"]?
Based on the JSON, here's code that'll walk the hash:
hash = JSON.parse(json)
def sum_hash(h)
sum = 0
h.each do |k, v|
sum += v.is_a?(Hash) ? sum_hash(v) : v.first
end
sum
end
sum_hash(hash) # => 1964
The hash has to be walked, and each value inspected since it's irregular. If the value is another hash sum_hash calls itself with that sub-hash, which then begins walking the sub-hash received.
For each hash value that isn't a hash, the integer is retrieved from the array using first and added to sum. When the method exits it returns the current value of sum, so, once the hash has been descended into, successive sum values get added.
Reducing the JSON makes it a LOT easier to make sure the code is doing the right thing:
json = '
{
"C": {
"1": {
"1": {
"A": [1],
"B": [1]
},
"2": {
"A": [1],
"B": [1]
},
"Review 1": {
"A": [1]
},
"Review 2": {
"A": [1]
}
}
}
}
'
Running the above code with that says the sum is 6.

Is there a quick and easy way to create a checksum from Ruby's basic data structures?

I have a data structure (Hash) that looks something like this:
{
foo: "Test string",
bar: [475934759, 5619827847]
}
I'm trying to create a checksum from that Hash to check for equality in the future. I tried using the hash method of the Hash, which resulted in a satisfyingly nice-looking hash, but it turns out that the same Hash will produce a different hash after the interpreter has been restarted.
I really just want to be able to create a ~128 bit checksum from a Hash, String or Array instance.
Is this possible?
You could calculate your own hash based on the object's Marshal dump or JSON representation.
This calculates the MD5 hash of a Marshal dump:
require 'digest/md5'
hash = {
foo: "Test string",
bar: [475934759, 5619827847]
}
Marshal::dump(hash)
#=> "\x04\b{\a:\bfooI\"\x10Test string\x06:\x06ET:\bbar[\ai\x04'0^\x1Cl+\b\x87\xC4\xF7N\x01\x00"
Digest::MD5.hexdigest(Marshal::dump(hash))
#=> "1b6308abdd8f5f6290e2825a078a1a02"
Update
You can implement your own strategy, although I would not recommend to change core functionality:
class Hash
def _dump(depth)
# this doesn't cause a recursion because sort returns an array
Marshal::dump(self.sort, depth)
end
def self._load(marshaled_hash)
Hash[Marshal::load(marshaled_hash)]
end
end
Marshal::dump({foo:1, bar:2})
#=> "\x04\bu:\tHash\e\x04\b[\a[\a:\bbari\a[\a:\bfooi\x06"
Marshal::dump({bar:2, foo:1})
#=> "\x04\bu:\tHash\e\x04\b[\a[\a:\bbari\a[\a:\bfooi\x06"
Marshal::load(Marshal::dump({foo:1, bar:2}))
#=> {:bar=>2, :foo=>1}
To build on #Stefan's answer above, if order of the hash is important, sort the output before pushing it through Mashall.
require 'digest/md5'
hash = {
'foo'=> "Test string",
'bar'=> [475934759, 5619827847]
}
puts Digest::MD5.hexdigest(Marshal::dump(hash.collect{|k,v| [k,v]}.sort{|a,b| a[0] <=> b[0]}))
# 8509c564c0ae8dcb6c2b9b564ba6a03f
hash = {
'bar'=> [475934759, 5619827847],
'foo'=> "Test string"
}
puts Digest::MD5.hexdigest(Marshal::dump(hash.collect{|k,v| [k,v]}.sort{|a,b| a[0] <=> b[0]}))
# 8509c564c0ae8dcb6c2b9b564ba6a03f
If you need to generate the checksum for the content of the hash, whatever the order of the data, using Marshal or sort or other techniques won't work.
The only solid way I found so far is the following:
require 'digest/md5'
hash1 = { "a" => 1, "b" => "2", c: { d: "3" } }
hash2 = { c: { d: "3" }, "a" => 1, "b" => "2" }
Digest::MD5.hexdigest(Marshal.dump(hash1)) # => "5def3b2cbdddd3aa6730b6d0527c2d79"
Digest::MD5.hexdigest(Marshal.dump(hash2)) # => "8155698ccfb05b8db01490e9b9634fd9"
Digest::MD5.hexdigest(hash1.to_s.chars.sort.join) # => "812bb65d65380fc1e620a9596806cc35"
Digest::MD5.hexdigest(hash2.to_s.chars.sort.join) # => "812bb65d65380fc1e620a9596806cc35"

Resources