For my rails app, SQL query result is received in the below format.
#data= JSON.parse(request,symbolize_names: true)[:data]
# #data sample
[{"time":"2017-11-14","A":0,"B":0,"C":0,"D":0,"E":0},
{"time":"2017-11-15","A":0,"B":0,"C":0,"D":0,"E":0},
{"time":"2017-11-16","A":2,"B":1,"C":1,"D":0,"E":1},
{"time":"2017-11-17","A":0,"B":0,"C":1,"D":0,"E":1},
{"time":"2017-11-20","A":0,"B":0,"C":0,"D":0,"E":0},
{"time":"2017-11-21","A":6,"B":17,"C":0,"D":0,"E":1}]
But I want the data in the format
[{"name":"A","data":{"2017-11-16":2,"2017-11-21":6}},
{"name":"B","data":{"2017-11-16":1,"2017-11-21":17}},
{"name":"C","data":{"2017-11-16":1,"2017-11-17":1}},
{"name":"D","data":{}},
{"name":"E","data":{"2017-11-16":1,"2017-11-17":1,"2017-11-21":1}}]
What is the best way to parse this in Ruby?
I tried using #data.each method, but it is lengthy.
I am totally new to Ruby. Any help would be appreciated.
Oddly specific question, but kinda an interesting problem so I took a stab at it. If this is coming from a SQL database I feel like the better solution would be to have SQL format the data for you as opposed to transforming it in ruby.
#data = JSON.parse(request,symbolize_names: true)[:data]
intermediate = {}
#data.each do |row|
time = row.delete(:time)
row.each do |key, val|
intermediate[key] ||= {data: {}}
intermediate[key][:data][time] = val if val > 0
end
end
transformed = []
intermediate.each do |key, val|
transformed << {name: key.to_s, data: val}
end
At the end of this transformed will contain the transformed data. Horrible variable names, and I hate having to do this in two passes. But got something working and figured I would share in case it is helpful.
I agree with csexton that it looks like a better query to source the data would be the ultimate solution here.
Anyway, here's a solution that's similar to csexton's but uses nested default Hash procs to simplify some of the operations:
def pivot(arr, column)
results = Hash.new do |hash, key|
hash[key] = Hash.new(0)
end
arr.each do |hash|
data = hash.dup
pivot = data.delete(column)
data.each_pair do |name, value|
results[name][pivot] += value
end
end
results.map { |name, data| {
name: name.to_s,
data: data.delete_if { |_, sum| sum.zero? }
}}
end
pivot(#data, :time) # => [{:name=>"A", :data=>{"2017-11-16"=>2, "2017-11-21"=>6}}, ..
Here's a more "Ruby-ish" (depending on who you ask) solution:
def pivot(arr, column)
arr
.flat_map do |hash|
hash
.to_a
.delete_if { |key, _| key == column }
.map! { |data| data << hash[column] }
end
.group_by(&:shift)
.map { |name, outer| {
name: name.to_s,
data: outer
.group_by(&:last)
.transform_values! { |inner| inner.sum(&:first) }
.delete_if { |_, sum| sum.zero? }
}}
end
pivot(#data, :time) # => [{:name=>"A", :data=>{"2017-11-16"=>2, "2017-11-21"=>6}}, ..
Quite frankly, I find it pretty unreadable and I wouldn't want to support it. :)
arr = [{"time":"2017-11-14","A":0,"B":0,"C":0,"D":0,"E":0},
{"time":"2017-11-15","A":0,"B":0,"C":0,"D":0,"E":0},
{"time":"2017-11-16","A":2,"B":1,"C":1,"D":0,"E":1},
{"time":"2017-11-17","A":0,"B":0,"C":1,"D":0,"E":1},
{"time":"2017-11-20","A":0,"B":0,"C":0,"D":0,"E":0},
{"time":"2017-11-21","A":6,"B":17,"C":0,"D":0,"E":1}]
(arr.first.keys - [:time]).map do |key|
{ name: key.to_s,
data: arr.select { |h| h[key] > 0 }.
each_with_object({}) { |h,g| g.update(h[:time]=>h[key]) } }
end
#=> [{:name=>"A", :data=>{"2017-11-16"=>2, "2017-11-21"=>6}},
# {:name=>"B", :data=>{"2017-11-16"=>1, "2017-11-21"=>17}},
# {:name=>"C", :data=>{"2017-11-16"=>1, "2017-11-17"=>1}},
# {:name=>"D", :data=>{}},
# {:name=>"E", :data=>{"2017-11-16"=>1, "2017-11-17"=>1, "2017-11-21"=>1}}]
Note that
arr.first.keys - [:time]
#=> [:A, :B, :C, :D, :E]
I have a terribly nested Json response.
[[{:test=>[{:id=>1, :b=>{id: '2'}}]}]]
There's more arrays than that but you get the idea.
Is there a way to recursively search through and find all the items that have a key I need?
I tried using this function extract_list() but it doesn't handle arrays well.
def nested_find(obj, needed_keys)
return {} unless obj.is_a?(Array) || obj.is_a?(Hash)
obj.inject({}) do |hash, val|
if val.is_a?(Hash) && (tmp = needed_keys & val.keys).length > 0
tmp.each { |key| hash[key] = val[key] }
elsif val.is_a?(Array)
hash.merge!(obj.map { |v| nested_find(v, needed_keys) }.reduce(:merge))
end
hash
end
end
Example
needed_keys = [:id, :another_key]
nested_find([ ['test', [{id:1}], [[another_key: 5]]]], needed_keys)
# {:id=>1, :another_key=>5}
The following is not what I'd suggest, but just to give a brief alternative to the other solutions provided:
2.1.1 :001 > obj = [[{:test=>[{:id=>1, :b=>{id: '2'}}]}]]
=> [[{:test=>[{:id=>1, :b=>{:id=>"2"}}]}]]
2.1.1 :002 > key = :id
=> :id
2.1.1 :003 > obj.inspect.scan(/#{key.inspect}=>([^,}]*)[,}]/).flatten.map {|s| eval s}
=> [1, "2"]
Note: use of eval here is just for an example. It would fail/produce incorrect results on anything whose inspect value was not eval-able back to the same instance, and it can execute malicious code:
You'll need to write your own recursive handler. Assuming that you've already converted your JSON to a Ruby data structure (via JSON.load or whatnot):
def deep_find_value_with_key(data, desired_key)
case data
when Array
data.each do |value|
if found = deep_find_value_with_key value, desired_key
return found
end
end
when Hash
if data.key?(desired_key)
data[desired_key]
else
data.each do |key, val|
if found = deep_find_value_with_key(val, desired_key)
return found
end
end
end
end
return nil
end
The general idea is that given a data structure, you check it for the key (if it's a hash) and return the matching value if found. Otherwise, you iterate it (if it's an Array or Hash) and perform the same check on each of it's children.
This will find the value for the first occurrence of the given key, or nil if the key doesn't exist in the tree. If you need to find all instances then it's slightly different - you basically need to pass an array that will accumulate the values:
def deep_find_value_with_key(data, desired_key, hits = [])
case data
when Array
data.each do |value|
deep_find_value_with_key value, desired_key, hits
end
when Hash
if data.key?(desired_key)
hits << data[desired_key]
else
data.each do |key, val|
deep_find_value_with_key(val, desired_key)
end
end
end
return hits
end
How can I have a hash of hash of hash?
My test returns
undefined method `[]' for nil:NilClass (NoMethodError)
Any tips?
found = Hash.new()
x = 1;
while x < 4 do
found[x] = Hash.new()
y = 1
while y < 4 do
found[x][y] = Hash.new()
found[x][y]['name1'] = 'abc1'
found[x][y]['name2'] = 'abc2'
found[x][y]['name3'] = 'abc3'
y += 1
end
x += 1
end
found.each do |k, v, y|
puts "k : #{k}"
puts " : #{v[y['name1']]}"
puts " : #{v[y['name2']]}"
puts " : #{v[y['name3']]}"
puts
end
I think you want something like this:
First of all create the data structure. You want nested hashes so you need to define default values for each hash key.
found = Hash.new do |hash,key|
hash[key] = Hash.new do |hash,key|
hash[key] = Hash.new
end
end
Run the search
(1..3).each do |x|
(1..3).each do |y|
found[x][y]['name1'] = 'abc1'
found[x][y]['name2'] = 'abc1'
found[x][y]['name3'] = 'abc1'
end
end
Then display the results
found.each do |x, y_hash|
y_hash.each do |y, name_hash|
name_hash.each do |name, value|
puts "#{x} => #{y} => #{name} => #{value}"
end
end
end
The way you build the hash seems to be functional. What probably causes the error is this loop:
found.each do |k, v, y|
Hash#each yields key/value pairs, so y will be assigned nil, thus causing the error two lines below. What you probably meant is a nested loop like
found.each do |x, h1|
h1.each do |y, h2|
puts h2['name1']
end
end
You should also be aware that you can write these kinds of counting loops more concisely in Ruby:
found = Hash.new { |h,k| h[k] = {} }
1.upto(3) do |x|
1.upto(3) do |y|
found[x][y] = {
'name1' => 'abc1',
'name2' => 'abc2',
'name3' => 'abc3',
}
end
end
I want to convert something like this:
class NestedItem
attr_accessor :key, :children
def initialize(key, &block)
self.key = key
self.children = []
self.instance_eval(&block) if block_given?
end
def keys
[key] + children.keys
end
end
root = NestedItem.new("root") do
children << NestedItem.new("parent_a") do
children << NestedItem.new("child_a")
children << NestedItem.new("child_c")
end
children << NestedItem.new("parent_b") do
children << NestedItem.new("child_y")
children << NestedItem.new("child_z")
end
end
require 'pp'
pp root
#=>
# #<NestedItem:0x1298a0
# #children=
# [#<NestedItem:0x129814
# #children=
# [#<NestedItem:0x129788 #children=[], #key="child_a">,
# #<NestedItem:0x12974c #children=[], #key="child_c">],
# #key="parent_a">,
# #<NestedItem:0x129738
# #children=
# [#<NestedItem:0x129698 #children=[], #key="child_y">,
# #<NestedItem:0x12965c #children=[], #key="child_z">],
# #key="parent_b">],
# #key="root">
Into this:
root.keys #=>
[
"root",
"root.parent_a",
"root.parent_a.child_a",
"root.parent_a.child_c",
"root.parent_b",
"root.parent_b.child_y",
"root.parent_b.child_z",
]
...using a recursive method.
What's the simplest way to go about this?
Update
I did this:
def keys
[key] + children.map(&:keys).flatten.map do |node|
"#{key}.#{node}"
end
end
Anything better?
Would Array.flatten work for you?
self.children.flatten should return the flattened results.
Yes, .flatten will produce what I think you really want.
But if you want exactly the string output you typed, this will do it:
def keys x
here = key
here = x + '.' + here if x
[ here ] + children.inject([]) { |m,o| m += o.keys here }
end
pp root.keys nil
Or, alternatively, replace the last line in #keys with:
([ here ] + children.map { |o| o.keys here }).flatten
I need a chunk of Ruby code to combine an array of contents like such:
[{:dim_location=>[{:dim_city=>:dim_state}]},
:dim_marital_status,
{:dim_location=>[:dim_zip, :dim_business]}]
into:
[{:dim_location => [:dim_business, {:dim_city=>:dim_state}, :dim_zip]},
:dim_marital_status]
It needs to support an arbitrary level of depth, though the depth will rarely be beyond 8 levels deep.
Revised after comment:
source = [{:dim_location=>[{:dim_city=>:dim_state}]}, :dim_marital_status, {:dim_location=>[:dim_zip, :dim_business]}]
expected = [{:dim_location => [:dim_business, {:dim_city=>:dim_state}, :dim_zip]}, :dim_marital_status]
source2 = [{:dim_location=>{:dim_city=>:dim_state}}, {:dim_location=>:dim_city}]
def merge_dim_locations(array)
return array unless array.is_a?(Array)
values = array.dup
dim_locations = values.select {|x| x.is_a?(Hash) && x.has_key?(:dim_location)}
old_index = values.index(dim_locations[0]) unless dim_locations.empty?
merged = dim_locations.inject({}) do |memo, obj|
values.delete(obj)
x = merge_dim_locations(obj[:dim_location])
if x.is_a?(Array)
memo[:dim_location] = (memo[:dim_location] || []) + x
else
memo[:dim_location] ||= []
memo[:dim_location] << x
end
memo
end
unless merged.empty?
values.insert(old_index, merged)
end
values
end
puts "source1:"
puts source.inspect
puts "result1:"
puts merge_dim_locations(source).inspect
puts "expected1:"
puts expected.inspect
puts "\nsource2:"
puts source2.inspect
puts "result2:"
puts merge_dim_locations(source2).inspect
I don't think there's enough detail in your question to give you a complete answer, but this might get you started:
class Hash
def recursive_merge!(other)
other.keys.each do |k|
if self[k].is_a?(Array) && other[k].is_a?(Array)
self[k] += other[k]
elsif self[k].is_a?(Hash) && other[k].is_a?(Hash)
self[k].recursive_merge!(other[k])
else
self[k] = other[k]
end
end
self
end
end