I am working with a large data set with multiple arrays of hashes which all have a key-value pair in common ("date" & the date value) as the first element of the hash.
The array of hashes I need to parse (#data["snapshot"]) is in the following format. Note that #data["snapshot"][0], #data["snapshot"][1], and #data["snapshot"][2] are in identical format with identical dates but their total's differ. In the resulting hash I need to have a key-value pair which identifies where the data came from.
#data["snapshot"][0] is as follows:
[{"date"=>"1455672010", "total"=>"**817**", "I"=>"1", "L"=>"3", "M"=>"62", "H"=>"5", "C"=>"0"},
{"date"=>"1455595298", "total"=>"**40**", "I"=>"8", "L"=>"5", "M"=>"562", "H"=>"125", "C"=>"0"},
{"date"=>"1455336016", "total"=>"**555**", "I"=>"10", "L"=>"1", "M"=>"93", "H"=>"121", "C"=>"0"}]
#data["snapshot"][1] is as follows:
[{"date"=>"1455672010", "total"=>"**70**", "I"=>"1", "L"=>"9", "M"=>"56", "H"=>"25", "C"=>"0"},
{"date"=>"1455595298", "total"=>"**54**", "I"=>"8", "L"=>"2", "M"=>"5", "H"=>"5", "C"=>"0"},
{"date"=>"1455336016", "total"=>"**25**", "I"=>"0", "L"=>"9", "M"=>"93", "H"=>"12", "C"=>"0"}]
#data["snapshot"][2] is as follows:
[{"date"=>"1455672010", "total"=>"**70**", "I"=>"12", "L"=>"5", "M"=>"5662", "H"=>"125", "C"=>"0"},
{"date"=>"1455595298", "total"=>"**43212**", "I"=>"56", "L"=>"6", "M"=>"5662", "H"=>"125", "C"=>"0"},
{"date"=>"1455336016", "total"=>"**55525**", "I"=>"100", "L"=>"19", "M"=>"5593", "H"=>"121", "C"=>"0"}]
My Question Is Ultimately:
How do I convert (flatten?) the 3 existing array of hashes (#data["snapshot"][0], #data["snapshot"][1], and #data["snapshot"][2]) into a single array of hashes in the following format?
[{"date"=>"1455672010", "CameFromDataSource0"=>"817", "CameFromDataSource1"=>"70", "CameFromDataSource2"=>"70"},
{"date"=>"1455595298", "CameFromDataSource0"=>"40", "CameFromDataSource1"=>"54", "CameFromDataSource2"=>"43212"},
{"date"=>"1455336016", "CameFromDataSource0"=>"555", "CameFromDataSource1"=>"25", "CameFromDataSource2"=>"55525"}]
TL;DR
snapshots.each_with_object(Hash.new {|hsh, date| hsh[date] = { "date" => date } })
.with_index do |(snapshot, hsh), i|
snapshot["data"].each {|datum| hsh[datum["date"]]["data#{i}"] = datum["total"] }
end.values
How it works
I'll break it down so you see how each part works. Here's our data (extraneous keys elided for clarity):
snapshots = [
{ "dataSourceID" => "152970",
"data" => [ { "date" => "1455672010", "total" => "817" },
{ "date" => "1455595298", "total" => "40" },
{ "date" => "1455336016", "total" => "555" } ]
}
{ "dataSourceID" => "33151",
"data" => [ { "date" => "1455672010", "total" => "70" },
{ "date" => "1455595298", "total" => "54" },
{ "date" => "1455336016", "total" => "25" } ]
},
{ "dataSourceID" => "52165",
"data" => [ { "date" => "1455672010", "total" => "70" },
{ "date" => "1455595298", "total" => "43212" },
{ "date" => "1455336016", "total" => "55525" } ]
}
]
Most of the magic is here:
result_hash = Hash.new {|hsh, date| hsh[date] = { "date" => date } }
Here we're using the Hash's default proc to automatically initialize new keys in the following way:
result_hash = Hash.new {|hsh, date| hsh[date] = { "date" => date } }
p result_hash["1455672010"]
# => { "date" => "1455672010" }
p result_hash
# => { "1455672010" => { "date" => "1455672010" } }
Simply accessing result_hash[foo] creates the hash { "date" => foo } and assigns it to result_hash[foo]. This enables the following:
result_hash["1455672010"]["data0"] = "817"
p result_hash
# => { "1455672010" => { "date" => "1455672010", "data0" => "817" } }
Magic!
Now suppose we have the following data:
data = [ { "date" => "1455672010", "total" => "817" },
{ "date" => "1455595298", "total" => "40" },
{ "date" => "1455336016", "total" => "555" } ]
Using our magic result_hash, we can do this:
data.each do |datum|
result_hash[datum["date"]]["data0"] = datum["total"]
end
p result_hash
# => { "1455672010" => { "date" => "1455672010", "data0" => "817" },
# "1455595298" => { "date" => "1455595298", "data0" => "40" },
# "1455336016" => { "date" => "1455336016", "data0" => "555" } }
See where I'm going with this? Here's all of our data, finally:
snapshots = [
{ "dataSourceID" => "152970",
"data" => [ { "date" => "1455672010", "total" => "817" },
{ "date" => "1455595298", "total" => "40" },
{ "date" => "1455336016", "total" => "555" } ]
}
{ "dataSourceID" => "33151",
"data" => [ { "date" => "1455672010", "total" => "70" },
{ "date" => "1455595298", "total" => "54" },
{ "date" => "1455336016", "total" => "25" } ]
},
{ "dataSourceID" => "52165",
"data" => [ { "date" => "1455672010", "total" => "70" },
{ "date" => "1455595298", "total" => "43212" },
{ "date" => "1455336016", "total" => "55525" } ]
}
]
Instead of hard-coding "data0", we can iterate over the snapshots hashes using each_with_index and build that key ("data0", then "data1", and so on) for each iteration. Inside that loop we can do exactly what we did above but with the "data" array from each snapshots hash:
result_hash = Hash.new {|hsh, date| hsh[date] = { "date" => date } }
snapshots.each_with_index do |snapshot, i|
data_key = "data#{i}"
snapshot["data"].each do |datum|
date = datum["date"]
result_hash[date][data_key] = datum["total"]
end
end
p result_hash.values
# => [ { "date" => "1455672010", "data0" => "817", "data1" => "70", "data2" => "70" },
# { "date" => "1455595298", "data0" => "40", "data1" => "54", "data2" => "43212" },
# { "date" => "1455336016", "data0" => "555", "data1" => "25", "data2" => "55525" } ]
Of course, this can be condensed some, which I've done in TL;DR above.
This is one way to do it.
Code
def convert(data)
data.each_with_object({}) { |a,h|
a.each { |g| h.update(g["date"]=>[g["total"][/\d+/]]) { |_,o,n| o+n } } }.
map { |date, arr| arr.each_with_index.with_object({"date"=>date}) { |(e,i),h|
h["key#{i}"] = e } }
end
Example
convert(data)
#=> [{"date"=>"1455672010", "key0"=>"817", "key1"=>"70", "key2"=>"70"},
# {"date"=>"1455595298", "key0"=>"40", "key1"=>"54", "key2"=>"43212"},
# {"date"=>"1455336016", "key0"=>"555", "key1"=>"25", "key2"=>"55525"}]
Two steps
You can see that I've done this in two steps. First construct a hash:
f = data.each_with_object({}) { |a,h| a.each { |g|
h.update(g["date"]=>[g["total"][/\d+/]]) { |_,o,n| o+n } } }
#=> {"1455672010"=>["817", "70", "70"],
# "1455595298"=>["40", "54", "43212"],
# "1455336016"=>["555", "25", "55525"]}
Here I have used the form of Hash#update (aka merge!) that employs a block ({ |_,o,n| o+n }) to determine the values of keys that are present in both hashes being merged.
Then convert the hash to the desired format:
f.map { |date, arr| arr.each_with_index.with_object({"date"=>date}) { |(e,i),h|
h["key#{i}"] = e } }
#=> [{"date"=>"1455672010", "key0"=>"817", "key1"=>"70", "key2"=>"70"},
# {"date"=>"1455595298", "key0"=>"40", "key1"=>"54", "key2"=>"43212"},
# {"date"=>"1455336016", "key0"=>"555", "key1"=>"25", "key2"=>"55525"}]
I have the following two hashes:
db = {"1" => "first_name", "2" => "last_name", "5" => "status", "10" => "city" }
csv = {"1" => "first_name", "2" => "last_name", "5" => "status", "7" => "address", "10" => "city" }
I want to order csv based on db, and if there are any keys in csv not in db, then I want to move them to the end of csv, so in the above example the result would look like this:
{"1" => "first_name", "2" => "last_name", "5" => "status", "10" => "city", "7" => "address" }
Since the key "7" wasn't in db hash, we just moved it to the end of the csv hash.
This is what I tried:
db = {"1" => "first_name", "2" => "last_name", "5" => "status", "10" => "city" }
csv = {"1" => "first_name", "2" => "last_name", "5" => "status", "7" => "address", "10" => "city" }
rejects = csv.reject {|k| db.include? k }
result = csv.keep_if {|k,_| db.include? k }
result.merge!(rejects)
result
=> {"1"=>"first_name", "2"=>"last_name", "5"=>"status", "10"=>"city", "7"=>"address"}
It seems to work. But is it guaranteed to work? Will the merger always put the second hash at the end, or is there a possibility that the merger could mix the hashes together without consideration of order?
You could do the following:
db_keys = db.keys
#=> ["1", "2", "5", "10"]
keys = db_keys + (csv.keys-db_keys)
#=> ["1", "2", "5", "10", "7"]
Hash[keys.zip(csv.values_at(*keys))]
#=> { "1"=>"first_name", "2"=>"last_name", "5"=>"status",
# "10"=>"city", "7"=>"address"}
I am struggling with some arrays with hashes inside. I want to parse them into a new object but have no idea how to do this.
Here is the data:
[
{
"name" => "itemHref",
"value" => "https://192.168.75.145:8281/api/workflows/16da1fa1-7c8b-4602-8d53-17fc5e1fa3ff/"
},
{
"name" => "id",
"value" => "16da1fa1-7c8b-4602-8d53-17fc5e1fa3ff"
},
{
"name" => "categoryName",
"value" => "FinanzInformatik"
},
{
"name" => "canExecute",
"value" => "true"
},
{
"name" => "categoryHref",
"value" => "https://192.168.75.145:8281/api/catalog/System/WorkflowCategory/ff8080813b90a145013b90cac51b0006/"
},
{
"name" => "description",
"value" => "bekommt alle VMs"
},
{
"name" => "name",
"value" => "getAllVms"
},
{
"name" => "type",
"value" => "Workflow"
},
{
"name" => "canEdit",
"value" => "true"
}
]
And, here is my code:
require 'rest-client'
require 'json'
class Workflow
def initialize(itemHref, id, categoryName, canExecute, categoryHref, description, name, type, canEdit)
#itemHref = itemHref
#id = id
#categoryName = categoryName
#canExecute = canExecute
#categoryHref = categoryHref
#description = description
#name = name
#type = type
#canEdit = canEdit
end
end
json_string = RestClient.get( "http://vcoadmin:vcoadmin#192.168.75.145:8280/api/workflows", :content_type => 'application/json', :accept => 'application/json')
parsed = JSON.parse(json_string)
parsed.each do |a, b|
if(b.class == Array)
b.flatten.each do |c|
p c['attributes']
#c['attributes'].each
{
|f| p f['name'], f['value'] }
end
end
end
How do I put the hash value into the object? I think about something based on the 'name' which is the identifier for the value.
Any ideas?
Assuming that the order of attributes shouldn't be changed:
Workflow.new(*parsed.map {|attr| attr['value']})
I would implement a PORO that can be initialized with a hash. So then you are able to pass your hash directly in to creating the workflow.
An example of this is can be seen: http://pullmonkey.com/2008/01/06/convert-a-ruby-hash-into-a-class-object/
a = {"rows" => [{"id" => "231-z", "name" => 'jon', "age"=> 27, "state" => 'AL'},
{"id" => "4121-x", "name" => 'ton', "age"=> 37, "state" => 'VA'}
]
}
b = {"rows" => [{"key" => ["xyz","4121-x"], "value" =>{"sum" => 12312, "realage" => 29}},
{"key" => ["xyz","231-z"], "value" =>{"sum" => 1212, "realage" => 33}}
]
}
In hash a, age is incorrect
In hash b, realage is correct. Also in hash b id is the second value in the first array that maps to id of hash a . Those are 4121-x, 231-z correspond to hash a
I want to correct the age in hash a and swap it with the realage of hash b
I can do it in multiple steps, but is it possible to do it in one liner or very short? So finally correct hash a should look like
a = {"rows" => [{"id" => "231-z", "name" => 'jon', "age"=> 33, "state" => 'AL'},
{"id" => "4121-x", "name" => 'ton', "age"=> 29, "state" => 'VA'}
]
}
does this look reasonable?
a['rows'].each_with_index do |ah, i|
(bh = b['rows'].select {|h| h['key'].last == ah['id'] }.first) &&
a['rows'][i] = ah.update('age' => bh['value']['realage'])
end
p a
{
"rows" => [
[0] {
"id" => "231-z",
"name" => "jon",
"age" => 33,
"state" => "AL"
},
[1] {
"id" => "4121-x",
"name" => "ton",
"age" => 29,
"state" => "VA"
}
]
}
Please note it will update a only if corresponding id found in b.
Also, the rows order does not matter, nor matter the rows number, it is only important b to have a row with same id as processed row in a
Here is a Working Demo