Merging records in JSON with Ruby - ruby

I have two json files that I'm trying to merge. The JSONs have different formatting (see below). I'd like to merge records, so [0] from file one and [0] from file two would become one record [0] in the new merged file.
The first JSON (file_a.json), appears like so:
{
"query": {
"count": 4,
"created": "2012-11-21T23:07:00Z",
"lang": "en-US",
"results": {
"quote": [
{
"Name": "Bill",
"Age": "46",
"Number": "3.55"
},
{
"Name": "Jane",
"Age": "33",
"Number": nil
},
{
"Name": "Jack",
"Age": "55",
"Number": nil
},
{
"Name": "Xavier",
"Age": nil,
"Number": "153353535"
}
]
}
}
}
The second JSON (file_b.json) appears like so:
[
{
"Number2": 25253,
"Number3": 435574,
"NAME": "Bill"
},
{
"Number2": 345353,
"Number3": 5566,
"NAME": "Jane"
},
{
"Number2": 56756,
"Number3": 232435,
"NAME": "Jack"
},
{
"Number2": 7457,
"Number3": 45425,
"NAME": "Xavier"
}
]
None of the keys are the same in both JSONs (well, actually "Name" is a key in both, but in the first the key is "Name" and in the second its "NAME" - just so I can check that the merge works correctly - so I want "Name" and "NAME" in the final JSON), the first record in the first file matches with the first record in the second file, and so on.
So far, I tried merging like this:
merged = %w[a b].inject([]) { |m,f| m << JSON.parse(File.read("file_#{f}.json")) }.flatten
But this of course merged them, but not how I wanted them merged (they are merged sucessively, and because of the different formatting, it gets quite ugly).
I also tried merging like this:
a = JSON.parse(File.read("file_a.json"))
b = JSON.parse(File.read("file_b.json"))
merged = a.zip(b)
Came closer but still not correct and the formatting was still horrendous.
In the end, what I want is this (formatting of second JSON - headers from first JSON can be junked):
[
{
"Name": "Bill",
"Age": 46,
"Number": 3.55,
"Number2": 25253,
"Number3": 435574,
"NAME": "Bill"
},
{
"Name": "Jane",
"Age": 33,
"Number": nil,
"Number2": 345353,
"Number3": 5566,
"NAME": "Jane"
},
{
"Name": "Jack",
"Age": 55,
"Number": nil,
"Number2": 56756,
"Number3": 232435,
"NAME": "Jack"
},
{
"Name": "Xavier",
"Age": nil,
"Number": 153353535,
"Number2": 7457,
"Number3": 45425,
"NAME": "Xavier"
}
]
Any help is appreciated. Thanks a lot.

Hеllo, seems format changed from last time :)
UPDATE: more readable version that also convert corresponding values to integers/floats:
require 'json'
require 'ap'
a = JSON.parse(File.read('./a.json'))['query']['results']['quote'] rescue []
b = JSON.parse(File.read('./b.json'))
final = []
a.each_with_index do |ah,i|
unless bh = b[i]
bh = {}
puts "seems b has no #{i} key, merging skipped"
end
final << ah.merge(bh).inject({}) do |f, (k,v)|
if v.is_a?(String)
if v =~ /\A\d+\.\d+\Z/
v = v.to_f
elsif v =~ /\A\d+\Z/
v = v.to_i
end
end
f.update k => v
end
end
ap final
will display:
[
[0] {
"Name" => "Bill",
"Age" => 46,
"Number" => 3.55,
"Number2" => 25253,
"Number3" => 435574,
"NAME" => "Bill"
},
[1] {
"Name" => "Jane",
"Age" => 33,
"Number" => nil,
"Number2" => 345353,
"Number3" => 5566,
"NAME" => "Jane"
},
[2] {
"Name" => "Jack",
"Age" => 55,
"Number" => nil,
"Number2" => 56756,
"Number3" => 232435,
"NAME" => "Jack"
},
[3] {
"Name" => "Xavier",
"Age" => nil,
"Number" => 153353535,
"Number2" => 7457,
"Number3" => 45425,
"NAME" => "Xavier"
}
]
Here is a working demo
Btw, your json is a bit wrong in both files.
See the fixed versions here and here

Related

Sorting an array of objects based on object property

I have an array like so:
[
{
"name": "aabb",
"commit": {
"id": "1",
"message": "aabb ",
"committed_date": "2018-04-04T15:11:04.000+05:30",
"committer_name": "ak",
"committer_email": "ak#ak.in"
},
"protected": false
},
{
"name": "aacc",
"commit": {
"id": "2",
"message": "aacc ",
"committed_date": "2018-02-04T15:11:04.000+05:30",
"committer_name": "ak",
"committer_email": "ak#ak.in"
},
"protected": false
},
{
"name": "aadd",
"commit": {
"id": "3",
"message": "aadd ",
"committed_date": "2018-04-01T15:11:04.000+05:30",
"committer_name": "ak",
"committer_email": "ak#ak.in"
},
"protected": false
}
]
I need to sort this array based on committed_date. How do I do that?
Do I have to loop and write a custom sorting function or does Ruby offers something out-of-box?
Using sort_by
array.sort_by {|obj| obj.attribute}
Or more concise
array.sort_by(&:attribute)
In your case
array.sort_by {|obj| obj[:commit][:committed_date]}

Transferring JSON Data into an array using ruby

This is my JSON code
{
"jobs": [
{
"id": 1,
"title": "Software Developer",
"applicants": [
{
"id": 1,
"name": "Rich Hickey",
"tags": ["clojure", "java", "immutability", "datomic", "transducers"]
},
{
"id": 2,
"name": "Guido van Rossum",
"tags": ["python", "google", "bdfl", "drop-box"]
}
]
},
{
"id": 2,
"title": "Software Architect",
"applicants": [
{
"id": 42,
"name": "Rob Pike",
"tags": ["plan-9", "TUPE", "go", "google", "sawzall"]
},
{
"id": 2,
"name": "Guido van Rossum",
"tags": ["python", "google", "bdfl", "drop-box"]
},
{
"id": 1337,
"name": "Jeffrey Dean",
"tags": ["spanner", "BigTable", "MapReduce", "deep learning", "massive clusters"]
}
]
}
]
}
I want to put the list of "Jobs" in an array using ruby.
I have the following code so far.
require 'json'
file = File.read(filepath)
data_hash = JSON.parse(file)
How do I iterate on the data_hash and chose what information I want and place it in an array?
You can use Array#each because data_hash['jobs'] contains an array of jobs:
data_hash['jobs'].each {|job| ... }
Like this,
arr = Array.new
data_hash.each { |job|
arr.insert(job['name'])
}
use Array#map for shorter code
data_hash['jobs'].map do |job|
# Do whatever you want with the job here
properties = %w(title applicants)
job.select{ |key| properties.include?(key) }
end

Extracting data from Deeply Nested JSON in Ruby on Rails

Have a Json out put like
{
"query": {
"results": {
"industry": [
{
"id": "112",
"name": "Agricultural Chemicals",
"company": [
{
"name": "Adarsh Plant",
"symbol": "ADARSHPL"
},
{
"name": "Agrium Inc",
"symbol": "AGU"
}
},
]
{
"id": "914",
"name": "Water Utilities",
"company": [
{
"name": "Acque Potabili",
"symbol": "ACP"
},
{
"name": "Water Resources Group",
"symbol": "WRG"
}
]
}
]
}
}
}
Need the out put like - Company Name, Company Symbol, Company id,
Company id name
and example of output would be
Adarsh Plant, ADARSHPL, 112, Agricultural Chemicals
Agrium Inc, AGU, 112, Agricultural Chemicals
Acque Potabili, ACP, 914, Water Utilities
Water Resources Group, WRG, 914, Water Utilities
Any suggestions
There's a typo in your sample json, but we'll talk about it later.
Assuming your json data converted to hash object already like this:
json={
"query"=> {
"results"=> {
"industry"=> [
{
"id"=> "112",
"name"=> "Agricultural Chemicals",
"company"=> [
{
"name"=> "Adarsh Plant",
"symbol"=> "ADARSHPL"
},
{
"name"=> "Agrium Inc",
"symbol"=> "AGU"
}
]
},
{
"id"=> "914",
"name"=> "Water Utilities",
"company"=> [
{
"name"=> "Acque Potabili",
"symbol"=> "ACP"
},
{
"name"=> "Water Resources Group",
"symbol"=> "WRG"
}
]
}
]
}
}
}
You can use inject and map to handle the two level array of industry, inject will iterate the outer array:
json["query"]["results"]["industry"].inject([]){|m,o|
m += o["company"].map{|x| [x["name"],x["symbol"],o["id"],o["name"]]}
}
result is an array of arrays with the order as you wish:
=> [["Adarsh Plant", "ADARSHPL", "112", "Agricultural Chemicals"],
["Agrium Inc", "AGU", "112", "Agricultural Chemicals"],
["Acque Potabili", "ACP", "914", "Water Utilities"],
["Water Resources Group", "WRG", "914", "Water Utilities"]]
If you want get a string delimited by comma, you could chain on .flatten.join(",") at the end.
json["query"]["results"]["industry"].inject([]){|m,o|
m += o["company"].map{|x| [x["name"],x["symbol"],o["id"],o["name"]]}
}.flatten.join(",")
Result:
=> Adarsh Plant,ADARSHPL,112,Agricultural Chemicals,Agrium Inc,AGU,112,Agricultural Chemicals,Acque Potabili,ACP,914,Water Utilities,Water Resources Group,WRG,914,Water Utilities
The typo of your json data:
In the middle }, ] { should be changed to ] },{ .
Convert json to hash
https://stackoverflow.com/a/7964378/3630826

How do I access JSON array data?

I have the following array:
[ { "attributes": {
"id": "usdeur",
"code": 4
},
"name": "USD/EUR"
},
{ "attributes": {
"id": "eurgbp",
"code": 5
},
"name": "EUR/GBP"
}
]
How can I get both ids for futher processing as output?
I tried a lot but no success. My problem is I always get only one id as output:
Market.all.select.each do |market|
present market.id
end
Or:
Market.all.each{|attributes| present attributes[:id]}
which gives me only "eurgbp" as a result while I need both ids.
JSON#parse should help you with this
require 'json'
json = '[ { "attributes": {
"id": "usdeur",
"code": 4
},
"name": "USD/EUR"
},
{ "attributes": {
"id": "eurgbp",
"code": 5
},
"name": "EUR/GBP"
}]'
ids = JSON.parse(json).map{|hash| hash['attributes']['id'] }
#=> ["usdeur", "eurgbp"]
JSON#parse turns a jSON response into a Hash then just use standard Hash methods for access.
I'm going to assume that the data is JSON that you're parsing (with JSON.parse) into a Ruby Array of Hashes, which would look like this:
hashes = [ { "attributes" => { "id" => "usdeur", "code" => 4 },
"name" => "USD/EUR"
},
{ "attributes" => { "id" => "eurgbp", "code" => 5 },
"name" => "EUR/GBP"
} ]
If you wanted to get just the first "id" value, you'd do this:
first_hash = hashes[0]
first_hash_attributes = first_hash["attributes"]
p first_hash_attributes["id"]
# => "usdeur"
Or just:
p hashes[0]["attributes"]["id"]
# => "usdeur"
To get them all, you'll do this:
all_attributes = hashes.map {|hash| hash["attributes"] }
# => [ { "id" => "usdeur", "code" => 4 },
# { "id" => "eurgbp", "code" => 5 } ]
all_ids = all_attributes.map {|attrs| attrs["id"] }
# => [ "usdeur", "eurgbp" ]
Or just:
p hashes.map {|hash| hash["attributes"]["id"] }
# => [ "usdeur", "eurgbp" ]
JSON library what using Rails is very slowly...
I prefer to use:
gem 'oj'
from https://github.com/ohler55/oj
fast and simple! LET'S GO!

Convert csv to json in ruby

CSV
id,modifier1_name,modifier2_price,modifier2_name,modifier2_price,modifier2_status
1,'Small',10,'Large',20,'YYY'
2,'Small',20,'Large',30,'YYY'
JSON
[
{
id: 1,
modifier: [
{name: 'Small', price: 10},
{name: 'Large', price: 20, status: 'YYY'}]
},
{
id: 2,
modifier: [
{name: 'Small', price: 20},
{name: 'Large', price: 30, status: 'YYY'}],
}
]
How to convert CSV to Json in this case when modifiers can be different ?
You will need to map the modifiers yourself, as there is no built-in method to map hash values into an array from your logic:
JSON.pretty_generate(CSV.open('filename.csv', headers: true).map do |row|
modifier = {}
row.each do |k, v|
if k =~ /modifier(.)_(.*)$/
(modifier[$1] ||= {})[$2] = v
end
end
{ id: row['id'],
modifier: modifier.sort_by { |k, v| k }.map {|k, v| v }
}
end)
For the file*
id,modifier1_name,modifier1_price,modifier2_name,modifier2_price,modifier2_status
1,Small,10,Large,20,YYY
2,Small,20,Large,30,YYY
*I made some changes to the file you show, since it will not give you the required result - you state modifier2_price twice, for example
You will get:
[
{
"id": "1",
"modifier": [
{
"name": "Small",
"price": "10"
},
{
"name": "Large",
"price": "20",
"status": "YYY"
}
]
},
{
"id": "2",
"modifier": [
{
"name": "Small",
"price": "20"
},
{
"name": "Large",
"price": "30",
"status": "YYY"
}
]
}
]
require 'csv'
require 'json'
CSV.open('filename.csv', :headers => true).map { |x| x.to_h }.to_json

Resources