Convert csv to json in ruby - ruby

CSV
id,modifier1_name,modifier2_price,modifier2_name,modifier2_price,modifier2_status
1,'Small',10,'Large',20,'YYY'
2,'Small',20,'Large',30,'YYY'
JSON
[
{
id: 1,
modifier: [
{name: 'Small', price: 10},
{name: 'Large', price: 20, status: 'YYY'}]
},
{
id: 2,
modifier: [
{name: 'Small', price: 20},
{name: 'Large', price: 30, status: 'YYY'}],
}
]
How to convert CSV to Json in this case when modifiers can be different ?

You will need to map the modifiers yourself, as there is no built-in method to map hash values into an array from your logic:
JSON.pretty_generate(CSV.open('filename.csv', headers: true).map do |row|
modifier = {}
row.each do |k, v|
if k =~ /modifier(.)_(.*)$/
(modifier[$1] ||= {})[$2] = v
end
end
{ id: row['id'],
modifier: modifier.sort_by { |k, v| k }.map {|k, v| v }
}
end)
For the file*
id,modifier1_name,modifier1_price,modifier2_name,modifier2_price,modifier2_status
1,Small,10,Large,20,YYY
2,Small,20,Large,30,YYY
*I made some changes to the file you show, since it will not give you the required result - you state modifier2_price twice, for example
You will get:
[
{
"id": "1",
"modifier": [
{
"name": "Small",
"price": "10"
},
{
"name": "Large",
"price": "20",
"status": "YYY"
}
]
},
{
"id": "2",
"modifier": [
{
"name": "Small",
"price": "20"
},
{
"name": "Large",
"price": "30",
"status": "YYY"
}
]
}
]

require 'csv'
require 'json'
CSV.open('filename.csv', :headers => true).map { |x| x.to_h }.to_json

Related

How to change output of the array hash using the map or group_by in ruby?

I have this array of hash where i have to separate same value object and group it by country base on date.
I try to use group_by but i only manage to get the countries i cannot separate the reported_date
outside of array hash.
notice one of the object is thesame
"reported_date": "2020-04-01"
i want it to be outside of every hash so it only print once.
heres my hash
[
{
"reported_date": "2020-04-01",
"country": "Italy",
"confirmed": 110574,
"deaths": 13155,
"recovered": 16847
},
{
"reported_date": "2020-04-01",
"country": "Spain",
"confirmed": 104118,
"deaths": 9387,
"recovered": 22647
},
{
"reported_date": "2020-04-01",
"country": "US",
"confirmed": 83948,
"deaths": 1941,
"recovered": 0
}
]
and i want to change something like this
{
"reported_date": "2020-04-01",
"countries": [
{
"country": "Italy",
"confirmed": 110574,
"deaths": 13155,
"recovered": 16847
},
{
"country": "Spain",
"confirmed": 104118,
"deaths": 9387,
"recovered": 22647
},
{
"country": "US",
"confirmed": 83948,
"deaths": 1941,
"recovered": 0
}
]
}
[
{
"reported_date": "2020-04-01",
"country": "Italy",
"confirmed": 110574,
"deaths": 13155,
"recovered": 16847
},
{
"reported_date": "2020-04-01",
"country": "Spain",
"confirmed": 104118,
"deaths": 9387,
"recovered": 22647
},
{
"reported_date": "2020-04-01",
"country": "US",
"confirmed": 83948,
"deaths": 1941,
"recovered": 0
}
].group_by { |h| h[:reported_date] }.map do |date, rest|
{
reported_date: date,
countries: rest.map { |h| h.reject! { |k| k == :reported_date } }
}
end
Returns
[
{
:reported_date => "2020-04-01",
:countries => [
{
:country => "Italy",
:confirmed => 110574,
:deaths => 13155,
:recovered => 16847
},
{
:country => "Spain",
:confirmed => 104118,
:deaths => 9387,
:recovered => 22647
},
{
:country => "US",
:confirmed => 83948,
:deaths => 1941,
:recovered => 0
}
]
}
]
Suppose your array were:
arr = [
{ date: "2020-04-01", country: "Italy", confirmed: 110574, deaths: 13155},
{ date: "2020-04-02", country: "Spain", confirmed: 104118, deaths: 9387},
{ date: "2020-04-01", country: "US", confirmed: 83948, deaths: 1941},
{ date: "2020-04-02", country: "Italy", confirmed: 120431, deaths: 13394}
]
Depending on your needs you might find it more convenient to reorganize your data, possibly in one of the following two ways.
#1
keeper_keys = arr.first.keys - [:date, :country]
#=> [:confirmed, :deaths]
h = arr.each_with_object({}) { |g,h| h[g[:date]] =
(h[g[:date]] || {}).merge(g[:country]=>g.slice(*keeper_keys)) }
#=> {"2020-04-01"=>{
# "Italy"=>{:confirmed=>110574, :deaths=>13155},
# "US"=>{:confirmed=>83948, :deaths=>1941}
# },
# "2020-04-02"=>{
# "Spain"=>{:confirmed=>104118, :deaths=>9387},
# "Italy"=>{:confirmed=>120431, :deaths=>13394}
# }
# }
h[g[:date]] || {} returns an empty hash if h has no key g[:date].
This allows us to easily compute, for example:
Deaths by date and country
p = h.transform_values { |g| g.transform_values { |f| f[:deaths] } }
#=> {"2020-04-01"=>{"Italy"=>13155, "US"=>1941},
# "2020-04-02"=>{"Spain"=>9387, "Italy"=>13394}}
Total deaths by date
p.transform_values { |g| g.values.sum }
#=> {"2020-04-01"=>15096, "2020-04-02"=>22781}
#2
keeper_keys = arr.first.keys - [:date, :country]
#=> [:confirmed, :deaths]
h = arr.each_with_object({}) { |g,h| h[g[:country]] =
(h[g[:country]] || {}).merge(g[:date]=>g.slice(*keeper_keys)) }
#=> {"Italy"=>{
# "2020-04-01"=>{:confirmed=>110574, :deaths=>13155},
# "2020-04-02"=>{:confirmed=>120431, :deaths=>13394}
# },
# "Spain"=>{
# "2020-04-02"=>{:confirmed=>104118, :deaths=>9387}
# },
# "US"=>{
# "2020-04-01"=>{:confirmed=>83948, :deaths=>1941}
# }
# }
h[g[:country]] || {} returns an empty hash if h has no key g[:country].
This allows us to easily compute:
Deaths by country and date
p = h.transform_values { |g| g.transform_values { |f| f[:deaths] } }
#=> {"Italy"=>{"2020-04-01"=>13155, "2020-04-02"=>13394},
# "Spain"=>{"2020-04-02"=>9387},
# "US"=>{"2020-04-01"=>1941}}
Notice that this expression is the same as the one used earlier to obtain deaths by date and country.
Total deaths by date
p.transform_values { |g| g.values.sum }
#=> {"Italy"=>26549, "Spain"=>9387, "US"=>1941}
Again, this is the same expression as the one used earlier to compute total deaths by country.
Better yet, use a database!

Unable to fetch deeply nested hash value

I have this rake task which uses rest-client to fetch some messy JSON from this API, and then uses hashie to make the code prettier.
Unfortunately I'm unable to fetch one of the deeply nested values, productGroup. If working correctly, it should output :category => "Jeans" or similar. Please see the JSON at the bottom.
This did not work:
mash.deep_fetch(:fields, 0).deep_locate(-> (key, value, object) { value.include?("product_group") }) { "ERROR: category" }
Example output:
% rake get_products
{:category=>nil, :name=>"Luxurous Jumpsuit", :image=>"http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/productLarge/129579-0012.jpg", :price=>"599", :description=>"Lorem ipsum dolor"}
Example mash:
#<Hashie::Mash brand="Dr Denim" categories=[#<Hashie::Mash name="Kvinne > KLÆR > Jeans > Slim">] description="Lorem ipsum dolor." fields=[#<Hashie::Mash name="sale" value="false">, #<Hashie::Mash name="product_id_original" value="226693-7698">, #<Hashie::Mash name="gender" value="Kvinne">, #<Hashie::Mash name="artNumber" value="226693-7698">, #<Hashie::Mash name="productGroup" value="Jeans">, #<Hashie::Mash name="productStyle" value="Slim">, #<Hashie::Mash name="extraImageProductSmall" value="http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/cart_thumb/226693-7698.jpg">, #<Hashie::Mash name="productClass" value="Klær">, #<Hashie::Mash name="extraImageProductLarge" value="http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/productLarge/226693-7698.jpg">, #<Hashie::Mash name="sizes" value="W24/L32,W25/L32,W26/L32,W27/L32,W28/L32,W29/L32,W30/L32,W31/L32,W32/L32,W26/L30,W27/L30,W28/L30,W29/L30,W24/L30,W25/L30,W32/L30,W31/L30,W30/L30">, #<Hashie::Mash name="color" value="Mid Blue">] identifiers=#<Hashie::Mash sku="226693-7698"> language="no" name="Regina Jeans" offers=[#<Hashie::Mash feed_id=10086 id="2820760a-c5b2-494a-b5dd-ab713f796cb9" in_stock=1 modified=1474947357838 price_history=[#<Hashie::Mash date=1474949513421 price=#<Hashie::Mash currency="NOK" value="599">>] product_url="http://pdt.tradedoubler.com/click?a1234" program_logo="http://hst.tradedoubler.com/file/17833/2014-logos/200X200.png" program_name="Nelly NO" source_product_id="226693-7698">] product_image=#<Hashie::Mash url="http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/productLarge/226693-7698.jpg">>
get_products.rake:
# encoding: utf-8
# Gets messy JSON from other store via REST client and cleans it up with Hashie
require "rest_client"
require "hashie"
Product = Struct.new(:category, :name, :image, :price, :description)
module ProductsFromOtherStore
CATEGORIES = [
"festkjoler",
"jakker",
"jeans",
"jumpsuit",
"vesker"
]
def self.fetch
CATEGORIES.map do |category|
Tradedoubler.fetch category
end
end
# Prettify, ie. `fooBar` => `foo_bar`
def self.prettify(x)
case x
when Hash
x.map { |key, value| [key.underscore, prettify(value)] }.to_h
when Array
x.map { |value| prettify(value) }
else
x
end
end
end
class ProductsFromOtherStore::Tradedoubler
KEY = "FE34B1309AB749F1578AEE87D9D74535513F6B54"
# Products to fetch from API
LIMIT = 2
def self.fetch category
new(category).filtered_products.take(LIMIT)
rescue RestClient::RequestTimeout => e
Array.new
end
def initialize category
#category = category
# API doesn't support gender or category searches, so do some filtering based on available JSON fields
#filters = Array.new
define_filter { |mash|
mash.fields.any? { |field|
field.name == "gender" && field.value.downcase == "kvinne"
}
}
define_filter { |mash|
mash.categories.any? { |category|
category.name.underscore.include? #category
}
}
end
def define_filter(&filter)
#filters << filter
end
def filtered_products
filtered_mashes.map { |mash|
# puts mash
Product.new(
# mash.deep_fetch(:fields, 0).find { |field| field[:name] == "product_group" }[:value],
mash.deep_fetch(:fields, 0).deep_locate(-> (key, value, object) { value.include?("product_group") }) { "ERROR: category" },
mash.deep_fetch(:name) { "ERROR: name" },
mash.deep_fetch(:product_image, :url) { "ERROR: image URL" },
mash.deep_fetch(:offers, 0, :price_history, 0, :price, :value) { "ERROR: price" },
mash.deep_fetch(:description) { "ERROR: description" }
)
}
end
private
def request
response = RestClient::Request.execute(
:method => :get,
:url => "http://api.tradedoubler.com/1.0/products.json;q=#{ URI.encode(#category) };limit=#{ LIMIT }?token=#{ KEY }",
:timeout => 0.4
)
end
def hashes
ProductsFromOtherStore.prettify(JSON.parse(request)["products"])
end
def mashes
hashes.map { |hash| Hashie::Mash.new(hash) }.each do |mash|
mash.extend Hashie::Extensions::DeepFetch
mash.extend Hashie::Extensions::DeepLocate
end
end
def filtered_mashes
mashes.select { |mash| mash_matches_filter? mash }
end
def mash_matches_filter? mash
# `.all?` requires all filters to match, `.any?` requires only one
#filters.all? { |filter| filter.call mash }
end
end
# All that for this
task :get_products => :environment do
#all_products_from_all_categories = ProductsFromOtherStore.fetch
#all_products_from_all_categories.each do |products|
products.each do |product|
puts product.to_h
end
end
end
The messy JSON we got via rest-client:
{
"productHeader": {
"totalHits": 367
},
"products": [{
"name": "501 CT Jeans For Women",
"productImage": {
"url": "http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/productLarge/441576-1056.jpg"
},
"language": "no",
"description": "Jeans fra Levi's. Noe kortere nederst, fem lommer. Normal høyde på midjen, med hemper i linningen og knappegylfen. Dekorative slitte partier foran og nederst på benet.<br />Laget av 100% bomull.",
"brand": "Levis",
"identifiers": {
"sku": "441576-1056"
},
"fields": [{
"name": "sale",
"value": "false"
}, {
"name": "sizes",
"value": "W24/L32,W25/L32,W26/L32,W27/L32,W28/L32,W29/L32,W30/L32,W31/L32,W25/L34,W26/L34,W27/L34,W28/L34,W29/L34,W30/L34"
}, {
"name": "productStyle",
"value": "Straight"
}, {
"name": "gender",
"value": "Kvinne"
}, {
"name": "product_id_original",
"value": "441576-1056"
}, {
"name": "productGroup",
"value": "Jeans"
}, {
"name": "extraImageProductLarge",
"value": "http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/productLarge/441576-1056.jpg"
}, {
"name": "extraImageProductSmall",
"value": "http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/cart_thumb/441576-1056.jpg"
}, {
"name": "artNumber",
"value": "441576-1056"
}, {
"name": "productClass",
"value": "Klær"
}, {
"name": "color",
"value": "Indigo"
}],
"offers": [{
"feedId": 10086,
"productUrl": "http://pdt.tradedoubler.com/click?a(2402331)p(80279)product(57d37b9ce4b085c06c38c96b)ttid(3)url(http%3A%2F%2Fnelly.com%2Fno%2Fkl%C3%A6r-til-kvinner%2Fkl%C3%A6r%2Fjeans%2Flevis-441%2F501-ct-jeans-for-women-441576-1056%2F)",
"priceHistory": [{
"price": {
"value": "1195",
"currency": "NOK"
},
"date": 1473477532181
}],
"modified": 1473477532181,
"inStock": 1,
"sourceProductId": "441576-1056",
"programLogo": "http://hst.tradedoubler.com/file/17833/2014-logos/200X200.png",
"programName": "Nelly NO",
"id": "57d37b9ce4b085c06c38c96b"
}],
"categories": [{
"name": "Kvinne > KLÆR > Jeans > Straight"
}]
}, {
"name": "501 CT Jeans For Women",
"productImage": {
"url": "http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/productLarge/441576-6581.jpg"
},
"language": "no",
"description": "Jeans fra Levi's. Noe kortere nederst, fem lommer. Normal høyde på midjen, med hemper i linningen og knappegylfen. Dekorative slitte partier foran og nederst på benet.<br />Laget av 100% bomull.",
"brand": "Levis",
"identifiers": {
"sku": "441576-6581"
},
"fields": [{
"name": "sale",
"value": "false"
}, {
"name": "artNumber",
"value": "441576-6581"
}, {
"name": "productStyle",
"value": "Straight"
}, {
"name": "gender",
"value": "Kvinne"
}, {
"name": "extraImageProductLarge",
"value": "http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/productLarge/441576-6581.jpg"
}, {
"name": "extraImageProductSmall",
"value": "http://nlyscandinavia.scene7.com/is/image/nlyscandinavia/cart_thumb/441576-6581.jpg"
}, {
"name": "productGroup",
"value": "Jeans"
}, {
"name": "product_id_original",
"value": "441576-6581"
}, {
"name": "productClass",
"value": "Klær"
}, {
"name": "color",
"value": "Desert"
}, {
"name": "sizes",
"value": "W24/L32,W25/L32,W26/L32,W27/L32,W28/L32,W29/L32,W30/L32,W31/L32,W25/L34,W26/L34,W27/L34,W28/L34,W29/L34,W30/L34,W31/L34"
}],
"offers": [{
"feedId": 10086,
"productUrl": "http://pdt.tradedoubler.com/click?a(2402331)p(80279)product(57b3cafbe4b06cf59bc254bf)ttid(3)url(http%3A%2F%2Fnelly.com%2Fno%2Fkl%C3%A6r-til-kvinner%2Fkl%C3%A6r%2Fjeans%2Flevis-441%2F501-ct-jeans-for-women-441576-6581%2F)",
"priceHistory": [{
"price": {
"value": "1195",
"currency": "NOK"
},
"date": 1471400699283
}],
"modified": 1471400699283,
"inStock": 1,
"sourceProductId": "441576-6581",
"programLogo": "http://hst.tradedoubler.com/file/17833/2014-logos/200X200.png",
"programName": "Nelly NO",
"id": "57b3cafbe4b06cf59bc254bf"
}],
"categories": [{
"name": "Kvinne > KLÆR > Jeans > Straight"
}]
}]
}
There is a lot of things going on in your code sample. I tried to split in parts and restructure it. It does not do the same as your code but I think it should get you started and perhaps you can come back when you have a more specific question.
Note that I did not use hashie, I think that accessing some deeply nested hash structures in a few places does not justify adding a new library to a project.
Questions/Ideas/Hints:
are prices Integers or Floats?
Is the JSON consistent (all elements present all the time?)
Are you using Ruby 2.3? Then look into Hash#dig
Why did you prettify the JSON keys? Does not make sense to me as you build Product objects to work with anyway?
Unless there are performance issues i would convert all products to Ruby objects first and filter then. Just easier and more readable.
Code
Product (same as yours)
Product = Struct.new(:category, :name, :image, :price, :description)
JsonProductBuilder converts the parsed JSON to Product Objects.
class JsonProductBuilder
def initialize(json)
#json = json
end
def call
json.fetch('products', []).map do |item|
Product.new(
extract_category(item),
item['name'],
item.fetch('productImage', {})['url'],
extract_price(item),
item['description']
)
end
end
private
attr_reader :json
def extract_category(item)
field = item['fields'].find do |field|
field['name'] == 'productGroup'
end
field['value'] if field
end
def extract_price(item)
offer = item['offers'].first
history = offer['priceHistory'].first
value = history['price']['value']
Integer(value) # Or use Float?
end
end
CategoryFilter returns a limited subset of the products. You can easily add other filters and combine them. Perhaps you might want to look into lazy for performance improvements.
class CategoryFilter
def initialize(products, *categories)
#products = products
#categories = categories
end
def call
products.select do |product|
categories.include?(product.category)
end
end
private
attr_reader :products, :categories
end
Use it like this:
limit = 10
categories = ['laptop', 'something']
params = {
q: categories.join(','),
limit: limit,
}
paramsString = params.map do |key, value|
"#{key}=#{value}"
end.join(';')
response = RestClient.get(
"http://api.tradedoubler.com/1.0/products.json;#{paramsString}?token=#{token}"
)
json = JSON.parse(response)
products = JsonProductBuilder.new(json).call
puts products.size
products = CategoryFilter.new(products, 'Klær', 'Sko', 'Jeans').call
puts products.size
products.each do |product|
puts product.to_h
end

Transferring JSON Data into an array using ruby

This is my JSON code
{
"jobs": [
{
"id": 1,
"title": "Software Developer",
"applicants": [
{
"id": 1,
"name": "Rich Hickey",
"tags": ["clojure", "java", "immutability", "datomic", "transducers"]
},
{
"id": 2,
"name": "Guido van Rossum",
"tags": ["python", "google", "bdfl", "drop-box"]
}
]
},
{
"id": 2,
"title": "Software Architect",
"applicants": [
{
"id": 42,
"name": "Rob Pike",
"tags": ["plan-9", "TUPE", "go", "google", "sawzall"]
},
{
"id": 2,
"name": "Guido van Rossum",
"tags": ["python", "google", "bdfl", "drop-box"]
},
{
"id": 1337,
"name": "Jeffrey Dean",
"tags": ["spanner", "BigTable", "MapReduce", "deep learning", "massive clusters"]
}
]
}
]
}
I want to put the list of "Jobs" in an array using ruby.
I have the following code so far.
require 'json'
file = File.read(filepath)
data_hash = JSON.parse(file)
How do I iterate on the data_hash and chose what information I want and place it in an array?
You can use Array#each because data_hash['jobs'] contains an array of jobs:
data_hash['jobs'].each {|job| ... }
Like this,
arr = Array.new
data_hash.each { |job|
arr.insert(job['name'])
}
use Array#map for shorter code
data_hash['jobs'].map do |job|
# Do whatever you want with the job here
properties = %w(title applicants)
job.select{ |key| properties.include?(key) }
end

Please help me with my Ruby script (ruby 2.0.0p481)

I am writing a ruby script to accept an input file that is a CSV. I want to make a pretty JSON file. I feel that I am so close but I can't seem to get there. My original project was in JS but the requirements have changed to make it a Ruby file.
My input file looks like this
item id,description,price,cost,price_type,quantity_on_hand,size_1_name,size_1_price,size_2_name,size_2_price,size_3_name,size_3_price
one thing to note is that some of the values in the CSV file maybe missing because it doesn't exist.
require 'csv'
require 'json'
def is_int(str)
return !!(str =~ /^[-+]?[1-9]([0-9]*)?$/)
end
lines = CSV.open(ARGV[0],{:col_sep => ","}).readlines
# remove first entry of the lines array
keys = lines.shift
lines.each do |values|
# convert the line into a hash and transform string into int
#hash=Hash[keys.zip(values.map{|val| is_int(val) ? val.to_i : val}) ]
hash = keys.zip(values.map{ val})
# Write a file with the hash results
File.open("#{hash['NAME']}.json", "w") do |f|
f.write JSON.pretty_generate [hash]
end
end
The output I am trying to get is
[
{
id: 111010,
description: 'Coffee',
price: 1.25,
cost: 0.80,
price_type: 'system',
quantity_on_hand: 100000,
modifiers: [
{
name: 'Small',
price: -0.25
},{
name: 'Medium',
price: 0.00
},{
name: 'Large',
price: 0.30
}
]
the version of Ruby I'm using is 2.0.0p481
Error was
usr/lib/ruby/2.0.0/csv.rb:1254:in `initialize': no implicit conversion of nil into String (TypeError)
from /usr/lib/ruby/2.0.0/csv.rb:1254:in `open'
from /usr/lib/ruby/2.0.0/csv.rb:1254:in `open'
from stockimporter.rb:8:in `<main>'
csv.csv:
id,description,price,cost,price_type,quantity_on_hand,size_1_name,size_1_price,size_2_name,size_2_price,size_3_name,size_3_price
111010,Coffee,1.25,0.80,system,10000,Small,-0.25,Medium,0.00,Large,0.30
111011,Tea,1.00,0.50,system,100,Small,-0.10,Medium,0.00,Large,0.10
111012,MissingInfo,1.00,0.50,,100,,,Medium,,Large,0.10
...
require 'csv'
require 'pp'
require 'json'
csv_options = {
headers: true, #skip first line of csv file
converters: [:numeric] #convert strings that look like integers or floats to integers or floats
}
results = []
CSV.foreach('csv.csv', csv_options) do |row|
record = Hash[row.take(6)]
modifiers = [
{
name: row["size_1_name"],
price: row["size_1_price"]
},
{
name: row["size_2_name"],
price: row["size_2_price"],
},
{
name: row["size_3_name"],
price: row["size_3_price"]
}
]
record['modifiers'] = modifiers
results << record
end
pp results
--output:--
[{"id"=>111010,
"description"=>"Coffee",
"price"=>1.25,
"cost"=>0.8,
"price_type"=>"system",
"quantity_on_hand"=>10000,
"modifiers"=>
[{:name=>"Small", :price=>-0.25},
{:name=>"Medium", :price=>0.0},
{:name=>"Large", :price=>0.3}]},
{"id"=>111011,
"description"=>"Tea",
"price"=>1.0,
"cost"=>0.5,
"price_type"=>"system",
"quantity_on_hand"=>100,
"modifiers"=>
[{:name=>"Small", :price=>-0.1},
{:name=>"Medium", :price=>0.0},
{:name=>"Large", :price=>0.1}]},
{"id"=>111012,
"description"=>"MissingInfo",
"price"=>1.0,
"cost"=>0.5,
"price_type"=>nil,
"quantity_on_hand"=>100,
"modifiers"=>
[{:name=>nil, :price=>nil},
{:name=>"Medium", :price=>nil},
{:name=>"Large", :price=>0.1}]}]
json = JSON.pretty_generate(results)
puts json
--output:--
[
{
"id": 111010,
"description": "Coffee",
"price": 1.25,
"cost": 0.8,
"price_type": "system",
"quantity_on_hand": 10000,
"modifiers": [
{
"name": "Small",
"price": -0.25
},
{
"name": "Medium",
"price": 0.0
},
{
"name": "Large",
"price": 0.3
}
]
},
{
"id": 111011,
"description": "Tea",
"price": 1.0,
"cost": 0.5,
"price_type": "system",
"quantity_on_hand": 100,
"modifiers": [
{
"name": "Small",
"price": -0.1
},
{
"name": "Medium",
"price": 0.0
},
{
"name": "Large",
"price": 0.1
}
]
},
{
"id": 111012,
"description": "MissingInfo",
"price": 1.0,
"cost": 0.5,
"price_type": null,
"quantity_on_hand": 100,
"modifiers": [
{
"name": null,
"price": null
},
{
"name": "Medium",
"price": null
},
{
"name": "Large",
"price": 0.1
}
]
}
]
You could also do it like this:
CSV.foreach('csv.csv', csv_options) do |row|
record = Hash[row.take(6)]
price_adjustments = row.drop(6)
# [["size_1_name", "Small"], ["size_1_price", -0.25]]
# |---------------------------------------------------|
# ^
# |
modifiers = price_adjustments.each_slice(2).map do |size_price|
size_price.first[0] = 'name'
size_price.last[0] = 'price'
Hash[size_price]
end
p modifiers #e.g. [{"name"=>"Small", "price"=>-0.25}, {"name"=>"Medium", "price"=>0.0}, {"name"=>"Large", "price"=>0.3}]
record['modifiers'] = modifiers
results << record
end
pp results

Merging records in JSON with Ruby

I have two json files that I'm trying to merge. The JSONs have different formatting (see below). I'd like to merge records, so [0] from file one and [0] from file two would become one record [0] in the new merged file.
The first JSON (file_a.json), appears like so:
{
"query": {
"count": 4,
"created": "2012-11-21T23:07:00Z",
"lang": "en-US",
"results": {
"quote": [
{
"Name": "Bill",
"Age": "46",
"Number": "3.55"
},
{
"Name": "Jane",
"Age": "33",
"Number": nil
},
{
"Name": "Jack",
"Age": "55",
"Number": nil
},
{
"Name": "Xavier",
"Age": nil,
"Number": "153353535"
}
]
}
}
}
The second JSON (file_b.json) appears like so:
[
{
"Number2": 25253,
"Number3": 435574,
"NAME": "Bill"
},
{
"Number2": 345353,
"Number3": 5566,
"NAME": "Jane"
},
{
"Number2": 56756,
"Number3": 232435,
"NAME": "Jack"
},
{
"Number2": 7457,
"Number3": 45425,
"NAME": "Xavier"
}
]
None of the keys are the same in both JSONs (well, actually "Name" is a key in both, but in the first the key is "Name" and in the second its "NAME" - just so I can check that the merge works correctly - so I want "Name" and "NAME" in the final JSON), the first record in the first file matches with the first record in the second file, and so on.
So far, I tried merging like this:
merged = %w[a b].inject([]) { |m,f| m << JSON.parse(File.read("file_#{f}.json")) }.flatten
But this of course merged them, but not how I wanted them merged (they are merged sucessively, and because of the different formatting, it gets quite ugly).
I also tried merging like this:
a = JSON.parse(File.read("file_a.json"))
b = JSON.parse(File.read("file_b.json"))
merged = a.zip(b)
Came closer but still not correct and the formatting was still horrendous.
In the end, what I want is this (formatting of second JSON - headers from first JSON can be junked):
[
{
"Name": "Bill",
"Age": 46,
"Number": 3.55,
"Number2": 25253,
"Number3": 435574,
"NAME": "Bill"
},
{
"Name": "Jane",
"Age": 33,
"Number": nil,
"Number2": 345353,
"Number3": 5566,
"NAME": "Jane"
},
{
"Name": "Jack",
"Age": 55,
"Number": nil,
"Number2": 56756,
"Number3": 232435,
"NAME": "Jack"
},
{
"Name": "Xavier",
"Age": nil,
"Number": 153353535,
"Number2": 7457,
"Number3": 45425,
"NAME": "Xavier"
}
]
Any help is appreciated. Thanks a lot.
Hеllo, seems format changed from last time :)
UPDATE: more readable version that also convert corresponding values to integers/floats:
require 'json'
require 'ap'
a = JSON.parse(File.read('./a.json'))['query']['results']['quote'] rescue []
b = JSON.parse(File.read('./b.json'))
final = []
a.each_with_index do |ah,i|
unless bh = b[i]
bh = {}
puts "seems b has no #{i} key, merging skipped"
end
final << ah.merge(bh).inject({}) do |f, (k,v)|
if v.is_a?(String)
if v =~ /\A\d+\.\d+\Z/
v = v.to_f
elsif v =~ /\A\d+\Z/
v = v.to_i
end
end
f.update k => v
end
end
ap final
will display:
[
[0] {
"Name" => "Bill",
"Age" => 46,
"Number" => 3.55,
"Number2" => 25253,
"Number3" => 435574,
"NAME" => "Bill"
},
[1] {
"Name" => "Jane",
"Age" => 33,
"Number" => nil,
"Number2" => 345353,
"Number3" => 5566,
"NAME" => "Jane"
},
[2] {
"Name" => "Jack",
"Age" => 55,
"Number" => nil,
"Number2" => 56756,
"Number3" => 232435,
"NAME" => "Jack"
},
[3] {
"Name" => "Xavier",
"Age" => nil,
"Number" => 153353535,
"Number2" => 7457,
"Number3" => 45425,
"NAME" => "Xavier"
}
]
Here is a working demo
Btw, your json is a bit wrong in both files.
See the fixed versions here and here

Resources