ruby write json file from an object - ruby

I'm getting an error expected ':' after property name in object at line 1 column 15 how can I get rid of '=>'? when I replace "=>" by ":" in the to_json methods I'm getting an error syntax error, unexpected ':', expecting =>
require 'json'
class Province
attr_accessor :provOrigine, :destination, :total,
:q1, :q2, :q3, :q4
def initialize(line)
#provOrigine = line.split(';').first.split(",").first
#destination = line.split(';').at(1).split(',').first
#q1 = (line.split(';').at(4)).to_i
#q2 = (line.split(';').at(5)).to_i
#q3 = (line.split(';').at(6)).to_i
#q4 = (line.split(';').at(7)).to_i
end
def to_json
{'provOrigine' => #provOrigine.to_s, 'destination' => #destination.to_s, 'q1' => #q1.to_s, 'q2' => #q2.to_s, 'q3' => #q3.to_s, 'q4' => #q4.to_s}
end
end
...
prov_instances = contains All the instances of Province
...
File.open("file_json_complete.json", "w") do |f|
prov_instances.each do |n|
f.write(n.to_json)
end
end
this is the result I'm getting
{"provOrigine"=>"Alberta", "destination"=>"Terre-Neuve-et-Labrador", "q1"=>"777", "q2"=>"1089", "q3"=>"553", "q4"=>"474"}{"provOrigine"=>"Alberta", "destination"=>"Nunavut", "q1"=>"24", "q2"=>"70", "q3"=>"29", "q4"=>"29"}{"provOrigine"=>"Alberta", "destination"=>"Île-du-Prince-Édouard", "q1"=>"116", "q2"=>"69", "q3"=>"150", "q4"=>"64"
}
and there is no commas between each object?

You'd need to add the commas yourself. f.write(n.to_json) is going to write out a single Province. It has no way to know you're going to keep writing more and need a comma.
Is there a reason you can't do this instead?
File.open('file_json_complete.json', 'w') do |f|
f.puts prov_instances.to_json
end

I needed to take the hash {} and cast it to_json in order to create a json string
class Province
def to_json(*a)
{'provOrigine' => #provOrigine.to_s,
'destination' => #destination.to_s,
'q1' => #q1.to_s, 'q2' => #q2.to_s,
'q3' => #q3.to_s, 'q4' => #q4.to_s
}.to_json(*a)
end
end
I don't need to loop thought each instances. I could take the object Array and cast it to_json
File.open("file_json_complete.json", "w") do |f|
f.write(prov_instances.to_json)
end

Related

puppet - unexpected result from 'each' in a custom function

I have a simple function which takes a JSON and 'does something' with it. The main part works good BUT the function returns not only what I want but additionally the result of .each loop!
The code:
module Puppet::Parser::Functions
newfunction(:mlh, :type => :rvalue) do |args|
lvm_default_hash = args[0]
lvm_additional_hash = args[1]
if lvm_additional_hash.keys.length == 1
if lvm_additional_hash.keys.include? 'logical_volumes'
# do stuff - we have only 'logical_volumes'
lvm_default_hash.keys.each do |key|
pv_array = Hash['physical_volumes' => lvm_default_hash[key]['physical_volumes']]
lv_hash = lvm_default_hash[key]['logical_volumes']
new_lv_hash = lvm_additional_hash['logical_volumes']
merged_lv_hash = Hash['logical_volumes' => lv_hash.merge(new_lv_hash)]
# this is what I want to return to init.pp
puts Hash[key => pv_array.merge(merged_lv_hash)]
end
end
end
end
end
Variables in the init.pp are:
$default_volume_groups = {
'sys' => {
'physical_volumes' => [
'/dev/sda2',
],
'logical_volumes' => {
'root' => {'size' => '4G'},
'swap' => {'size' => '256M'},
'var' => {'size' => '8G'},
'docker' => {'size' => '16G'},
},
},
}
and the second argument from a hieradata:
modified_volume_groups:
logical_volumes:
cloud_log:
size: '16G'
In the init.pp I have something like this to test it:
notice(mlh($default_volume_groups, $modified_volume_groups))
which gives me a result:
syslogical_volumesvarsize8Gdockersize16Gcloud_logsize16Gswapsize256Mrootsize4Gphysical_volumes/dev/sda2
Notice: Scope(Class[Ops_lvm]): sys
The "long" part before the Notice is the proper result from the puts but the Notice: Scope(): sys is this what I do not want to!
I know that this is the result of this each loop over the default_volumes_groups:
lvm_default_hash.keys.each do |key|
# some stuff
end
How to block of this unwanted result? It blows my puppet's logic because my init.pp sees this sys and not what I want.
Does someone knows how to handle such problem?
Thank you!
I found how to handle this problem but maybe someone could explain me why it works in this way :)
This does not work (short version):
module Puppet::Parser::Functions
newfunction(:mlh, :type => :rvalue) do |args|
lvm_default_hash = args[0]
lvm_additional_hash = args[1]
if lvm_additional_hash.keys.length == 1
if lvm_additional_hash.keys.include? 'logical_volumes'
lvm_default_hash.keys.each do |key|
pv_array = Hash['physical_volumes' => lvm_default_hash[key]['physical_volumes']]
lv_hash = lvm_default_hash[key]['logical_volumes']
new_lv_hash = lvm_additional_hash['logical_volumes']
merged_lv_hash = Hash['logical_volumes' => lv_hash.merge(new_lv_hash)]
puts Hash[key => pv_array.merge(merged_lv_hash)]
end
end
end
end
end
but this works:
module Puppet::Parser::Functions
newfunction(:mlh, :type => :rvalue) do |args|
lvm_default_hash = args[0]
lvm_additional_hash = args[1]
# empty Hash
hash_to_return = {}
if lvm_additional_hash.keys.length == 1
if lvm_additional_hash.keys.include? 'logical_volumes'
lvm_default_hash.keys.each do |key|
pv_array = Hash['physical_volumes' => lvm_default_hash[key]['physical_volumes']]
lv_hash = lvm_default_hash[key]['logical_volumes']
new_lv_hash = lvm_additional_hash['logical_volumes']
merged_lv_hash = Hash['logical_volumes' => lv_hash.merge(new_lv_hash)]
# assigned value in the 'each' loop we want to return to puppet
hash_to_return = Hash[key => pv_array.merge(merged_lv_hash)]
end
# returned Hash - instead of previous 'puts'
return hash_to_return
end
end
end
end
Now I have what I need!
Notice: Scope(Class[Ops_lvm]): sysphysical_volumes/de
You've got it -- the first one doesn't work because in Ruby, the return value of a block or function is the last evaluated statement. In the case of the one that didn't work, the last evaluated statement was the .each. As it turns out, each evaluates to the enumerable that it was looping through.
A simple example:
def foo
[1, 2, 3].each do |n|
puts n
end
end
If I were to run this, the return value of the function would be the array:
> foo
1
2
3
=> [1, 2, 3]
So what you have works, because the last thing evaluated is return hash_to_return. You could even just go hash_to_return and it'd work.
If you wanted to get rid of the return and clean that up a little bit (and if you're using Ruby 1.9 or above), you could replace your each line with:
lvm_default_hash.keys.each_with_object({}) do |key, hash_to_return|
This is because each_with_object evaluates to the "object" (in this case the empty hash passed into the method, and referred to as hash_to_return in the block params). If you do this you can remove the return as well as the initialization hash_to_return = {}.
Hope this helps!
Your custom function has rvalue type which means it needs to return value. If you don't specify return <something> by default, your last statement is implicitly your return.
In the example above, first one that does not work correctly, has last statement inside each block:
puts Hash[key => pv_array.merge(merged_lv_hash)]
Your second example is correct simply because you set value for hash_to_return in each block and then "return" it outside of each block. Not sure if this is the behavior you want since last assigned hash value (in last loop inside each block) will be the one that will be returned from this function.

Ruby CSV: Comparison of columns (from two csvs), write new column in one

I've searched and haven't found a method for this particular conundrum. I have two CSV files of data that sometimes relate to the same thing. Here's an example:
CSV1 (500 lines):
date,reference,amount,type
10/13/2015,,1510.40,sale
10/13/2015,,312.90,sale
10/14/2015,,928.50,sale
10/15/2015,,820.25,sale
10/12/2015,,702.70,credit
CSV2 (20000 lines):
reference,date,amount
243534985,10/13/2015,312.90
345893745,10/15/2015,820.25
086234523,10/14/2015,928.50
458235832,10/13/2015,1510.40
My goal is to match the date and amount from CSV2 with the date and amount in CSV1, and write the reference from CSV2 to the reference column in the corresponding row.
This is a simplified view, as CSV2 actually contains many many more columns - these are just the relevant ones, so ideally I'd like to refer to them by header name or maybe index somehow?
Here's what I've attempted, but I'm a bit stuck.
require 'csv'
data1 = {}
data2 = {}
CSV.foreach("data1.csv", :headers => true, :header_converters => :symbol, :converters => :all) do |row|
data1[row.fields[0]] = Hash[row.headers[1..-1].zip(row.fields[1..-1])]
end
CSV.foreach("data2.csv", :headers => true, :header_converters => :symbol, :converters => :all) do |row|
data2[row.fields[0]] = Hash[row.headers[1..-1].zip(row.fields[1..-1])]
end
data1.each do |data1_row|
data2.each do |data2_row|
if (data1_row['comparitive'] == data2_row['comparitive'])
puts data1_row['identifier'] + data2_row['column_thats_important_and_wanted']
end
end
end
Result:
22:in `[]': no implicit conversion of String into Integer (TypeError)
I've also tried:
CSV.foreach('data2.csv') do |data2|
CSV.foreach('data1.csv') do |data1|
if (data1[3] == data2[4])
data1[1] << data2[1]
puts "Change made!"
else
puts "nothing changed."
end
end
end
This however did not match anything inside the if statement, so perhaps not the right approach?
The headers method should help you match columns--from there it's a matter of parsing and writing the modified data back out to a file.
Solved.
data1 = CSV.read('data1.csv')
data2 = CSV.read('data2.csv')
data2.each do |data2|
data1.each do |data1|
if (data1[5] == data2[4])
data1[1] = data2[1]
puts "Change made!"
puts data1
end
end
end
File.open('referenced.csv','w'){ |f| f << data1.map(&:to_csv).join("")}

compose objects without initializing objects that are not in hash

I am trying to compose an object Transaction from objects TranFee and Rate.
class Transaction
attr_reader :tranfee, :rate
def initialize(hash)
#tranfee = PaymentType::TranFee.new(hash)
#rate = PaymentType::Rate.new(hash)
end
end
module PaymentType
def initialize(args = {}, regex)
args.each do |key,value|
if key =~ regex
instance_variable_set("##{key}", value) unless value.nil?
eigenclass = class << self; self; end
eigenclass.class_eval do
attr_reader key
end
end
end
end
class TranFee
include PaymentType
def initialize(args, regex = /\Atran.*/)
super(args, regex)
end
end
class Rate
include PaymentType
def initialize(args, regex = /\Arate.*/)
super(args, regex)
end
end
end
The rate and TranFee objects are created from a hash like the one below.
reg_debit = {"name" => "reg_debit", "rate_base" => 0.0005,
"tran_fee" => 0.21, "rate_basis_points" => 0.002, "tran_auth_fee" => 0.10}
I am initializing the objects based on regex because the hash will eventually contain more values and I want the program to adjust as more items/classes are added.
Additionally there will be some instances where there are no key's starting with "tran". Does anyone know how to make Transaction create only a Rate object if TranFee has no instance variables inside of it? (in otherwords, if the hash returns nothing when keys =~ /\Atran.*/)
an example would be when the hash looks like this reg_debit = {"name" => "reg_debit", "rate_base" => 0.0005, "rate_basis_points" => 0.002}, right now the output is
#<Transaction:0x007ff98c070548 #tranfee=#<PaymentType::TranFee:0x007ff98c070520>, #rate=#<PaymentType::Rate:0x007ff98c0704a8 #rate_base=0.0005, #rate_basis_points=0.002>>
So I am getting a TranFee object with nothing in it and I would like for that to drop off in this situation. not sure if there may be a better way to design this? I was trying to think of a way to use ostruct or struct, but I havnt been able to figure it out. thanks for any help here.
I believe your strategy is very problematic - creating attributes to a class from user input doesn't sound like a very good idea.
Furthermore, adding methods (like attr_reader) to every instances can have severe performance issues.
If all you want is a data structure to hold your data, keep using a Hash. If you want a structure you can query using a dot notation instead of bracket notation, you might want to consider a gem like hashie or hashr.
If you want some code to make the flat data-structure hierarchical, I can suggest something like this:
hierarchical_hash = hash.each_with_object({}) do |(k, v), h|
if k.match(/^([^_]+)_(.+)$/)
root_key = $1
child_key = $2
h[root_key] ||= {}
h[root_key][child_key] = v
else
h[k] = v
end
end
# => {
# => "name" => "reg_debit",
# => "rate" => {
# => "base" => 0.0005,
# => "basis_points" => 0.002
# => },
# => "tran" => {
# => "fee" => 0.21,
# => "auth_fee" => 0.1
# => }
# => }
Your question raises some interesting issues. I will try to explain how you can fix it, but, as #Uri mentions, there may be better ways to address your problem.
I've assumed #tranfee is to be set equal to the first value in the hash whose key begins with "tran" and that #rate is to be set equal to the first value in the hash whose key begins with "rate". If that interpretation is not correct, please let me know.
Note that I've put initialize in the PaymentType module in a class (Papa) and made TranFee and Rate subclasses. That's the only way you can use super within initialize in the subclasses of that class.
Code
class Transaction
attr_reader :tranfee, :rate
def initialize(hash={})
o = PaymentType::TranFee.new(hash)
#tranfee = o.instance_variable_get(o.instance_variables.first)
o = PaymentType::Rate.new(hash)
#rate = o.instance_variable_get(o.instance_variables.first)
end
end
.
module PaymentType
class Papa
def initialize(hash, prefix)
key, value = hash.find { |key,value| key.start_with?(prefix) && value }
(raise ArgumentError, "No key beginning with #{prefix}") unless key
instance_variable_set("##{key}", value)
self.class.singleton_class.class_eval { attr_reader key }
end
end
class TranFee < Papa
def initialize(hash)
super hash, "tran"
end
end
class Rate < Papa
def initialize(hash)
super hash, "rate"
end
end
end
I believe the method Object#singleton_class has been available since Ruby 1.9.3.
Example
reg_debit = {"name" => "reg_debit", "rate_base" => 0.0005, "tran_fee" => 0.21,
"rate_basis_points" => 0.002, "tran_auth_fee" => 0.10}
a = Transaction.new reg_debit
p Transaction.instance_methods(false) #=> [:tranfee, :rate]
p a.instance_variables #=> [:#tranfee, :#rate]
p a.tranfee #=> 0.21
p a.rate #=> 0.0005

Ruby yaml custom domain type does not keep class

I'm trying to dump duration objects (from the ruby-duration gem) to yaml with a custom type, so they are represented in the form hh:mm:ss. I've tried to modify the answer from this question, but when parsing the yaml with YAML.load, a Fixnum is returned instead of a Duration. Interestingly, the Fixnum is the total number of seconds in the duration, so the parsing seems to work, but convert to Fixnum after that.
My code so far:
class Duration
def to_yaml_type
"!example.com,2012-06-28/duration"
end
def to_yaml(opts = {})
YAML.quick_emit( nil, opts ) { |out|
out.scalar( to_yaml_type, to_string_representation, :plain )
}
end
def to_string_representation
format("%h:%m:%s")
end
def Duration.from_string_representation(string_representation)
split = string_representation.split(":")
Duration.new(:hours => split[0], :minutes => split[1], :seconds => split[2])
end
end
YAML::add_domain_type("example.com,2012-06-28", "duration") do |type, val|
Duration.from_string_representation(val)
end
To clarify, what results I get:
irb> Duration.new(27500).to_yaml
=> "--- !example.com,2012-06-28/duration 7:38:20\n...\n"
irb> YAML.load(Duration.new(27500).to_yaml)
=> 27500
# should be <Duration:0xxxxxxx #seconds=20, #total=27500, #weeks=0, #days=0, #hours=7, #minutes=38>
It look like you’re using the older Syck interface, rather that the newer Psych. Rather than using to_yaml and YAML.quick_emit, you can use encode_with, and instead of add_domain_type use add_tag and init_with. (The documentation for this is pretty poor, the best I can offer is a link to the source).
class Duration
def to_yaml_type
"tag:example.com,2012-06-28/duration"
end
def encode_with coder
coder.represent_scalar to_yaml_type, to_string_representation
end
def init_with coder
split = coder.scalar.split ":"
initialize(:hours => split[0], :minutes => split[1], :seconds => split[2])
end
def to_string_representation
format("%h:%m:%s")
end
def Duration.from_string_representation(string_representation)
split = string_representation.split(":")
Duration.new(:hours => split[0], :minutes => split[1], :seconds => split[2])
end
end
YAML.add_tag "tag:example.com,2012-06-28/duration", Duration
p s = YAML.dump(Duration.new(27500))
p YAML.load s
The output from this is:
"--- !<tag:example.com,2012-06-28/duration> 7:38:20\n...\n"
#<Duration:0x00000100e0e0d8 #seconds=20, #total=27500, #weeks=0, #days=0, #hours=7, #minutes=38>
(The reason the result you’re seeing is the total number of seconds in the Duration is because it is being parsed as sexagesimal integer.)

Ruby 1.9: turn these 4 arrays into hash of key/value pairs

I have four arrays that are coming in from the client. Let's say that there is an array of names, birth dates, favorite color and location. The idea is I want a hash later where each name will have a hash with respective attributes:
Example date coming from the client:
[name0, name1, name2, name3]
[loc0, loc1]
[favcololor0, favcolor1]
[bd0, bd1, bd2, bd3, bd4, bd5]
Output I'd like to achieve:
name0 => { location => loc0, favcolor => favcolor0, bd => bd0 }
name1 => { location => loc1, favcolor => favcolor1, bd => bd1 }
name2 => { location => nil, favcolor => nil, bd => bd2 }
name3 => { location => nil, favcolor => nil, bd => bd3 }
I want to have an array at the end of the day where I can iterate and work on each particular person hash.
There need not be an equivalent number of values in each array. Meaning, names are required.. and I might receive 5 of them, but I only might receive 3 birth dates, 2 favorite colors and 1 location. Every missing value will result in a nil.
How does one make that kind of data structure with Ruby 1.9?
I would probably do it like this
# assuming names, fav_colors, birth_dates, and locations are your arrays
name_collection = {}
names.zip(birth_dates, fav_colors, locations) do |name, birth_date, fav_color, location|
name_collection[name] = { :birth_date => birth_date,
:fav_color => fav_color,
:location => location }
end
# usage
puts name_collection['jack'][:fav_color] # => 'blue'
A small class to represent a person
class Person
attr_accessor :name, :color, :loc, :bd
def initialize(args = {})
#name = args[:name]
#color = args[:color]
#loc = args[:loc]
#bd = args[:bd]
end
def pp()
puts "*********"
puts "Name: #{#name}"
puts "Location: #{#loc}"
puts "Birthday: #{#bd}"
puts "Fav. Color: #{#color}"
puts "*********"
end
end
another to represent people, which is mainly just a listing of Persons.
class People
attr_accessor :list_of_people
def initialize()
#list_of_people = {}
end
def load_people(names, locations, favcolors, birthdates)
names.each_with_index do |name, index|
#list_of_people[name] = Person.new(:name => name, :color => favcolors[index], :loc => locations[index], :bd => birthdates[index])
end
end
def pp()
#list_of_people.each_pair do |key, value|
value.pp()
end
end
end
I threw in a pretty print function for each so you can see their data. With a starting point like this it will be really easy to modify and add methods that do all sorts of useful things.
if __FILE__ == $0
names = ["name0", "name1", "name2", "name3"]
locs = ["loc0","loc1"]
favcolors = ["favcolor0", "favcolor1"]
bds = ["bd0","bd1","bd2","bd3","bd4"]
a = People.new()
a.load_people(names,locs,favcolors,bds)
a.pp()
end
I think the kind of data structure you're looking for is -ahem- a Struct.
# setup data
names = %w(name0 name1 name2 name3)
locations = %w(loc0 loc1)
colors = %w(favcololor0 favcolor1)
bd = %w(bd0 bd1 bd2 bd3 bd4 bd5)
# let's go
Person = Struct.new( :name, :location, :fav_color, :bd )
all_persons = names.zip( locations, colors, bd ).map{|p| Person.new( *p)}
# done
puts all_persons
someone= all_persons.find{|p| p.name == "name1"}
puts someone.location unless someone.nil?

Resources