What's the simplest method to convert YAML to dot-separated strings in Ruby?
So this:
root:
child_a: Hello
child_b:
nested_child_a: Nesting
nested_child_b: Nesting Again
child_c: K
To this:
{
"ROOT.CHILD_A" => "Hello",
"ROOT.CHILD_B.NESTED_CHILD_A" => "Nesting",
"ROOT.CHILD_B.NESTED_CHILD_B" => "Nesting Again",
"ROOT.CHILD_C" => "K"
}
It's not a one-liner, but perhaps it will fit your needs
def to_dotted_hash(source, target = {}, namespace = nil)
prefix = "#{namespace}." if namespace
case source
when Hash
source.each do |key, value|
to_dotted_hash(value, target, "#{prefix}#{key}")
end
when Array
source.each_with_index do |value, index|
to_dotted_hash(value, target, "#{prefix}#{index}")
end
else
target[namespace] = source
end
target
end
require 'pp'
require 'yaml'
data = YAML.load(DATA)
pp data
pp to_dotted_hash(data)
__END__
root:
child_a: Hello
child_b:
nested_child_a: Nesting
nested_child_b: Nesting Again
child_c: K
prints
{"root"=>
{"child_a"=>"Hello",
"child_b"=>{"nested_child_a"=>"Nesting", "nested_child_b"=>"Nesting Again"},
"child_c"=>"K"}}
{"root.child_c"=>"K",
"root.child_b.nested_child_a"=>"Nesting",
"root.child_b.nested_child_b"=>"Nesting Again",
"root.child_a"=>"Hello"}
Related
I have a Hash which needs to be converted in a String with escaped characters.
{name: "fakename"}
and should end up like this:
'name:\'fakename\'
I don't know how this type of string is called. Maybe there is an already existing method, which I simply don't know...
At the end I would do something like this:
name = {name: "fakename"}
metadata = {}
metadata['foo'] = 'bar'
"#{name} AND #{metadata}"
which ends up in that:
'name:\'fakename\' AND metadata[\'foo\']:\'bar\''
Context: This query a requirement to search Stripe API: https://stripe.com/docs/api/customers/search
If possible I would use Stripe's gem.
In case you can't use it, this piece of code extracted from the gem should help you encode the query parameters.
require 'cgi'
# Copied from here: https://github.com/stripe/stripe-ruby/blob/a06b1477e7c28f299222de454fa387e53bfd2c66/lib/stripe/util.rb
class Util
def self.flatten_params(params, parent_key = nil)
result = []
# do not sort the final output because arrays (and arrays of hashes
# especially) can be order sensitive, but do sort incoming parameters
params.each do |key, value|
calculated_key = parent_key ? "#{parent_key}[#{key}]" : key.to_s
if value.is_a?(Hash)
result += flatten_params(value, calculated_key)
elsif value.is_a?(Array)
result += flatten_params_array(value, calculated_key)
else
result << [calculated_key, value]
end
end
result
end
def self.flatten_params_array(value, calculated_key)
result = []
value.each_with_index do |elem, i|
if elem.is_a?(Hash)
result += flatten_params(elem, "#{calculated_key}[#{i}]")
elsif elem.is_a?(Array)
result += flatten_params_array(elem, calculated_key)
else
result << ["#{calculated_key}[#{i}]", elem]
end
end
result
end
def self.url_encode(key)
CGI.escape(key.to_s).
# Don't use strict form encoding by changing the square bracket control
# characters back to their literals. This is fine by the server, and
# makes these parameter strings easier to read.
gsub("%5B", "[").gsub("%5D", "]")
end
end
params = { name: 'fakename', metadata: { foo: 'bar' } }
Util.flatten_params(params).map { |k, v| "#{Util.url_encode(k)}=#{Util.url_encode(v)}" }.join("&")
I use it now with that string, which works... Quite straigt forward:
"email:\'#{email}\'"
email = "test#test.com"
key = "foo"
value = "bar"
["email:\'#{email}\'", "metadata[\'#{key}\']:\'#{value}\'"].join(" AND ")
=> "email:'test#test.com' AND metadata['foo']:'bar'"
which is accepted by Stripe API
I have a CSV in the following format:
name,contacts.0.phone_no,contacts.1.phone_no,codes.0,codes.1
YK,1234,4567,AB001,AK002
As you can see, this is a nested structure. The CSV may contain multiple rows. I would like to convert this into an array of hashes like this:
[
{
name: 'YK',
contacts: [
{
phone_no: '1234'
},
{
phone_no: '4567'
}
],
codes: ['AB001', 'AK002']
}
]
The structure uses numbers in the given format to represent arrays. There can be hashes inside arrays. Is there a simple way to do that in Ruby?
The CSV headers are dynamic. It can change. I will have to create the hash on the fly based on the CSV file.
There is a similar node library called csvtojson to do that for JavaScript.
Just read and parse it line-by-line. The arr variable in the code below will hold an array of Hash that you need
arr = []
File.readlines('README.md').drop(1).each do |line|
fields = line.split(',').map(&:strip)
hash = { name: fields[0], contacts: [fields[1], fields[2]], address: [fields[3], fields[4]] }
arr.push(hash)
end
Let's first construct a CSV file.
str = <<~END
name,contacts.0.phone_no,contacts.1.phone_no,codes.0,IQ,codes.1
YK,1234,4567,AB001,173,AK002
ER,4321,7654,BA001,81,KA002
END
FName = 't.csv'
File.write(FName, str)
#=> 121
I have constructed a helper method to construct a pattern that will be used to convert each row of the CSV file (following the first, containing the headers) to an element (hash) of the desired array.
require 'csv'
def construct_pattern(csv)
csv.headers.group_by { |col| col[/[^.]+/] }.
transform_values do |arr|
case arr.first.count('.')
when 0
arr.first
when 1
arr
else
key = arr.first[/(?<=\d\.).*/]
arr.map { |v| { key=>v } }
end
end
end
In the code below, for the example being considered:
construct_pattern(csv)
#=> {"name"=>"name",
# "contacts"=>[{"phone_no"=>"contacts.0.phone_no"},
# {"phone_no"=>"contacts.1.phone_no"}],
# "codes"=>["codes.0", "codes.1"],
# "IQ"=>"IQ"}
By tacking if pattern.empty? onto the above expression we ensure the pattern is constructed only once.
We may now construct the desired array.
pattern = {}
CSV.foreach(FName, headers: true).map do |csv|
pattern = construct_pattern(csv) if pattern.empty?
pattern.each_with_object({}) do |(k,v),h|
h[k] =
case v
when Array
case v.first
when Hash
v.map { |g| g.transform_values { |s| csv[s] } }
else
v.map { |s| csv[s] }
end
else
csv[v]
end
end
end
#=> [{"name"=>"YK",
# "contacts"=>[{"phone_no"=>"1234"}, {"phone_no"=>"4567"}],
# "codes"=>["AB001", "AK002"],
# "IQ"=>"173"},
# {"name"=>"ER",
# "contacts"=>[{"phone_no"=>"4321"}, {"phone_no"=>"7654"}],
# "codes"=>["BA001", "KA002"],
# "IQ"=>"81"}]
The CSV methods I've used are documented in CSV. See also Enumerable#group_by and Hash#transform_values.
I'm storing configuration data in hashes written in flat files. I want to import the hashes into my Class so that I can invoke corresponding methods.
example.rb
{
:test1 => { :url => 'http://www.google.com' },
:test2 => {
{ :title => 'This' } => {:failure => 'sendemal'}
}
}
simpleclass.rb
class Simple
def initialize(file_name)
# Parse the hash
file = File.open(file_name, "r")
#data = file.read
file.close
end
def print
#data
end
a = Simple.new("simpleexample.rb")
b = a.print
puts b.class # => String
How do I convert any "Hashified" String into an actual Hash?
You can use eval(#data), but really it would be better to use a safer and simpler data format like JSON.
You can try YAML.load method
Example:
YAML.load("{test: 't_value'}")
This will return following hash.
{"test"=>"t_value"}
You can also use eval method
Example:
eval("{test: 't_value'}")
This will also return same hash
{"test"=>"t_value"}
Hope this will help.
I would to this using the json gem.
In your Gemfile you use
gem 'json'
and then run bundle install.
In your program you require the gem.
require 'json'
And then you may create your "Hashfield" string by doing:
hash_as_string = hash_object.to_json
and write this to your flat file.
Finally, you may read it easily by doing:
my_hash = JSON.load(File.read('your_flat_file_name'))
This is simple and very easy to do.
Should it not be clear, it is only the hash that must be contained in a JSON file. Suppose that file is "simpleexample.json":
puts File.read("simpleexample.json")
# #{"test1":{"url":"http://www.google.com"},"test2":{"{:title=>\"This\"}":{"failure":"sendemal"}}}
The code can be in a normal Ruby source file, "simpleclass.rb":
puts File.read("simpleclass.rb")
# class Simple
# def initialize(example_file_name)
# #data = JSON.parse(File.read(example_file_name))
# end
# def print
# #data
# end
# end
Then we can write:
require 'json'
require_relative "simpleclass"
a = Simple.new("simpleexample.json")
#=> #<Simple:0x007ffd2189bab8 #data={"test1"=>{"url"=>"http://www.google.com"},
# "test2"=>{"{:title=>\"This\"}"=>{"failure"=>"sendemal"}}}>
a.print
#=> {"test1"=>{"url"=>"http://www.google.com"},
# "test2"=>{"{:title=>\"This\"}"=>{"failure"=>"sendemal"}}}
a.class
#=> Simple
To construct the JSON file from the hash:
h = { :test1=>{ :url=>'http://www.google.com' },
:test2=>{ { :title=>'This' }=>{:failure=>'sendemal' } } }
we write:
File.write("simpleexample.json", JSON.generate(h))
#=> 95
Suppose I have following hash or nested hash:
h = { :a1 => { :b1 => "c1" },
:a2 => { :b2 => "c2"},
:a3 => { :b3 => "c3"} }
I want to create a method that takes hash as a parameter and recursively convert all the keys (keys that are symbol eg. :a1) to String (eg. "a1"). So far I have come up with the following method which doesn't work and returns {"a1"=>{:b1=>"c1"}, "a2"=>{:b2=>"c2"}, "a3"=>{:b3=>"c3"}}.:
def stringify_all_keys(hash)
stringified_hash = {}
hash.each do |k, v|
stringified_hash[k.to_s] = v
if v.class == Hash
stringify_all_keys(stringified_hash[k.to_s])
end
end
stringified_hash
end
What am I doing wrong and how do a get all the keys converted to string like this:
{"a1"=>{"b1"=>"c1"}, "a2"=>{"b2"=>"c2"}, "a3"=>{"b3"=>"c3"}}
If you are using ActiveSupport already or are open to using it, then deep_stringify_keys is what you're looking for.
hash = { person: { name: 'Rob', age: '28' } }
hash.deep_stringify_keys
# => {"person"=>{"name"=>"Rob", "age"=>"28"}}
Quick'n'dirty if your values are basic objects like strings, numbers, etc:
require 'json'
JSON.parse(JSON.dump(hash))
Didn't test this, but looks about right:
def stringify_all_keys(hash)
stringified_hash = {}
hash.each do |k, v|
stringified_hash[k.to_s] = v.is_a?(Hash) ? stringify_all_keys(v) : v
end
stringified_hash
end
using plain ruby code, the below code could help.
you can monkey patched it to the ruby Hash, to use it like this my_hash.deeply_stringfy_keys
however, I do not recommend monkey batching ruby.
you can adjust the method to provide the deeply_strigify_keys! (bang) version of it.
in case you want to make a different method witch does not stringify recursively, or to control the level of stringifying then consider re-writing the below method logic so you can have it written better with considering the other variation mentioned above.
def deeply_stringify_keys(hash)
stringified_hash = {}
hash.each do |k, v|
if v.is_a?(Hash)
stringified_hash[k.to_s] = deeply_stringify_keys(v)
elsif v.is_a?(Array)
stringified_hash[k.to_s] = v.map {|i| i.is_a?(Hash)? deeply_stringify_keys(i) : i}
else
stringified_hash[k.to_s] = v
end
end
stringified_hash
end
I need to create a signature string for a variable in Ruby, where the variable can be a number, a string, a hash, or an array. The hash values and array elements can also be any of these types.
This string will be used to compare the values in a database (Mongo, in this case).
My first thought was to create an MD5 hash of a JSON encoded value, like so: (body is the variable referred to above)
def createsig(body)
Digest::MD5.hexdigest(JSON.generate(body))
end
This nearly works, but JSON.generate does not encode the keys of a hash in the same order each time, so createsig({:a=>'a',:b=>'b'}) does not always equal createsig({:b=>'b',:a=>'a'}).
What is the best way to create a signature string to fit this need?
Note: For the detail oriented among us, I know that you can't JSON.generate() a number or a string. In these cases, I would just call MD5.hexdigest() directly.
I coding up the following pretty quickly and don't have time to really test it here at work, but it ought to do the job. Let me know if you find any issues with it and I'll take a look.
This should properly flatten out and sort the arrays and hashes, and you'd need to have to some pretty strange looking strings for there to be any collisions.
def createsig(body)
Digest::MD5.hexdigest( sigflat body )
end
def sigflat(body)
if body.class == Hash
arr = []
body.each do |key, value|
arr << "#{sigflat key}=>#{sigflat value}"
end
body = arr
end
if body.class == Array
str = ''
body.map! do |value|
sigflat value
end.sort!.each do |value|
str << value
end
end
if body.class != String
body = body.to_s << body.class.to_s
end
body
end
> sigflat({:a => {:b => 'b', :c => 'c'}, :d => 'd'}) == sigflat({:d => 'd', :a => {:c => 'c', :b => 'b'}})
=> true
If you could only get a string representation of body and not have the Ruby 1.8 hash come back with different orders from one time to the other, you could reliably hash that string representation. Let's get our hands dirty with some monkey patches:
require 'digest/md5'
class Object
def md5key
to_s
end
end
class Array
def md5key
map(&:md5key).join
end
end
class Hash
def md5key
sort.map(&:md5key).join
end
end
Now any object (of the types mentioned in the question) respond to md5key by returning a reliable key to use for creating a checksum, so:
def createsig(o)
Digest::MD5.hexdigest(o.md5key)
end
Example:
body = [
{
'bar' => [
345,
"baz",
],
'qux' => 7,
},
"foo",
123,
]
p body.md5key # => "bar345bazqux7foo123"
p createsig(body) # => "3a92036374de88118faf19483fe2572e"
Note: This hash representation does not encode the structure, only the concatenation of the values. Therefore ["a", "b", "c"] will hash the same as ["abc"].
Here's my solution. I walk the data structure and build up a list of pieces that get joined into a single string. In order to ensure that the class types seen affect the hash, I inject a single unicode character that encodes basic type information along the way. (For example, we want ["1", "2", "3"].objsum != [1,2,3].objsum)
I did this as a refinement on Object, it's easily ported to a monkey patch. To use it just require the file and run "using ObjSum".
module ObjSum
refine Object do
def objsum
parts = []
queue = [self]
while queue.size > 0
item = queue.shift
if item.kind_of?(Hash)
parts << "\\000"
item.keys.sort.each do |k|
queue << k
queue << item[k]
end
elsif item.kind_of?(Set)
parts << "\\001"
item.to_a.sort.each { |i| queue << i }
elsif item.kind_of?(Enumerable)
parts << "\\002"
item.each { |i| queue << i }
elsif item.kind_of?(Fixnum)
parts << "\\003"
parts << item.to_s
elsif item.kind_of?(Float)
parts << "\\004"
parts << item.to_s
else
parts << item.to_s
end
end
Digest::MD5.hexdigest(parts.join)
end
end
end
Just my 2 cents:
module Ext
module Hash
module InstanceMethods
# Return a string suitable for generating content signature.
# Signature image does not depend on order of keys.
#
# {:a => 1, :b => 2}.signature_image == {:b => 2, :a => 1}.signature_image # => true
# {{:a => 1, :b => 2} => 3}.signature_image == {{:b => 2, :a => 1} => 3}.signature_image # => true
# etc.
#
# NOTE: Signature images of identical content generated under different versions of Ruby are NOT GUARANTEED to be identical.
def signature_image
# Store normalized key-value pairs here.
ar = []
each do |k, v|
ar << [
k.is_a?(::Hash) ? k.signature_image : [k.class.to_s, k.inspect].join(":"),
v.is_a?(::Hash) ? v.signature_image : [v.class.to_s, v.inspect].join(":"),
]
end
ar.sort.inspect
end
end
end
end
class Hash #:nodoc:
include Ext::Hash::InstanceMethods
end
These days there is a formally defined method for canonicalizing JSON, for exactly this reason: https://datatracker.ietf.org/doc/html/draft-rundgren-json-canonicalization-scheme-16
There is a ruby implementation here: https://github.com/dryruby/json-canonicalization
Depending on your needs, you could call ary.inspect or ary.to_yaml, even.