Reading a hash from a file on disk [duplicate] - ruby

This question already has answers here:
Saving hashes to file on Ruby
(3 answers)
Closed 5 years ago.
Here is a hash that I save to a file to later be read.
my_hash = {-1 => 20, -2 => 30, -3 => 40}
File.open("my_file.txt", "w") { |f| f.write my_hash }
#how it looks opening the text file
{-1 => 20, -2 => 30, -3 => 40}
When I go to read it, is where my problem is. (following code is separate from top)
my_hash = File.foreach("my_file.txt") { |f| print f }
p my_hash
#=> {-1 => 20, -2 => 30, -3 => 40}nil
that nil messes up the rest of my code..not sure how to get rid of if. Just for clarity the rest of the code...
back_up_hash = {-1 => 20}
if my_hash.nil?
my_hash = back_up_hash
end
That little nil always makes my_hash equal to back_up_hash. I need that .nil? just in case the file is doesn't have the hash, otherwise the problem just gets pushed further down.
I also tried to read (slurp?..it's a small file) the file like this....
my_hash = File.read("my_file.txt") { |f| print f }
p my_hash
=> "{-1 => 20, -2 => 30, -3 => 40}"
# not sure how to get it out of string form...and I have searched for it.

You could use the eval method on the string (source)
eval("{-1 => 20, -2 => 30, -3 => 40}")
=> {-1 => 20, -2 => 30, -3 => 40}

If you want to take a file on disk whose contents are {-1 => 20, -2 => 30, -3 => 40} and make a hash from it, you want:
hash_str = File.read('my_file.txt')
my_hash = eval(hash_str) # Treat a string like Ruby code and evaluate it
# or, as a one-liner
my_hash = eval(File.read('my_file.txt'))
What you were doing is reading in the file and printing it to the screen, one line at a time. The 'print' command does not transform the data, and the foreach method does not map the data it yields to your block into any result. This is why you get nil for your my_hash.
As I recommended in a comment, if you have a Ruby object (like a Hash) and you need to save it to disk and load it later, you may want to use the Marshal module (built into Ruby):
$ irb
irb(main):001:0> h = {-1 => 20, -2 => 30, -3 => 40}
#=> {-1=>20, -2=>30, -3=>40}
irb(main):002:0> File.open('test.marshal','wb'){ |f| Marshal.dump(h, f) }
#=> #<File:test.marshal (closed)>
$ irb # later, a new irb session with no knowledge of h
irb(main):001:0> h = File.open('test.marshal'){ |f| Marshal.load(f) }
#=> {-1=>20, -2=>30, -3=>40}

The proper way to save simple data structures to a file is to serialize them. In this particular case, using JSON is probably a good choice:
# save hash to file:
f.write MultiJson.dump(my_hash)
# load it back:
p MultiJson.load(file_contents)
Keep in mind that JSON is only able to serialize simple, built-in data types (strings, numbers, arrays, hashes and the like). You will not be able to serialize and deserialize custom objects this way without some additional work.
If you don't have MultiJson, try it with JSON instead.

I have had success with these 2 simple methods:
def create_json_copy
File.open("db/json_records/stuff.json","w") do |f|
f.write("#{#existing_data.to_json}")
end
end
def read_json_copy
#json = JSON.parse(File.read("db/json_records/stuff.json")).as_json.with_indifferent_access
#json.each do |identifier,record|
existing_record = Something.find_by(some_column: identifier)
if !existing_record
Something.create!(record.except(:id).except(:created_at).except(:updated_at))
end
end
end
note: #existing_data is a Ruby Hash organised as { some_identifier: record_objet, ... } . I call .to_json on it before writing it to file and then when reading I JSON.parse it followed by .as_json, with_indifferent_access isn't really needed here so you can take it off as long as you substitute the symbols inside the excepts.

Related

Ruby: what's the difference using property with colons and quotes

I'm a newbie on Ruby, just doing a fast upgrade in a existent project and I'm wondering what's the difference between object record and to_validate.
puts to_validated.class
# Hash
puts to_validated
# {"data"=>{"name"=>"david"}, "metadata"=>{"body_size"=>"16", "collector_ip"=>"172.22.0.1", "collector_timestamp"=>1579608324863, "event"=>"whatever", "version"=>"1.0"}}
puts record.class
# Hash
puts record
# {:data=>{"name"=>"david"}, :metadata=>{"body_size"=>"16", "collector_ip"=>"172.22.0.1", "collector_timestamp"=>1579610268940, "event"=>"default", "version"=>"1.0.0"}}
The only difference on those objects is colons on data and metadata. Is possible to convert colons into quotes?
I know it's a dumb question but I'm applying a fix in this project and using a third party library that is failing using record object.
You can use Hash.transform_keys which was introduced to Ruby in version 2.5 to change symbolized keys into strings:
record.transform_keys { |k| k.to_s }
In Ruby hashes are a key value data structure. The keys can actually be a mix of any kind of objects:
hash_with_numerical_keys = {
1 => 'A',
2 => 'B',
3 => 'C'
3.5 => 'D'
}
hash_with_string_keys = {
'a' => 1,
'b' => 2,
'c' => 3
}
hash_with_symbols = {
:a => 1,
:b => 2,
:c => 3
}
# can also be declared as
hash_with_symbols = {
a: 1,
b: 2,
c: 3
}
hash_with_singletons = {
nil => 0,
true => 1,
false => 2
}
Symbols are commonly used as only one instance of a symbol ever exists. They are thus really effective to compare. String keys mostly come into play when you're dealing with JSON or some kind of external data.
The hash you are looking at contains a mixture of string and symbol keys:
{:data=>{"name"=>"david"}, :metadata=>{"body_size"=>"16", "collector_ip"=>"172.22.0.1", "collector_timestamp"=>1579610268940, "event"=>"default", "version"=>"1.0.0"}}
In Ruby 2.5 you can use hash#transform_keys to change the keys into strings:
hash.transform_keys(&:to_s)
In earlier versions you can do:
hash.each_with_object({}) do |(k,v), new_hash|
new_hash[k.to_s] = v
end
If you are using Rails or just ActiveSupport you can use Hash#stringify_keys, Hash#deep_stringify_keys or HashWithIndifferentAccess

Parsing a Ruby Array to JSON

I have some results:
puts result
That look like this output:
Allowed
20863963
1554906
Denied
3607325
0
Quarantined
156240
0
Debug
p results
output
[["Allowed", 20863963, 1554906], ["Denied", 3607325, 0], ["Quarantined", 156194, 0]]
The headers are:
status,hits,page_views
I need to convert this to json. If the results was in standard csv format then it would be straight forward but how would one approach it if the results format looked like above?
Expected output something similar to this:
[{"status":"Allowed","hits":"20863963","page_views":"1554906"},{"status":"Denied","hits":"3607325","page_views":"0"},{"status":"Quarantined","hits":"156240","page_views":"0"}]
Solution
a = result.map{|s| {status: s[0], hits: s[1].to_i, page_views: s[2].to_i} }
puts a.to_json
Look at to_json method.
require 'json'
# => true
h = {a: 1, b: 2,c: 3}
# => {a: 1, b: 2,c: 3}
h.to_json
# => "{\"a\":1,\"b\":2,\"c\":3}"
output = "Allowed
20863963
1554906
Denied
3607325
0
Quarantined
156240
0"
a = output.split("\n").each_slice(3).map{|s| {status: s[0], hits: s[1].to_i, page_views: s[2].to_i} } # => [{:status=>"Allowed", :hits=>20863963, :page_views=>1554906}, {:status=>"Denied", :hits=>3607325, :page_views=>0}, {:status=>"Quarantined", :hits=>156240, :page_views=>0}]
a.to_json # => => "[{\"status\":\"Allowed\",\"hits\":20863963,\"page_views\":1554906},{\"status\":\"Denied\",\"hits\":3607325,\"page_views\":0},{\"status\":\"Quarantined\",\"hits\":156240,\"page_views\":0}]"
You assign your "headers" into the attr_accessor and then tell JSON to parse that symbol. Here's an example:
class Document
attr_accessor :content
def content
metadata[:content] || metadata['content']
end
def self.parse_contents
txt = File.read(path, {mode: 'r:bom|utf-8'})
page = Document.new
page.metadata = JSON.parse(txt)
page.content = page.metadata['content']
return page
end
end
Hope it helps!

Iterating over Ruby hash while comparing values to another Ruby hash

I have 2 Ruby objects that I am converting to hashes: one from XML and another from JSON. When I puts the variable name I get hash, so it appears that I'm doing that correctly.
The format is several records in the format below.
Format of hash one (smithjj being a unique username):
{ smithjj => {office => 331, buidling => 1} }
Format of hash 2:
{"Data"=>{"xmlns:dmd"=>"http://www.xyz.com/schema/data-metadata",
"dmd:date"=>"2012-03-06", "Record"=>{"PCI"=>{"DPHONE3"=>nil, "OPHONE3"=>"111",
"DTY_DOB"=>"1956", "TEACHING_INTERESTS"=>nil, "FAX1"=>"123", "ROOMNUM"=>"111",
"DTD_DOB"=>"5", "DTM_DOB"=>"11", "WEBSITE"=>"www.test.edu", "FAX2"=>"324",
"ENDPOS"=>"Director", "LNAME"=>"Smith", "FAX3"=>"4891", "MNAME"=>"Thomas",
"GENDER"=>"Male", "ALT_NAME"=>nil, "PFNAME"=>"TG", "id"=>"14101823488",
"RESEARCH_INTERESTS"=>nil, "BIO"=>"", "CITIZEN"=>"Yes", "EMAIL"=>"test#email",
"SUFFIX"=>nil, "DPHONE1"=>nil}, "termId"=>"234", "IndexEntry"=>{"text"=>"Other",
"indexKey"=>"DEPARTMENT", "entryKey"=>"Other"}, "dmd:surveyId"=>"23424",
"username"=>"smithers", "userId"=>"23324"}, "xmlns"=>"http://www.adsfda.com/"}}
I want to iterate over each unique username in the first hash and compare values from the PCI section of the second hash to the values in the first hash. The keys are different names so I planned on pairing them up.
I've tried several ways of doing it, but I keep getting a string integer error, so I must not be iterating correctly. I'm doing an .each do block, but all the examples I see show a simple hash, not a key => key => value, key => value.
Any direction is much appreciated.
Here's how you should be able to do the iteration properly for hashes, if you're not already using this:
h = {:foo => 42, :bar => 43, 44 => :baz}
h.each {|key, val| puts "The value at #{key} is #{val}."}
This produces:
The value at foo is 42.
The value at bar is 43.
The value at 44 is baz.
As for the last part of your question, could you please make it a little clearer? e.g., what error are you getting exactly? This could help us understand the issue more.
EDIT: If what you'd like to do is compare certain values against others, try something like this:
h1 = {:foo => 42, :bar => 43, 44 => :baz, :not_used => nil}
h2 = {:life_universe => 42, :fourty_three => 45, :another_num => 44, :unused => :pancakes}
# Very easy to add more
comparisons = {:foo => :life_universe, :bar => :fourty_three, :baz => :another_num}
def all_match_up?(hash1, hash2, comps)
comparisons.each {|key, val|
if key != val
# They don't match!
puts "#{key} doesnt match #{val}!"
return false
end
}
# They all match!
true
end
all_match_up?(h1, h2, comparisons)
Try this to see what happens ;)

Ruby equivalent of Perl Data::Dumper

I am learning Ruby & Perl has this very convenient module called Data::Dumper, which allows you to recursively analyze a data structure (like hash) & allow you to print it. This is very useful while debugging. Is there some thing similar for Ruby?
Look into pp
example:
require 'pp'
x = { :a => [1,2,3, {:foo => bar}]}
pp x
there is also the inspect method which also works quite nicely
x = { :a => [1,2,3, {:foo => bar}]}
puts x.inspect
I normally use a YAML dump if I need to quickly check something.
In irb the syntax is simply y obj_to_inspect. In a normal Ruby app, you may need to add a require 'YAML' to the file, not sure.
Here is an example in irb:
>> my_hash = {:array => [0,2,5,6], :sub_hash => {:a => 1, :b => 2}, :visible => true}
=> {:sub_hash=>{:b=>2, :a=>1}, :visible=>true, :array=>[0, 2, 5, 6]}
>> y my_hash # <----- THE IMPORTANT LINE
---
:sub_hash:
:b: 2
:a: 1
:visible: true
:array:
- 0
- 2
- 5
- 6
=> nil
>>
The final => nil just means the method didn't return anything. It has nothing to do with your data structure.
you can use Marshal, amarshal, YAML

Best way to convert strings to symbols in hash

What's the (fastest/cleanest/straightforward) way to convert all keys in a hash from strings to symbols in Ruby?
This would be handy when parsing YAML.
my_hash = YAML.load_file('yml')
I'd like to be able to use:
my_hash[:key]
Rather than:
my_hash['key']
In Ruby >= 2.5 (docs) you can use:
my_hash.transform_keys(&:to_sym)
Using older Ruby version? Here is a one-liner that will copy the hash into a new one with the keys symbolized:
my_hash = my_hash.inject({}){|memo,(k,v)| memo[k.to_sym] = v; memo}
With Rails you can use:
my_hash.symbolize_keys
my_hash.deep_symbolize_keys
Here's a better method, if you're using Rails:
params.symbolize_keys
The end.
If you're not, just rip off their code (it's also in the link):
myhash.keys.each do |key|
myhash[(key.to_sym rescue key) || key] = myhash.delete(key)
end
For the specific case of YAML in Ruby, if the keys begin with ':', they will be automatically interned as symbols.
require 'yaml'
require 'pp'
yaml_str = "
connections:
- host: host1.example.com
port: 10000
- host: host2.example.com
port: 20000
"
yaml_sym = "
:connections:
- :host: host1.example.com
:port: 10000
- :host: host2.example.com
:port: 20000
"
pp yaml_str = YAML.load(yaml_str)
puts yaml_str.keys.first.class
pp yaml_sym = YAML.load(yaml_sym)
puts yaml_sym.keys.first.class
Output:
# /opt/ruby-1.8.6-p287/bin/ruby ~/test.rb
{"connections"=>
[{"port"=>10000, "host"=>"host1.example.com"},
{"port"=>20000, "host"=>"host2.example.com"}]}
String
{:connections=>
[{:port=>10000, :host=>"host1.example.com"},
{:port=>20000, :host=>"host2.example.com"}]}
Symbol
if you're using Rails, it is much simpler - you can use a HashWithIndifferentAccess and access the keys both as String and as Symbols:
my_hash.with_indifferent_access
see also:
http://api.rubyonrails.org/classes/ActiveSupport/HashWithIndifferentAccess.html
Or you can use the awesome "Facets of Ruby" Gem, which contains a lot of extensions to Ruby Core and Standard Library classes.
require 'facets'
> {'some' => 'thing', 'foo' => 'bar'}.symbolize_keys
=> {:some=>"thing", :foo=>"bar}
see also:
http://rubyworks.github.io/rubyfaux/?doc=http://rubyworks.github.io/facets/docs/facets-2.9.3/core.json#api-class-Hash
Even more terse:
Hash[my_hash.map{|(k,v)| [k.to_sym,v]}]
Since Ruby 2.5.0 you can use Hash#transform_keys or Hash#transform_keys!.
{'a' => 1, 'b' => 2}.transform_keys(&:to_sym) #=> {:a => 1, :b => 2}
http://api.rubyonrails.org/classes/Hash.html#method-i-symbolize_keys
hash = { 'name' => 'Rob', 'age' => '28' }
hash.symbolize_keys
# => { name: "Rob", age: "28" }
If you are using json, and want to use it as a hash, in core Ruby you can do it:
json_obj = JSON.parse(json_str, symbolize_names: true)
symbolize_names: If set to true, returns symbols for the names (keys) in a JSON object. Otherwise strings are returned. Strings are the default.
Doc: Json#parse symbolize_names
Here's a way to deep symbolize an object
def symbolize(obj)
return obj.inject({}){|memo,(k,v)| memo[k.to_sym] = symbolize(v); memo} if obj.is_a? Hash
return obj.inject([]){|memo,v | memo << symbolize(v); memo} if obj.is_a? Array
return obj
end
I really like the Mash gem.
you can do mash['key'], or mash[:key], or mash.key
A modification to #igorsales answer
class Object
def deep_symbolize_keys
return self.inject({}){|memo,(k,v)| memo[k.to_sym] = v.deep_symbolize_keys; memo} if self.is_a? Hash
return self.inject([]){|memo,v | memo << v.deep_symbolize_keys; memo} if self.is_a? Array
return self
end
end
params.symbolize_keys will also work. This method turns hash keys into symbols and returns a new hash.
In Rails you can use:
{'g'=> 'a', 2 => {'v' => 'b', 'x' => { 'z' => 'c'}}}.deep_symbolize_keys!
Converts to:
{:g=>"a", 2=>{:v=>"b", :x=>{:z=>"c"}}}
So many answers here, but the one method rails function is hash.symbolize_keys
This is my one liner for nested hashes
def symbolize_keys(hash)
hash.each_with_object({}) { |(k, v), h| h[k.to_sym] = v.is_a?(Hash) ? symbolize_keys(v) : v }
end
In case the reason you need to do this is because your data originally came from JSON, you could skip any of this parsing by just passing in the :symbolize_names option upon ingesting JSON.
No Rails required and works with Ruby >1.9
JSON.parse(my_json, :symbolize_names => true)
You could be lazy, and wrap it in a lambda:
my_hash = YAML.load_file('yml')
my_lamb = lambda { |key| my_hash[key.to_s] }
my_lamb[:a] == my_hash['a'] #=> true
But this would only work for reading from the hash - not writing.
To do that, you could use Hash#merge
my_hash = Hash.new { |h,k| h[k] = h[k.to_s] }.merge(YAML.load_file('yml'))
The init block will convert the keys one time on demand, though if you update the value for the string version of the key after accessing the symbol version, the symbol version won't be updated.
irb> x = { 'a' => 1, 'b' => 2 }
#=> {"a"=>1, "b"=>2}
irb> y = Hash.new { |h,k| h[k] = h[k.to_s] }.merge(x)
#=> {"a"=>1, "b"=>2}
irb> y[:a] # the key :a doesn't exist for y, so the init block is called
#=> 1
irb> y
#=> {"a"=>1, :a=>1, "b"=>2}
irb> y[:a] # the key :a now exists for y, so the init block is isn't called
#=> 1
irb> y['a'] = 3
#=> 3
irb> y
#=> {"a"=>3, :a=>1, "b"=>2}
You could also have the init block not update the hash, which would protect you from that kind of error, but you'd still be vulnerable to the opposite - updating the symbol version wouldn't update the string version:
irb> q = { 'c' => 4, 'd' => 5 }
#=> {"c"=>4, "d"=>5}
irb> r = Hash.new { |h,k| h[k.to_s] }.merge(q)
#=> {"c"=>4, "d"=>5}
irb> r[:c] # init block is called
#=> 4
irb> r
#=> {"c"=>4, "d"=>5}
irb> r[:c] # init block is called again, since this key still isn't in r
#=> 4
irb> r[:c] = 7
#=> 7
irb> r
#=> {:c=>7, "c"=>4, "d"=>5}
So the thing to be careful of with these is switching between the two key forms. Stick with one.
Would something like the following work?
new_hash = Hash.new
my_hash.each { |k, v| new_hash[k.to_sym] = v }
It'll copy the hash, but you won't care about that most of the time. There's probably a way to do it without copying all the data.
a shorter one-liner fwiw:
my_hash.inject({}){|h,(k,v)| h.merge({ k.to_sym => v}) }
How about this:
my_hash = HashWithIndifferentAccess.new(YAML.load_file('yml'))
# my_hash['key'] => "val"
# my_hash[:key] => "val"
This is for people who uses mruby and do not have any symbolize_keys method defined:
class Hash
def symbolize_keys!
self.keys.each do |k|
if self[k].is_a? Hash
self[k].symbolize_keys!
end
if k.is_a? String
raise RuntimeError, "Symbolizing key '#{k}' means overwrite some data (key :#{k} exists)" if self[k.to_sym]
self[k.to_sym] = self[k]
self.delete(k)
end
end
return self
end
end
The method:
symbolizes only keys that are String
if symbolize a string means to lose some informations (overwrite part of hash) raise a RuntimeError
symbolize also recursively contained hashes
return the symbolized hash
works in place!
The array we want to change.
strings = ["HTML", "CSS", "JavaScript", "Python", "Ruby"]
Make a new variable as an empty array so we can ".push" the symbols in.
symbols = [ ]
Here's where we define a method with a block.
strings.each {|x| symbols.push(x.intern)}
End of code.
So this is probably the most straightforward way to convert strings to symbols in your array(s) in Ruby. Make an array of strings then make a new variable and set the variable to an empty array. Then select each element in the first array you created with the ".each" method. Then use a block code to ".push" all of the elements in your new array and use ".intern or .to_sym" to convert all the elements to symbols.
Symbols are faster because they save more memory within your code and you can only use them once. Symbols are most commonly used for keys in hash which is great. I'm the not the best ruby programmer but this form of code helped me a lot.If anyone knows a better way please share and you can use this method for hash too!
If you would like vanilla ruby solution and as me do not have access to ActiveSupport here is deep symbolize solution (very similar to previous ones)
def deep_convert(element)
return element.collect { |e| deep_convert(e) } if element.is_a?(Array)
return element.inject({}) { |sh,(k,v)| sh[k.to_sym] = deep_convert(v); sh } if element.is_a?(Hash)
element
end
Starting on Psych 3.0 you can add the symbolize_names: option
Psych.load("---\n foo: bar")
# => {"foo"=>"bar"}
Psych.load("---\n foo: bar", symbolize_names: true)
# => {:foo=>"bar"}
Note: if you have a lower Psych version than 3.0 symbolize_names: will be silently ignored.
My Ubuntu 18.04 includes it out of the box with ruby 2.5.1p57
ruby-1.9.2-p180 :001 > h = {'aaa' => 1, 'bbb' => 2}
=> {"aaa"=>1, "bbb"=>2}
ruby-1.9.2-p180 :002 > Hash[h.map{|a| [a.first.to_sym, a.last]}]
=> {:aaa=>1, :bbb=>2}
This is not exactly a one-liner, but it turns all string keys into symbols, also the nested ones:
def recursive_symbolize_keys(my_hash)
case my_hash
when Hash
Hash[
my_hash.map do |key, value|
[ key.respond_to?(:to_sym) ? key.to_sym : key, recursive_symbolize_keys(value) ]
end
]
when Enumerable
my_hash.map { |value| recursive_symbolize_keys(value) }
else
my_hash
end
end
I like this one-liner, when I'm not using Rails, because then I don't have to make a second hash and hold two sets of data while I'm processing it:
my_hash = { "a" => 1, "b" => "string", "c" => true }
my_hash.keys.each { |key| my_hash[key.to_sym] = my_hash.delete(key) }
my_hash
=> {:a=>1, :b=>"string", :c=>true}
Hash#delete returns the value of the deleted key
Facets' Hash#deep_rekey is also a good option, especially:
if you find use for other sugar from facets in your project,
if you prefer code readability over cryptical one-liners.
Sample:
require 'facets/hash/deep_rekey'
my_hash = YAML.load_file('yml').deep_rekey
In ruby I find this to be the most simple and easy to understand way to turn string keys in hashes to symbols :
my_hash.keys.each { |key| my_hash[key.to_sym] = my_hash.delete(key)}
For each key in the hash we call delete on it which removes it from the hash (also delete returns the value associated with the key that was deleted) and we immediately set this equal to the symbolized key.
Similar to previous solutions but written a bit differently.
This allows for a hash that is nested and/or has arrays.
Get conversion of keys to a string as a bonus.
Code does not mutate the hash been passed in.
module HashUtils
def symbolize_keys(hash)
transformer_function = ->(key) { key.to_sym }
transform_keys(hash, transformer_function)
end
def stringify_keys(hash)
transformer_function = ->(key) { key.to_s }
transform_keys(hash, transformer_function)
end
def transform_keys(obj, transformer_function)
case obj
when Array
obj.map{|value| transform_keys(value, transformer_function)}
when Hash
obj.each_with_object({}) do |(key, value), hash|
hash[transformer_function.call(key)] = transform_keys(value, transformer_function)
end
else
obj
end
end
end

Resources