Store Ruby YAML results into an array - ruby

So I have an empty array and a .yml file. I have managed to output the results of that file with this code
puts YAML.load_file('some_yml_file.yml').inspect
I was wondering, how can I pull out each of the data and store them into an empty array?
Is it
emptyarray = []
YAML.load_file('some_yml_file.yml').inspect do |entry|
emptyarray << entry
end
Any help would be appreciated! Thanks!

YAML.load_file returns a Ruby object corresponding to the type of data structure the YAML represents. If the YAML contains a sequence, YAML.load_file will return a Ruby array. You don't need to do anything further to put the data into an array, because it's already an array:
yaml = <<END
---
- I am
- a YAML
- sequence
END
data = YAML.load(yaml)
puts data.class
# => Array
puts data == ["I am", "a YAML", "sequence"]
# => true
(You'll notice that I used YAML.load to load the data from a string rather than a file, but the result is the same as using YAML.load_file on a file with the same contents.)
If the top-level structure in the YAML is not a sequence (e.g. if it's a mapping, analogous to a Ruby hash), then you will have to do additional work to turn it into an array, but we can't tell you what that code would look like without seeing your YAML.

Change YAML.load_file('some_yml_file.yml').inspect do |entry| with YAML.load_file('some_yml_file.yml').each do |entry| and it should work as you expect it (assuming it's not a string).
If you post a sample of your data structure inside the YAML file and what you wish to extract and put in an array then that would help.

Related

Reading text file, parsing it, and storing it into a hash

I want to open a text file name test.txt and turn into hash which has the condition value into 1111 only
Instance Id:xxxxx, value: 123
Instance Id:xxxxx, value: 1111
Instance Id:xxxxx, value: 1111
can any one please help me.
This my sample code:
File.open('test.txt').each_line do |line|
puts line if line.match(/1111/)
end
# define a array in the outside scope so you can access it
array = []
# run your loop that reads the file
File.open('test.txt').each_line do |line|
# split lines into two parts, instance_id - value pairs
instance_id, value = line.split(',')
# only add to array if the value is the one you're looking for
# also, split instance_id to only get the value of the ID
array << instance_id.split(':')[1] if value.match(/1111/)
end
puts array
# => ["xxxxx", "xxxxx"]
EDIT: updated the suggestion to better suit the updated request in the comments
Also worth noting is that it serves no purpose to have the values in a hash since you would have different IDs for the same value, you would want to put this in an array.

Permanently storing arrays Ruby

so I've written a code that iterates through several hundred CSV files, and then stores the last element of each into a new array.
module Example
#array = []
def example(file_names) #where file_names is an array of strings for the csv files
file_names.each { |x|
#array << (CSV.parse open("#{x}.csv").read)[-1] if File.exists?("{x}.csv") == true }
return #array
end
end
Executing this code can take some time, and I want to be able to refer to this newly-created array in other methods without having to run this code again. Is there a way to permanently store the #array variable?
It depends on just how permanent you want your results to be. If you just don't want to parse the CSV files for the lifetime of your program, then you can simply cache the result in a member variable (as you are with #array), and only execute your code if that array is empty for example:
module Example
def example(file_names)
# ||= will only calculate a result if #array is nil, otherwise
# it will return the saved value
#array ||= file_names.map { |x| CSV.parse open("#{x}.csv").read)[-1] if File.exists?("{x}.csv") }
end
end
If you want your work to be saved in-between executions of the program you can try saving your results to a (single) file and reading it back in, using perhaps on of the following:
JSON: http://ruby-doc.org/stdlib-2.2.2/libdoc/json/rdoc/JSON.html
YAML: http://ruby-doc.org/stdlib-2.2.2/libdoc/yaml/rdoc/YAML.html
Marshal: http://ruby-doc.org/core-2.2.2/Marshal.html
See mu-is-too-short's comment for some of the drawbacks of using Marshal

Parsing JSON from text file

I have a log file that appears to have a time stamp followed by a tab, then a string of JSON.
10/28/2014 00:04:51 {"servers":[{"id":833495,"account_id":39033,"name":"USTSMASCMSP113","host":"USTSMASCMSP113","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":28.7,"cpu_stolen":0,"disk_io":0.17,"memory":60.8,"memory_used":10444865536,"memory_total":17176723456,"fullest_disk":84.0,"fullest_disk_free":8171000000},"links":{"alert_policy":28370}},{"id":831246,"account_id":39033,"name":"USTSMASCMSP118","host":"USTSMASCMSP118","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":30.7,"cpu_stolen":0,"disk_io":2.57,"memory":54.3,"memory_used":9329180672,"memory_total":17176723456,"fullest_disk":83.1,"fullest_disk_free":8653000000},"links":{"alert_policy":28370}},{"id":833455,"account_id":39033,"name":"USTSMASCMSP119","host":"USTSMASCMSP119","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":36.3,"cpu_stolen":0,"disk_io":0.38,"memory":54.5,"memory_used":9362735104,"memory_total":17176723456,"fullest_disk":73.7,"fullest_disk_free":17912000000},"links":{"alert_policy":28370}},{"id":838342,"account_id":39033,"name":"USTSMASCMSP120","host":"USTSMASCMSP120","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":29.6,"cpu_stolen":0,"disk_io":0,"memory":55.6,"memory_used":9550430208,"memory_total":17176723456,"fullest_disk":84.9,"fullest_disk_free":10259000000},"links":{"alert_policy":28370}},{"id":833500,"account_id":39033,"name":"USTSMASCMSP121","host":"USTSMASCMSP121","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":38.9,"cpu_stolen":0,"disk_io":5.47,"memory":56.7,"memory_used":9743368192,"memory_total":17176723456,"fullest_disk":84.9,"fullest_disk_free":10288000000},"links":{"alert_policy":28370}},{"id":831221,"account_id":39033,"name":"USTSMASCMSP123","host":"USTSMASCMSP123","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":32.4,"cpu_stolen":0,"disk_io":0.42,"memory":53.8,"memory_used":9232711680,"memory_total":17176723456,"fullest_disk":87.2,"fullest_disk_free":8705000000},"links":{"alert_policy":28370}},{"id":833466,"account_id":39033,"name":"USTSMASCMSP124","host":"USTSMASCMSP124","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":25.1,"cpu_stolen":0,"disk_io":4.28,"memory":58.6,"memory_used":10065281024,"memory_total":17176723456,"fullest_disk":74.2,"fullest_disk_free":19387000000},"links":{"alert_policy":28370}},{"id":838346,"account_id":39033,"name":"USTSMASCMSP125","host":"USTSMASCMSP125","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":32.2,"cpu_stolen":0,"disk_io":0.13,"memory":56.5,"memory_used":9703522304,"memory_total":17176723456,"fullest_disk":89.1,"fullest_disk_free":7443000000},"links":{"alert_policy":28370}},{"id":833504,"account_id":39033,"name":"USTSMASCMSP126","host":"USTSMASCMSP126","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":29.6,"cpu_stolen":0,"disk_io":0,"memory":56.9,"memory_used":9776922624,"memory_total":17176723456,"fullest_disk":85.7,"fullest_disk_free":9761000000},"links":{"alert_policy":28370}},{"id":831212,"account_id":39033,"name":"USTSMASCMSP127","host":"USTSMASCMSP127","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":30.5,"cpu_stolen":0,"disk_io":0,"memory":50.9,"memory_used":8735686656,"memory_total":17176723456,"fullest_disk":85.0,"fullest_disk_free":10222000000},"links":{"alert_policy":28370}},{"id":833510,"account_id":39033,"name":"USTSMASCMSP128","host":"USTSMASCMSP128","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":26.8,"cpu_stolen":0,"disk_io":0,"memory":57.8,"memory_used":9936306176,"memory_total":17176723456,"fullest_disk":79.7,"fullest_disk_free":10393000000},"links":{"alert_policy":28370}},{"id":833473,"account_id":39033,"name":"USTSMASCMSP129","host":"USTSMASCMSP129","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":31.9,"cpu_stolen":0,"disk_io":0.17,"memory":56.4,"memory_used":9693036544,"memory_total":17176723456,"fullest_disk":86.7,"fullest_disk_free":9095000000},"links":{"alert_policy":28370}},{"id":3922351,"account_id":39033,"name":"USTSMASCMSP136","host":"USTSMASCMSP136","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":16.2,"cpu_stolen":0,"disk_io":0.28,"memory":13.3,"memory_used":4552916992,"memory_total":34356592640,"fullest_disk":73.3,"fullest_disk_free":56043000000},"links":{"alert_policy":28369}},{"id":831184,"account_id":39033,"name":"USTSMASCMSP144","host":"USTSMASCMSP144","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":30.8,"cpu_stolen":0,"disk_io":0.88,"memory":61.8,"memory_used":10617880576,"memory_total":17176723456,"fullest_disk":90.9,"fullest_disk_free":8237000000},"links":{"alert_policy":28370}},{"id":833478,"account_id":39033,"name":"USTSMASCMSP145","host":"USTSMASCMSP145","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":30.5,"cpu_stolen":0,"disk_io":0,"memory":50.3,"memory_used":8631877632,"memory_total":17176723456,"fullest_disk":81.5,"fullest_disk_free":9712000000},"links":{"alert_policy":28370}},{"id":838357,"account_id":39033,"name":"USTSMASCMSP146","host":"USTSMASCMSP146","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":29.7,"cpu_stolen":0,"disk_io":0,"memory":52.8,"memory_used":9069133824,"memory_total":17176723456,"fullest_disk":80.6,"fullest_disk_free":9934000000},"links":{"alert_policy":28370}},{"id":838360,"account_id":39033,"name":"USTSMASCMSP147","host":"USTSMASCMSP147","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":28.7,"cpu_stolen":0,"disk_io":0,"memory":73.6,"memory_used":12640583680,"memory_total":17176723456,"fullest_disk":80.8,"fullest_disk_free":9825000000},"links":{"alert_policy":28370}},{"id":3964940,"account_id":39033,"name":"USTSMASCMSP148","host":"USTSMASCMSP148","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":12.6,"cpu_stolen":0,"disk_io":0.08,"memory":10.6,"memory_used":3636461568,"memory_total":34356592640,"fullest_disk":73.2,"fullest_disk_free":56213000000},"links":{"alert_policy":28369}},{"id":832099,"account_id":39033,"name":"USTSMASCMSP196","host":"USTSMASCMSP196","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":24.8,"cpu_stolen":0,"disk_io":0,"memory":30.2,"memory_used":5187305472,"memory_total":17176723456,"fullest_disk":77.2,"fullest_disk_free":11962000000},"links":{"alert_policy":28370}},{"id":832093,"account_id":39033,"name":"USTSMASCMSP197","host":"USTSMASCMSP197","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":24.8,"cpu_stolen":0,"disk_io":0.34,"memory":30.1,"memory_used":5169479680,"memory_total":17176723456,"fullest_disk":74.5,"fullest_disk_free":13362000000},"links":{"alert_policy":28370}},{"id":832082,"account_id":39033,"name":"USTSMASCMSP198","host":"USTSMASCMSP198","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":23.5,"cpu_stolen":0,"disk_io":0,"memory":26.8,"memory_used":4604297216,"memory_total":17176723456,"fullest_disk":72.8,"fullest_disk_free":14268000000},"links":{"alert_policy":28370}},{"id":832077,"account_id":39033,"name":"USTSMASCMSP199","host":"USTSMASCMSP199","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":21.0,"cpu_stolen":0,"disk_io":0,"memory":31.3,"memory_used":5379194880,"memory_total":17176723456,"fullest_disk":71.5,"fullest_disk_free":14950000000},"links":{"alert_policy":28370}},{"id":832832,"account_id":39033,"name":"USTSMASCMSP200","host":"USTSMASCMSP200","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":50.3,"cpu_stolen":0,"disk_io":0,"memory":28.9,"memory_used":4966055936,"memory_total":17176723456,"fullest_disk":74.3,"fullest_disk_free":13477000000},"links":{"alert_policy":28370}},{"id":831194,"account_id":39033,"name":"USTSMASCMSP201","host":"USTSMASCMSP201","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":27.0,"cpu_stolen":0,"disk_io":0,"memory":28.6,"memory_used":4918870016,"memory_total":17176723456,"fullest_disk":74.4,"fullest_disk_free":13397000000},"links":{"alert_policy":28370}},{"id":832059,"account_id":39033,"name":"USTSMASCMSP202","host":"USTSMASCMSP202","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":25.1,"cpu_stolen":0,"disk_io":0,"memory":26.6,"memory_used":4564451328,"memory_total":17176723456,"fullest_disk":68.3,"fullest_disk_free":16645000000},"links":{"alert_policy":28370}},{"id":832043,"account_id":39033,"name":"USTSMASCMSP203","host":"USTSMASCMSP203","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":24.7,"cpu_stolen":0,"disk_io":1.17,"memory":25.2,"memory_used":4326424576,"memory_total":17176723456,"fullest_disk":68.2,"fullest_disk_free":16646000000},"links":{"alert_policy":28370}},{"id":832034,"account_id":39033,"name":"USTSMASCMSP204","host":"USTSMASCMSP204","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":26.5,"cpu_stolen":0,"disk_io":0,"memory":27.1,"memory_used":4653580288,"memory_total":17176723456,"fullest_disk":68.5,"fullest_disk_free":16510000000},"links":{"alert_policy":28370}},{"id":832031,"account_id":39033,"name":"USTSMASCMSP205","host":"USTSMASCMSP205","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":27.8,"cpu_stolen":0,"disk_io":0,"memory":24.9,"memory_used":4285530112,"memory_total":17176723456,"fullest_disk":68.8,"fullest_disk_free":16376000000},"links":{"alert_policy":28370}},{"id":832024,"account_id":39033,"name":"USTSMASCMSP206","host":"USTSMASCMSP206","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":30.2,"cpu_stolen":0,"disk_io":0,"memory":26.5,"memory_used":4548722688,"memory_total":17176723456,"fullest_disk":67.2,"fullest_disk_free":17181000000},"links":{"alert_policy":28370}},{"id":831202,"account_id":39033,"name":"USTSMASCMSP207","host":"USTSMASCMSP207","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":23.9,"cpu_stolen":0,"disk_io":0,"memory":25.6,"memory_used":4394582016,"memory_total":17176723456,"fullest_disk":67.3,"fullest_disk_free":17146000000},"links":{"alert_policy":28370}},{"id":3979556,"account_id":39033,"name":"USTSMASCMSP227","host":"USTSMASCMSP227","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":8.25,"cpu_stolen":0,"disk_io":5.96,"memory":18.1,"memory_used":3116367872,"memory_total":17174626304,"fullest_disk":65.1,"fullest_disk_free":18301000000},"links":{"alert_policy":28369}},{"id":3965125,"account_id":39033,"name":"USTSMASCMSP247","host":"USTSMASCMSP247","health_status":"green","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":18.2,"cpu_stolen":0,"disk_io":34.0,"memory":32.3,"memory_used":5543821312,"memory_total":17174626304,"fullest_disk":59.9,"fullest_disk_free":28765000000},"links":{"alert_policy":28369}},{"id":3965022,"account_id":39033,"name":"USTSMASCMSP248","host":"USTSMASCMSP248","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":43.9,"cpu_stolen":0,"disk_io":8.61,"memory":28.1,"memory_used":4821352448,"memory_total":17143169024,"fullest_disk":86.5,"fullest_disk_free":9645000000},"links":{"alert_policy":28369}},{"id":3851100,"account_id":39033,"name":"ustsmvscmsp888","host":"ustsmvscmsp888","health_status":"orange","reporting":true,"last_reported_at":"2014-10-28T04:04:17+00:00","summary":{"cpu":19.7,"cpu_stolen":0,"disk_io":0,"memory":42.9,"memory_used":1841299456,"memory_total":4293918720,"fullest_disk":72.9,"fullest_disk_free":14243000000},"links":{"alert_policy":28369}}],"links":{"server.alert_policy":"/v2/alert_policies/{alert_policy_id}"}}
I have code that reads the input file and I am attempting to get values from the string like so.
require 'json'
input = File.open("input.txt", 'r')
input.each do |line|
line = line.split("\t")
my_json = line[1].to_json
puts my_json["cpu"]
end
The end result looks like this:
C:\>ruby test.rb
cpu
I am not entirely sure if the string itself is not formatted correctly or if I'm not calling on the value correctly.
Run this code inside console and check the return but i'm pretty sure you can't call
my_json["cpu"]
You will need to call something like:
my_json["servers"].first["cpu"]
my_json = line[1].to_json
Will take an object and convert it to JSON, not parse an existing JSON string into a ruby object. You're going to wind up with an escaped string within a JSON string.
You need something like
my_json = JSON.parse(line[1])
you can verify this has worked correctly by outputting my_json to stdout:
$stdout << my_json.class.name
You should have an instance of Hash or Openstruct.
Consider this:
require 'json'
line = '10/28/2014 00:04:51 {"servers":[{"id":833495,"account_id":39033}]}'
JSON[line[/\{.+}/]]
# => {"servers"=>[{"id"=>833495, "account_id"=>39033}]}
That means you can reduce your parsing code to something like:
File.foreach("input.txt") do |line|
data = JSON[line[/\{.+}/]]
# do something with the data
end
Regular expressions are greedy, so /\{.+}/ will start at the first { and look until it finds the last }, and return the starting and ending braces, and everything in between.
The JSON [] method is smart enough to know that if its parameter is a string it should try to parse it and return a Ruby array or hash. If the parameter is an Array or Hash, it will serialize it.
Note: It isn't necessary, and definitely not desirable to include huge data samples like you did. Reduce it to the bare minimum necessary to demonstrate the problem.

How can I use a ruby program to save an array of hashes to a csv file?

I have an array of hashes that contain sales data in a ruby program and would like to write code that would save this data to a csv file that I can later access or update. Any suggestions on how I can accomplish this? Thanks for any and all help!
You can use Ruby Marshal builtin class for serialization.
# load array from array.bin or initialize new array
array = if File.exists?('array.bin')
File.open('array.bin') do|file|
Marshal.load(file)
end
else
[]
end
# see what's in array
puts array.inspect
# modify array
array << ["test"]
# save into array.bin file
File.open('array.bin','w') do|file|
Marshal.dump(array, file)
end
I think you can consider using class CSV from Ruby standard library.
http://ruby-doc.org/stdlib-1.9.2/libdoc/csv/rdoc/CSV.html

Comparing values in two hashes

I am having trouble in comparing values in two hashes, getting the error "Can't convert String into Integer".
First hash has values captured from a web page using the method "capture_page_data(browser)" and the second hash has data parsed from a report.
Code looks like below:
# Open the web application
# Navigate to a specific page and capture page data
loan_data = Hash.new
loan_data = capture_page_data(browser)
Second hash has values captured from a report generated from the web application.
Code looks like below:
#report_data[page] = Hash.new
# we have written some logic to parse the data from the report into hash variable
Now I am trying to compare the values in theses two hashes to ensure the data in report is matching with the data in application using below code which is giving me the error "Can't convert String into Integer".
loan_data.map{|ld| ld['MainContent_cphContent_LoanOverViewGeneralInfoCtrl_lblRelName']} &
#report_data.map{|rd| rd['Relationship']}
Please help me out in resolving this issue.
Regards,
Veera.
Hash#map iterates through the hash like it was an array of key/value pairs.
{a:1,b:2}.map{|x| puts x.inspect }
# prints
# [:a,1]
# [:b,2]
{a:1,b:2}.map{|k,v| puts "#{k} => #{v}" }
# prints
# a => 1
# b => 2
It applies the block you provide to each pair and collects the results into a new array.
result = {a:1,b:2}.map{|k,v| "#{k} => #{v}" }
puts result.inspect
# prints
# [ "a => 1", "b => 2" ]
I would guess what you are trying to do is compare a single key from each array... in which case...
if loan_data[:id][:span]['MainContent_cphContent_LoanOverViewGeneralInfoCtrl_lblR‌​elName'] == #report_data[1]['Relationship']
log_message("pass")
else
log_message("fail")
end
might be what you are trying to do.. but I am only guessing.
It all depends on the shape of your data.
If you inspect the ld variable inside your block, you will find that it is an array. You can get an element of it with ld[0] or ld[1], but ld[string] does not make sense and results in the exception you are seeing. The ld array will actually be an array with two elements: key and value.
Thanks for your suggestions.. but I found a different solution to compare a single key from two hashes/Arrays using the below code which worked fine.
string_equals?(loan_data[:id][:span]['MainContent_cphContent_LoanOverViewGeneralInfoCtrl_lblRelName'], #report_data[1]['Relationship'] )
Thanks,
Veera.
It's best to debug the content of loan_data and #report_data directly, but you can try .to_sym to convert the key into symbol.
loan_data.map{|ld| ld['MainContent_cphContent_LoanOverViewGeneralInfoCtrl_lblRelName'.to_sym]} &
#report_data.map{|rd| rd['Relationship'.to_sym]}

Resources