Convert Hash to csv file - ruby

I have the following hash named fcs, and want to save it to a CSV.
{
"bla"=>{
"lower"=>[17.12241, 18.79847, 17.71413, 18.49174, 18.30381, 18.78557, 18.93176, 19.33453, 19.62619, 20.02301],
"point"=>[21.84838, 23.86319, 23.93176, 25.19658, 25.72613, 26.70761, 27.41132, 28.28576, 29.05525, 29.88925],
"upper"=>[26.57434, 28.92791, 30.14938, 31.90142, 33.14845, 34.62966, 35.89088, 37.23698, 38.48432, 39.7555]
},
"blo"=>{
"lower"=>[17.12241, 18.79847, 17.71413, 18.49174, 18.30381, 18.78557, 18.93176, 19.33453, 19.62619, 20.02301],
"point"=>[21.84838, 23.86319, 23.93176, 25.19658, 25.72613, 26.70761, 27.41132, 28.28576, 29.05525, 29.88925],
"upper"=>[26.57434, 28.92791, 30.14938, 31.90142, 33.14845, 34.62966, 35.89088, 37.23698, 38.48432, 39.7555]
}
}
This is the desired content of the output file:
lower.bla,bla,upper.bla,lower.blo,blo,upper.blo
17.12241,21.84838,26.57434,17.12241,21.84838,26.57434
18.79847,23.86319,28.92791,18.79847,23.86319,28.92791
.
.
.
I am using the following code to generate it:
def to_csv
fcs = self.fcs
CSV.generate(col_sep: self.delim) do |csv|
names = Array.new
cols = Array.new
fcs.keys.each do |ts|
names << "lower." + ts
names << ts
names << "upper." + ts
end
csv << names
fcs.values.each do |ts|
cols << ts.values
end
cols.flatten(1).transpose.each do |row|
csv << row
end
end
end
However it uses three different loops. I'm feeling this is not the best code to achieve the desired output. Is there a best way to rewrite it?

header, body =
fcs
.flat_map{|k1, h| h.map{|k2, v| ["#{k2}.#{k1}".sub(/point\./, ""), v]}}
.transpose
[header, *body.transpose].map{|e| e.join(",")}.join($/)
gives:
lower.bla,bla,upper.bla,lower.blo,blo,upper.blo
17.12241,21.84838,26.57434,17.12241,21.84838,26.57434
18.79847,23.86319,28.92791,18.79847,23.86319,28.92791
17.71413,23.93176,30.14938,17.71413,23.93176,30.14938
18.49174,25.19658,31.90142,18.49174,25.19658,31.90142
18.30381,25.72613,33.14845,18.30381,25.72613,33.14845
18.78557,26.70761,34.62966,18.78557,26.70761,34.62966
18.93176,27.41132,35.89088,18.93176,27.41132,35.89088
19.33453,28.28576,37.23698,19.33453,28.28576,37.23698
19.62619,29.05525,38.48432,19.62619,29.05525,38.48432
20.02301,29.88925,39.7555,20.02301,29.88925,39.7555

Related

Generate CSV from Ruby results

I currently have this script that generates usernames from a given CSV. Rather than printing these results to the console, how can I write a new CSV with these results?
This is the script I currently have, runs with no errors. I am assuming if I write a new CSV in the do |row| block it is going to create x amount of new files which I do not want.
require 'csv'
CSV.foreach('data.csv', :headers => true) do |row|
id = row['id']
fn = row['first_name']
ln = row['last_name']
p fn[0] + ln + id[3,8]
end
Just manage the CSV file to write around the reading:
CSV.open("path/to/file.csv", "wb") do |csv|
CSV.foreach('data.csv', :headers => true) do |row|
id = row['id']
fn = row['first_name']
ln = row['last_name']
csv << [fn[0], ln, id[3,8]]
# or, to output it as a single column:
# csv << ["#{fn[0]}#{ln}#{id[3,8]}"]
end
end
Writing CSV to a file.

Write an array to multi column CSV format using Ruby

I have an array of arrays in Ruby that i'm trying to output to a CSV file (or text). That I can then easily transfer over to another XML file for graphing.
I can't seem to get the output (in text format) like so. Instead I get one line of data which is just a large array.
0,2
0,3
0,4
0,5
I originally tried something along the lines of this
File.open('02.3.gyro_trends.text' , 'w') { |file| trend_array.each { |x,y| file.puts(x,y)}}
And it outputs
0.2
46558
0
46560
0
....etc etc.
Can anyone point me in the "write" direction for getting either:
(i) .text file that can put my data like so.
trend_array[0][0], trend_array[0][1]
trend_array[1][0], trend_array[1][1]
trend_array[2][0], trend_array[2][1]
trend_array[3][0], trend_array[3][1]
(ii) .csv file that would put this data in separate columns.
edit I recently added more than two values into my array, check out my answer combining Cameck's solution.
This is currently what I have at the moment.
trend_array=[]
j=1
# cycle through array and find change in gyro data.
while j < gyro_array.length-2
if gyro_array[j+1][1] < 0.025 && gyro_array[j+1][1] > -0.025
trend_array << [0, gyro_array[j][0]]
j+=1
elsif gyro_array[j+1][1] > -0.025 # if the next value is increasing by x1.2 the value of the previous amount. Log it as +1
trend_array << [0.2, gyro_array[j][0]]
j+=1
elsif gyro_array[j+1][1] < 0.025 # if the next value is decreasing by x1.2 the value of the previous amount. Log it as -1
trend_array << [-0.2, gyro_array[j][0]]
j+=1
end
end
#for graphing and analysis purposes (wanted to print it all as a csv in two columns)
File.open('02.3test.gyro_trends.text' , 'w') { |file| trend_array.each { |x,y| file.puts(x,y)}}
File.open('02.3test.gyro_trends_count.text' , 'w') { |file| trend_array.each {|x,y| file.puts(y)}}
I know it's something really easy, but for some reason I'm missing it. Something with concatenation, but I found that if I try and concatenate a \\n in my last line of code, it doesn't output it to the file. It outputs it in my console the way I want it, but not when I write it to a file.
Thanks for taking the time to read this all.
File.open('02.3test.gyro_trends.text' , 'w') { |file| trend_array.each { |a| file.puts(a.join(","))}}
Alternately using the CSV Class:
def write_to_csv(row)
if csv_exists?
CSV.open(#csv_name, 'a+') { |csv| csv << row }
else
# create and add headers if doesn't exist already
CSV.open(#csv_name, 'wb') do |csv|
csv << CSV_HEADER
csv << row
end
end
end
def csv_exists?
#exists ||= File.file?(#csv_name)
end
Call write_to_csv with an array [col_1, col_2, col_3]
Thank you both #cameck & #tkupari, both answers were what I was looking for. Went with Cameck's answer in the end, because it "cut out" cutting and pasting text => xml. Here's what I did to get an array of arrays into their proper places.
require 'csv'
CSV_HEADER = [
"Apples",
"Oranges",
"Pears"
]
#csv_name = "Test_file.csv"
def write_to_csv(row)
if csv_exists?
CSV.open(#csv_name, 'a+') { |csv| csv << row }
else
# create and add headers if doesn't exist already
CSV.open(#csv_name, 'wb') do |csv|
csv << CSV_HEADER
csv << row
end
end
end
def csv_exists?
#exists ||= File.file?(#csv_name)
end
array = [ [1,2,3] , ['a','b','c'] , ['dog', 'cat' , 'poop'] ]
array.each { |row| write_to_csv(row) }

How to map and edit a CSV file with Ruby

Is there a way to edit a CSV file using the map method in Ruby? I know I can open a file using:
CSV.open("file.csv", "a+")
and add content to it, but I have to edit some specific lines.
The foreach method is only useful to read a file (correct me if I'm wrong).
I checked the Ruby CSV documentation but I can't find any useful info.
My CSV file has less than 1500 lines so I don't mind reading all the lines.
Another answer using each.with_index():
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5].sort # these are rows you would like to modify
rows_array.each.with_index(desired_indices[0]) do |row, index|
if desired_indices.include?(index)
# modify over here
rows_array[index][target_column] = 'modification'
end
end
# now update the file
CSV.open('sample3.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
You can also use each_with_index {} insead of each.with_index {}
Is there a way to edit a CSV file using the map method in Ruby?
Yes:
rows = CSV.open('sample.csv')
rows_array = rows.to_a
or
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5] # these are rows you would like to modify
edited_rows = rows_array.each_with_index.map do |row, index|
if desired_indices.include?(index)
# simply return the row
# or modify over here
row[3] = 'shiva'
# store index in each edited rows to keep track of the rows
[index, row]
end
end.compact
# update the main row_array with updated data
edited_rows.each{|row| rows_array[row[0]] = row[1]}
# now update the file
CSV.open('sample2.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
This is little messier. Is not it? I suggest you to use each_with_index with out map to do this. See my another answer
Here is a little script I wrote as an example on how read CSV data, do something to data, and then write out the edited text to a new file:
read_write_csv.rb:
#!/usr/bin/env ruby
require 'csv'
src_dir = "/home/user/Desktop/csvfile/FL_insurance_sample.csv"
dst_dir = "/home/user/Desktop/csvfile/FL_insurance_sample_out.csv"
puts " Reading data from : #{src_dir}"
puts " Writing data to : #{dst_dir}"
#create a new file
csv_out = File.open(dst_dir, 'wb')
#read from existing file
CSV.foreach(src_dir , :headers => false) do |row|
#then you can do this
# newrow = row.each_with_index { |rowcontent , row_num| puts "# {rowcontent} #{row_num}" }
# OR array to hash .. just saying .. maybe hash of arrays..
#h = Hash[*row]
#csv_out << h
# OR use map
#newrow = row.map(&:capitalize)
#csv_out << h
#OR use each ... Add and end
#newrow.each do |k,v| puts "#{k} is #{v}"
#Lastly, write back the edited , regexed data ..etc to an out file.
#csv_out << newrow
end
# close the file
csv_out.close
The output file has the desired data:
USER#USER-SVE1411EGXB:~/Desktop/csvfile$ ls
FL_insurance_sample.csv FL_insurance_sample_out.csv read_write_csv.rb
The input file data looked like this:
policyID,statecode,county,eq_site_limit,hu_site_limit,fl_site_limit,fr_site_limit,tiv_2011,tiv_2012,eq_site_deductible,hu_site_deductible,fl_site_deductible,fr_site_deductible,point_latitude,point_longitude,line,construction,point_granularity
119736,FL,CLAY COUNTY,498960,498960,498960,498960,498960,792148.9,0,9979.2,0,0,30.102261,-81.711777,Residential,Masonry,1
448094,FL,CLAY COUNTY,1322376.3,1322376.3,1322376.3,1322376.3,1322376.3,1438163.57,0,0,0,0,30.063936,-81.707664,Residential,Masonry,3
206893,FL,CLAY COUNTY,190724.4,190724.4,190724.4,190724.4,190724.4,192476.78,0,0,0,0,30.089579,-81.700455,Residential,Wood,1
333743,FL,CLAY COUNTY,0,79520.76,0,0,79520.76,86854.48,0,0,0,0,30.063236,-81.707703,Residential,Wood,3
172534,FL,CLAY COUNTY,0,254281.5,0,254281.5,254281.5,246144.49,0,0,0,0,30.060614,-81.702675,Residential,Wood,1

Concatenate array and string for CSV

I'm attempting to concatenate a string to an array value. Since the URL/domain is the same, I simply store the users email prefix and append the url/domain. I need to export the full email address out to CSV:
CSV.generate(options) do |csv|
columns = %w(name, email_address)
url = "#example.com"
all.each do |location|
csv << location.attributes.values_at(*columns) + [url]
end
end
Currently the resulting output is:
Joe, user1, #example.com
Bob, user2, #example.com
What I need is:
Joe, user1#example.com
Bob, user2#example.com
How can I achieve the above?
location is a model object, right?
CSV.generate(options) do |csv|
domain = "example.com"
all.each do |location|
csv << [location.name, "#{location.email_address}##{domain}"]
end
end
Update:
IMO that's cleaner as well. But if you want to keep your version, then I suggest you create a full_email_address method in your Location model which returns something like username#domain.com.
Then, you can vary the columns data later on and easily modify your CSV output. Like so:
class Location << ActiveRecord::Base
def full_email_address
return "" if self.email_address.blank?
domain = "example.com" # or save this as a constant in the class
"#{self.email_address}##{domain}"
end
end
CSV.generate(options) do |csv|
columns = %w{name full_email_address} # add other methods or attributes here
all.each do |location|
csv << columns.map{ |moa| location.public_send(moa) }
end
end
Try this:
CSV.generate(options) do |csv|
columns = %w(name, email_address)
url = "#example.com"
all.each do |location|
row = location.attributes.values_at(*columns)
row[-1] = row[-1] + url
csv << row
end
end
Currently the array written to CSV is ['Joe', 'user1'] + ['#example.com'], so instead of adding url to the attributes array I am adding it to the last attribute.

Output many arrays to CSV-files in Ruby

I have a question about Ruby. What I want to do is first to sort my items ascending and then write them out to a CSV-file. Now, the problem is further complicated by the fact that I want to iterate over a lot of CSV-files. I found this thread and the answer looks fine, but I am not able to get more than the last line written to my output file.
How can I get the whole data sorted and written to different CSV-files?
My code:
require 'date'
require 'csv'
class Daily <
# Daily has a open
Struct.new(:open)
# a method to print out a csv record for the current Daily.
def print_csv_record
printf("%s,", open)
printf("\n")
end
end
#------#
# MAIN #
#------#
# This is where I iterate over my csv-files:
foobar = ['foo', 'bar']
foobar.each do |foobar|
# get the input filename from the command line
input_file = "#{foobar}.csv"
# define an array to hold the Daily records
arr = Array.new
# loop through each record in the csv file, adding
# each record to my array while overlooking the header.
f = File.open(input_file, "r")
f.each_with_index { |row, i|
next if i == 0
words = row.split(',')
p = Daily.new
# do a little work here to convert my numbers
p.open = words[1].to_f
arr.push(p)
}
# sort the data by ascending opens
arr.sort! { |a,b| a.open <=> b.open }
# print out all the sorted records (just print to stdout)
arr.each { |p|
CSV.open("#{foobar}_new.csv", "w") do |csv|
csv << p.print_csv_record
end
}
end
My input CSV-file:
Open
52.23
52.45
52.36
52.07
52.69
52.38
51.2
50.99
51.41
51.89
51.38
50.94
49.55
50.21
50.13
50.14
49.49
48.5
47.92
My output CSV-file:
47.92
You need to put the iteration inside the open CSV file:
CSV.open("#{foobar}_new.csv", "w") do |csv|
arr.each { |p|
csv << p.print_csv_record
}
end

Resources