How to create multiple columns per row in CSV with Ruby - ruby

Ok, I have a hash which contains several properties. I wanted certain properties of this hash to be added to a CSV file.
Here's what I've written:
require 'csv'
require 'curb'
require 'json'
arr = []
CSV.foreach('test.csv') do | row |
details = []
details << result['results'][0]['formatted_address']
result['results'][0]['address_components'].each do | w |
details << w['short_name']
end
arr << details
end
CSV.open('test_result.csv', 'w') do | csv |
arr.each do | e |
csv << [e]
end
end
end
All works fine apart from the fact the I get each like so:
["something", "300", "something", "something", "something", "something", "something", "GB", "something"]
As an array, which I do not want. I want each element of the array in a new column. The problem is that I do not know how many items I'll have otherwise I could something like this:
CSV.open('test_result.csv', 'w') do | csv |
arr.each do | e |
csv << [e[0], e[1], ...]
end
end
end
Any ideas?


Change csv << [e] to csv << e.

Related

Nested Ruby Hash to CSV

I have a nested hash as follow example:
{
"0001" => {
"All nodes" => [N001, N002, N003],
"All links" => [N001.1, N002.1, N003.1],
"Pumps" => [N001.2]
},
"0002" => {
"All nodes" => [N004, N005, N006],
"All links" => [N004.1, N005.1, N006.1],
"Pumps" => [N005.2]
},
"0003" => {
"All nodes" => [N007, N008, N009],
"All links" => [N007.1, N008.1, N009.1],
"Pumps" => [N007.2]
}
}
Under Nodes are stored information like coordinates, under Links are stored inverts, diameters and storage and under Pumps are stored On/Off levels and discharge.
I would like to know if you have any idea for how to export to CSV the information that are stored in the hash but in the right column (which will be the hash keys (0001, 0002 and 0003)).
As example, this is what I managed to make till now:
require 'CSV'
net=WSApplication.current_network
CSVsaveloc=WSApplication.file_dialog(false, "csv", "Comma Separated Variable File", "testexportcsv",false,true)
f = File.new(CSVsaveloc, "w")
CSV.open(CSVsaveloc,"wb") do |csv|
csv << Hash.keys
Hash.each do |key,values|
csv << x.mean
csv << y.mean
csv << diameter.min
csv << discharge.min
end
end
Now I'm getting the export like this:
0001,0002,0003
246164.2646
518466.7589
300mm
0.01
246181.6492
518444.1727
250mm
0.005
246171.5763
518509.8948
500mm
0.1
BUT, I would like to have it like this:
0001,0002,0003
246164.2646,246181.6492,246171.5763
518466.7589,518444.1727,518509.8948
300mm,250mm,500mm
0.01,0.005,0.1

From what I'm noticing, it seems that CSV inserts to a new line every time you run csv << . It might help to create an array for each line you're expecting, then append to CSV from there.
Expanding from the example code you gave earlier, maybe you can try this instead?
require 'CSV'
net=WSApplication.current_network
CSVsaveloc=WSApplication.file_dialog(false, "csv", "Comma Separated Variable File", "testexportcsv",false,true)
f = File.new(CSVsaveloc, "w")
# Create arrays
x_mean = []
y_mean = []
diameter_min = []
discharge_min = []
# Insert each value to corresponding array
Hash.each do |key, values|
x_mean << x.mean
y_mean << y.mean
diameter_min << diameter.min
discharge_min << discharge.min
end
# Insert each array to each line of CSV
CSV.open(CSVsaveloc,"wb") do |csv|
csv << Hash.keys
csv << x_mean
csv << y_mean
csv << diameter_min
csv << discharge_min
end

Ruby iterating over hash of hash

I have the following array and am struggling to format it for my needs.
consolidated = [
{:name=>"Bob", :details=>{"work"=>"Carpenter", "age"=>"26", "Experience"=>"6"} },
{:name=>"Colin", :details=>{"work"=>"painting", "age"=>"20", "Experience"=>"4"} }
]
I am trying to format it as below:
Bob work Carpenter
age 26
Experience 6
Colin work painting
age 20
Experience 4
I tried the following:
require 'csv'
CSV.open("output.csv", "wb") do |csv|
csv << ["name", "nature", "details"]
consolidated.each do |val|
csv << [val[:name], val[:details]]
end
end
#=> [{:name=>"Bob", :details=>{"work"=>"Carpenter", "age"=>"26", "Experience"=>"6"}},
# {:name=>"Colin", :details=>{"work"=>"painting", "age"=>"20", "Experience"=>"4"}}]
but it prints the following
name nature details
Bob "work"=>"Carpenter", "age"=>"26", "Experience"=>"6"
Colin "work"=>"painting", "age"=>"20", "Experience"=>"4"
I'm not exactly sure how to iterate hash of hash from the 1st loop only to get the expected format.
Thanks.

Here's something to get you started:
require 'csv'
data = [
{:name => "Bob", :details=>{"work"=>"Carpenter", "age"=>"26", "Experience"=>"6"}},
{:name => "Colin", :details=>{"work"=>"painting", "age"=>"20", "Experience"=>"4"}}
]
str = CSV.generate do |csv|
data.each do |datum|
datum[:details].each do |detail_key, detail_value|
csv << [datum[:name], detail_key, detail_value]
end
end
end
puts str
# >> Bob,work,Carpenter
# >> Bob,age,26
# >> Bob,Experience,6
# >> Colin,work,painting
# >> Colin,age,20
# >> Colin,Experience,4
Simply iterate all details and emit a new row for each key-value pair there, adding a name of a person.
This will get you almost what you need. Missing only blank rows between sections and person's name is duplicated on each line. It'll be your homework to find out how to add those improvements.

I don't know about CSV generation (so, assuming it works as you have written), you can iterate on your object this way:
consolidated = [{:name => "Bob", :details=>{"work"=>"Carpenter", "age"=>"26", "Experience"=>"6"}}, {:name => "Colin", :details=> {"work"=>"painting", "age"=>"20", "Experience"=>"4"}}]
CSV.open("output.csv", "wb") do |csv|
csv << ["name", "nature", "details"]
consolidated.each do |val|
details = val[:details]
nature_1 = details.keys.first
detail_1 = details.delete(nature_1)
csv << [val[:name], nature_1, detail_1]
details.each do |k, v|
csv << [nil, k, v]
end
end
end
Note: This will corrupt your original data array consolidated. So, if you want to preserve it, dup it first. Or modify the logic to not delete the first key-value from val[:details].

You need to iterate the embedded hash by each_pair iterator.
Something like this:
data = {:name => "Bob", :details=>{"work"=>"Carpenter", "age"=>"26", "Experience"=>"6"}}
CSV.open("output.csv", "wb") do |csv|
csv << ["name", "nature", "details"]
data.each do |val|
csv << [ val[:name], val[:details]['work'] ]
data[:details].each_pair do |key, value]
# here we have to drop the first pair because i've used it earlier
next if key == 'work'
csv << [ "", key, value ]
end
end
end

Read from a CSV file, multiply two columns, and then write back to the CSV file in ruby?

I have created csv file with values.I am able to read rows but don't know how to access individual values of a column.
require "csv"
CSV.open("file.csv", "w")
do |csv|
csv << ["val1", "val2","mul"]
csv << ["53", "27"]
csv<<["32","20"]
end

You probably need to ignore the header row if you have one. But the general idea is this:
CSV.open('dest.csv', 'w') do |csv|
csv << ["val1", "val2","mul"]
CSV.foreach('source.csv') do |row|
c1 = row[0]
c2 = row[1]
csv << [c1, c2, c1*c2]
end
end
If you have headers, you could do this:
CSV.open('dest.csv', 'w') do |csv|
csv << ["val1", "val2", "mul"]
CSV.foreach('source.csv', headers: true) do |row|
c1 = row['val1']
c2 = row['val2']
csv << [c1, c2, c1*c2]
end
end
You can use the one below for a non-ruby solution too:
awk -F "," '{print $1,$2,$1*$2}' source.csv > dest.csv

Counting every line and adding to the end of each line in a csv

What I want to do is count how many # are in each row and put this value into a total field at the end.
I mean it sort of works, but its only adding the value of the last line it counts.
My Csv
Header,Header,Header
Info#,Info,Info
Info,Info##,Info
Info,Info,Info###
My Code
require "csv"
table = CSV.read("my_test.csv", {
headers: true,
col_sep: ","
})
File.readlines('my_test.csv').each do |line|
table.each do |row|
at_count = line.count('#')
row["Total"] = at_count
end
end
CSV.open("my_test.csv", "w") do |f|
f << table.headers
table.each { | row | f << row }
end
Current Result
Header,Header,Header,Total
Info#,Info,Info,3
Info,Info##,Info,3
Info,Info,Info###,3

You don't need File.readlines; CSV already read it.
require "csv"
table = CSV.read("test.csv", { headers: true}) #just shorter
table.each do |row | #no readlines
at_count = row.to_s.count('#') # note the to_s
row["Total"] = at_count
end
CSV.open("my_test.csv", "w") do |f |
f << table.headers
table.each { | row | f << row}
end

Output array to CSV in Ruby

It's easy enough to read a CSV file into an array with Ruby but I can't find any good documentation on how to write an array into a CSV file. Can anyone tell me how to do this?
I'm using Ruby 1.9.2 if that matters.

To a file:
require 'csv'
CSV.open("myfile.csv", "w") do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# ...
end
To a string:
require 'csv'
csv_string = CSV.generate do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# ...
end
Here's the current documentation on CSV: http://ruby-doc.org/stdlib/libdoc/csv/rdoc/index.html

If you have an array of arrays of data:
rows = [["a1", "a2", "a3"],["b1", "b2", "b3", "b4"], ["c1", "c2", "c3"]]
Then you can write this to a file with the following, which I think is much simpler:
require "csv"
File.write("ss.csv", rows.map(&:to_csv).join)

I've got this down to just one line.
rows = [['a1', 'a2', 'a3'],['b1', 'b2', 'b3', 'b4'], ['c1', 'c2', 'c3'], ... ]
csv_str = rows.inject([]) { |csv, row| csv << CSV.generate_line(row) }.join("")
#=> "a1,a2,a3\nb1,b2,b3\nc1,c2,c3\n"
Do all of the above and save to a csv, in one line.
File.open("ss.csv", "w") {|f| f.write(rows.inject([]) { |csv, row| csv << CSV.generate_line(row) }.join(""))}
NOTE:
To convert an active record database to csv would be something like this I think
CSV.open(fn, 'w') do |csv|
csv << Model.column_names
Model.where(query).each do |m|
csv << m.attributes.values
end
end
Hmm #tamouse, that gist is somewhat confusing to me without reading the csv source, but generically, assuming each hash in your array has the same number of k/v pairs & that the keys are always the same, in the same order (i.e. if your data is structured), this should do the deed:
rowid = 0
CSV.open(fn, 'w') do |csv|
hsh_ary.each do |hsh|
rowid += 1
if rowid == 1
csv << hsh.keys# adding header row (column labels)
else
csv << hsh.values
end# of if/else inside hsh
end# of hsh's (rows)
end# of csv open
If your data isn't structured this obviously won't work

If anyone is interested, here are some one-liners (and a note on loss of type information in CSV):
require 'csv'
rows = [[1,2,3],[4,5]] # [[1, 2, 3], [4, 5]]
# To CSV string
csv = rows.map(&:to_csv).join # "1,2,3\n4,5\n"
# ... and back, as String[][]
rows2 = csv.split("\n").map(&:parse_csv) # [["1", "2", "3"], ["4", "5"]]
# File I/O:
filename = '/tmp/vsc.csv'
# Save to file -- answer to your question
IO.write(filename, rows.map(&:to_csv).join)
# Read from file
# rows3 = IO.read(filename).split("\n").map(&:parse_csv)
rows3 = CSV.read(filename)
rows3 == rows2 # true
rows3 == rows # false
Note: CSV loses all type information, you can use JSON to preserve basic type information, or go to verbose (but more easily human-editable) YAML to preserve all type information -- for example, if you need date type, which would become strings in CSV & JSON.

Building on #boulder_ruby's answer, this is what I'm looking for, assuming us_eco contains the CSV table as from my gist.
CSV.open('outfile.txt','wb', col_sep: "\t") do |csvfile|
csvfile << us_eco.first.keys
us_eco.each do |row|
csvfile << row.values
end
end
Updated the gist at https://gist.github.com/tamouse/4647196

Struggling with this myself. This is my take:
https://gist.github.com/2639448:
require 'csv'
class CSV
def CSV.unparse array
CSV.generate do |csv|
array.each { |i| csv << i }
end
end
end
CSV.unparse [ %w(your array), %w(goes here) ]

Resources