If a csv file has multiple columns with same name, how to get all the values of these columns.
# file.csv
name,name
one,two
Because you cannot use [] method to access all the duplicate columns, you can just break it up into an Array
>> csv = CSV.table('file.csv')
# if you just need the values
>> csv.to_a.last
=> ["one", "two"]
# if you need to know the column name call to `to_a` on each row
>> csv.flat_map(&:to_a)
=> [[:name, "one"], [:name, "two"]]
# to aggregate duplicate column values (assuming you know the duplicate column name ahead of time)
>> csv.map{|row| row.inject({names: []}){|h,(column,value)| h[:names] << value if column == :name; h }}
=> [{:names=>["one", "two"]}]
Related
I have a 2D array... is their any way to create CSV::Table with first row considered as headers and assuming all rows has same number of headers.
You can create a CSV::Table object with headers from a 2D array using CSV.parse.
First convert your 2d array to a string where the values in each row are joined by a comma, and each row is joined by a newline, then pass that string to CSV.parse along with the headers: true option
require 'csv'
sample_array = [
["column1", "column2", "column3"],
["r1c1", "r1c2", "r1c3"],
["r2c1", "r2c2", "r2c3"],
["r3c1", "r3c2", "r3c3"],
]
csv_data = sample_array.map {_1.join(",")}.join("\n")
table = CSV.parse(csv_data, headers: true)
p table
p table.headers
p table[0]
p table[1]
p table[2]
=>
#<CSV::Table mode:col_or_row row_count:4>
["column1", "column2", "column3"]
#<CSV::Row "column1":"r1c1" "column2":"r1c2" "column3":"r1c3">
#<CSV::Row "column1":"r2c1" "column2":"r2c2" "column3":"r2c3">
#<CSV::Row "column1":"r3c1" "column2":"r3c2" "column3":"r3c3">
Below is the basic example to create CSV file in ruby:
hash = {a: [1, 2, 3], b: [4, 5, 6]}
require 'csv'
CSV.open("my_file.csv", "wb") do |csv|
csv << %w(header1 header2 header3)
hash.each_value do |array|
csv << array
end
end
#diwanshu-tyagi will this help to resolve your question? if not please add example of your input value, I'll update this answer.
Thanks
Right now I can extract 1 column (column 6) from the csv file. How could I edit the script below to extract more than 1 column? Let's say I also want to extract column 9 and 10 as well as 6. I would want the output to be such that column 6 ends up in column 1 of the output file, 9 in the 2nd column of the output file, and column 10 in the 3rd column of the output file.
ruby -rcsv -e 'CSV.foreach(ARGV.shift) {|row| puts row [5]}' input.csv &> output.csv
Since row is an array, your question boils down to how to pick certain elements from an array; this is not related to CSV.
You can use values_at:
row.values_at(5,6,9,10)
returns the fields 5,6,9 and 10.
If you want to present these picked fields in a different order, it is however easier to map each index explicitly:
output_row = Array.new(row.size) # Or row.dup, depending on your needs
output_row[1] = row[6]
# Or, if you have used row.dup and want to swap the rows:
output_row[1],output_row[6] = row[6],row[1]
# and so on
out_csv.puts(output_row)
This assumes that you have defined before
out_csv=CSV.new(STDOUT)
since you want to have your new CSV be created on standard output.
Let's first create a (header-less) CSV file:
enum = 1.step
FNameIn = 't_in.csv'
CSV.open(FNameIn, "wb") { |csv| 3.times { csv << 5.times.map { enum.next } } }
#=> 3
I've assumed the file contains string representations of integers.
The file contains the three lines:
File.read(FNameIn).each_line { |line| p line }
"1,2,3,4,5\n"
"6,7,8,9,10\n"
"11,12,13,14,15\n"
Now let's extract the columns at indices 1 and 3. These columns are to be written to the output file in that order.
cols = [1, 3]
Now write to the CSV output file.
arr = CSV.read(FNameIn, converters: :integer).
map { |row| row.values_at(*cols) }
#=> [[2, 4], [7, 9], [12, 14]]
FNameOut = 't_out.csv'
CSV.open(FNameOut, 'wb') { |csv| arr.each { |row| csv << row } }
We have written three lines:
File.read(FNameOut).each_line { |line| p line }
"2,4\n"
"7,9\n"
"12,14\n"
which we can read back into an array:
CSV.read(FNameOut, converters: :integer)
#=> [[2, 4], [7, 9], [12, 14]]
A straightforward transformation of these operations is required to perform these operations from the command line.
Open two csv files.The field name and values label, sample are the same in both files, combine the average_old and average_new to to third csv.
I could find some of the results indicating faster csv but it's totally new to me. Any small snippet is appreciated.If the row exists in only one file I want to keep it in the new file.
For example:
File1.csv
label,sample,average_old
t1,10,12
t2,11,13
File2.csv
label,sample,average_new
t1,10,16
t2,11,15
File3.csv should be
label,sample,average_old,average_new
t1,10,12,16
t2,11,13,15
require 'csv'
CSV.foreach('/tmp/File2.csv').inject(
CSV.foreach('/tmp/File1.csv').inject({}) do |memo, row|
(memo[row.first] ||= []) << row
memo
end
) do |memo, row|
(memo[row.first] ||= []) << row
memo
end.map(&:flatten).map(&:uniq)
#⇒ [
# ["label", "sample", "average_old", "average_new"],
# ["t1", "10", "12", "16"],
# ["t2", "11", "13", "15"]
# ]
Writing this array to file should be easy.
Hash to csv
hash :
{
"employee" => [
{
"name" => "Claude",
"lastname"=> "David",
"profile" => [
"age" => "43",
"jobs" => [
{
"name" => "Ingeneer",
"year" => "5"
}
],
"graduate" => [
{
"place" => "Oxford",
"year" => "1990"
},
],
"kids" => [
{
"name" => "Viktor",
"age" => "18",
}
]
}
}]
this is an example of an hash I would work on. So, as you can see, there is many level of array in it.
My question is, how do I put it properly in a CSV file?
I tried this :
column_names = hash['employee'].first.keys
s=CSV.generate do |csv|
csv << column_names
hash['scrap'].each do |x|
csv << x.values
end
end
File.write('myCSV.csv', s)
but I only get name, lastname and profile as keys, when I would catch all of them (age, jobs, name , year, graduate, place...).
Beside, how can I associate one value per case?
Because I actually have all employee[x] which take a cell alone. Is there any parameters I have missed?
Ps: This could be the following of this post
A valid CSV output has a fixed number of columns, your hash has a variable number of values. The keys jobs, graduate and kids could all have multiple values.
If your only goal is to make a CSV output that can be read in Excel for example, you could enumerate your Hash, take the maximum number of key/value pairs per key, total it and then write your CSV output, filling the blank values with "".
There are plenty of examples here on Stack Overflow, search for "deep hash" to start with.
Your result would have a different number of columns with each Hash you provide it.
That's too much work if you ask me.
If you just want to present a readable result, your best and easiest option is to convert the Hash to YAML which is created for readability:
require 'yaml'
hash = {.....}
puts hash.to_yaml
employee:
- name: Claude
lastname: David
profile:
- age: '43'
jobs:
- name: Ingeneer
year: '5'
graduate:
- place: Oxford
year: '1990'
kids:
- name: Viktor
age: '18'
If you want to convert the hash to a CSV file or record, you'll need to get a 'flat' representation of your keys and values. Something like the following:
h = {
a: 1,
b: {
c: 3,
d: 4,
e: {
f: 5
},
g: 6
}
}
def flat_keys(h)
h.keys.reject{|k| h[k].is_a?(Hash)} + h.values.select{|v| v.is_a?(Hash)}.flat_map{|v| flat_keys(v)}
end
flat_keys(h)
# [:a, :c, :d, :g, :f]
def flat_values(h)
h.values.flat_map{|v| v.is_a?(Hash) ? flat_values(v) : v}
end
flat_values(h)
# [1, 3, 4, 5, 6]
Then you can apply that to create a CSV output.
It depends on how those fields are represented in the database.
For example, your jobs has a hash with name key and your kids also has a hash with name key, so you can't just 'flatten' them, because keys have to be unique.
jobs is probably another model (database table), so you probably would have to (depending on the database) write it separately, including things like the id of the related object and so on.
Are you sure you're not in over your head? Judging from your last question and because you seem to treat csv's as simple key-values pair omitting all the database representation and relations.
Hello everyone I'm currently trying to get my Dashboard working properly, however I cannot figure out a way to get the values into something my List Widget can read.
begin
id = 1
names.each do |item|
label = names[id][0] #names = names.csv path
value = host_status[id]['status'] #host_status = host_status.csv path
items = { label: label, value: value }
id += 1
end
rescue
end
send_event('hosts', { items: items })
So what this script should do is :
write the host_status.csv with the values it gets from the status.cgi (working)
iterate through both the host_status.csv and names.csv getting values from both of them
output should be something like this (label comes from names.csv, value from host_status.csv) =>
{label: "localhost", value: "UP"}, {label: "USV", value: "UP"}
The list widget needs something like an Array in a Hash with the keys label and value as far as I can tell, however my script doesnt return anything is there something like a push method for hashes?
I'm assuming here that names is an array of arrays [['foo'], ['bar']]
and that host_status is an array of objects [{'status' => 'foo'}, {'status' => 'bar'}]
you should just be able to do
names = [['foo'], ['bar']]
host_status = [{'status' => 'foo'}, {'status' => 'bar'}]
labels = names.map(&:first) # ['foo', 'bar']
values = host_status.map {|s| s['status'] } # ['foo', 'bar']
# zip zips together two arrays [['foo', 'foo'], ['bar', 'bar']]
# inject iterates over the array and returns a new data structure
items = labels.zip(values).inject([]) do |memo, (k,v)|
memo.push({label: k, value:v})
memo
end
You should just be able to run that code sample in an irb session.
Enumerable#map:http://www.ruby-doc.org/core-1.9.3/Enumerable.html#method-i-map
Enumerable#zip: http://www.ruby-doc.org/core-1.9.3/Enumerable.html#method-i-zip
Enumerable#inject: http://www.ruby-doc.org/core-1.9.3/Enumerable.html#method-i-inject