Given the following CSV file, how would you remove all rows that contain the word 'true' in the column 'foo'?
Date,foo,bar
2014/10/31,true,derp
2014/10/31,false,derp
I have a working solution, however it requires making a secondary CSV object csv_no_foo
#csv = CSV.read(#csvfile, headers: true) #http://bit.ly/1mSlqfA
#headers = CSV.open(#csvfile,'r', :headers => true).read.headers
# Make a new CSV
#csv_no_foo = CSV.new(#headers)
#csv.each do |row|
# puts row[5]
if row[#headersHash['foo']] == 'false'
#csv_no_foo.add_row(row)
else
puts "not pushing row #{row}"
end
end
Ideally, I would just remove the offending row from the CSV like so:
...
if row[#headersHash['foo']] == 'false'
#csv.delete(true) #Doesn't work
...
Looking at the ruby documentation, it looks like the row class has a delete_if function. I'm confused on the syntax that that function requires. Is there a way to remove the row without making a new csv object?
http://ruby-doc.org/stdlib-1.9.2/libdoc/csv/rdoc/CSV/Row.html#method-i-each
You should be able to use CSV::Table#delete_if, but you need to use CSV::table instead of CSV::read, because the former will give you a CSV::Table object, whereas the latter results in an Array of Arrays. Be aware that this setting will also convert the headers to symbols.
table = CSV.table(#csvfile)
table.delete_if do |row|
row[:foo] == 'true'
end
File.open(#csvfile, 'w') do |f|
f.write(table.to_csv)
end
You might want to filter rows in a ruby manner:
require 'csv'
csv = CSV.parse(File.read(#csvfile), {
:col_sep => ",",
:headers => true
}
).collect { |item| item[:foo] != 'true' }
Hope it help.
Related
I'm trying to access the csv data, which I recive if I make a http-request.
I don't save it to a csv file, so I save it to the variable.
Let's say this is the response I get, how can I print food?
uuid,event_id,category
12,1,food
13,2,cars
And this is the part of the ruby code which is important.
That's something I found, but it was originally used with a file, so it doesn't work.
csvdata = request(action,parameter)
#data_hash = {}
CSV.foreach(csvdata) do |row|
uuid, event_id, category = row
#data_hash[uuid] = event_id
end
Do I really need files for that or is there a easy way I can access the values?
Update
CSV.parse(csvdata,data = Hash.new) do |row|
puts data
end
The hash should look like this so I can use the column names
{"uuid" => "12,13", "event_id" => "323,3243", "category" => "food,cars"}
csv_data = Hash.new{|k, v| k[v] = []}
CSV.parse(csv_string, headers: true) do |row|
row.each{|k, v| csv_data[k] << v}
end
csv_data = Hash[csv_data.map{|k, v| [k, v.join(",")]}]
Update after specification Requested output.
Try this:
csvdata = request(action,parameter)
#data_hash = {}
CSV.parse(csvdata, headers: true) do |row|
#data_hash[row['uuid']] = row['event_id']
end
#data_hash
# => {"12"=>"1", "13"=>"2"}
When you parse a CSV, the seconds parameter (data = Hash.new in your code) is actually an options parameter. You can see the available options here:
:headers
If set to :first_row or true, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of ::parse_line with the same :col_sep, :row_sep, and :quote_char as this instance to produce an Array of headers. This setting causes #shift to return rows as CSV::Row objects instead of Arrays and #read to return CSV::Table objects instead of an Array of Arrays.
When passing headers: true - values are parsed into a Row object, where they can be accessed by name.
i tried almost everything but I am feeling cornered.
I have a CSV and reading a line from it:
CSV.foreach(file, quote_char: '"', col_sep: ',', row_sep: :auto, headers: true) { |line|
newLine = []
newLine = line.values #undefined method .values
...
}
line is aparently hash, because line['column_name'] is working fine and also line.to_a returns ["col","value","col2","value2",...]
please help, thank you!
You can use #fields on the class CSV::Row
http://ruby-doc.org/stdlib-1.9.3/libdoc/csv/rdoc/CSV/Row.html
It is not a regular hash, it is an instance of CSV::Row, see here for the API
As you can see in the result of the following code the method values isn't there. Your solution of using line['column_name'] is fine.
You can get all the fields with the method fields without parameter.
CSV.parse(DATA, :col_sep => ",", :headers => true).each do |row|
puts row.class
puts row.methods - Object.methods
end
__END__
kId,kName,kURL
1,Google UK,http://google.co.uk
2,Yahoo UK,http://yahoo.co.uk
It is a CSV row which is part array and part hash and doesn't have the .values method available. Use .to_hash first and then you will be able to use .values. (Note that this will remove the field ordering and any duplicate fields)
newLine = line.to_hash.values
I have a string with extra whitespace:
First,Last,Email ,Mobile Phone ,Company,Title ,Street,City,State,Zip,Country, Birthday,Gender ,Contact Type
I want to parse this line and remove the whitespaces.
My code looks like:
namespace :db do
task :populate_contacts_csv => :environment do
require 'csv'
csv_text = File.read('file_upload_example.csv')
csv = CSV.parse(csv_text, :headers => true)
csv.each do |row|
puts "First Name: #{row['First']} \nLast Name: #{row['Last']} \nEmail: #{row['Email']}"
end
end
end
#prices = CSV.parse(IO.read('prices.csv'), :headers=>true,
:header_converters=> lambda {|f| f.strip},
:converters=> lambda {|f| f ? f.strip : nil})
The nil test is added to the row but not header converters assuming that the headers are never nil, while the data might be, and nil doesn't have a strip method. I'm really surprised that, AFAIK, :strip is not a pre-defined converter!
You can strip your hash first:
csv.each do |unstriped_row|
row = {}
unstriped_row.each{|k, v| row[k.strip] = v.strip}
puts "First Name: #{row['First']} \nLast Name: #{row['Last']} \nEmail: #{row['Email']}"
end
Edited to strip hash keys too
CSV supports "converters" for the headers and fields, which let you get inside the data before it's passed to your each loop.
Writing a sample CSV file:
csv = "First,Last,Email ,Mobile Phone ,Company,Title ,Street,City,State,Zip,Country, Birthday,Gender ,Contact Type
first,last,email ,mobile phone ,company,title ,street,city,state,zip,country, birthday,gender ,contact type
"
File.write('file_upload_example.csv', csv)
Here's how I'd do it:
require 'csv'
csv = CSV.open('file_upload_example.csv', :headers => true)
[:convert, :header_convert].each { |c| csv.send(c) { |f| f.strip } }
csv.each do |row|
puts "First Name: #{row['First']} \nLast Name: #{row['Last']} \nEmail: #{row['Email']}"
end
Which outputs:
First Name: 'first'
Last Name: 'last'
Email: 'email'
The converters simply strip leading and trailing whitespace from each header and each field as they're read from the file.
Also, as a programming design choice, don't read your file into memory using:
csv_text = File.read('file_upload_example.csv')
Then parse it:
csv = CSV.parse(csv_text, :headers => true)
Then loop over it:
csv.each do |row|
Ruby's IO system supports "enumerating" over a file, line by line. Once my code does CSV.open the file is readable and the each reads each line. The entire file doesn't need to be in memory at once, which isn't scalable (though on new machines it's becoming a lot more reasonable), and, if you test, you'll find that reading a file using each is extremely fast, probably equally fast as reading it, parsing it then iterating over the parsed file.
Here's what I'm trying to accomplish. I need to have a single CSV with headers and several rows. I'm iterating through the headers and storing then and then associating the row data to the header. I need to be able to iterate through each of the rows in the CSV to use for constructing an XML's data. The constructed XML is then dumped as a .xml file and the program starts on the next row in the CSV. Each row has a column that provides the name of the XML file.
Here's what I've got so far.
Read in the data from the CSV file. Collect the header and row data.
def get_rows
raw_data = CSV.read('test.csv', {:skip_blanks => false, :headers => true})
data = []
raw_data.each { |row| data << row}
return build_header(data, raw_data)
end
take the header and row data and marry them up.
def build_header(data, raw_data)
(0..(data.length - 1)).each do |ri|
h = {}
raw_data.headers.each_with_index do |v, i|
h[v] = data[ri].fields[i]
end
return build_ostruct(h)
end
end
take the hash h and make an ostruct of it.
def build_ostruct(h)
x = OpenStruct.new(h)
uniq = x.tc_name
y = uniq_name.to_s + ".xml"
#marshal dump for debugging
x.marshal_dump.each{ |k,v| puts "#{k} => #{v}" }
return xml_builder(x, y)
end
Below this I'm taking the new ostruct "x" and calling the column headers from the CSV to #populate the XML nodes
For example: x.column1, x.column2, x.column3
Now the part I'm getting hung up on is getting the ostruct to receive the new row of data per iteration run. The objective is to have the ostruct populate with each row from the CSV. Currently the hash is displaying the proper data set and my XML is populating as expected but only with the first row of data. How do I get this to iterate through all the rows and populate the ostruct with the data per iteration so I can create a bulk set of XML's?
Thanks in advance for any and all help!
Something like this should work:
require 'csv'
require 'nokogiri'
CSV.foreach('test.csv', :headers => true) do |row|
builder = Nokogiri::XML::Builder.new do |xml|
xml.root do |root|
row.each do |k, v|
root.send k, v
end
end
end
File.open("#{row['tc_name']}.xml", 'w'){|f| f << builder.to_xml}
end
you are calling return in build_header, which ends the call. you need to collect your results in some way without immediately returning the first one, so that build_header can run for the entire set of rows.
Looking at the documentation for the CSV library of Ruby, I'm pretty sure this is possible and easy.
I simply need to delete the first three columns of a CSV file using Ruby but I haven't had any success getting it run.
csv_table = CSV.read(file_path_in, :headers => true)
csv_table.delete("header_name")
csv_table.to_csv # => The new CSV in string format
Check the CSV::Table documentation: http://ruby-doc.org/stdlib-1.9.2/libdoc/csv/rdoc/CSV/Table.html
csv_table = CSV.read("../path/to/file.csv", :headers => true)
keep = ["x", "y", "z"]
new_csv_table = csv_table.by_col!.delete_if do |column_name,column_values|
!keep.include? column_name
end
new_csv_table.to_csv
What about:
require 'csv'
File.open("resfile.csv","w+") do |f|
CSV.foreach("file.csv") do |row|
f.puts(row[3..-1].join(","))
end
end
I have built on a few of the questions (really liked what #fguillen did with CSV::Table) here but just made it a bit simpler to drop it into an existing project, target a file and make a quick change.
Have added byebug cause ... yes. Then also retained the headers from the original file (assuming they exist for anyone wanting to use this snippet).
The file is overwritten each time in case you want to test/tinker.
require 'csv'
require 'byebug'
in_file = './db/data/inbox/change__to_file_name.csv'
out_file = in_file + ".out"
target_col = "change_to_column_name"
csv_table = CSV.read(in_file, headers: true)
csv_table.delete(target_col)
CSV.open(out_file, 'w+', force_quotes: true) do |csv|
csv << csv_table.headers
csv_table.each_with_index do |row|
csv << row
end
end