How to parse a CSV file, update a field, then save - ruby

I need to read a CSV file, update a field, then save the changes. I have everything working fine except saving my changes to the field I'm updating:
require 'csv'
#parsed_file = CSV::Reader.parse(File.open("#{RAILS_ROOT}/doc/some.csv"))
#parsed_file.each_with_index do |row, x|
address = row[5]
l = Location.address_find(address)
if l != nil
puts "#{l.name} at #{l.address}"
row[14] = l.store_code
puts row[14]
else
puts "No matching address Found!!!"
end
#What do I do here? Something like this? CSV::Writer.generate(#parsed_file)
end
What do I do here? How do I save the changes I made and update the file?

What I would do is write the updated records to a new file and then, if you want, after you have finished your program can delete the old file and rename the new file to have the original file name.
To do this I would open the output file at the top of your code, outside the each_with_index loop. e.g.
csv_out = CSV::Writer.generate(File.open('new.csv', 'wb'))
Then inside your each_with_index loop you can write the current row to the new file like this:
csv_out << row
Then at the end you can close the new file:
csv_out.close
As mentioned in the comments, CSV::Writer is no longer in the standard library. An equivalent version of the code using the newer CSV.foreach (for reading) and CSV.open (for writing) is:
CSV.open("path/to/updated_file.csv", "wb") do |csv_out|
CSV.foreach("#{RAILS_ROOT}/doc/some.csv") do |row|
address = row[5]
l = Location.address_find(address)
if l != nil
puts "#{l.name} at #{l.address}"
row[14] = l.store_code
puts row[14]
else
puts "No matching address Found!!!"
end
csv_out << row
end
end

Related

How to generate a column in a CSV file instead of a row

So what I'm trying to do is read a specific column from an existing CSV file, parse some information out of the pulled data, then make a new CSV file with the newly parsed information in a single column. The header and first entry of the generated array go into the CSV file correctly, but after those, every other entry goes into the adjacent cells of the same row instead of creating a column, so it's like an L shape instead of just a line. Any ideas?
#!ruby.exe
require 'csv'
puts "Please enter a file name:" #user input file name (must be in same
folder as this file)
file = gets.chomp
begin
File.open(file, 'r')
rescue
print "Failed to open #{file}\n"
exit
end #makes sure that the file exists, if it does not it posts an error
data_file = File.new(file)
data = [] #initializes array for addresses from .csv
counter=0 #set counter up to allow for different sized files to be used
without issue
CSV.foreach(data_file, headers: true) do |row|
data << row.to_hash
counter+=1
end #goes through .csv one line ar a time
data.reject(&:empty?)
puts "Which column do you want to parse?"
column = gets.chomp
i=0
streets = []
while (i<counter)
address = data[i][column]
street_name = address.gsub(/^((\d[a-zA-Z])|[^a-zA-Z])*/, '')
streets.push(street_name)
i+=1
end
streets.reject(&:empty?)
puts "What do you want the output to be called?"
new_file = gets.chomp
CSV.open(new_file, "w", :write_headers=> true, :headers => [column]) do |hdr|
hdr << streets
end
You should scan the street array and insert it as a row, which means you need to place the line of data into an array before to send to the csv. Ok, maybe the code is simpler than the explanation:
CSV.open(new_file, "w", :write_headers=> true, :headers => [column]) do |csv_line|
streets.each { |street| csv_line << [street] }
end

How to open a CSV file and increment a specific value by one

I'm trying to open a simple CSV file in Ruby, find a particular key, increment its value by one, then re-save it.
Example CSV file:
store1,0
store2,0
store3,0
...etc.
Ruby code:
require 'csv'
currentStore = store # store is passed as a parameter
if currentStore.nil? && currentStore.empty?
currentStore = "nil"
store_data = {}
File.open('store_count.csv').each_line {|line|
line_data = line.split(",")
if !line_data[1].nil? && !line_data[1].empty?
store[line_data[0]] = line_data[1].strip.to_i
else
next
end
}
if store_data.key?(currentStore)
store_data[currentStore] += 1
CSV.open("store_count.csv", "wb") {
|csv| store_data.to_a.each {
|elem| csv << elem
}
}
end
So for example, if I increment 'store3', I need my file to look like:
store1,0
store2,0
store3,1
etc...
After I increment the value, I need to re-save to CSV.
You cannot read and write at the same time.
What are the Ruby File.open modes and options?
If you are dealing with a small file, where you can fit everything into the memory, this should be enough.
temp_file = File.open('temp.csv', 'w')
CSV.readlines('temp.csv').each do |line|
temp_file << "#{line[0]},#{line[1].to_i + 1}\n"
end
temp_file.flush
temp_file.close
If you want to rename the file, you can do this
`rm store.csv`
`mv temp.csv store.csv`

How to map and edit a CSV file with Ruby

Is there a way to edit a CSV file using the map method in Ruby? I know I can open a file using:
CSV.open("file.csv", "a+")
and add content to it, but I have to edit some specific lines.
The foreach method is only useful to read a file (correct me if I'm wrong).
I checked the Ruby CSV documentation but I can't find any useful info.
My CSV file has less than 1500 lines so I don't mind reading all the lines.
Another answer using each.with_index():
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5].sort # these are rows you would like to modify
rows_array.each.with_index(desired_indices[0]) do |row, index|
if desired_indices.include?(index)
# modify over here
rows_array[index][target_column] = 'modification'
end
end
# now update the file
CSV.open('sample3.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
You can also use each_with_index {} insead of each.with_index {}
Is there a way to edit a CSV file using the map method in Ruby?
Yes:
rows = CSV.open('sample.csv')
rows_array = rows.to_a
or
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5] # these are rows you would like to modify
edited_rows = rows_array.each_with_index.map do |row, index|
if desired_indices.include?(index)
# simply return the row
# or modify over here
row[3] = 'shiva'
# store index in each edited rows to keep track of the rows
[index, row]
end
end.compact
# update the main row_array with updated data
edited_rows.each{|row| rows_array[row[0]] = row[1]}
# now update the file
CSV.open('sample2.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
This is little messier. Is not it? I suggest you to use each_with_index with out map to do this. See my another answer
Here is a little script I wrote as an example on how read CSV data, do something to data, and then write out the edited text to a new file:
read_write_csv.rb:
#!/usr/bin/env ruby
require 'csv'
src_dir = "/home/user/Desktop/csvfile/FL_insurance_sample.csv"
dst_dir = "/home/user/Desktop/csvfile/FL_insurance_sample_out.csv"
puts " Reading data from : #{src_dir}"
puts " Writing data to : #{dst_dir}"
#create a new file
csv_out = File.open(dst_dir, 'wb')
#read from existing file
CSV.foreach(src_dir , :headers => false) do |row|
#then you can do this
# newrow = row.each_with_index { |rowcontent , row_num| puts "# {rowcontent} #{row_num}" }
# OR array to hash .. just saying .. maybe hash of arrays..
#h = Hash[*row]
#csv_out << h
# OR use map
#newrow = row.map(&:capitalize)
#csv_out << h
#OR use each ... Add and end
#newrow.each do |k,v| puts "#{k} is #{v}"
#Lastly, write back the edited , regexed data ..etc to an out file.
#csv_out << newrow
end
# close the file
csv_out.close
The output file has the desired data:
USER#USER-SVE1411EGXB:~/Desktop/csvfile$ ls
FL_insurance_sample.csv FL_insurance_sample_out.csv read_write_csv.rb
The input file data looked like this:
policyID,statecode,county,eq_site_limit,hu_site_limit,fl_site_limit,fr_site_limit,tiv_2011,tiv_2012,eq_site_deductible,hu_site_deductible,fl_site_deductible,fr_site_deductible,point_latitude,point_longitude,line,construction,point_granularity
119736,FL,CLAY COUNTY,498960,498960,498960,498960,498960,792148.9,0,9979.2,0,0,30.102261,-81.711777,Residential,Masonry,1
448094,FL,CLAY COUNTY,1322376.3,1322376.3,1322376.3,1322376.3,1322376.3,1438163.57,0,0,0,0,30.063936,-81.707664,Residential,Masonry,3
206893,FL,CLAY COUNTY,190724.4,190724.4,190724.4,190724.4,190724.4,192476.78,0,0,0,0,30.089579,-81.700455,Residential,Wood,1
333743,FL,CLAY COUNTY,0,79520.76,0,0,79520.76,86854.48,0,0,0,0,30.063236,-81.707703,Residential,Wood,3
172534,FL,CLAY COUNTY,0,254281.5,0,254281.5,254281.5,246144.49,0,0,0,0,30.060614,-81.702675,Residential,Wood,1

Parsing data to a csv file on a new line - ruby

I am trying to export data that I 'get' into a new csv file. Currently, my code below posts everyone onto a single line until it fills up and then it continues to the next line.
I would like to have it where when data is imported, it starts on the following line below, creating a list of transactions.
def export_data
File.open('coffee_orders.csv', 'a+') do |csv|
puts #item_quantity = [Time.now, #item_name, #amount]
csv << #item_quantity
end
end
Basing it on your starting code, I'd do something like:
def export_data
File.open('coffee_orders.csv', 'a') do |csv|
csv << [Time.now, #item_name, #amount].join(', ')
end
end
Or:
def export_data
File.open('coffee_orders.csv', 'a') do |csv|
csv << '%s, %s, %s' % [Time.now, #item_name, #amount].map(&:to_s)
end
end
Notice, it's not necessary to use 'a+' to append to a file. Instead use 'a' only unless you absolutely need "read" mode while the file is open also. Here's what the IO.new documentation says:
"a" Write-only, starts at end of file if file exists,
otherwise creates a new file for writing.
"a+" Read-write, starts at end of file if file exists,
otherwise creates a new file for reading and
writing.
The way I'd write it for myself would be something like:
CSV_FILENAME = 'coffee_orders.csv'
def export_data
csv_has_content = File.size?(CSV_FILENAME)
CSV.open(CSV_FILENAME, 'a') do |csv|
csv << %w[Time Item Amount] unless csv_has_content
csv << [Time.now, #item_name, #amount]
end
end
This uses Ruby's CSV class to handle all the ins-and-outs. It checks to see if the file already exists, and if it has no content it writes the header before writing the content.
Try this. It will add a new line after each transaction. When you append to it next, it will be from a new line.
def export_data
File.open('coffee_orders.csv', 'a+') do |csv|
csv.puts #item_quantity = [Time.now, #item_name, #amount]
end
end
Although by looking the extension, you would probably want to confine it to csv format.
def export_data
File.open('coffee_orders.csv', 'a+') do |csv|
#item_quantity = [Time.now, #item_name, #amount]
csv.puts #item_quantity.join(',')
end
end

trying to find the 1st instance of a string in a CSV using fastercsv

I'm trying to open a CSV file, look up a string, and then return the 2nd column of the csv file, but only the the first instance of it. I've gotten as far as the following, but unfortunately, it returns every instance. I'm a bit flummoxed.
Can the gods of Ruby help? Thanks much in advance.
M
for the purpose of this example, let's say names.csv is a file with the following:
foo, happy
foo, sad
bar, tired
foo, hungry
foo, bad
#!/usr/local/bin/ruby -w
require 'rubygems'
require 'fastercsv'
require 'pp'
FasterCSV.open('newfile.csv', 'w') do |output|
FasterCSV.foreach('names.csv') do |lookup|
index_PL = lookup.index('foo')
if index_PL
output << lookup[2]
end
end
end
ok, so, if I want to return all instances of foo, but in a csv, then how does that work?
so what I'd like as an outcome is happy, sad, hungry, bad. I thought it would be:
FasterCSV.open('newfile.csv', 'w') do |output|
FasterCSV.foreach('names.csv') do |lookup|
index_PL = lookup.index('foo')
if index_PL
build_str << "," << lookup[2]
end
output << build_str
end
end
but it does not seem to work
Replace foreach with open (to get an Enumerable) and find:
FasterCSV.open('newfile.csv', 'w') do |output|
output << FasterCSV.open('names.csv').find { |r| r.index('foo') }[2]
end
The index call will return nil if it doesn't find anything; that means that the find will give you the first row that has 'foo' and you can pull out the column at index 2 from the result.
If you're not certain that names.csv will have what you're looking for then a bit of error checking would be advisable:
FasterCSV.open('newfile.csv', 'w') do |output|
foos_row = FasterCSV.open('names.csv').find { |r| r.index('foo') }
if(foos_row)
output << foos_row[2]
else
# complain or something
end
end
Or, if you want to silently ignore the lack of 'foo' and use an empty string instead, you could do something like this:
FasterCSV.open('newfile.csv', 'w') do |output|
output << (FasterCSV.open('names.csv').find { |r| r.index('foo') } || ['','',''])[2]
end
I'd probably go with the "complain if it isn't found" version though.

Resources