So what I'm trying to do is read a specific column from an existing CSV file, parse some information out of the pulled data, then make a new CSV file with the newly parsed information in a single column. The header and first entry of the generated array go into the CSV file correctly, but after those, every other entry goes into the adjacent cells of the same row instead of creating a column, so it's like an L shape instead of just a line. Any ideas?
#!ruby.exe
require 'csv'
puts "Please enter a file name:" #user input file name (must be in same
folder as this file)
file = gets.chomp
begin
File.open(file, 'r')
rescue
print "Failed to open #{file}\n"
exit
end #makes sure that the file exists, if it does not it posts an error
data_file = File.new(file)
data = [] #initializes array for addresses from .csv
counter=0 #set counter up to allow for different sized files to be used
without issue
CSV.foreach(data_file, headers: true) do |row|
data << row.to_hash
counter+=1
end #goes through .csv one line ar a time
data.reject(&:empty?)
puts "Which column do you want to parse?"
column = gets.chomp
i=0
streets = []
while (i<counter)
address = data[i][column]
street_name = address.gsub(/^((\d[a-zA-Z])|[^a-zA-Z])*/, '')
streets.push(street_name)
i+=1
end
streets.reject(&:empty?)
puts "What do you want the output to be called?"
new_file = gets.chomp
CSV.open(new_file, "w", :write_headers=> true, :headers => [column]) do |hdr|
hdr << streets
end
You should scan the street array and insert it as a row, which means you need to place the line of data into an array before to send to the csv. Ok, maybe the code is simpler than the explanation:
CSV.open(new_file, "w", :write_headers=> true, :headers => [column]) do |csv_line|
streets.each { |street| csv_line << [street] }
end
I made a ruby program to copy content of one CSV file to a new CSV file.
This is my code -
require 'csv'
class CopyFile
def self.create_duplicate_file(file_name)
CSV.open(file_name, "wb") do |output_row|
output_row << CSV.open('input.csv', 'r') { |csv| csv.first }
CSV.foreach('input.csv', headers: true) do |row|
output_row << row
end
end
end
end
puts "Insert duplicate file name"
file_name = gets.chomp
file_name = file_name+".csv"
CopyFile.create_duplicate_file(file_name)
puts "\nDuplicate File Created."
I am opening the input.csv file twice, one to copy headers and then to copy content.
I want to optimise my code. So is there a way to optimise it further?
Just use the cp method:
FileUtils.cp(src, destination, options), no need to reinvent the wheel, like this:
class CopyFile
def self.create_duplicate_file(file_name)
FileUtils.cp('input.csv',file_name)
end
end
or better yet:
file_name = gets.chomp
file_name = file_name+".csv"
FileUtils.cp('input.csv', file_name)
Is there a way to edit a CSV file using the map method in Ruby? I know I can open a file using:
CSV.open("file.csv", "a+")
and add content to it, but I have to edit some specific lines.
The foreach method is only useful to read a file (correct me if I'm wrong).
I checked the Ruby CSV documentation but I can't find any useful info.
My CSV file has less than 1500 lines so I don't mind reading all the lines.
Another answer using each.with_index():
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5].sort # these are rows you would like to modify
rows_array.each.with_index(desired_indices[0]) do |row, index|
if desired_indices.include?(index)
# modify over here
rows_array[index][target_column] = 'modification'
end
end
# now update the file
CSV.open('sample3.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
You can also use each_with_index {} insead of each.with_index {}
Is there a way to edit a CSV file using the map method in Ruby?
Yes:
rows = CSV.open('sample.csv')
rows_array = rows.to_a
or
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5] # these are rows you would like to modify
edited_rows = rows_array.each_with_index.map do |row, index|
if desired_indices.include?(index)
# simply return the row
# or modify over here
row[3] = 'shiva'
# store index in each edited rows to keep track of the rows
[index, row]
end
end.compact
# update the main row_array with updated data
edited_rows.each{|row| rows_array[row[0]] = row[1]}
# now update the file
CSV.open('sample2.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
This is little messier. Is not it? I suggest you to use each_with_index with out map to do this. See my another answer
Here is a little script I wrote as an example on how read CSV data, do something to data, and then write out the edited text to a new file:
read_write_csv.rb:
#!/usr/bin/env ruby
require 'csv'
src_dir = "/home/user/Desktop/csvfile/FL_insurance_sample.csv"
dst_dir = "/home/user/Desktop/csvfile/FL_insurance_sample_out.csv"
puts " Reading data from : #{src_dir}"
puts " Writing data to : #{dst_dir}"
#create a new file
csv_out = File.open(dst_dir, 'wb')
#read from existing file
CSV.foreach(src_dir , :headers => false) do |row|
#then you can do this
# newrow = row.each_with_index { |rowcontent , row_num| puts "# {rowcontent} #{row_num}" }
# OR array to hash .. just saying .. maybe hash of arrays..
#h = Hash[*row]
#csv_out << h
# OR use map
#newrow = row.map(&:capitalize)
#csv_out << h
#OR use each ... Add and end
#newrow.each do |k,v| puts "#{k} is #{v}"
#Lastly, write back the edited , regexed data ..etc to an out file.
#csv_out << newrow
end
# close the file
csv_out.close
The output file has the desired data:
USER#USER-SVE1411EGXB:~/Desktop/csvfile$ ls
FL_insurance_sample.csv FL_insurance_sample_out.csv read_write_csv.rb
The input file data looked like this:
policyID,statecode,county,eq_site_limit,hu_site_limit,fl_site_limit,fr_site_limit,tiv_2011,tiv_2012,eq_site_deductible,hu_site_deductible,fl_site_deductible,fr_site_deductible,point_latitude,point_longitude,line,construction,point_granularity
119736,FL,CLAY COUNTY,498960,498960,498960,498960,498960,792148.9,0,9979.2,0,0,30.102261,-81.711777,Residential,Masonry,1
448094,FL,CLAY COUNTY,1322376.3,1322376.3,1322376.3,1322376.3,1322376.3,1438163.57,0,0,0,0,30.063936,-81.707664,Residential,Masonry,3
206893,FL,CLAY COUNTY,190724.4,190724.4,190724.4,190724.4,190724.4,192476.78,0,0,0,0,30.089579,-81.700455,Residential,Wood,1
333743,FL,CLAY COUNTY,0,79520.76,0,0,79520.76,86854.48,0,0,0,0,30.063236,-81.707703,Residential,Wood,3
172534,FL,CLAY COUNTY,0,254281.5,0,254281.5,254281.5,246144.49,0,0,0,0,30.060614,-81.702675,Residential,Wood,1
I am currently new to Ruby and am having a hard time writing to an excel file.
I want to parse through a CSV file, extract data where the 'food' column in the csv file = butter and put the rows where 'food' column = butter into a new excel workbook. I can write the data that contains butter in the 'food' column just fine into a CSV file but am having trouble writing it to a workbook (excel format).
require 'rubygems'
require 'csv'
require 'spreadsheet'
csv_fname = 'commissions.csv'
options = { headers: :first_row }
food_type = { 'food' => 'butter'}
food_type_match = nil
CSV.open(csv_fname, 'r', options) do |csv|
food_type_match = csv.find_all do |row|
Hash[row].select { |k,v| food_type[k] } == food_type
end
end
#writing the 'butter' data to a CSV file
#CSV.open('butter.csv', 'w') do |csv_object|
# food_type_match.each do |row_array|
# csv_object << row_array
# end
#end
book = Spreadsheet::Workbook.new
sheet1 = book.create_worksheet
food_type_match.each do |csv|
csv.each_with_index do |row, i|
sheet1.row(i).replace(row)
end
end
The spreadsheet generates but comes out blank. I have searched through numerous topics on ruby spreadsheet but I cannot get it to work. Any help would be greatly appreciated.
Updated Completely
What if you try this:
book = Spreadsheet::Workbook.new
sheet1 = book.create_worksheet
food_type_match.each do |csv|
csv.each_with_index do |row, i|
sheet1.insert_row(i,row)
end
end
book.write('/path_to_output_location/book.xls')
Also where does this output to? I cannot see a give path for this so I would think that is the issue but you say it generates? I added the write line because the code states this for #write
Write this Workbook to a File, IO Stream or Writer Object. The latter will
make more sense once there are more than just an Excel-Writer available.
Like I said I am completely unfamiliar with this gem and the documentation is terrible with axslx it would be something like this
package = Axlsx::Package.new
book = package.workbook
book.add_worksheet do |sheet|
food_type_match.each do |csv|
sheet.add_row csv
end
end
package.serialize('/path_to_output_location/book.xlsx')
Try write_xlsx gem. Here is my simple csvtoxlsx.rb script to combine *.csv in a folder to a single.xlsx:
require "csv"
require "write_xlsx"
def csvtoxls(csv, xlsx)
count = 0
workbook = WriteXLSX.new(xlsx)
Dir[csv].sort.each do | file |
puts file
name = File.basename(file, ".csv")
worksheet = workbook.add_worksheet(name)
i = 0
CSV.foreach(file) do | row |
worksheet.write_row(i, 0, row)
i = i + 1
count = count + 1
end
end
workbook.close
count
end
abort("Syntax: ruby -W0 csvtoxlsx.rb 'folder/*.csv' single.xlsx") if ARGV.length < 2
time_begin = Time.now
count = csvtoxls(ARGV[0], ARGV[1])
time_spent = Time.now - time_begin
puts "csvtoxlsx process #{ARGV[0]} with #{count} rows in #{time_spent.round(2)} seconds"
I am trying to export data that I 'get' into a new csv file. Currently, my code below posts everyone onto a single line until it fills up and then it continues to the next line.
I would like to have it where when data is imported, it starts on the following line below, creating a list of transactions.
def export_data
File.open('coffee_orders.csv', 'a+') do |csv|
puts #item_quantity = [Time.now, #item_name, #amount]
csv << #item_quantity
end
end
Basing it on your starting code, I'd do something like:
def export_data
File.open('coffee_orders.csv', 'a') do |csv|
csv << [Time.now, #item_name, #amount].join(', ')
end
end
Or:
def export_data
File.open('coffee_orders.csv', 'a') do |csv|
csv << '%s, %s, %s' % [Time.now, #item_name, #amount].map(&:to_s)
end
end
Notice, it's not necessary to use 'a+' to append to a file. Instead use 'a' only unless you absolutely need "read" mode while the file is open also. Here's what the IO.new documentation says:
"a" Write-only, starts at end of file if file exists,
otherwise creates a new file for writing.
"a+" Read-write, starts at end of file if file exists,
otherwise creates a new file for reading and
writing.
The way I'd write it for myself would be something like:
CSV_FILENAME = 'coffee_orders.csv'
def export_data
csv_has_content = File.size?(CSV_FILENAME)
CSV.open(CSV_FILENAME, 'a') do |csv|
csv << %w[Time Item Amount] unless csv_has_content
csv << [Time.now, #item_name, #amount]
end
end
This uses Ruby's CSV class to handle all the ins-and-outs. It checks to see if the file already exists, and if it has no content it writes the header before writing the content.
Try this. It will add a new line after each transaction. When you append to it next, it will be from a new line.
def export_data
File.open('coffee_orders.csv', 'a+') do |csv|
csv.puts #item_quantity = [Time.now, #item_name, #amount]
end
end
Although by looking the extension, you would probably want to confine it to csv format.
def export_data
File.open('coffee_orders.csv', 'a+') do |csv|
#item_quantity = [Time.now, #item_name, #amount]
csv.puts #item_quantity.join(',')
end
end