How to map and edit a CSV file with Ruby - ruby
Is there a way to edit a CSV file using the map method in Ruby? I know I can open a file using:
CSV.open("file.csv", "a+")
and add content to it, but I have to edit some specific lines.
The foreach method is only useful to read a file (correct me if I'm wrong).
I checked the Ruby CSV documentation but I can't find any useful info.
My CSV file has less than 1500 lines so I don't mind reading all the lines.
Another answer using each.with_index():
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5].sort # these are rows you would like to modify
rows_array.each.with_index(desired_indices[0]) do |row, index|
if desired_indices.include?(index)
# modify over here
rows_array[index][target_column] = 'modification'
end
end
# now update the file
CSV.open('sample3.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
You can also use each_with_index {} insead of each.with_index {}
Is there a way to edit a CSV file using the map method in Ruby?
Yes:
rows = CSV.open('sample.csv')
rows_array = rows.to_a
or
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5] # these are rows you would like to modify
edited_rows = rows_array.each_with_index.map do |row, index|
if desired_indices.include?(index)
# simply return the row
# or modify over here
row[3] = 'shiva'
# store index in each edited rows to keep track of the rows
[index, row]
end
end.compact
# update the main row_array with updated data
edited_rows.each{|row| rows_array[row[0]] = row[1]}
# now update the file
CSV.open('sample2.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
This is little messier. Is not it? I suggest you to use each_with_index with out map to do this. See my another answer
Here is a little script I wrote as an example on how read CSV data, do something to data, and then write out the edited text to a new file:
read_write_csv.rb:
#!/usr/bin/env ruby
require 'csv'
src_dir = "/home/user/Desktop/csvfile/FL_insurance_sample.csv"
dst_dir = "/home/user/Desktop/csvfile/FL_insurance_sample_out.csv"
puts " Reading data from : #{src_dir}"
puts " Writing data to : #{dst_dir}"
#create a new file
csv_out = File.open(dst_dir, 'wb')
#read from existing file
CSV.foreach(src_dir , :headers => false) do |row|
#then you can do this
# newrow = row.each_with_index { |rowcontent , row_num| puts "# {rowcontent} #{row_num}" }
# OR array to hash .. just saying .. maybe hash of arrays..
#h = Hash[*row]
#csv_out << h
# OR use map
#newrow = row.map(&:capitalize)
#csv_out << h
#OR use each ... Add and end
#newrow.each do |k,v| puts "#{k} is #{v}"
#Lastly, write back the edited , regexed data ..etc to an out file.
#csv_out << newrow
end
# close the file
csv_out.close
The output file has the desired data:
USER#USER-SVE1411EGXB:~/Desktop/csvfile$ ls
FL_insurance_sample.csv FL_insurance_sample_out.csv read_write_csv.rb
The input file data looked like this:
policyID,statecode,county,eq_site_limit,hu_site_limit,fl_site_limit,fr_site_limit,tiv_2011,tiv_2012,eq_site_deductible,hu_site_deductible,fl_site_deductible,fr_site_deductible,point_latitude,point_longitude,line,construction,point_granularity
119736,FL,CLAY COUNTY,498960,498960,498960,498960,498960,792148.9,0,9979.2,0,0,30.102261,-81.711777,Residential,Masonry,1
448094,FL,CLAY COUNTY,1322376.3,1322376.3,1322376.3,1322376.3,1322376.3,1438163.57,0,0,0,0,30.063936,-81.707664,Residential,Masonry,3
206893,FL,CLAY COUNTY,190724.4,190724.4,190724.4,190724.4,190724.4,192476.78,0,0,0,0,30.089579,-81.700455,Residential,Wood,1
333743,FL,CLAY COUNTY,0,79520.76,0,0,79520.76,86854.48,0,0,0,0,30.063236,-81.707703,Residential,Wood,3
172534,FL,CLAY COUNTY,0,254281.5,0,254281.5,254281.5,246144.49,0,0,0,0,30.060614,-81.702675,Residential,Wood,1
Related
Ruby CSV - Write on same row without overwriting?
I'm using CSV.open(filename, "w") do |csv| to create and write to a csv file in one ruby.rb file and now I need to open it and edit it in a second .rb file. Right now I'm using CSV.open(filename, "a") do |csv| but that creates new rows rather than adding the new content to the end of the existing rows. If I use CSV.open(filename, "w") do |csv| the second time it overwrites the first rows. edit: # Create export CSV final_export_csv = "filepath_final.csv" # Create filename for CSV file imported_csv_filename = "imported_file.csv" CSV.open(final_export_csv, "w", headers: ["several", "headers"] + [:new_header], write_headers: true) do |final_csv| # Read existing CSV file CSV.foreach(imported_csv_filename) do |old_csv_row| # Read a row, add the new column, write it to the new row CSV.open(denominator_csv_filename, "r+") do |new_csv_col| # gathering some data code data = { passed.in } # Write data new_csv_col << [ passedin[:data] ] old_csv_row[:new_header] = passedin[:data] final_export_csv << old_csv_row end end end end end
As tadman comments, you can't actually edit a file in place. Well, you can but all the lines have to remain the same length. You're not doing that. Instead, read a row, modify it, and write it to a new CSV. Then replace the old file with the new one. Be careful to avoid slurping the entire CSV into memory, CSV files can get quite large. require 'csv' require 'tempfile' require 'fileutils' csv_file = "test.csv" # Write the new file to a tempfile to avoid polluting the directory. temp = Tempfile.new # Read the header line. old_csv = CSV.open(csv_file, "r", headers: true, return_headers: true) old_csv.readline # Open the new CSV with the existing headers plus a new one. new_csv = CSV.open( temp, "w", headers: old_csv.headers + [:new], write_headers: true ) # Read a row, add the new column, write it to the new CSV. old_csv.each do |row| row[:new] = 42 new_csv << row end old_csv.close new_csv.close # Replace the old CSV with the new one. FileUtils.move(temp.path, csv_file)
Write an array to multi column CSV format using Ruby
I have an array of arrays in Ruby that i'm trying to output to a CSV file (or text). That I can then easily transfer over to another XML file for graphing. I can't seem to get the output (in text format) like so. Instead I get one line of data which is just a large array. 0,2 0,3 0,4 0,5 I originally tried something along the lines of this File.open('02.3.gyro_trends.text' , 'w') { |file| trend_array.each { |x,y| file.puts(x,y)}} And it outputs 0.2 46558 0 46560 0 ....etc etc. Can anyone point me in the "write" direction for getting either: (i) .text file that can put my data like so. trend_array[0][0], trend_array[0][1] trend_array[1][0], trend_array[1][1] trend_array[2][0], trend_array[2][1] trend_array[3][0], trend_array[3][1] (ii) .csv file that would put this data in separate columns. edit I recently added more than two values into my array, check out my answer combining Cameck's solution. This is currently what I have at the moment. trend_array=[] j=1 # cycle through array and find change in gyro data. while j < gyro_array.length-2 if gyro_array[j+1][1] < 0.025 && gyro_array[j+1][1] > -0.025 trend_array << [0, gyro_array[j][0]] j+=1 elsif gyro_array[j+1][1] > -0.025 # if the next value is increasing by x1.2 the value of the previous amount. Log it as +1 trend_array << [0.2, gyro_array[j][0]] j+=1 elsif gyro_array[j+1][1] < 0.025 # if the next value is decreasing by x1.2 the value of the previous amount. Log it as -1 trend_array << [-0.2, gyro_array[j][0]] j+=1 end end #for graphing and analysis purposes (wanted to print it all as a csv in two columns) File.open('02.3test.gyro_trends.text' , 'w') { |file| trend_array.each { |x,y| file.puts(x,y)}} File.open('02.3test.gyro_trends_count.text' , 'w') { |file| trend_array.each {|x,y| file.puts(y)}} I know it's something really easy, but for some reason I'm missing it. Something with concatenation, but I found that if I try and concatenate a \\n in my last line of code, it doesn't output it to the file. It outputs it in my console the way I want it, but not when I write it to a file. Thanks for taking the time to read this all.
File.open('02.3test.gyro_trends.text' , 'w') { |file| trend_array.each { |a| file.puts(a.join(","))}}
Alternately using the CSV Class: def write_to_csv(row) if csv_exists? CSV.open(#csv_name, 'a+') { |csv| csv << row } else # create and add headers if doesn't exist already CSV.open(#csv_name, 'wb') do |csv| csv << CSV_HEADER csv << row end end end def csv_exists? #exists ||= File.file?(#csv_name) end Call write_to_csv with an array [col_1, col_2, col_3]
Thank you both #cameck & #tkupari, both answers were what I was looking for. Went with Cameck's answer in the end, because it "cut out" cutting and pasting text => xml. Here's what I did to get an array of arrays into their proper places. require 'csv' CSV_HEADER = [ "Apples", "Oranges", "Pears" ] #csv_name = "Test_file.csv" def write_to_csv(row) if csv_exists? CSV.open(#csv_name, 'a+') { |csv| csv << row } else # create and add headers if doesn't exist already CSV.open(#csv_name, 'wb') do |csv| csv << CSV_HEADER csv << row end end end def csv_exists? #exists ||= File.file?(#csv_name) end array = [ [1,2,3] , ['a','b','c'] , ['dog', 'cat' , 'poop'] ] array.each { |row| write_to_csv(row) }
How to open a CSV file and increment a specific value by one
I'm trying to open a simple CSV file in Ruby, find a particular key, increment its value by one, then re-save it. Example CSV file: store1,0 store2,0 store3,0 ...etc. Ruby code: require 'csv' currentStore = store # store is passed as a parameter if currentStore.nil? && currentStore.empty? currentStore = "nil" store_data = {} File.open('store_count.csv').each_line {|line| line_data = line.split(",") if !line_data[1].nil? && !line_data[1].empty? store[line_data[0]] = line_data[1].strip.to_i else next end } if store_data.key?(currentStore) store_data[currentStore] += 1 CSV.open("store_count.csv", "wb") { |csv| store_data.to_a.each { |elem| csv << elem } } end So for example, if I increment 'store3', I need my file to look like: store1,0 store2,0 store3,1 etc... After I increment the value, I need to re-save to CSV.
You cannot read and write at the same time. What are the Ruby File.open modes and options? If you are dealing with a small file, where you can fit everything into the memory, this should be enough. temp_file = File.open('temp.csv', 'w') CSV.readlines('temp.csv').each do |line| temp_file << "#{line[0]},#{line[1].to_i + 1}\n" end temp_file.flush temp_file.close If you want to rename the file, you can do this `rm store.csv` `mv temp.csv store.csv`
Output many arrays to CSV-files in Ruby
I have a question about Ruby. What I want to do is first to sort my items ascending and then write them out to a CSV-file. Now, the problem is further complicated by the fact that I want to iterate over a lot of CSV-files. I found this thread and the answer looks fine, but I am not able to get more than the last line written to my output file. How can I get the whole data sorted and written to different CSV-files? My code: require 'date' require 'csv' class Daily < # Daily has a open Struct.new(:open) # a method to print out a csv record for the current Daily. def print_csv_record printf("%s,", open) printf("\n") end end #------# # MAIN # #------# # This is where I iterate over my csv-files: foobar = ['foo', 'bar'] foobar.each do |foobar| # get the input filename from the command line input_file = "#{foobar}.csv" # define an array to hold the Daily records arr = Array.new # loop through each record in the csv file, adding # each record to my array while overlooking the header. f = File.open(input_file, "r") f.each_with_index { |row, i| next if i == 0 words = row.split(',') p = Daily.new # do a little work here to convert my numbers p.open = words[1].to_f arr.push(p) } # sort the data by ascending opens arr.sort! { |a,b| a.open <=> b.open } # print out all the sorted records (just print to stdout) arr.each { |p| CSV.open("#{foobar}_new.csv", "w") do |csv| csv << p.print_csv_record end } end My input CSV-file: Open 52.23 52.45 52.36 52.07 52.69 52.38 51.2 50.99 51.41 51.89 51.38 50.94 49.55 50.21 50.13 50.14 49.49 48.5 47.92 My output CSV-file: 47.92
You need to put the iteration inside the open CSV file: CSV.open("#{foobar}_new.csv", "w") do |csv| arr.each { |p| csv << p.print_csv_record } end
Reading every line in a CSV and using it to query an API
I have the following Ruby code: require 'octokit.rb' require 'csv.rb' CSV.foreach("actors.csv") do |row| CSV.open("node_attributes.csv", "wb") do |csv| csv << [Octokit.user "userid"] end end I have a csv called actors.csv where every row has one entry - a string with a userid. I want to go through all the rows, and for each row do Octokit.user "userid", and then store the output from each query on a separate row in a CSV - node_attributes.csv. My code does not seem to do this? How can I modify it to make this work?
require 'csv' DOC = 'actors.csv' DOD = 'new_output.csv' holder = CSV.read(DOC) You can navigate it by calling holder[0][0] => data in the array holder[1][0] => moar data in array make sense? #make this a loop profile = [] profile[0] = holder[0][0] profile[1] = holder[1][0] profile[2] = 'whatever it is you want to store in the new cell' CSV.open(DOD, "a") do |data| data << profile.map end #end the loop here That last bit of code will print whatever you want into a new csv file