I'm using
CSV.open(filename, "w") do |csv|
to create and write to a csv file in one ruby.rb file and now I need to open it and edit it in a second .rb file. Right now I'm using CSV.open(filename, "a") do |csv| but that creates new rows rather than adding the new content to the end of the existing rows.
If I use CSV.open(filename, "w") do |csv| the second time it overwrites the first rows.
edit:
# Create export CSV
final_export_csv = "filepath_final.csv"
# Create filename for CSV file
imported_csv_filename = "imported_file.csv"
CSV.open(final_export_csv, "w", headers: ["several", "headers"] + [:new_header], write_headers: true) do |final_csv|
# Read existing CSV file
CSV.foreach(imported_csv_filename) do |old_csv_row|
# Read a row, add the new column, write it to the new row
CSV.open(denominator_csv_filename, "r+") do |new_csv_col|
# gathering some data code
data = { passed.in }
# Write data
new_csv_col <<
[
passedin[:data]
]
old_csv_row[:new_header] = passedin[:data]
final_export_csv << old_csv_row
end
end
end
end
end
As tadman comments, you can't actually edit a file in place. Well, you can but all the lines have to remain the same length. You're not doing that.
Instead, read a row, modify it, and write it to a new CSV. Then replace the old file with the new one. Be careful to avoid slurping the entire CSV into memory, CSV files can get quite large.
require 'csv'
require 'tempfile'
require 'fileutils'
csv_file = "test.csv"
# Write the new file to a tempfile to avoid polluting the directory.
temp = Tempfile.new
# Read the header line.
old_csv = CSV.open(csv_file, "r", headers: true, return_headers: true)
old_csv.readline
# Open the new CSV with the existing headers plus a new one.
new_csv = CSV.open(
temp, "w",
headers: old_csv.headers + [:new],
write_headers: true
)
# Read a row, add the new column, write it to the new CSV.
old_csv.each do |row|
row[:new] = 42
new_csv << row
end
old_csv.close
new_csv.close
# Replace the old CSV with the new one.
FileUtils.move(temp.path, csv_file)
I currently have this script that generates usernames from a given CSV. Rather than printing these results to the console, how can I write a new CSV with these results?
This is the script I currently have, runs with no errors. I am assuming if I write a new CSV in the do |row| block it is going to create x amount of new files which I do not want.
require 'csv'
CSV.foreach('data.csv', :headers => true) do |row|
id = row['id']
fn = row['first_name']
ln = row['last_name']
p fn[0] + ln + id[3,8]
end
Just manage the CSV file to write around the reading:
CSV.open("path/to/file.csv", "wb") do |csv|
CSV.foreach('data.csv', :headers => true) do |row|
id = row['id']
fn = row['first_name']
ln = row['last_name']
csv << [fn[0], ln, id[3,8]]
# or, to output it as a single column:
# csv << ["#{fn[0]}#{ln}#{id[3,8]}"]
end
end
Writing CSV to a file.
Is there a way to edit a CSV file using the map method in Ruby? I know I can open a file using:
CSV.open("file.csv", "a+")
and add content to it, but I have to edit some specific lines.
The foreach method is only useful to read a file (correct me if I'm wrong).
I checked the Ruby CSV documentation but I can't find any useful info.
My CSV file has less than 1500 lines so I don't mind reading all the lines.
Another answer using each.with_index():
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5].sort # these are rows you would like to modify
rows_array.each.with_index(desired_indices[0]) do |row, index|
if desired_indices.include?(index)
# modify over here
rows_array[index][target_column] = 'modification'
end
end
# now update the file
CSV.open('sample3.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
You can also use each_with_index {} insead of each.with_index {}
Is there a way to edit a CSV file using the map method in Ruby?
Yes:
rows = CSV.open('sample.csv')
rows_array = rows.to_a
or
rows_array = CSV.read('sample.csv')
desired_indices = [3, 4, 5] # these are rows you would like to modify
edited_rows = rows_array.each_with_index.map do |row, index|
if desired_indices.include?(index)
# simply return the row
# or modify over here
row[3] = 'shiva'
# store index in each edited rows to keep track of the rows
[index, row]
end
end.compact
# update the main row_array with updated data
edited_rows.each{|row| rows_array[row[0]] = row[1]}
# now update the file
CSV.open('sample2.csv', 'wb') { |csv| rows_array.each{|row| csv << row}}
This is little messier. Is not it? I suggest you to use each_with_index with out map to do this. See my another answer
Here is a little script I wrote as an example on how read CSV data, do something to data, and then write out the edited text to a new file:
read_write_csv.rb:
#!/usr/bin/env ruby
require 'csv'
src_dir = "/home/user/Desktop/csvfile/FL_insurance_sample.csv"
dst_dir = "/home/user/Desktop/csvfile/FL_insurance_sample_out.csv"
puts " Reading data from : #{src_dir}"
puts " Writing data to : #{dst_dir}"
#create a new file
csv_out = File.open(dst_dir, 'wb')
#read from existing file
CSV.foreach(src_dir , :headers => false) do |row|
#then you can do this
# newrow = row.each_with_index { |rowcontent , row_num| puts "# {rowcontent} #{row_num}" }
# OR array to hash .. just saying .. maybe hash of arrays..
#h = Hash[*row]
#csv_out << h
# OR use map
#newrow = row.map(&:capitalize)
#csv_out << h
#OR use each ... Add and end
#newrow.each do |k,v| puts "#{k} is #{v}"
#Lastly, write back the edited , regexed data ..etc to an out file.
#csv_out << newrow
end
# close the file
csv_out.close
The output file has the desired data:
USER#USER-SVE1411EGXB:~/Desktop/csvfile$ ls
FL_insurance_sample.csv FL_insurance_sample_out.csv read_write_csv.rb
The input file data looked like this:
policyID,statecode,county,eq_site_limit,hu_site_limit,fl_site_limit,fr_site_limit,tiv_2011,tiv_2012,eq_site_deductible,hu_site_deductible,fl_site_deductible,fr_site_deductible,point_latitude,point_longitude,line,construction,point_granularity
119736,FL,CLAY COUNTY,498960,498960,498960,498960,498960,792148.9,0,9979.2,0,0,30.102261,-81.711777,Residential,Masonry,1
448094,FL,CLAY COUNTY,1322376.3,1322376.3,1322376.3,1322376.3,1322376.3,1438163.57,0,0,0,0,30.063936,-81.707664,Residential,Masonry,3
206893,FL,CLAY COUNTY,190724.4,190724.4,190724.4,190724.4,190724.4,192476.78,0,0,0,0,30.089579,-81.700455,Residential,Wood,1
333743,FL,CLAY COUNTY,0,79520.76,0,0,79520.76,86854.48,0,0,0,0,30.063236,-81.707703,Residential,Wood,3
172534,FL,CLAY COUNTY,0,254281.5,0,254281.5,254281.5,246144.49,0,0,0,0,30.060614,-81.702675,Residential,Wood,1
I'm still fairly new to coding and I'm trying to learn about manipulating CSV files.
The code below opens a specified CSV file, goes to each url in the CSV file in column B (header = url), and finds the price on the webpage.
Example data from CSV file:
Store,URL,Price
Walmart,http://www.walmart.com/ip/HP-11.6-Stream-Laptop-PC-with-Intel-Celeron-Processor-2GB-Memory-32GB-Hard-Drive-Windows-8.1-and-Microsoft-Office-365-Personal-1-yr-subscription/39073484
Walmart,http://www.walmart.com/ip/Nextbook-10.1-Intel-Quad-Core-2-In-1-Detachable-Windows-8.1-Tablet/39092206
Walmart,http://www.walmart.com/ip/Nextbook-10.1-Intel-Quad-Core-2-In-1-Detachable-Windows-8.1-Tablet/39092206
I'm having trouble writing that price to the adjacent column C (header = price) in the same CSV.
require 'nokogiri'
require 'open-uri'
require 'csv'
contents = CSV.open "mp_lookup.csv", headers: true, header_converters: :symbol
contents.each do |row|
row_url = row[:url]
goto_url = Nokogiri::HTML(open(row_url))
new_price = goto_url.css('meta[itemprop="price"]')[0]['content']
#----
#In this section, I'm looking to write the value of new_price to the 3rd column in the same CSV file
#----
end
In the past, I've been able to use:
in_file = open("mp_lookup.csv", 'w')
in_file.write(new_price)
But this doesn't seem to work in this situation.
Any help is appreciated!
The simple answer is that you can refer to the :price column in the CSV file, just like you refer to the :url column. Try this code to set the price in the CSV object in memory:
row[:price] = new_price
After you've read through all of the records, you'll want to save the CSV file again. You can save it to any filename, but we'll simply overwrite the previous file in this example:
CSV.open("mp_lookup.csv", "wb") do |csv|
contents.each do |row|
csv << row
end
end
In a real production environment, you'd want to be more fault tolerant than this, and preserve the original file until the end of the process. However, this shows how to update the values in the price column for each row, and then save the changes to a file.
Here's what I'm trying to accomplish. I need to have a single CSV with headers and several rows. I'm iterating through the headers and storing then and then associating the row data to the header. I need to be able to iterate through each of the rows in the CSV to use for constructing an XML's data. The constructed XML is then dumped as a .xml file and the program starts on the next row in the CSV. Each row has a column that provides the name of the XML file.
Here's what I've got so far.
Read in the data from the CSV file. Collect the header and row data.
def get_rows
raw_data = CSV.read('test.csv', {:skip_blanks => false, :headers => true})
data = []
raw_data.each { |row| data << row}
return build_header(data, raw_data)
end
take the header and row data and marry them up.
def build_header(data, raw_data)
(0..(data.length - 1)).each do |ri|
h = {}
raw_data.headers.each_with_index do |v, i|
h[v] = data[ri].fields[i]
end
return build_ostruct(h)
end
end
take the hash h and make an ostruct of it.
def build_ostruct(h)
x = OpenStruct.new(h)
uniq = x.tc_name
y = uniq_name.to_s + ".xml"
#marshal dump for debugging
x.marshal_dump.each{ |k,v| puts "#{k} => #{v}" }
return xml_builder(x, y)
end
Below this I'm taking the new ostruct "x" and calling the column headers from the CSV to #populate the XML nodes
For example: x.column1, x.column2, x.column3
Now the part I'm getting hung up on is getting the ostruct to receive the new row of data per iteration run. The objective is to have the ostruct populate with each row from the CSV. Currently the hash is displaying the proper data set and my XML is populating as expected but only with the first row of data. How do I get this to iterate through all the rows and populate the ostruct with the data per iteration so I can create a bulk set of XML's?
Thanks in advance for any and all help!
Something like this should work:
require 'csv'
require 'nokogiri'
CSV.foreach('test.csv', :headers => true) do |row|
builder = Nokogiri::XML::Builder.new do |xml|
xml.root do |root|
row.each do |k, v|
root.send k, v
end
end
end
File.open("#{row['tc_name']}.xml", 'w'){|f| f << builder.to_xml}
end
you are calling return in build_header, which ends the call. you need to collect your results in some way without immediately returning the first one, so that build_header can run for the entire set of rows.