I want to drop the top two rows from a CSV file and add my own header. I have wrapped this in a rake task.
task :fix_csv do
# copy to temp file
cp ENV['source'], TMP_FILE
# drop header rows
table = CSV.table(TMP_FILE)
File.open(TMP_FILE, 'w') do |f|
f.write(table.drop(2).to_csv)
end
# add new header
CSV.open(TMP_FILE, 'w', force_quotes: true) do |csv|
csv << HEADERS if csv.count.eql? 0
end
puts 'Done!'
end
However, this fails with an error:
rake aborted!
IOError: not opened for reading
../rakefile.rb:54:in `count'
Line 54 is:
csv << HEADERS if csv.count.eql? 0
Why can't it read the file? Do I need to explicitly close the file after I've removed the first two rows?
The second time you open the file for writing only, but then you are trying to iterate getting an access to content (namely by querying the row count):
# ⇓⇓⇓
CSV.open(TMP_FILE, 'w', force_quotes: true) do |csv|
# ⇓⇓⇓⇓⇓
csv << HEADERS if csv.count.eql? 0
end
while it’s easy to fix, may I ask what would be wrong with forgetting about CSV in total, in favor of somewhat like:
old = File.readlines(FILE_NAME).drop(2)
old[0...0] = HEADERS.join(',')
File.write(FILE_NAME, old)
?
Related
I'm using
CSV.open(filename, "w") do |csv|
to create and write to a csv file in one ruby.rb file and now I need to open it and edit it in a second .rb file. Right now I'm using CSV.open(filename, "a") do |csv| but that creates new rows rather than adding the new content to the end of the existing rows.
If I use CSV.open(filename, "w") do |csv| the second time it overwrites the first rows.
edit:
# Create export CSV
final_export_csv = "filepath_final.csv"
# Create filename for CSV file
imported_csv_filename = "imported_file.csv"
CSV.open(final_export_csv, "w", headers: ["several", "headers"] + [:new_header], write_headers: true) do |final_csv|
# Read existing CSV file
CSV.foreach(imported_csv_filename) do |old_csv_row|
# Read a row, add the new column, write it to the new row
CSV.open(denominator_csv_filename, "r+") do |new_csv_col|
# gathering some data code
data = { passed.in }
# Write data
new_csv_col <<
[
passedin[:data]
]
old_csv_row[:new_header] = passedin[:data]
final_export_csv << old_csv_row
end
end
end
end
end
As tadman comments, you can't actually edit a file in place. Well, you can but all the lines have to remain the same length. You're not doing that.
Instead, read a row, modify it, and write it to a new CSV. Then replace the old file with the new one. Be careful to avoid slurping the entire CSV into memory, CSV files can get quite large.
require 'csv'
require 'tempfile'
require 'fileutils'
csv_file = "test.csv"
# Write the new file to a tempfile to avoid polluting the directory.
temp = Tempfile.new
# Read the header line.
old_csv = CSV.open(csv_file, "r", headers: true, return_headers: true)
old_csv.readline
# Open the new CSV with the existing headers plus a new one.
new_csv = CSV.open(
temp, "w",
headers: old_csv.headers + [:new],
write_headers: true
)
# Read a row, add the new column, write it to the new CSV.
old_csv.each do |row|
row[:new] = 42
new_csv << row
end
old_csv.close
new_csv.close
# Replace the old CSV with the new one.
FileUtils.move(temp.path, csv_file)
I've been trying to use Ruby to create a CSV file from json data. I was able to create the file, but I need to add a few headers. I tried following suggestions and answers from similar questions posted here on Stack Overflow, but I keep getting errors. Can anyone give me some pointers?
Here's my code.
require 'csv'
require 'json'
CSV.open("your_csv.csv", "w") do |csv|
JSON.parse(File.open("tojson.txt").read).each do |hash|
csv << hash.values
#csv.each { |line| line['New_header'] = line[0].to_i + line[1].to_i }
end
end
And here is the error I'm getting:
Anyone have any suggestions?
This is not how you add headers to a csv file. When you generate csv content, a header row is just a regular row. And should be generated as such. Example:
CSV.open("your_csv.csv", "w") do |csv|
csv << ['new_header', 'value1', 'value2'] # the headers
JSON.parse(File.open("tojson.txt").read).each do |hash|
row = [generate, values, for, headers, above]
csv << row
end
end
You don't have a #csv variable. You have a csv one.
I want to open a TSV (tab-separated-value) file, and save specific rows to a new CSV (comma-separated-value) file.
If the row contains 'NLD' in a field with the header 'Actor1Code', I want to save the row to a CSV; if not, I want to iterate to the next row. This is what I have so far, but apparently that is not enough:
require 'csv'
CSV.open("path/to.csv", "wb") do |csv| #csv to save to
CSV.open('data.txt', 'r', '\t').each do |row| #csv to scrape
if row['Actor1Code'] == 'NLD'
csv << row
else
end
end
end
Are you sure that you're calling CSV.open correctly? The documentation seems to suggest arguments are passed in as hashes:
CSV.open('data.txt', 'r', col_sep: "\t")
The error you're seeing is probably the result of '\t' being interpreted as a hash and referenced with [].
I am having trouble writing columns to a csv file with Ruby. Below is my snippet of code.
calc = numerator/denominator.to_f
data_out = "#{numerator}, #{denominator}, #{calc}"
File.open('cdhu3_X.csv','a+') do|hdr|
hdr << ["numerator","denominator","calculation\n"] #< column header
hdr << "#{data_out}\n"
end
The code adds the column headers to every line and I only need it at the top of each column of data. I have searched here and other places but can't find a clear answer to how its done.
Any help would be greatly appreciated.
I would recommend to use the CSV-library instead:
require 'csv'
CSV.open('test.csv','w',
:write_headers=> true,
:headers => ["numerator","denominator","calculation"] #< column header
) do|hdr|
1.upto(12){|numerator|
1.upto(12){ |denominator|
data_out = [numerator, denominator, numerator/denominator.to_f]
hdr << data_out
}
}
end
If you can't use the w option and you really need the a+ (e.g., the data isn't available all at once), then you could try the following trick:
require 'csv'
column_header = ["numerator","denominator","calculation"]
1.upto(12){|numerator|
1.upto(12){ |denominator|
CSV.open('test.csv','a+',
:write_headers=> true,
:headers => column_header
) do|hdr|
column_header = nil #No header after first insertion
data_out = [numerator, denominator, numerator/denominator.to_f]
hdr << data_out
end
}
}
The cleanest way to do this is to open the file once, in mode 'w', write the headers, and then write the data.
If there's some technical reason that can't do this (e.g., the data isn't available all at once), then you can use the IO#tell method on the file to return the current file position. When you open the file for appending, the position is set to the end of the file, so if the current file position is zero, then the file was newly created and has no headers:
File.open('cdhu3_X.csv', 'a+') do |hdr|
if hdr.tell() == 0 # file is empty, so write header
hdr << "numerator, denominator, calculation\n"
end
hdr << "#{data_out}\n"
end
Best way to handle csv file is to use Ruby's CSV module.
I had same problem after reading CSV code I came across this solution which i find most efficient.
headers = ['col1','col2','col3']
CSV.open(file_path, 'a+', {force_quotes: true}) do |csv|
csv << headers if csv.count.eql? 0 # csv.count method gives number of lines in file if zero insert headers
end
This works for me
headers = ["Reference Number", "Vendor Line Code"]
CSV.open(file_path, "wb") do |csv|
csv << headers
#vendor.vendor_items.each do |vi|
row_data = [vi.reference_number, vi.line_code]
csv << row_data
end
end
I'm trying to query a table, fetch all records, and save the result as a CSV file.
This is what I've done so far:
require 'OCI8'
conn = OCI8.new('scott','tiger','020')
file = File.open('output.csv','w') do |f|
conn.exec('select * from emp') do |e|
f.write log.join(',')
end
end
.. And while it does generate a CSV file, the problem is that all records get saved onto a single line. How can I put the data such that each record goes onto a new line ?
Well, you can use f.puts instead of f.write there, but I'd recommend you take a look at CSV module:
http://ruby-doc.org/stdlib/libdoc/csv/rdoc/index.html
outfile = File.open('csvout', 'wb')
CSV::Writer.generate(outfile) do |csv|
csv << ['c1', nil, '', '"', "\r\n", 'c2']
...
end
outfile.close
PS: Actually, there is another CSV library called FasterCSV, which became CSV in standard library in Ruby 1.9. But in general, any should be better than writing it yourself.