CSV writing Ruby Encoding Error - ruby

I am trying to write a UTF-8 character in a CSV file with csv library of Ruby. And I have got an error:
csv ruby write problem ASCII-8BIT (Encoding::CompatibilityError)
#create csv file
CSV.open(CSV_file,"wb",) do |csv|
csv << First_line
rows.each do |r|
csv << r.generate_array
end
end
That's the code where UTF-8 conflicts with ASCII-8BIT.
Example text that fails:
demás

Here is an example of CSV writing and reading with UTF-8:
fn="/tmp/f.csv"
require "csv"
d1=DATA.read.split(/\n/).map {|e| e.split}
CSV.open(fn, "w:utf-8") do |row|
d1.each { |dr| row << dr }
end
d2=[]
CSV.foreach(fn) do |row|
d2 << row
end
puts d1==d2
# true
__END__
privé face à face à un tête-à-tête
Face to face with one-on-one
demás
Without a more detailed example from you, I cannot help further.

Related

Is there a way to make Ruby CSV gem generate CSVs with Windows (CR LF) End Of Lines?

For some reason CSV gem is generating CSVs with Unix EOL (see screenshot) here:
https://www.dropbox.com/s/4re7tpp4pj9psov/ice_screenshot_20171230-162304.png?dl=0
Screenshot made in Notepad++ (View all Characters)
Code I use:
require 'csv'
all_the_things = []
all_the_things << ["item1.1","item1.2","item1.3"]
all_the_things << ["item2.1","item2.1","item2.1"]
all_the_things << ["item3.1","item3.1","item3.1"]
CSV.open("test.csv", "wb" ) do |row|
row << ["Column1", "Column2", "Column3"] #just headers
all_the_things.each do |data|
row << data
end
end
Is there a way to make it use Windows EOL (CR LF) instead of UNIX (LF) ones ?
I'm using Windows 10, and if I just output some lines to file using puts everything working just fine (albeight managing proper data structure without CSV gem is nightmare):
....
File.open("test.csv", "w") do |line|
myarray.each do |data|
line.puts data
end
end
Thank you in advance for any ideas and Happy New Year !
As it is clearly stated in the documentation, one might use the row_sep option to specify the row separator:
require 'csv'
# ⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓ here
CSV.open("/tmp/file.csv", "wb", row_sep: "\r\n") do |csv|
csv << %w|1 2 3 4|
csv << %w|a b c d|
end
Also, there is no “CSV gem,” it’s ruby standard library.

Read a csv in ruby with UTF-8 literal

i have this csv file
file data.csv:
data.csv: ASCII text
This file has ~10000 lines with some UTF-8 literal chars.
For example:
1388357672.209253000,48:a2:2d:78:84:10,\xe5\x87\xb6\xe5\xb7\xb4\xe5\xb7\xb4\xe8\x87\xad\xe7\x98\xaa\xe7\x98\xaa\xe7\x9a\x84\xe6\x80\xaa\xe5\x85\xbd\xe5\x87\xba
I iterate over this file in Ruby and save every line in my postgresql db
File.open(filename, "r").each_line do |line|
CSV.parse(line, encoding: 'UTF-8') do |row|
//Save to Postgresql
end
end
I have now the problem that the UTF-8 literal string is saved in the db and not the correct UTF-8 string. I can convert every line with echo -e "line" but this takes much time. Is ther a way that ruby can do this task?
Try this:
CSV.parse(line, encoding: 'UTF-8') do |row|
row = row.map do |elem|
elem.gsub(/\\x../) {|s| [s[2..-1].hex].pack("C")}.force_encoding("UTF-8")
end
//Save to Postgresql
end
Just put each cell in double quotes:
"\xe5\x87\xb6\xe5\xb7\xb4\xe5\xb7\xb4\xe8\x87\xad\xe7\x98\xaa\xe7\x98\xaa\xe7\x9a\x84\xe6\x80\xaa\xe5\x85\xbd\xe5\x87\xba"
=> "凶巴巴臭瘪瘪的怪兽出"

Problems with FasterCSV (Ruby)

I'm writing a program, that creates a csv-File. And I have a problem right at the beginning.
So, my code is
def create_csv
destfile = Rails.root.join("public", "reports", "statistic_csv#{id}.csv")
csv_string = FasterCSV.generate do |out|
out << ["row", "of", "CSV", "data"]
end
FasterCSV.open(destfile, "w") do |csv|
csv << csv_string
end
end
I thought, I will get 4 columns in the output file, smth like this row|of|csv|data. But what I get is "row,of,CSV,data" in one cell A1. How can i solve the Problem? Thanks in advance!
PS. I use ruby 1.8.7 and FasterCSV 1.5.5
You are encoding the CSV string twice. This should work:
def create_csv
destfile = Rails.root.join("public", "reports", "statistic_csv#{id}.csv")
FasterCSV.open(destfile, "wb") do |csv|
csv << ["row", "of", "CSV", "data"]
end
end
You can also specify a custom column separator:
FasterCSV.open(destfile, "wb", { :col_sep => "|" }) do |csv|
# ...
end
I presume you're opening this in Excel. Excel may not be detecting the file as a CSV file. Try importing the data into an excel workbook as opposed to opening the file in Excel.

Split output data using CSV in Ruby 1.9

I have a csv file that has 7000+ records that I process/manipulate and export to a new csv file. I have no issues doing that and everything works as expected.
I would like to change the process to where it breaks the output into multiple files. So instead of writing all 7000+ rows to the new csv file it would write the first 1000 rows to newexport1.csv and the next 1000 rows to newexport2.csv until it reaches the end of the data.
Is there an easy way to do this with CSV in Ruby 1.9?
My current write method:
CSV.open("#{PATH_TO_EXPORT_FILE}/newexport.csv", "w+", :col_sep => '|', :headers => true) do |f|
export_rows.each do |row|
f << row
The short answer is "no". You'll want to adjust your current code to split up the set and then dump each subset to a different file. This ought to be pretty close:
export_rows.each_slice(1000).with_index do |rows, idx|
CSV.open("#{PATH_TO_EXPORT_FILE}/newexport-#{idx.to_s}.csv", "w+", :col_sep => '|', :headers => true) do |f|
rows.each { |row| f << row }
end
end
Yes, there is.
It's embedded in Ruby 1.9
Check this link
To read:
CSV.foreach("path/to/file.csv") do |row|
# manipulate the content
end
To write:
CSV.open("path/to/file.csv", "wb") do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# something else
end
I think that you'll need to combine one inside the other.
FasterCSV is the standard CSV library since ruby 1.9, you can find a lot of example code in the examples folder:
https://github.com/JEG2/faster_csv/tree/master/examples
For the example code to work, you should change:
require "faster_csv"
for
require "csv"

Ruby 1.9.2 export a CSV string without generating a file

I just can't get the 'To a String' example under 'Writing' example in the documentation to work at all.
ruby -v returns:
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin10.8.0]
The example from the documentation I can't working is here:
csv_string = CSV.generate do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
end
The error I get is:
wrong number of arguments (0 for 1)
So it seems like I am missing an argument, in the documentation here it states:
This method wraps a String you provide, or an empty default String
But when I pass in a empty string, it gives me the following error:
No such file or directory -
I am not looking to generate a csv file, I just wanted to create a string of csv that I send as text to the user.
Here is code I know works against Ruby 1.9.2 with Rails 3.0.1
def export_csv(filename, header, rows)
require 'csv'
file = CSV.generate do |csv|
csv << header if not header.blank?
rows.map {|row| csv << row}
end
send_data file, :type => 'text/csv; charset=iso-8859-1; header=present', :disposition => "attachment;filename=#{filename}.csv"
end

Resources