Wrapping output of an array to CSV conversion in quotations in Ruby - ruby

What I'm wanting to find out is how to have every entry passed from the array to the CSV at the end of the program be wrapped by " "'s to allow Excel to read it correctly. I know this needs to be done before or during the "push" at line 34, but doing "streets.push('"'+street_name+'"')" results in every entry being surrounded by THREE quotation marks, which doesn't make much sense to me.
#!ruby.exe
require 'csv'
puts "Please enter a file name:" #user input file name (must be in same
folder as this file)
file = gets.chomp
begin
File.open(file, 'r')
rescue
print "Failed to open #{file}\n"
exit
end #makes sure that the file exists, if it does not it posts an error
data_file = File.new(file)
data = [] #initializes array for addresses from .csv
counter=0 #set counter up to allow for different sized files to be used
without issue
CSV.foreach(data_file, headers: true) do |row|
data << row.to_hash
counter+=1
end #goes through .csv one line ar a time
data.reject(&:empty?)
puts "Which column do you want to parse?"
column = gets.chomp
i=0
streets = []
while (i<counter)
address = data[i][column]
street_name = address.gsub(/^((\d[a-zA-Z])|[^a-zA-Z])*/, '')
streets.push(street_name)
i+=1
end
streets.reject(&:empty?)
puts "What do you want the output to be called?"
new_file = gets.chomp
CSV.open(new_file, "w", :write_headers=> true, :headers => [column]) do |hdr|
hdr << streets
end

You can pass the :force_quotes option to the CSV library to have it quote everything in the csv for you:
base_options = {headers: ['first,col', 'second column'], write_headers: true}
options = [{}, {force_quotes: true}]
data = [
['a', 'b'],
['c', 'd'],
['e', 'f']
]
options.each do |option|
result = CSV.generate(base_options.merge(option)) do |csv|
data.each do |datum|
csv << datum
end
end
puts "#{option}:\n#{result}"
end
For instance, in this small script, by default, the only thing that gets quoted is the first column header because it contains a comma. By passing in force_quotes: true, in the second pass though, everything gets quoted.
Output:
{}:
"first,col",second column
a,b
c,d
e,f
{:force_quotes=>true}:
"first,col","second column"
"a","b"
"c","d"
"e","f"

You can use map to process the array before putting it in csv.
streets.map!{|s| '"'+s+'"'}

Related

manipulating csv with ruby

I have a CSV from which I've removed the irrelevant data.
Now I need to split "Name and surname" into 2 columns by space but ignoring a 3rd column in case there are 3 names, then invert the order of the columns "Name and surname" and "Phone" (phone first) and then put them into a file ignoring the headers. I've never actually learned Ruby but I've played with Python 10 years ago. Can you help me? This is what I was able to do until now:
E.g.
require 'csv'
csv_table = CSV.read(ARGV[0], :headers => true)
keep = ["Name and surname", "Phone", "Email"]
new_csv_table = csv_table.by_col!.delete_if do |column_name,column_values|
!keep.include? column_name
end
new_csv_table.to_csv
Begin by creating a CSV file.
str =<<~END
Name and surname,Phone,Email
John Doe,250-256-3145,John#Doe.com
Marsha Magpie,250-256-3154,Marsha#Magpie.com
END
File.write('t_in.csv', str)
#=> 109
Initially, let's read the file, add two columns, "Name" and "Surname", and optionally delete the column, "Name and surname", without regard to column order.
First read the file into a CSV::Table object.
require 'csv'
tbl = CSV.read('t_in.csv', headers: true)
#=> #<CSV::Table mode:col_or_row row_count:3>
Add the new columns.
tbl.each do |row|
row["Name"], row["Surname"] = row["Name and surname"].split
end
#=> #<CSV::Table mode:col_or_row row_count:3>
Note that if row["Name and surname"] had equaled “John Paul Jones”, we would have obtained row["Name"] #=> “John” and row["Surname"] #=> “Paul”.
If the column "Name and surname" is no longer required we can delete it.
tbl.delete("Name and surname")
#=> ["John Doe", "Marsha Magpie"]
Write tbl to a new CSV file.
CSV.open('t_out.csv', "w") do |csv|
csv << tbl.headers
tbl.each { |row| csv << row }
end
#=> #<CSV::Table mode:col_or_row row_count:3>
Let's see what was written.
puts File.read('t_out.csv')
displays
Phone,Email,Name,Surname
250-256-3145,John#Doe.com,John,Doe
250-256-3154,Marsha#Magpie.com,Marsha,Magpie
Now let's rearrange the order of the columns.
header_order = ["Phone", "Name", "Surname", "Email"]
CSV.open('t_out.csv', "w") do |csv|
csv << header_order
tbl.each { |row| csv << header_order.map { |header| row[header] } }
end
puts File.read('t_out.csv')
#=> #<CSV::Table mode:col_or_row row_count:3>
displays
Phone,Name,Surname,Email
250-256-3145,John,Doe,John#Doe.com
250-256-3154,Marsha,Magpie,Marsha#Magpie.com

How to read a .txt file and compare it to user input, to see if it matches in Ruby?

I am trying to read a .txt file and see if the user input provided (a barcode) is found in the file, if it isn't I want to tell them and then not include it in the receipt file (which'll be outputted at the end).
(Product is an array [barcode (String), quantity (String)]
while line = file.gets
line = line.split(",")
products.each do |product|
if line[0] != product[0]
puts "Your item #{product[0]} could not be found in the stockfile. It will not be included in the receipt."
end
if line[0] == product[0]
receipt << [line[0],line[1],line[2]]
end
Stockfile:
12636723,BenQ XL2411Z Monitor,29.99
12345670,Razer Deathadder Mouse,4.49
77766236,Lenovo Thinkpad X1 Carbon Laptop,65.00
Realised now that logic is flawed, and that if I were to do the above it'd print out not found a bunch of times. I don't know if I can ask for help with logic.
Don't reïnvent the wheel, the csv gem is made for reading such files, I do it here from the DATA portion of the file so that you can run this sample.
Products could be a struct or class, for shortness I use a Hash here.
require "csv"
products = [{name: 'milk', barcode: 1},{name: 'butter',barcode: 2},{name: 'flour', barcode: 3}]
receipt = []
stock = File.read('stock.csv', :col_sep => ",", :headers => true)
stock = CSV.parse(DATA, :col_sep => ",", :headers => true)
products.each do |product|
if stock.find {|row| row['Name'] == product[:name]}
receipt << product.to_a
else
puts "Your item #{product[:name]} could not be found in the stockfile. It will not be included in the receipt."
end
end
__END__
Name, Quantifier, Amount
milk, liter, 2
butter, gram, 250
Which gives
# Your item flour could not be found in the stockfile. It will not be included in the receipt.
# receipt: #[[[:name, "milk"], [:barcode, 1]], [[:name, "butter"], [:barcode, 2]]]

Ruby CSV: Comparison of columns (from two csvs), write new column in one

I've searched and haven't found a method for this particular conundrum. I have two CSV files of data that sometimes relate to the same thing. Here's an example:
CSV1 (500 lines):
date,reference,amount,type
10/13/2015,,1510.40,sale
10/13/2015,,312.90,sale
10/14/2015,,928.50,sale
10/15/2015,,820.25,sale
10/12/2015,,702.70,credit
CSV2 (20000 lines):
reference,date,amount
243534985,10/13/2015,312.90
345893745,10/15/2015,820.25
086234523,10/14/2015,928.50
458235832,10/13/2015,1510.40
My goal is to match the date and amount from CSV2 with the date and amount in CSV1, and write the reference from CSV2 to the reference column in the corresponding row.
This is a simplified view, as CSV2 actually contains many many more columns - these are just the relevant ones, so ideally I'd like to refer to them by header name or maybe index somehow?
Here's what I've attempted, but I'm a bit stuck.
require 'csv'
data1 = {}
data2 = {}
CSV.foreach("data1.csv", :headers => true, :header_converters => :symbol, :converters => :all) do |row|
data1[row.fields[0]] = Hash[row.headers[1..-1].zip(row.fields[1..-1])]
end
CSV.foreach("data2.csv", :headers => true, :header_converters => :symbol, :converters => :all) do |row|
data2[row.fields[0]] = Hash[row.headers[1..-1].zip(row.fields[1..-1])]
end
data1.each do |data1_row|
data2.each do |data2_row|
if (data1_row['comparitive'] == data2_row['comparitive'])
puts data1_row['identifier'] + data2_row['column_thats_important_and_wanted']
end
end
end
Result:
22:in `[]': no implicit conversion of String into Integer (TypeError)
I've also tried:
CSV.foreach('data2.csv') do |data2|
CSV.foreach('data1.csv') do |data1|
if (data1[3] == data2[4])
data1[1] << data2[1]
puts "Change made!"
else
puts "nothing changed."
end
end
end
This however did not match anything inside the if statement, so perhaps not the right approach?
The headers method should help you match columns--from there it's a matter of parsing and writing the modified data back out to a file.
Solved.
data1 = CSV.read('data1.csv')
data2 = CSV.read('data2.csv')
data2.each do |data2|
data1.each do |data1|
if (data1[5] == data2[4])
data1[1] = data2[1]
puts "Change made!"
puts data1
end
end
end
File.open('referenced.csv','w'){ |f| f << data1.map(&:to_csv).join("")}

How to use CSV.open and CSV.foreach methods to convert specific data in a csv file?

The Old.csv file contains these headers, "article_category_id", "articleID", "timestamp", "udid", but some of the values in those columns are strings. So, I am trying to convert them to integers and store in another CSV file, New.csv. This is my code:
require 'csv'
require 'time'
CSV.foreach('New.csv', "wb", :write_headers=> true, :headers =>["article_category_id", "articleID", "timestamp", "udid"]) do |csv|
CSV.open('Old.csv', :headers=>true) do |row|
csv['article_category_id']=row['article_category_id'].to_i
csv['articleID']=row['articleID'].to_i
csv['timestamp'] = row['timestamp'].to_time.to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
csv['udid'] = udids.index(row['udid']) + 1
csv<<row
end
end
But, I am getting the following error: in 'foreach': ruby wrong number of arguments (3 for 1..2) (ArgumentError).
When I change the foreach to open, I get the following error: undefined method '[]' for #<CSV:0x36e0298> (NoMethodError). Why is that? And how can I resolve it? Thanks.
CSV#foreach does not accept file access rights as second parameter:
CSV.open('New.csv', :headers=>true) do |csv|
CSV.foreach('Old.csv',
:write_headers => true,
:headers => ["article_category_id", "articleID", "timestamp", "udid"]
) do |row|
row['article_category_id'] = row['article_category_id'].to_i
...
csv << row
end
end
CSV#open should be placed before foreach. You are to iterate the old one and produce the new one. Inside the loop you should change row and than append it to the output.
You can refer my code:
require 'csv'
require 'time'
CSV.open('New.csv', "wb") do |csv|
csv << ["article_category_id", "articleID", "timestamp", "udid"]
CSV.foreach('Old.csv', :headers=>true) do |row|
array = []
article_category_id=row['article_category_id'].to_i
articleID=row['articleID'].to_i
timestamp = row['timestamp'].to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
udid = udids.index(row['udid']) + 1
array << [article_category_id, articleID, timestamp, udid]
csv<<array
end
end
The problem with Vinh answer is that at the end array variable is an array which has array inside.
So what is inserted indo CVS looks like
[[article_category_id, articleID, timestamp, udid]]
And that is why you get results in double quotes.
Please try something like this:
require 'csv'
require 'time'
CSV.open('New.csv', "wb") do |csv|
csv << ["article_category_id", "articleID", "timestamp", "udid"]
CSV.foreach('Old.csv', :headers=>true) do |row|
article_category_id = row['article_category_id'].to_i
articleID = row['articleID'].to_i
timestamp = row['timestamp'].to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
udid = udids.index(row['udid']) + 1
output_row = [article_category_id, articleID, timestamp, udid]
csv << output_row
end
end

How to Get Specific Row Value From CSV?

I have a vertical CSV file that looks like this:
name,value
case,"123Case0001"
custodian,"Doe_John"
PDate,"10/30/2013"
I can read the file like this:
CSV.foreach("#{batch_File_Dir_cdata}", :quote_char => '"', :col_sep =>',', :row_sep =>:auto, :headers => true) do |record|
ev_info = record[0]
ev_val = record[1]
The problem is, I need to get a specific ev_val for just one specific ev_info. I could potentially use the row number, but foresight tells me that this could change. What will be the same is the name of information. I want to find the row with the specific information name and get that value.
When I do the foreach, it gets that value and then goes past it and leaves me with an empty variable, because it went on to the other rows.
Can anyone help?
You've got a lot of choices, but the easiest is to assign to a variable based on the contents, as in:
ev_info = record[0]
ev_val = record[1] if ev_info='special name'
Note, though, that you need to define whatever variable you are assigning to outside of the block as it will otherwise be created as a local variable and be inaccessible to you afterwards.
Alternatively, you can read in the entire array and then select the record you're interested in with index or select.
I'd do it something like:
require 'pp'
require 'csv'
ROWS_IN_RECORD = 4
data = []
File.open('test.dat', 'r') do |fi|
loop do
record = {}
ROWS_IN_RECORD.times do
row = fi.readline.parse_csv
record[row.first] = row.last
end
data << record
break if fi.eof?
end
end
pp data
Running that outputs:
[{"name"=>"value",
"case"=>"123Case0001",
"custodian"=>"Doe_John",
"PDate"=>"10/30/2013"},
{"name"=>"value_2",
"case"=>"123Case0001 2",
"custodian"=>"Doe_John 2",
"PDate"=>"10/30/2013 2"}]
It returns an array of hashes, so each hash is the record you'd normally get from CSV if the file was a normal CSV file.
There are other ways of breaking down the input file into logical groups, but this is scalable, with a minor change, to work on huge data files. For a huge file just process each record at the end of the loop instead of pushing it onto the data variable.
I got it to work. I original had the following:
CSV.foreach("#{batch_File_Dir_cdata}", :quote_char => '"', :col_sep =>',', :row_sep =>:auto, :headers => true) do |record|
ev_info = record[0]
c_val = record[1]
case when ev_info == "Custodian"
cust = cval
end
end
puts cust
what I needed to do was this:
CSV.foreach("#{batch_File_Dir_cdata}", :quote_char => '"', :col_sep =>',', :row_sep =>:auto, :headers => true) do |record|
ev_info = record[0]
case when ev_info == "Custodian"
c_val = record[1]
end
end
puts c_val

Resources