no implicit conversion of nil into String error - ruby

I have a ruby script that will create two files by taking and merging values from another file.
#Resources
require 'rubygems'
require 'csv'
col_date = []
col_constant1 = []
col_constant2 = []
col_appYear = []
col_statsDesc = []
col_keyStats =[]
col_weeklyTotal=[]
weekly_total = []
fname = "finalStats.csv" #variable for capture file
finalStatsFile = File.open(fname, "w") #write to capture file
fname2 = "weeklyStats.csv"
weeklyStatsFile = File.open(fname2, "w")
CSV.foreach('compareData.csv', converters: :numeric) do |row|
weekly_total << row[0] - row[1]
weekly_total.each do |data|
data << weekly_total.shift
weeklyStatsFile.puts data
end
end
#retrieve stats from original document
CSV.foreach("autoCapture.csv") {|row| col_date << row[0]}
CSV.foreach("autoCapture.csv") {|row| col_constant1 << row[1]}
CSV.foreach("autoCapture.csv") {|row| col_appYear << row[2]}
CSV.foreach("autoCapture.csv") {|row| col_statsDesc << row[3]}
CSV.foreach("autoCapture.csv") {|row| col_constant2 << row[4]}
CSV.foreach("autoCapture.csv") {|row| col_keyStats << row[5]}
CSV.foreach("weeklyStats.csv") {|row| col_weeklyTotal << row[0]}
col_date.zip(col_constant1, col_appYear, col_statsDesc, col_constant2, col_keyStats, col_weeklyTotal).each do |col_date, col_constant1, col_appYear, col_statsDesc, col_constant2,
col_keyStats, col_weeklyTotal|
finalStatsFile.puts col_date+", "+col_constant1+", "+ col_appYear+", "+col_statsDesc+", "+col_constant2+", "+col_keyStats+", "+col_weeklyTotal
end
In one file I wish to subtract the values in row[1] from the values in row[0] to create a new 'weekly_total' value. I then output this array of values in a file called weeklyStats.csv. This will output a column of values fine.
However, I want to join these values with another set from another file (autoCapture.csv) and when I try to zip them as arrays so that they read across in corresponding rows I get the error:
weeklyStats_csv.rb:42:in `+': no implicit conversion of nil into String (TypeError)
from weeklyStats_csv.rb:42:in `block in <main>'
from weeklyStats_csv.rb:40:in `each'
from weeklyStats_csv.rb:40:in `<main>'
I gather this means that the array zip will not catch an exception if the one of the values is nil and therefore cannot convert to string. The problem is, I have tried converting weekly_total to string and array as I thought that it may be the problem (a mismatch of types) but I just dont where to go from here. Can anyone help?

One of (or more) values in string
finalStatsFile.puts col_date+", "+col_constant1+", "+ col_appYear+", "+col_statsDesc+", "+col_constant2+", "+col_keyStats+", "+col_weeklyTotal
became nil. To fix the output you should explicitly cast them to strings:
finalStatsFile.puts col_date.to_s + ", " +
col_constant1.to_s + ", " +
col_appYear.to_s + ", " +
col_statsDesc.to_s + ", " +
col_constant2.to_s + ", " +
col_keyStats.to_s + ", " +
col_weeklyTotal.to_s
BTW, the whole clause might be rewritten in more rubyish manner:
finalStatsFile.puts [ col_date,
col_constant1,
col_appYear,
col_statsDesc,
col_constant2,
col_keyStats,
col_weeklyTotal ].map(&:to_s).join(', ')

Related

Writing data into a CSV file by two different CSV files

So, i'm learning ruby and i've been stuck with this for a long time and i need some help.
I need to write to a CSV file from two different CSV files and i have the code to do it but in 2 different functions and i need the two files together in one.
So thats the code:
require 'CSV'
class Plantas <
Struct.new( :code)
end
class Especies <
Struct.new(:id, :type, :code, :name_es, :name_ca, :name_en, :latin_name, :customer_id )
end
def ecode
f_inECODE = File.open("pflname.csv", "r") #get EPPOCODE
f_out=CSV.open("plantas.csv", "w+", :headers => true) #outputfile
f_inECODE.each_line do |line|
fields = line.split(',')
newPlant = Plantas.new
newPlant.code = fields[2].tr_s('"', '').strip #eppocode
plant = [newPlant.code] #linies a imprimir
f_out << plant
end
end
def data
f_dataspices=File.open("spices.csv", "r")
f_out=CSV.open("plantas.csv", "w+", :headers => true) #outputfile
f_dataspices.each_line do |line|
fields = line.split(',')
newEspecies = Especies.new
newEspecies.id = fields[0].tr_s('"', '').strip
newEspecies.type = fields[1].tr_s('"', '').strip
newEspecies.code = fields[2].tr_s('"', '').strip
newEspecies.name_es = fields[3].tr_s('"', '').strip
newEspecies.name_ca = fields[4].tr_s('"', '').strip
newEspecies.name_en = fields[5].tr_s('"', '').strip
newEspecies.latin_name = fields[6].tr_s('"', '').strip
newEspecies.customer_id = fields[7].tr_s('"', '').strip
especia = [newEspecies.id,newEspecies.type,newEspecies.code,newEspecies.name_es,newEspecies.name_ca,newEspecies.name_en,newEspecies.latin_name,newEspecies.customer_id]
f_out << especia
end
end
data
ecode
And the wished output would be like this: species.csv + ecode.csv
"id","type","code","name_es","name_ca","name_en","latin_name","customer_id","ecode"
7205,"DunSpecies",NULL,"0","0","0","",11630,LEECO
7437,"DunSpecies",NULL,"0","Xicoira","0","",5273,LEE3O
7204,"DunSpecies",NULL,"0","0","0","",11630,L4ECO
And the actual is this:
"id","type","code","name_es","name_ca","name_en","latin_name","customer_id"
7205,"DunSpecies",NULL,"0","0","0","",11630
7437,"DunSpecies",NULL,"0","Xicoira","0","",5273
7204,"DunSpecies",NULL,"0","0","0","",11630
(without ecode)
From one side i have the ecode and from the other the whole data i just need to put it together.
I'd like to put all together in the same file (plantas.csv)
I did in two different functions because I don't know how to put all together with one foreach I would like to put all in the same function but I don't how doing it.
If someone could help me to get this code all in one function and writing the results in the same file I would be so grateful.
An example of the input of the file ecode.csv (in which I just want the ecode field) is this:
"""identifier"",""datatype"",""code"",""lang"",""langno"",""preferred"",""status"",""creation"",""modification"",""country"",""fullname"",""authority"",""shortname"""
"""N1952"",""PFL"",""LEECO"",""la"",""1"",""0"",""N"",""06/06/2000"",""09/03/2010"","""",""Leea coccinea non"",""Planchon"",""Leea coccinea non"""
"""N2974"",""PFL"",""LEECO"",""en"",""1"",""0"",""N"",""06/06/2000"",""21/02/2011"","""",""west Indian holly"","""",""West Indian holly"""
An example of the input of the file data.csv (in which I want all the fields) is this:
"id","type","code","name_es","name_ca","name_en","latin_name","customer_id"
7205,"DunSpecies",NULL,"0","0","0","",11630
7437,"DunSpecies",NULL,"0","Xicoira","0","",5273
And the way to link both files is by creating a third file in which i write everything in it!
At least this is my idea, i dont know if there is a simpler way to do it.
Thanks!
Cleaning up ecode.csv made it more challenging, but here is what I came up with:
In case, data.csv and ecode.csv are matched by row numbers:
require 'csv'
data = CSV.read('data.csv', headers: true).to_a
headers = data.shift << 'eppocode'
double_quoted_ecode = CSV.read('ecode.csv')
ecodeIO = StringIO.new
ecodeIO.puts double_quoted_ecode.to_a
ecodeIO.rewind
ecode = CSV.parse(ecodeIO, headers: true)
CSV.open('plantas.csv', 'w+') do |plantas|
plantas << headers
data.each.with_index do |row, idx|
planta = row + [ecode['code'][idx]]
plantas << planta
end
end
Using your example files, this gives you the following plantas.csv:
id,type,code,name_es,name_ca,name_en,latin_name,customer_id,eppocode
7205,DunSpecies,NULL,0,0,0,"",11630,LEECO
7437,DunSpecies,NULL,0,Xicoira,0,"",5273,LEECO
In case, entries are matched by data.csv's id and ecode.csv's identifier:
require 'csv'
data = CSV.read('data.csv', headers: true)
headers = data.headers << 'eppocode'
double_quoted_ecode = CSV.read('ecode.csv')
ecodeIO = StringIO.new
ecodeIO.puts double_quoted_ecode.to_a
ecodeIO.rewind
ecode = CSV.parse(ecodeIO, headers: true)
CSV.open('plantas.csv', 'w+') do |plantas|
plantas << headers
data.each do |row|
id = row['id']
ecode_row = ecode.find { |entry| entry['identifier'] == id } || {}
planta = row << ecode_row['code']
plantas << planta
end
end
I hope you find this helpful.
Data
Let's begin by creating the two CSV files. To make the results easier to follow I have arbitrarily removed some of the fields in each file, and changed one field value.
ecode.csv
ecode = '"""identifier"",""datatype"",""code"",""lang"",""langno"",""preferred"",""status"",""creation"",""modification"",""country"",""fullname"",""authority"",""shortname""" """N1952"",""PFL"",""LEECO"",""la"",""1"",""0"",""N"",""06/06/2000"",""09/03/2010"","""",""Leea coccinea non"",""Planchon"",""Leea coccinea non""" """N2974"",""PFL"",""LEEC1"",""en"",""1"",""0"",""N"",""06/06/2000"",""21/02/2011"","""",""west Indian holly"","""",""West Indian holly"""'
File.write('ecode.csv', ecode)
#=> 452
data.csv
data = '"id","type","code","customer_id"\n7205,"DunSpecies",NULL,11630\n7437,"DunSpecies",NULL,,5273'
File.write('data.csv', data)
#=> 90
Code
CSV.open('plantas.csv', 'w') do |csv_out|
converter = ->(s) { s.delete('"') }
epposcode = CSV.foreach('ecode.csv',
headers:true,
header_converters: [converter],
converters: [converter]
).map { |csv| csv["code"] }
headers = CSV.open('data.csv', &:readline) << 'epposcode'
csv_out << headers
CSV.foreach('data.csv', headers:true) do |row|
csv_out << (row << epposcode.shift)
end
end
#=> 90
Result
Let's see what was written.
puts File.read('plantas.csv')
id,type,code,customer_id,epposcode
7205,DunSpecies,NULL,11630,LEECO
7437,DunSpecies,NULL,,5273,LEEC1
Explanation
The structure we want is the following.
CSV.open('plantas.csv', 'w') do |csv_out|
epposcode = <array of 'code' field values from 'ecode.csv'>
headers = <headers from 'data.csv' to which 'epposcode' is appended>
csv_out << headers
CSV.foreach('data.csv', headers:true) do |row|
csv_out << <row of 'data.csv' to which an element of epposcode is appended>>
end
end
CSV::open is the main CSV method for writing files and CSV::foreach is generally my method-of-choice for reading CSV files. I could have instead written the following.
csv_out = CSV.open('plantas.csv', 'w')
epposcode = <array of 'code' field values from 'ecode.csv'>
headers = <headers from 'data.csv' to which 'epposcode' is appended>
csv_out << headers
CSV.foreach('data.csv', headers:true) do |row|
csv_out << <row of 'data.csv' to which an element of epposcode is appended>>
end
csv_out.close
but using a block is convenient because the file is closed before returning from the block.
It is convenient to use a converter for both the header fields and the row fields:
converter = ->(s) { s.delete('"') }
This is a proc (I've defined a lambda) that removes double quotes from strings. They are specified as two of foreach's optional arguments:
epposcode = CSV.foreach('ecode.csv',
headers:true,
header_converters: [converter],
converters: [converter]
)
Search for "Data Converters" in the CSV doc.
We invoke foreach without a block to return an enumerator, so it can be chained to map:
epposcode = CSV.foreach('ecode.csv',
headers:true,
header_converters: [converter],
converters: [converter]
).map { |csv| csv["code"] }
For the example,
epposcode
#=> ["LEECO", "LEEC1"]

Ruby: Write CSV line by line instead of whole file

I am using the follow code to write to a CSV file. It writes the whole file at once. I would like to write the CSV file line by line by amending the file. How can I adjust my code?
CSV.open("#{#app_path}/Data_#{#filename}", "w") do |csv|
data_array.each do |r|
csv << r
end
end
As I understand, the problem is not the csv file, but the size of the array (and that after each fail you have to rebuild the array).
My attempt at solving that would be to process the array in chunks like below:
def process_array_by_chunks(array, starting_index = 0, chunk_size)
return if array.empty?
current_index = starting_index
size = array.size
stop = false
while !stop do
puts "doing index: #{current_index}"
yield(array[current_index, chunk_size])
stop = true if current_index >= size
current_index = current_index + chunk_size
end
rescue StandardError => e
puts "failed at index: #{current_index}"
puts "data left to process: "
return array[current_index, size]
end
# call function with a block in which we write csv file
process_array_by_chunks(array, start, chunk_size) do | array|
CSV.open(path, "w") do |csv|
array.each do |r|
csv << r
end
end
end
if that blows up for some reason the function will return an array with all the items that were not yet processed.

How to use CSV.open and CSV.foreach methods to convert specific data in a csv file?

The Old.csv file contains these headers, "article_category_id", "articleID", "timestamp", "udid", but some of the values in those columns are strings. So, I am trying to convert them to integers and store in another CSV file, New.csv. This is my code:
require 'csv'
require 'time'
CSV.foreach('New.csv', "wb", :write_headers=> true, :headers =>["article_category_id", "articleID", "timestamp", "udid"]) do |csv|
CSV.open('Old.csv', :headers=>true) do |row|
csv['article_category_id']=row['article_category_id'].to_i
csv['articleID']=row['articleID'].to_i
csv['timestamp'] = row['timestamp'].to_time.to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
csv['udid'] = udids.index(row['udid']) + 1
csv<<row
end
end
But, I am getting the following error: in 'foreach': ruby wrong number of arguments (3 for 1..2) (ArgumentError).
When I change the foreach to open, I get the following error: undefined method '[]' for #<CSV:0x36e0298> (NoMethodError). Why is that? And how can I resolve it? Thanks.
CSV#foreach does not accept file access rights as second parameter:
CSV.open('New.csv', :headers=>true) do |csv|
CSV.foreach('Old.csv',
:write_headers => true,
:headers => ["article_category_id", "articleID", "timestamp", "udid"]
) do |row|
row['article_category_id'] = row['article_category_id'].to_i
...
csv << row
end
end
CSV#open should be placed before foreach. You are to iterate the old one and produce the new one. Inside the loop you should change row and than append it to the output.
You can refer my code:
require 'csv'
require 'time'
CSV.open('New.csv', "wb") do |csv|
csv << ["article_category_id", "articleID", "timestamp", "udid"]
CSV.foreach('Old.csv', :headers=>true) do |row|
array = []
article_category_id=row['article_category_id'].to_i
articleID=row['articleID'].to_i
timestamp = row['timestamp'].to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
udid = udids.index(row['udid']) + 1
array << [article_category_id, articleID, timestamp, udid]
csv<<array
end
end
The problem with Vinh answer is that at the end array variable is an array which has array inside.
So what is inserted indo CVS looks like
[[article_category_id, articleID, timestamp, udid]]
And that is why you get results in double quotes.
Please try something like this:
require 'csv'
require 'time'
CSV.open('New.csv', "wb") do |csv|
csv << ["article_category_id", "articleID", "timestamp", "udid"]
CSV.foreach('Old.csv', :headers=>true) do |row|
article_category_id = row['article_category_id'].to_i
articleID = row['articleID'].to_i
timestamp = row['timestamp'].to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
udid = udids.index(row['udid']) + 1
output_row = [article_category_id, articleID, timestamp, udid]
csv << output_row
end
end

Ruby, replacing values in a string array with strings/ints

I'm trying to replace an array with a new array, replacing values as I go along.
Here's the code:
minesarray = [['*','.','.','.'],['.','.','*','.'],['.','.','.','.']]
def pp_board(board)
puts Array.new(board[0].size*2+1, '-').join('')
board.each do |row|
puts "|" + row.join("|") + "|"
puts Array.new(row.size*2+1, '-').join('')
end
end
pp_board(minesarray)
count = 0
minesarray.map{ |row|
row.each { |col|
if minesarray[row][col] = '*'
minesarray[row][col]="a"
elif minesarray[row][col] = '.'
minesarray[row][col] = 0
end
}
}
I receive the following error:
mines2.rb:17:in '[]': can't convert Array into Integer (TypeError)
from mines2.rb:17:in 'block (2 levels) in (main)'
from mines2.rb:16:in 'each'
from mines2.rb:16:in 'block in (main)'
from mines2.rb:15:in 'map'
from mines2.rb:15:in '(main)'
The row and col is not the index of the array, but the element itself.
You could do like below:
minesarray = [['*','.','.','.'],['.','.','*','.'],['.','.','.','.']]
def pp_board(board)
puts Array.new(board[0].size*2+1, '-').join('')
board.each do |row|
puts "|" + row.join("|") + "|"
puts Array.new(row.size*2+1, '-').join('')
end
end
pp_board(minesarray)
minesarray = minesarray.map { |row|
row.map { |v|
if v == '*'
'a'
elsif v == '.'
'0'
end
}
}
pp_board(minesarray)
The problem is that row and col are not indexes, they are the actual elements of the array.
You also have a couple of other issues. "elif" should be elsif and you're using assignment (=) in your conditionals where you should be using an equality check (==).

How to generate xml_builder ruby code from an XML file

I've got an xml file. How could I generate xml_builder ruby code out of that file?
Notice - I'm sort of going backwards here (instead of generating xml, I'm generating ruby code).
Pretty formatting isn't a big deal - I can always run it through a formatter later.
It's sort of impossible, not unlike if you asked "how to generate Ruby script that outputs number 3", for the answer could be:
puts 3
or
puts 2+1
or
puts [1,2,3].count
etc.
So, one answer to your question would be:
xml = File.read('your.xml')
puts "puts <<EOF\n#{xml}\nEOF"
Anyway, if you would just want to generate Builder based script which just generates your XML node-for-node, I guess it would be easiest using XSLT. That's a language constructed exactly for such purposes - transforming XMLs.
Here's what I eventually came up with:
#!/usr/bin/env ruby
require "rexml/document"
filename = ARGV[0]
if filename
f = File.read(filename)
else
raise "Couldn't read file: `#{filename}'"
end
doc = REXML::Document.new(f)
def self.output_hash(attributes={})
count = attributes.size
str = ""
index = 0
attributes.each do |key, value|
if index == 0
str << " "
end
str << "#{key.inspect} => "
str << "#{value.inspect}"
if index + 1 < count
str << ", "
end
index += 1
end
str
end
def self.make_xml_builder(doc, str = "")
doc.each do |element|
if element.respond_to?(:name)
str << "xml.#{element.name}"
str << "#{output_hash(element.attributes)}"
if element.length > 0
str << " do \n"
make_xml_builder(element, str)
str << "end\n"
else
str << "\n"
end
elsif element.class == REXML::Text
string = element.to_s
string.gsub!("\n", "")
string.gsub!("\t", "")
if !string.empty?
str << "xml.text!(#{string.inspect})\n"
end
end
end
str
end
puts make_xml_builder(doc)
After generating that, I then formatted it in Emacs.

Resources