I would like to know how to get the value from encoded data in Ruby.
In the sample below the value I would like to store in a variable is FTUV-38VO-FP. The puts method returns the value correctly:
Ruby Console
(byebug) row[0]
"\u0000F\u0000T\u0000U\u0000V\u0000-\u00003\u00008\u0000V\u0000O\u0000-\u0000F\u0000P\u0000"
(byebug) row[0].encoding
#<Encoding:ASCII-8BIT>
(byebug) puts row[0]
FTUV-38VO-FP
nil
Code
def self.import_csv(hd_wix_event_order_csv_upload_id)
csv_object = HdWixEventOrderCsvUpload.find(hd_wix_event_order_csv_upload_id)
CSV.parse(open(csv_object.csv_file.url,"r"), { :col_sep => "\t", :headers => :first_row}).each_with_index do |row,index|
puts "INDEX" + index.to_s
puts row
byebug
end
end
Thanks
Related
I've searched and haven't found a method for this particular conundrum. I have two CSV files of data that sometimes relate to the same thing. Here's an example:
CSV1 (500 lines):
date,reference,amount,type
10/13/2015,,1510.40,sale
10/13/2015,,312.90,sale
10/14/2015,,928.50,sale
10/15/2015,,820.25,sale
10/12/2015,,702.70,credit
CSV2 (20000 lines):
reference,date,amount
243534985,10/13/2015,312.90
345893745,10/15/2015,820.25
086234523,10/14/2015,928.50
458235832,10/13/2015,1510.40
My goal is to match the date and amount from CSV2 with the date and amount in CSV1, and write the reference from CSV2 to the reference column in the corresponding row.
This is a simplified view, as CSV2 actually contains many many more columns - these are just the relevant ones, so ideally I'd like to refer to them by header name or maybe index somehow?
Here's what I've attempted, but I'm a bit stuck.
require 'csv'
data1 = {}
data2 = {}
CSV.foreach("data1.csv", :headers => true, :header_converters => :symbol, :converters => :all) do |row|
data1[row.fields[0]] = Hash[row.headers[1..-1].zip(row.fields[1..-1])]
end
CSV.foreach("data2.csv", :headers => true, :header_converters => :symbol, :converters => :all) do |row|
data2[row.fields[0]] = Hash[row.headers[1..-1].zip(row.fields[1..-1])]
end
data1.each do |data1_row|
data2.each do |data2_row|
if (data1_row['comparitive'] == data2_row['comparitive'])
puts data1_row['identifier'] + data2_row['column_thats_important_and_wanted']
end
end
end
Result:
22:in `[]': no implicit conversion of String into Integer (TypeError)
I've also tried:
CSV.foreach('data2.csv') do |data2|
CSV.foreach('data1.csv') do |data1|
if (data1[3] == data2[4])
data1[1] << data2[1]
puts "Change made!"
else
puts "nothing changed."
end
end
end
This however did not match anything inside the if statement, so perhaps not the right approach?
The headers method should help you match columns--from there it's a matter of parsing and writing the modified data back out to a file.
Solved.
data1 = CSV.read('data1.csv')
data2 = CSV.read('data2.csv')
data2.each do |data2|
data1.each do |data1|
if (data1[5] == data2[4])
data1[1] = data2[1]
puts "Change made!"
puts data1
end
end
end
File.open('referenced.csv','w'){ |f| f << data1.map(&:to_csv).join("")}
The Old.csv file contains these headers, "article_category_id", "articleID", "timestamp", "udid", but some of the values in those columns are strings. So, I am trying to convert them to integers and store in another CSV file, New.csv. This is my code:
require 'csv'
require 'time'
CSV.foreach('New.csv', "wb", :write_headers=> true, :headers =>["article_category_id", "articleID", "timestamp", "udid"]) do |csv|
CSV.open('Old.csv', :headers=>true) do |row|
csv['article_category_id']=row['article_category_id'].to_i
csv['articleID']=row['articleID'].to_i
csv['timestamp'] = row['timestamp'].to_time.to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
csv['udid'] = udids.index(row['udid']) + 1
csv<<row
end
end
But, I am getting the following error: in 'foreach': ruby wrong number of arguments (3 for 1..2) (ArgumentError).
When I change the foreach to open, I get the following error: undefined method '[]' for #<CSV:0x36e0298> (NoMethodError). Why is that? And how can I resolve it? Thanks.
CSV#foreach does not accept file access rights as second parameter:
CSV.open('New.csv', :headers=>true) do |csv|
CSV.foreach('Old.csv',
:write_headers => true,
:headers => ["article_category_id", "articleID", "timestamp", "udid"]
) do |row|
row['article_category_id'] = row['article_category_id'].to_i
...
csv << row
end
end
CSV#open should be placed before foreach. You are to iterate the old one and produce the new one. Inside the loop you should change row and than append it to the output.
You can refer my code:
require 'csv'
require 'time'
CSV.open('New.csv', "wb") do |csv|
csv << ["article_category_id", "articleID", "timestamp", "udid"]
CSV.foreach('Old.csv', :headers=>true) do |row|
array = []
article_category_id=row['article_category_id'].to_i
articleID=row['articleID'].to_i
timestamp = row['timestamp'].to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
udid = udids.index(row['udid']) + 1
array << [article_category_id, articleID, timestamp, udid]
csv<<array
end
end
The problem with Vinh answer is that at the end array variable is an array which has array inside.
So what is inserted indo CVS looks like
[[article_category_id, articleID, timestamp, udid]]
And that is why you get results in double quotes.
Please try something like this:
require 'csv'
require 'time'
CSV.open('New.csv', "wb") do |csv|
csv << ["article_category_id", "articleID", "timestamp", "udid"]
CSV.foreach('Old.csv', :headers=>true) do |row|
article_category_id = row['article_category_id'].to_i
articleID = row['articleID'].to_i
timestamp = row['timestamp'].to_i unless row['timestamp'].nil?
unless udids.include?(row['udid'])
udids << row['udid']
end
udid = udids.index(row['udid']) + 1
output_row = [article_category_id, articleID, timestamp, udid]
csv << output_row
end
end
I have a file that looks like this:
[noahc:~/projects/wordsquares] master(+87/-50)* ± cat wordlist/test_word_list.txt
card
apple
joe
bird
card
dart
area
rear
birdbird
after
boat
swim
north
abbe
byes
beep
If I open up an irb session, I can do:
2.0.0p247 :001 > file = File.open('./wordlist/test_word_list.txt', 'r')
=> #<File:./wordlist/test_word_list.txt>
2.0.0p247 :002 > file.readlines
=> ["card\n", "apple\n", "joe\n", "bird\n", "card\n", "dart\n", "area\n", "rear\n", "birdbird\n", "after\n", "boat\n", "swim\n", "north\n", "abbe\n", "byes\n", "beep \n", "\n"]
But, now I have a class called WordSquareGenerator:
class WordSquareGenerator
require 'pry'
require 'pry-nav'
require './lib/word_list_builder.rb'
def initialize(n, file_location)
#size_of_square = n
#file = load_file(file_location)
#word_stem_hash = WordListBuilder.new(n, #file).word_stem_hash
#word_list = nil
end
def word_square_word_list
binding.pry
#file.each do |w|
binding.pry
#word_list ? break : solve_for_word_list([word])
end
binding.pry
end
def is_list_valid?(list)
(0..#size_of_square - 1).each do |n|
(0..#size_of_square - 2).each do |m|
return false if list[n][m] != list [m][n]
end
end
#generated_list = list unless #generated_list
end
def solve_for_word_list(word_array)
if word_array.length == 4
#word_list = word_array
elsif #word_list
else
next_words = #word_stem_hash[word_array.map{|w| w[word_array.length]}.join]
next_words.each do |word|
solve_for_word_list(word_array + [word])
end
end
end
private
def load_file(file_location)
File.open(file_location, 'r')
end
end
When I run the word_square_word_list method and hit the first binding.pry, I can do:
2.0.0 (#<WordSquareGenerator:0x007ff3f91c16b0>):0 > #file.readlines
=> []
and I get an empty array for readlines. How can it be that I'm getting two different results doing the same thing, except one is inside that class and the other isn't?
At WordListBuilder you are probably reading the file already and when the action gets back to WordSquareGenerator the file is already at it's end and there's nothing else to read. First, don't do what you're doing now, that is opening the file since this leaks the file handle (you're not closing it anywhere) and someone else reading the handle is causing your code to fail.
Here's how you could do it:
class WordSquareGenerator
require 'pry'
require 'pry-nav'
require './lib/word_list_builder.rb'
def initialize(n, file_location)
#size_of_square = n
#file_location = file_location
#word_stem_hash = WordListBuilder.new(n, file_location).word_stem_hash
#word_list = nil
end
def word_square_word_list
binding.pry
IO.foreach(#file_location) do |word|
binding.pry
#word_list ? break : solve_for_word_list([word])
end
binding.pry
end
def is_list_valid?(list)
(0..#size_of_square - 1).each do |n|
(0..#size_of_square - 2).each do |m|
return false if list[n][m] != list [m][n]
end
end
#generated_list = list unless #generated_list
end
def solve_for_word_list(word_array)
if word_array.length == 4
#word_list = word_array
elsif #word_list
else
next_words = #word_stem_hash[word_array.map{|w| w[word_array.length]}.join]
next_words.each do |word|
solve_for_word_list(word_array + [word])
end
end
end
end
And you would also need to update WordListBuilder to do the same. This also has the advantage of closing the file handle automatically for you so you don't have to care about closing it yourself.
Try running this before you read the lines:
#file.seek 0
You have probably already read the lines, and you have to seek back to the start of the file before you read them again.
Sample IRB session:
irb(main):001:0> f = File.new 'test.txt' # already existing file
=> #<File:test.txt>
irb(main):002:0> f.readlines
=> ["this", "is", "a", "test"]
irb(main):003:0> f.readlines
=> []
irb(main):004:0> f.seek 0
=> 0
irb(main):005:0> f.readlines
=> ["this", "is", "a", "test"]
You could even make it a method:
def readlines
#file.seek 0
#file.readlines
end
Also, make sure to close your file (#file.close)! You should only leave it open for as little time as possible. And you definitely should not leave it open for the whole program. If you don't want to worry about that, just store the lines in a variable instead of keeping the file:
#lines = File.open(file_location) {|f|
f.readlines
}
# you could also use the shortcut form
# #lines = File.open(file_location, &:readlines)
Edit: The issue is being unable to get the quantity of arrays within the hash, so it can be, x = amount of arrays. so it can be used as function.each_index{|x| code }
Trying to use the index of the amount of rows as a way of repeating an action X amount of times depending on how much data is pulled from a CSV file.
Terminal issued
=> Can't convert symbol to integer (TypeError)
Complete error:
=> ~/home/tests/Product.rb:30:in '[]' can't convert symbol into integer (TypeError) from ~home/tests/Product.rub:30:in 'getNumbRel'
from test.rb:36:in '<main>'
the function is that is performing the action is:
def getNumRel
if defined? #releaseHashTable
return #releaseHashTable[:releasename].length
else
#releaseHashTable = readReleaseCSV()
return #releaseHashTable[:releasename].length
end
end
The csv data pull is just a hash of arrays, nothing snazzy.
def readReleaseCSV()
$log.info("Method "+"#{self.class.name}"+"."+"#{__method__}"+" has started")
$log.debug("reading product csv file")
# Create a Hash where the default is an empty Array
result = Array.new
csvPath = "#{File.dirname(__FILE__)}"+"/../../data/addingProdRelProjIterTestSuite/releaseCSVdata.csv"
CSV.foreach(csvPath, :headers => true, :header_converters => :symbol) do |row|
row.each do |column, value|
if "#{column}" == "prodid"
proHash = Hash.new { |h, k| h[k] = [ ] }
proHash['relid'] << row[:relid]
proHash['releasename'] << row[:releasename]
proHash['inheritcomponents'] << row[:inheritcomponents]
productId = Integer(value)
if result[productId] == nil
result[productId] = Array.new
end
result[productId][result[productId].length] = proHash
end
end
end
$log.info("Method "+"#{self.class.name}"+"."+"#{__method__}"+" has finished")
#productReleaseArr = result
end
Sorry, couldn't resist, cleaned up your method.
# empty brackets unnecessary, no uppercase in method names
def read_release_csv
# you don't need + here
$log.info("Method #{self.class.name}.#{__method__} has started")
$log.debug("reading product csv file")
# you're returning this array. It is not a hash. [] is preferred over Array.new
result = []
csvPath = "#{File.dirname(__FILE__)}/../../data/addingProdRelProjIterTestSuite/releaseCSVdata.csv"
CSV.foreach(csvPath, :headers => true, :header_converters => :symbol) do |row|
row.each do |column, value|
# to_s is preferred
if column.to_s == "prodid"
proHash = Hash.new { |h, k| h[k] = [ ] }
proHash['relid'] << row[:relid]
proHash['releasename'] << row[:releasename]
proHash['inheritcomponents'] << row[:inheritcomponents]
# to_i is preferred
productId = value.to_i
# this notation is preferred
result[productId] ||= []
# this is identical to what you did and more readable
result[productId] << proHash
end
end
end
$log.info("Method #{self.class.name}.#{__method__} has finished")
#productReleaseArr = result
end
You haven't given much to go on, but it appears that #releaseHashTable contains an Array, not a Hash.
Update: Based on the implementation you posted, you can see that productId is an integer and that the return value of readReleaseCSV() is an array.
In order to get the releasename you want, you have to do this:
#releaseHashTable[productId][n][:releasename]
where productId and n are integers. Either you'll have to specify them specifically, or (if you don't know n) you'll have to introduce a loop to collect all the releasenames for all the products of a particular productId.
This is what Mark Thomas meant:
> a = [1,2,3] # => [1, 2, 3]
> a[:sym]
TypeError: can't convert Symbol into Integer
# here starts the backstrace
from (irb):2:in `[]'
from (irb):2
An Array is only accessible by an index like so a[1] this fetches the second element from the array
Your return a an array and thats why your code fails:
#....
result = Array.new
#....
#productReleaseArr = result
# and then later on you call
#releaseHashTable = readReleaseCSV()
#releaseHashTable[:releasename] # which gives you TypeError: can't convert Symbol into Integer
I'm writing an app that revolves around getting sets of numerical data from a file. However, since the data is acquired in string form, I have to convert it to floats, which is where the fun starts. The relevant section of my code is as shown (lines 65-73):
ft = []
puts "File Name: #{ARGV[0]}"
File.open(ARGV[0], "r") do |file|
file.each_line do |line|
ft << line.scan(/\d+/)
end
end
ft.collect! {|i| i.to_f}
This works just fine in irb, that is, the last line changes the array to floats.
irb(main):001:0> ft = ["10", "23", "45"]
=> ["10", "23", "45"]
irb(main):002:0> ft.collect! {|i| i.to_f}
=> [10.0, 23.0, 45.0]
However when I run my application I get this error:
ruby-statistics.rb:73:in `block in <main>': undefined method `to_f' for #<Array:
0x50832c> (NoMethodError)
from ruby-statistics.rb:73:in `collect!'
from ruby-statistics.rb:73:in `<main>'
Any help with this would be appreciated.
line.scan returns an array, so you are inserting an array into an array. The easiest thing to do would be to call flatten on the array before you convert the strings to floats.
ft = []
puts "File Name: #{ARGV[0]}"
File.open(ARGV[0], "r") do |file|
file.each_line do |line|
ft << line.scan(/\d+/)
end
end
ft = ft.flatten.collect { |i| i.to_f }
You should have a look at the format of "ft" after reading the file.
Each line gets stored in another array so in fact "ft" looks something like this:
[["1","2"],["3","4"]]
So you have to do something like this:
ft = []
puts "File Name: #{ARGV[0]}"
File.open(ARGV[0], "r") do |file|
file.each_line do |line|
ft << line.scan(/\d+/)
end
end
tmp = []
ft.each do |line|
line.each do |number|
tmp << number.to_f
end
end
puts tmp
This is just a guess since I don't know what your file format looks like.
Edit:
Here as a one-liner:
ft.flatten!.collect! { |i| i.to_f }