I'm trying to run the following code:
class RentLimit < ActiveRecord::Base
def self.load_data
rows = CSV.open("csvs/income_limits_2011_to_2015.csv").read
rows.shift
rows.each do |county, yr, date, _50pct_1br, _50pct_2br, _50pct_3br, _50pct_4br, _60pct_1br, _60pct_2br, _60pct_3br, _60pct_4br|
[50, 60].each do |ami|
[1, 2, 3, 4].each do |br|
r = new
r.county = county
r.state = "SC"
r.year = yr
r.effective_date = Date.parse(date)
r.pct_ami = ami
r.br = br
r.max_rent = self.send("_#{ami}pct_#{br}br".to_sym)
r.save
end#of brs
end# of amis
end# of rows
end
end
but am getting this error message when trying to run it:
NoMethodError: undefined method `_50pct_1br' for #<Class:0x007fe942ce3b18>
The send method isn't able to access those block parameters inside of the scope. Is there any way to give access to block parameters to send? If not, how else might I dynamically access block parameters?
How do I use send or its equivalent to access block parameters in Ruby?
This is much easier if you tell CSV.open what your column names are. It looks like your CSV file might have a header row that you're skipping with rows.shift, in which case you shouldn't skip it, and use the headers: true option. Then you can access each field by name with row["field_name"] or, in your case, row["_#{ami}pct_#{br}br"]:
CSV_PATH = "csvs/income_limits_2011_to_2015.csv"
DEFAULT_STATE = "SC"
def self.load_data
CSV.open(CSV_PATH, 'r', headers: true) do |csv|
csv.each do |row|
max_rent = row["_#{ami}pct_#{br}br"]
create(
county: row["county"],
state: DEFAULT_STATE,
year: row["yr"],
effective_date: Date.parse(row["date"]),
pct_ami: ami,
br: br,
max_rent: max_rent,
)
end
end
end
Note that I used CSV.open with a block to ensure that the file is closed after it's been read, which your original code wasn't doing. I also used create instead of new; ... save, since the latter is needlessly verbose.
If you're skipping the first row for some other reason, or you want to use field names other than those in the header row, you can set the options return_headers: false, headers: names, where names is an array of names, e.g.:
CSV_HEADERS = %w[
county yr date _50pct_1br _50pct_2br _50pct_3br _50pct_4br
_60pct_1br _60pct_2br _60pct_3br _60pct_4br
].freeze
def self.load_data
CSV.open(CSV_PATH, 'r', return_headers: false, headers: CSV_HEADERS) do |csv|
# ...
end
end
Finally, since some of your attributes are the same for every object created, I'd move those out of the loop:
def self.load_data
base_attrs = { state: DEFAULT_STATE, pct_ami: ami, br: br }
CSV.open(CSV_PATH, 'r', headers: true) do |csv|
csv.each do |row|
create(base_attrs.merge(
county: row["county"],
year: row["yr"],
effective_date: row["date"],
max_rent: row["_#{ami}pct_#{br}br"]
))
end
end
end
Related
I have .csv file with rows of which every row represents one call with certain duration, number etc. I need to create array of Call objects - every Call.new expects Hash of parameters, so it's easy - it just takes rows from CSV. But for some reason it doesn't work - when I invoke Call.new(raw_call) it's nil.
It's also impossible for me to see any output - I placed puts in various places in code (inside blocks etc) and it simply doesn't show anything. I obviously have another class - Call, which holds initialize for Call etc.
require 'csv'
class CSVCallParser
attr_accessor :io
def initialize(io)
self.io = io
end
NAMES = {
a: :date,
b: :service,
c: :phone_number,
d: :duration,
e: :unit,
f: :cost
}
def run
parse do |raw_call|
parse_call(raw_call)
end
end
private
def parse_call(raw_call)
NAMES.each_with_object({}) do |name, title, memo|
memo[name] = raw_call[title.to_s]
end
end
def parse(&block)
CSV.parse(io, headers: true, header_converters: :symbol, &block)
end
end
CSVCallParser.new(ARGV[0]).run
Small sample of my .csv file: headers and one row:
"a","b","c","d","e","f"
"01.09.2016 08:49","International","48627843111","0:29","","0,00"
I noticed a few things that isn't going as expected. In the parse_call method,
def parse_call(raw_call)
NAMES.each_with_object({}) do |name, title, memo|
memo[name] = raw_call[title.to_s]
end
end
I tried to print name, title, and memo. I expected to get :a, :date, and {}, but what I actually got was [:a,:date],{}, and nil.
Also, raw_call headers are :a,:b,:c..., not :date, :service..., so you should be using raw_call[name], and converting that to string will not help, since the key is a symbol in the raw_call.
So I modified the function to
def parse_call(raw_call)
NAMES.each_with_object({}) do |name_title, memo|
memo[name_title[1]] = raw_call[name_title[0]]
end
end
name_title[1] returns the title (:date, :service, etc)
name_title[0] returns the name (:a, :b, etc)
Also, in this method
def run
parse do |raw_call|
parse_call(raw_call)
end
end
You are not returning any results you get, so you are getting nil,
So, I changed it to
def run
res = []
parse do |raw_call|
res << parse_call(raw_call)
end
res
end
Now, if I output the line
p CSVCallParser.new(File.read("file1.csv")).run
I get (I added two more lines to the csv sample)
[{:date=>"01.09.2016 08:49", :service=>"International", :phone_number=>"48627843111", :duration=>"0:29", :unit=>"", :cost=>"0,00"},
{:date=>"02.09.2016 08:49", :service=>"International", :phone_number=>"48622454111", :duration=>"1:29", :unit=>"", :cost=>"0,00"},
{:date=>"03.09.2016 08:49", :service=>"Domestic", :phone_number=>"48627843111", :duration=>"0:29", :unit=>"", :cost=>"0,00"}]
If you want to run this program from the terminal like so
ruby csv_call_parser.rb calls.csv
(In this case, calls.csv is passed in as an argument to ARGV)
You can do so by modifying the last line of the ruby file.
p CSVCallParser.new(File.read(ARGV[0])).run
This will also return the array with hashes like before.
csv = CSV.parse(csv_text, :headers => true)
puts csv.map(&:to_h)
outputs:
[{a:1, b:1}, {a:2, b:2}]
I'm trying to access the csv data, which I recive if I make a http-request.
I don't save it to a csv file, so I save it to the variable.
Let's say this is the response I get, how can I print food?
uuid,event_id,category
12,1,food
13,2,cars
And this is the part of the ruby code which is important.
That's something I found, but it was originally used with a file, so it doesn't work.
csvdata = request(action,parameter)
#data_hash = {}
CSV.foreach(csvdata) do |row|
uuid, event_id, category = row
#data_hash[uuid] = event_id
end
Do I really need files for that or is there a easy way I can access the values?
Update
CSV.parse(csvdata,data = Hash.new) do |row|
puts data
end
The hash should look like this so I can use the column names
{"uuid" => "12,13", "event_id" => "323,3243", "category" => "food,cars"}
csv_data = Hash.new{|k, v| k[v] = []}
CSV.parse(csv_string, headers: true) do |row|
row.each{|k, v| csv_data[k] << v}
end
csv_data = Hash[csv_data.map{|k, v| [k, v.join(",")]}]
Update after specification Requested output.
Try this:
csvdata = request(action,parameter)
#data_hash = {}
CSV.parse(csvdata, headers: true) do |row|
#data_hash[row['uuid']] = row['event_id']
end
#data_hash
# => {"12"=>"1", "13"=>"2"}
When you parse a CSV, the seconds parameter (data = Hash.new in your code) is actually an options parameter. You can see the available options here:
:headers
If set to :first_row or true, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of ::parse_line with the same :col_sep, :row_sep, and :quote_char as this instance to produce an Array of headers. This setting causes #shift to return rows as CSV::Row objects instead of Arrays and #read to return CSV::Table objects instead of an Array of Arrays.
When passing headers: true - values are parsed into a Row object, where they can be accessed by name.
I have class Importer for my Rails application in which I use method to import CSV file.
def import
CSV.foreach(file.path, headers: true, encoding: "iso-8859-1") do |row|
mail = row["email"]||row["Email"]||row["e-mail"]||row["E-mail"]||row["mail"]||row["Mail"]
end
end
I set variable mail to perform actions inside a loop, I try to protect it from different names of mail column, but I have no clue how should break a loop and keep a code DRY in case when there is CSV without column with any of defined headers.
EDIT:
def import
header = nil
headers = CSV.open(file.path, encoding: "iso-8859-1") { |csv| csv.first }
headers.each { |e| header = e if e.downcase.gsub('-','')=~/^(|e)mail$/ }
if header != nil
CSV.foreach(file.path, headers: true, encoding: "iso-8859-1") do |row|
mail = row[header]
end
end
end
Solution to the problem
This should get you started. You'll need to change the regexp to match all of your cases.
def import
CSV.foreach(file.path, headers: true, encoding: "iso-8859-1") do |row|
if row.headers.none?{|e| e =~ /email/i}
raise "freak out"
end
end
end
I would also consider setting a variable has_email_headers that you can check since you don't want to have to scan every row's header since they are all the same.
According to the CSV documentation of Ruby 2.5.0 you can also use the return_headers:true to check for the header_row? later in a loop. Here's an example:
data = CSV.read("your.csv", headers: true, return_headers: true)
(0..(data.length-1)).each do |row|
if data[row].header_row? then
p "yes header!"
end
end
Once could also try the header_converters: [:downcase, :symbol] option, to just have to check fewer values (i.e., case insensitive), such as [:email, :mail]:
CSV.foreach(file.path, headers: true, header_converters: [:downcase, :symbol], encoding: "iso-8859-1") do |row|
puts 'You are missing the "email" header!' unless [:email, :mail].all? { |header| row.headers.include? header }
# refine/refactor as necessary...
# do rest of function...
end
Documentation on :header_converters.
Given the following CSV file, how would you remove all rows that contain the word 'true' in the column 'foo'?
Date,foo,bar
2014/10/31,true,derp
2014/10/31,false,derp
I have a working solution, however it requires making a secondary CSV object csv_no_foo
#csv = CSV.read(#csvfile, headers: true) #http://bit.ly/1mSlqfA
#headers = CSV.open(#csvfile,'r', :headers => true).read.headers
# Make a new CSV
#csv_no_foo = CSV.new(#headers)
#csv.each do |row|
# puts row[5]
if row[#headersHash['foo']] == 'false'
#csv_no_foo.add_row(row)
else
puts "not pushing row #{row}"
end
end
Ideally, I would just remove the offending row from the CSV like so:
...
if row[#headersHash['foo']] == 'false'
#csv.delete(true) #Doesn't work
...
Looking at the ruby documentation, it looks like the row class has a delete_if function. I'm confused on the syntax that that function requires. Is there a way to remove the row without making a new csv object?
http://ruby-doc.org/stdlib-1.9.2/libdoc/csv/rdoc/CSV/Row.html#method-i-each
You should be able to use CSV::Table#delete_if, but you need to use CSV::table instead of CSV::read, because the former will give you a CSV::Table object, whereas the latter results in an Array of Arrays. Be aware that this setting will also convert the headers to symbols.
table = CSV.table(#csvfile)
table.delete_if do |row|
row[:foo] == 'true'
end
File.open(#csvfile, 'w') do |f|
f.write(table.to_csv)
end
You might want to filter rows in a ruby manner:
require 'csv'
csv = CSV.parse(File.read(#csvfile), {
:col_sep => ",",
:headers => true
}
).collect { |item| item[:foo] != 'true' }
Hope it help.
I have a total of 7 columns with 6 columns initially filled out in a CSV file that I'm writing. When I try to populate the 7th column with data, I keep running into this error:
NoMethodError: You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.<<
Why do I keep running into this error? I should be able to write values into a 'nil'/blank space on a CSV file. Below is my code:
#Finds two records in the database
accounts = Account.find(1,2)
spammer_status = []
#Makes a call into the akismet API and populates spammer_status array with
#true or false values if the person is a spammer or not.
accounts.each do |accounts|
spammer_status << client.comment_check(accounts.last_seen_ip, nil,
:comment_author => accounts.name,
:comment_author_email => accounts.email,
:comment_author_url => accounts.url,
:comment_content => accounts.about)
end
#Changes the values from booleans to strings
spammer_status.map! { |value| value.to_s }
#Populates the initial 6 columns from the database values
CSV.open("/var/local/openhub/current/script/akismet_results.csv","w") do |row|
row << ["id",
"name",
"email",
"url",
"description",
"last seen ip",
"spammer status"]
accounts.each do |accounts|
row << [accounts.id,
accounts.name,
accounts.email,
accounts.url,
accounts.about,
accounts.last_seen_ip]
end
end
#Attempts to populate the 7th column, nil error
CSV.foreach("/var/local/openhub/current/script/akismet_results.csv", headers:true) do |row|
# binding.pry
row[6] << spammer_status.shift
end
What am I doing wrong here? The error is on the foreach part of the program. All I want to do is to iterate a row at a time and then add the string converted booleans to the correct column. Any help would be appreciated?
You are trying to << to a nil object. row[6] is nil. I believe you just want to assign a value to row[6] or if you want to use <<, just do it to row itself.
CSV.foreach("/var/local/openhub/current/script/akismet_results.csv", headers:true) do |row|
# binding.pry
row[6] = spammer_status.shift
# or you could do row << spammer_status.shift
end
I eventually figured this issue out by refactoring my code. I have to admit that the above variation was not written very well. By extracting the spam status as a method and then creating a method call when the CSV is written, I was able to make the functionality work.
def is_spam?(account)
spammer_status = client.comment_check(account.last_seen_ip, nil,
:comment_author => account.name,
:comment_author_email => account.email,
:comment_author_url => account.url,
:comment_content => account.about)
spammer_status.to_s
end
CSV.foreach("/var/local/openhub/current/script/akismet_results.csv","a", headers: true) do |row|
row << [account.id,
account.name,
account.email,
account.url,
account.about,
account.last_seen_ip,
is_spam?(account)]
end