I have an Array of Arrays (imported from CSV file):
[[title1],[title2],[title3],[title4],[title5]],
[[song1],[author1],[bpm1],[key1],[energy1]],
...
[[song100],[author100],[bpm100],[key100],[energy100]].
and would like to convert it to an Array of Hashes like:
[{"title1"=>"song1","title2"=>"author1","title3"=>"bpm1","title4"=>"key1","title5"=>"energy1"}],
...
[{"title1"=>"song100","title2"=>"author100","title3"=>"bpm100","title4"=>"key100","title5"=>"energy100"}].
I used the code below but it doesn't work:
require 'csv'
csv=CSV.read('library.csv')
array_hash=[]
hash={}
for i in 1..(csv.size)
hash1={}
for n in 0..4
a=csv[0][n]
b=csv[i][n]
hash1[a]=b
hash.merge!(hash1)
end
array_hash.push(hash)
end
But I get:
> NoMethodError: undefined method `[]' for nil:NilClass from
> (irb):149:in `block (2 levels) in irb_binding' from (irb):146:in
> `each' from (irb):146:in `block in irb_binding' from (irb):143:in
> `each' from (irb):143 from
> /Users/user/.rvm/rubies/ruby-2.0.0-p481/bin/irb:12:in `<main>'
What is wrong with this?
How to do the same using .each ?
The ruby CSV library has a to_hash function on CSV::Row, so you can do as below instead:
require 'csv'
rows = CSV.read('library.csv', headers: true).map(&:to_hash) #rows would return a list of hashes
Related
When I am using Kiba ELT, I have followed the tutorials in YouTube as well as the tutorials provided by the owner. Yet, I am getting this error:
bitlasoft#Bitlasoft-TS-22:~/test01$ bundle exec kiba movies.etl
{
"title: Blade Runner" => "title: Minority Report"
}
/home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/themoviedb-1.0.1/lib/themoviedb/api.rb:37:in `set_response': Tmdb::InvalidApiKeyError (Tmdb::InvalidApiKeyError)
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/themoviedb-1.0.1/lib/themoviedb/search.rb:75:in `fetch_response'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/themoviedb-1.0.1/lib/themoviedb/search.rb:60:in `fetch'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/themoviedb-1.0.1/lib/themoviedb/resource.rb:41:in `search'
from /home/bitlasoft/test01/common.rb:30:in `process'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:35:in `block (3 levels) in process_rows'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:34:in `each'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:34:in `block (2 levels) in process_rows'
from /home/bitlasoft/test01/common.rb:12:in `block in each'
from /home/bitlasoft/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/csv.rb:1739:in `each'
from /home/bitlasoft/test01/common.rb:11:in `each'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:33:in `block in process_rows'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:32:in `each'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:32:in `process_rows'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:13:in `run'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/cli.rb:13:in `run'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/bin/kiba:5:in `<top (required)>'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/bin/kiba:23:in `load'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/bin/kiba:23:in `<main>'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/bin/ruby_executable_hooks:15:in `eval'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/bin/ruby_executable_hooks:15:in `<main>'
bitlasoft#Bitlasoft-TS-22:~/test01$
Here is my movies.etl and common.rb config:
require_relative 'common'
api_key = IO.read('.themoviedb')
source CSVSource, filename: 'movies.csv'
limit ENV['LIMIT']
show_me!
transform MovieDBlookup,
api_key: api_key,
title_field: 'title'
show_me!
require 'csv'
require 'awesome_print'
class CSVSource
def initialize(filename:)
#filename = filename
end
def each
csv = CSV.open(#filename, headers: true)
csv.each do |row|
yield(row.to_hash)
end
csv.close
end
end
require 'themoviedb'
class MovieDBlookup
def initialize(api_key:, title_field:)
#title_field = title_field
Tmdb::Api.key(api_key)
end
def process(row)
movie = Tmdb::Movie.find(row[#title_field]).first
row[:vote_average] = movie.vote_average
row[:vote_count] = movie.vote_count
row
end
end
def show_me!
transform do |row|
ap row
row
end
end
def limit(x)
x = Integer(x || -1)
return if x == -1
transform do |row|
#counter ||= 0
#counter += 1
abort("stopping....")if #counter >= x
row
end
end
(Kiba owner here) - the error you get isn't Kiba specific ; it looks like the API key you provided for themoviedb is invalid. Did you sign-up for the movie db here, and is the API key provided copy-pasted into the file you are loading (.themoviedb)?
One problem I could think of would be that you'd have a line ending character in that file (carriage return / line feed), in which case maybe just calling api_key = IO.read('.themoviedb').strip could help.
Again, this isn't Kiba specific, but hope this helps!
In application.rb I tried to read a YAML file:
config.before_configuration do
env_file = File.join(Rails.root, 'config', 'local_env.yml')
YAML.load(File.open(env_file)).each do |key, value|
ENV[key.to_s] = value
end if File.exists?(env_file)
end
but, I get this error:
/var/www/config/application.rb:26:in `block in <class:Application>': undefined method `each' for #<String:0x00000007afb7f0> (NoMethodError)
from /usr/local/rvm/gems/ruby-2.2.1/gems/activesupport-4.1.6/lib/active_support/lazy_load_hooks.rb:36:in `call'
from /usr/local/rvm/gems/ruby-2.2.1/gems/activesupport-4.1.6/lib/active_support/lazy_load_hooks.rb:36:in `execute_hook'
from /usr/local/rvm/gems/ruby-2.2.1/gems/activesupport-4.1.6/lib/active_support/lazy_load_hooks.rb:28:in `block in on_load'
from /usr/local/rvm/gems/ruby-2.2.1/gems/activesupport-4.1.6/lib/active_support/lazy_load_hooks.rb:27:in `each'
from /usr/local/rvm/gems/ruby-2.2.1/gems/activesupport-4.1.6/lib/active_support/lazy_load_hooks.rb:27:in `on_load'
from /usr/local/rvm/gems/ruby-2.2.1/gems/railties-4.1.6/lib/rails/railtie/configuration.rb:53:in `before_configuration'
from /var/www/config/application.rb:24:in `<class:Application>'
Any idea?
EDIT
My yaml file:
LOAD_JS_FROM_AMAZON:no
RACK_ENV:production
S3_BUCKET_NAME:bucket_name
S3_CMS_BUCKET_NAME:cms_bucket_name
YAML.load(File.open(env_file))
Your YAML is returning a String not a Hash
You need spaces between colon and value:
LOAD_JS_FROM_AMAZON: no
RACK_ENV: production
S3_BUCKET_NAME: bucket_name
S3_CMS_BUCKET_NAME: cms_bucket_name
I have series of zip files under #workingdir, and am trying to unzip the files that match #Regexp, and print the lines from them.
require 'zip/zip'
#workingdir = '/my/dir/structure/*.zip'
#Regexp = '/yup:maybe.*nope/i'
Dir.glob(#workingdir) do |zips|
Zip::ZipFile.open(zips) do |file|
file.each do |search|
tempFile = file.read(search)
tempFile.each do |line|
if (line =~ #Regexp ) then
p line
end
end
end
end
end
Below is the error message from IRB:
NoMethodError: undefined method `each' for #<String:0x0000000168bf40>
from (irb):70:in `block (3 levels) in irb_binding'
from /var/lib/gems/1.9.1/gems/rubyzip2-2.0.2/lib/zip/zip.rb:1122:in `each'
from /var/lib/gems/1.9.1/gems/rubyzip2-2.0.2/lib/zip/zip.rb:1122:in `each'
from /var/lib/gems/1.9.1/gems/rubyzip2-2.0.2/lib/zip/zip.rb:1265:in `each'
from (irb):68:in `block (2 levels) in irb_binding'
from /var/lib/gems/1.9.1/gems/rubyzip2-2.0.2/lib/zip/zip.rb:1381:in `open'
from (irb):67:in `block in irb_binding'
from (irb):66:in `glob'
from (irb):66
from /usr/bin/irb:12:in `<main>'
I tried tempFile.grep, and received the same error, except that grep was an undefined method. I believe I need to define a class.
Turns out my code had two problems. 1) My regular expression was being processed as a string (I should not have used the quotes). 2) Seeing as it runs fine otherwise on Ruby 1.8.7, I suspect the is a difference in how 1.8.7 and 1.9.1 process the 'each' method. If anyone has additional insights, I'm more than happy to hear them. The code below works fine on 1.8.7:
require 'zip/zip'
#workingdir = '/my/dir/structure/*.zip'
#Regexp = /regexp/i
Dir.glob(#workingdir) do |zips|
Zip::ZipFile.open(zips) do |file|
file.each do |search|
tempFile = file.read(search)
tempFile.each do |line|
if (line =~ #Regexp) then
puts zips + ': ' + line.chomp
end
end
end
end
end
Thanks again everyone!
I'm getting an exception in the following piece of code. Can someone please tell me what I'm doing wrong, and how I can prevent it?
def self.find_by_data(data = {})
where(name_canonical: data['name'].downcase.gsub(/\s+/, ''),
fuel: data['fuel'],
trim_canonical: data['trim'].downcase.gsub(/\s+/, ''),
year: data['year']).first
end
Exception:
/Users/charlie/Documents/WIP/projectx/ar_models.rb:35:in `find_by_data': undefined method `downcase' for nil:NilClass (NoMethodError)ooooooooooooooooooooooooo| ETA: 0:00:00
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-4.0.0/lib/active_record/relation/delegation.rb:36:in `block in find_by_data'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-4.0.0/lib/active_record/associations/collection_proxy.rb:845:in `block in scoping'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-4.0.0/lib/active_record/relation.rb:270:in `scoping'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-4.0.0/lib/active_record/associations/collection_proxy.rb:845:in `scoping'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-4.0.0/lib/active_record/relation/delegation.rb:36:in `find_by_data'
from /Users/charlie/Documents/WIP/projectx/ar_models.rb:132:in `create_or_assign_existing'
from /Users/charlie/Documents/WIP/projectx/app.rb:230:in `block (2 levels) in work'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-4.0.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:294:in `with_connection'
from /Users/charlie/Documents/WIP/projectx/app.rb:80:in `block in work'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/parallel-0.6.3/lib/parallel.rb:319:in `call'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/parallel-0.6.3/lib/parallel.rb:319:in `call_with_index'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/parallel-0.6.3/lib/parallel.rb:179:in `block (3 levels) in work_in_threads'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/parallel-0.6.3/lib/parallel.rb:326:in `with_instrumentation'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/parallel-0.6.3/lib/parallel.rb:177:in `block (2 levels) in work_in_threads'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/parallel-0.6.3/lib/parallel.rb:171:in `loop'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/parallel-0.6.3/lib/parallel.rb:171:in `block in work_in_threads'
from /Users/charlie/.rvm/gems/ruby-2.0.0-p0/gems/parallel-0.6.3/lib/parallel.rb:62:in `block (2 levels) in in_threads'
When you see "undefined method ... for nil:NilClass" it means you have a nil value you're trying to call the method on.
In this case, something like data['name'] is not defined.
To make this more bullet-proof:
data['name'].to_s.downcase.gsub(/\s+/, '')
This converts everything to string to start with. nil.to_s is an empty string by default, so it's safe.
Use ternary oprators perhaps:
def self.find_by_data(data = {})
where(name_canonical: data['name'] == nil ? '' : data['name'].downcase.gsub(/\s+/, ''),
fuel: data['fuel'],
trim_canonical: data['trim'] == nil ? '' : data['name'].downcase.gsub(/\s+/, ''),
year: data['year']).first
end
Your
data['name']
or
data['trim']
is a nil.
Check your input data.
I'm using mongomapper to store pages in a db, and I index them first. In the index method, I loop through each word, and check to see if it is already in the words hashmap. If not, I add an empty array to the hash, then push its location to the array.
def index_words
#words = self.body.split(" ")
#words.each_with_index do |word,i|
if self.words[word.stem].nil?
self.words[word.stem] = []
end
puts "Called from #{caller[0]}"
self.words[word.stem].push(i)
end
end
When I run this, I get an undefined method error, saying that self.words[word.stem] is nil. Furthermore, this method is actually being called from the loop, when it's only called once in the constructor:
def initialize(*args)
super
index_words
end
The error message is:
p = Page.new({author: 'Michael',url: 'michaelfine.me',title: 'Michael Fine',body: 'Body Text'})
called fromPage.rb:19:in `each'
NoMethodError: undefined method `push' for nil:NilClass
from Page.rb:24:in `block in index_words'
from Page.rb:19:in `each'
from Page.rb:19:in `each_with_index'
from Page.rb:19:in `index_words'
from Page.rb:14:in `initialize'
from (irb):103:in `new'
from (irb):103
from /Users/Michael/.rvm/rubies/ruby-1.9.3-p286/bin/irb:16:in `<main>'