Tmdb::InvalidApiKeyError (Tmdb::InvalidApiKeyError) - etl

When I am using Kiba ELT, I have followed the tutorials in YouTube as well as the tutorials provided by the owner. Yet, I am getting this error:
bitlasoft#Bitlasoft-TS-22:~/test01$ bundle exec kiba movies.etl
{
"title: Blade Runner" => "title: Minority Report"
}
/home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/themoviedb-1.0.1/lib/themoviedb/api.rb:37:in `set_response': Tmdb::InvalidApiKeyError (Tmdb::InvalidApiKeyError)
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/themoviedb-1.0.1/lib/themoviedb/search.rb:75:in `fetch_response'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/themoviedb-1.0.1/lib/themoviedb/search.rb:60:in `fetch'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/themoviedb-1.0.1/lib/themoviedb/resource.rb:41:in `search'
from /home/bitlasoft/test01/common.rb:30:in `process'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:35:in `block (3 levels) in process_rows'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:34:in `each'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:34:in `block (2 levels) in process_rows'
from /home/bitlasoft/test01/common.rb:12:in `block in each'
from /home/bitlasoft/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/csv.rb:1739:in `each'
from /home/bitlasoft/test01/common.rb:11:in `each'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:33:in `block in process_rows'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:32:in `each'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:32:in `process_rows'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/runner.rb:13:in `run'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/lib/kiba/cli.rb:13:in `run'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/gems/kiba-0.6.1/bin/kiba:5:in `<top (required)>'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/bin/kiba:23:in `load'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/bin/kiba:23:in `<main>'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/bin/ruby_executable_hooks:15:in `eval'
from /home/bitlasoft/.rvm/gems/ruby-2.2.2/bin/ruby_executable_hooks:15:in `<main>'
bitlasoft#Bitlasoft-TS-22:~/test01$
Here is my movies.etl and common.rb config:
require_relative 'common'
api_key = IO.read('.themoviedb')
source CSVSource, filename: 'movies.csv'
limit ENV['LIMIT']
show_me!
transform MovieDBlookup,
api_key: api_key,
title_field: 'title'
show_me!
require 'csv'
require 'awesome_print'
class CSVSource
def initialize(filename:)
#filename = filename
end
def each
csv = CSV.open(#filename, headers: true)
csv.each do |row|
yield(row.to_hash)
end
csv.close
end
end
require 'themoviedb'
class MovieDBlookup
def initialize(api_key:, title_field:)
#title_field = title_field
Tmdb::Api.key(api_key)
end
def process(row)
movie = Tmdb::Movie.find(row[#title_field]).first
row[:vote_average] = movie.vote_average
row[:vote_count] = movie.vote_count
row
end
end
def show_me!
transform do |row|
ap row
row
end
end
def limit(x)
x = Integer(x || -1)
return if x == -1
transform do |row|
#counter ||= 0
#counter += 1
abort("stopping....")if #counter >= x
row
end
end

(Kiba owner here) - the error you get isn't Kiba specific ; it looks like the API key you provided for themoviedb is invalid. Did you sign-up for the movie db here, and is the API key provided copy-pasted into the file you are loading (.themoviedb)?
One problem I could think of would be that you'd have a line ending character in that file (carriage return / line feed), in which case maybe just calling api_key = IO.read('.themoviedb').strip could help.
Again, this isn't Kiba specific, but hope this helps!

Related

ArgumentError when accessing pageobject widget

class MainTabsWidget < PageObject::Elements::UnorderedList
include PageObject
include PageObject::PageFactory
link(:first_link, :text => 'First')
link(:second_link, :text => 'Second')
link(:third_link, :text => 'Third')
link(:category_link, :text => 'Category')
link(:subcat1_link, :text => 'Subcat1')
link(:subcat2_link, :text => 'Subcat2')
def goto_tab_item(tab_item)
items = tab_item.split /\//
items.each do |item|
case item
when 'First'
first_link
when 'Second'
second_link
when 'Third'
third_link
when 'Category'
category_link
when 'Subcat1'
subcat1_link
when 'Subcat2'
subcat2_link
end
end
end
end
PageObject.register_widget :main_menu, MainTabsWidget , :ul
I have a page called landing page that uses the widget in the following manner....
class LandingPage
include PageObject
include PageObject::PageFactory
main_menu(:menu_tabs, :id => 'mainMenu')
def select_menu_item(item)
menu_tabs_element.go_to_tab_item item
end
end
This actually did work at one point. If you pass 'First' for select_menu_item, it will click to the page specified by first_link. If you specify 'Category/Subcat1' it will click on the category link and then subcat1.
Howerver, something changed. We are transition from watir-webdriver gem to watir (6.10) gem, and now the code gets the argument exeption
'ArgumentError: wrong number of arguments (0 for 1..2)'
This occurs when menu_tabs_element gets referenced in any way. I've run out of ideas to remedy this.
Stack Trace:
ArgumentError: wrong number of arguments (0 for 1..2)
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object/element_locators.rb:10:in `element'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object/elements/element.rb:191:in `respond_to_missing?'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object.rb:53:in `respond_to_missing?'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object.rb:70:in `respond_to?'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object.rb:70:in `initialize'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object/platforms/watir/page_object.rb:1047:in `new'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object/platforms/watir/page_object.rb:1047:in `find_watir_element'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object/platforms/watir/page_object.rb:1118:in `block in define_widget_singular_accessor'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page-object-2.2.4/lib/page-object/widgets.rb:38:in `block (3 levels) in define_accessors'
./lib/pages/menu.rb:13:in `select_menu_item'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page_navigation-0.10/lib/page_navigation.rb:134:in `block in navigate_through_pages'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page_navigation-0.10/lib/page_navigation.rb:129:in `each'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page_navigation-0.10/lib/page_navigation.rb:129:in `navigate_through_pages'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/page_navigation-0.10/lib/page_navigation.rb:67:in `navigate_to'
./spec/refactor/test_spec.rb:6:in `block (2 levels) in <top (required)>'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/hooks.rb:350:in `instance_exec'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/hooks.rb:350:in `run'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/hooks.rb:509:in `block in run_owned_hooks_for'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/hooks.rb:508:in `each'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/hooks.rb:508:in `run_owned_hooks_for'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/hooks.rb:460:in `run'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/example_group.rb:537:in `block in run_before_context_hooks'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/memoized_helpers.rb:186:in `block in isolate_for_context_hook'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/memoized_helpers.rb:182:in `instance_exec'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/memoized_helpers.rb:182:in `isolate_for_context_hook'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/example_group.rb:536:in `run_before_context_hooks'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/example_group.rb:589:in `run'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/runner.rb:118:in `block (3 levels) in run_specs'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/runner.rb:118:in `map'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/runner.rb:118:in `block (2 levels) in run_specs'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/configuration.rb:1896:in `with_suite_hooks'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/runner.rb:113:in `block in run_specs'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/reporter.rb:79:in `report'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/runner.rb:112:in `run_specs'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/runner.rb:87:in `run'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/runner.rb:71:in `run'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/lib/rspec/core/runner.rb:45:in `invoke'
C:/Ruby22/lib/ruby/gems/2.2.0/gems/rspec-core-3.7.0/exe/rspec:4:in `<top (required)>'
C:/Ruby22/bin/rspec:23:in `load'
C:/Ruby22/bin/rspec:23:in `<top (required)>'
-e:1:in `load'
-e:1:in `<main>'
1 example, 1 failure, 0 passed
Finished in 6.087 seconds
Process finished with exit code 1
It is not clear to me which change in Watir would cause this issue. It seems more likely a change in Page-Object. At any rate, here are 2 solutions. Note that this was tested with Page-Object v 2.2.4.
Using Widgets
As pointed out by Alexis in Page-Object Issue 263, including page object accessor methods in widgets is not as straight forward as one would expect. In the MainTabsWidget class, instead of include PageObject, you need to do:
extend PageObject::Accessors # to get the accessor methods
attr_reader :platform # the accessor methods don't work without a platform
With this change, your page object should work.
Using Page Section
Page-Object has a page section feature that might better suit your needs. It simplifies some of the code (ie address the above issue).
You define a class to represent your main tabs section. Note that you do not need to register the section or have the multi-line setup as with Widgets.
class MainTabs
include PageObject
include PageObject::PageFactory
link(:first_link, :text => 'First')
link(:second_link, :text => 'Second')
link(:third_link, :text => 'Third')
link(:category_link, :text => 'Category')
link(:subcat1_link, :text => 'Subcat1')
link(:subcat2_link, :text => 'Subcat2')
def select_menu_item(tab_item)
items = tab_item.split /\//
items.each do |item|
case item
when 'First'
first_link
when 'Second'
second_link
when 'Third'
third_link
when 'Category'
category_link
when 'Subcat1'
subcat1_link
when 'Subcat2'
subcat2_link
end
end
end
end
The page object can simply become:
class LandingPage
include PageObject
include PageObject::PageFactory
page_section(:menu_tabs, MainTabs, :id => 'mainMenu')
end
The clicking of the menu would change to:
page.menu_tabs.select_menu_item('First')
(or you could continue to define a #select_menu_item method in LandingPage to avoid downstream changes)

TypeError: wrong argument type class (expected module) in rspec

i'm developing a simple net application (packed as a gem) to learn Ruby and TDD.
I have this class (receiver.rb):
require 'eventmachine'
class Receiver < EM::Connection
def initialize(port)
EM.run do
EM.open_datagram_socket('0.0.0.0', port, self)
end
end
end
And this test (receiver_spec.rb):
require "spec_helper"
require "net/receiver"
describe "Receiver" do
it "can istantiate" do
#rec = Receiver.new(500)
end
end
Anyway, when i run rspec it prints out this:
1) Receiver can istantiate
Failure/Error: #rec = Receiver.new(500)
TypeError:
wrong argument type Receiver (expected Module)
# /var/lib/gems/1.9.1/gems/eventmachine- 1.0.7/lib/eventmachine.rb:1535:in `include'
# /var/lib/gems/1.9.1/gems/eventmachine-1.0.7/lib/eventmachine.rb:1535:in `block in klass_from_handler'
# /var/lib/gems/1.9.1/gems/eventmachine-1.0.7/lib/eventmachine.rb:1535:in `initialize'
# /var/lib/gems/1.9.1/gems/eventmachine-1.0.7/lib/eventmachine.rb:1535:in `new'
# /var/lib/gems/1.9.1/gems/eventmachine-1.0.7/lib/eventmachine.rb:1535:in `klass_from_handler'
# /var/lib/gems/1.9.1/gems/eventmachine- 1.0.7/lib/eventmachine.rb:867:in `open_datagram_socket'
# ./lib/rppc/net/receiver.rb:9:in `block in initialize'
# /var/lib/gems/1.9.1/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `call'
# /var/lib/gems/1.9.1/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run_machine'
# /var/lib/gems/1.9.1/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run'
# ./lib/rppc/net/receiver.rb:8:in `initialize'
# ./spec/net/receiver_spec.rb:6:in `new'
# ./spec/net/receiver_spec.rb:6:in `block (2 levels) in <top (required)>'
I'm quite new to the ruby environment, so if i missed something let me know.
I'm not sure what documentation you're working from, but it appears the open_datagram_socket requires a Module and cannot accept a Class as the third (handler) argument.
Per the comment in http://www.rubydoc.info/github/eventmachine/eventmachine/EventMachine.open_datagram_socket, it appears this restriction may have been loosened in a later version of EventMachine

Error while upgrading from Rails 3.1 to Rails 3.2

Upgrade steps were performed as mentioned in http://guides.rubyonrails.org/3_2_release_notes.html
Right now I'm on Ruby 1.8.7 and Rails 3.1.x (using system ruby). I installed RVM with ruby 1.8.7 and added to rails 3.2 and then I get the following error:
[app]$ rails console
Faraday: you may want to install system_timer for reliable timeouts
$HOME/src/qbol/tapp/config/environment.rb:16:in `add': undefined method `>' for nil:NilClass (NoMethodError)
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/tagged_logging.rb:55:in `add'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/tagged_logging.rb:61:in `info'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activerecord-3.2.17/lib/active_record/railtie.rb:86
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/lazy_load_hooks.rb:36:in `instance_eval'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/lazy_load_hooks.rb:36:in `execute_hook'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/lazy_load_hooks.rb:26:in `on_load'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/lazy_load_hooks.rb:25:in `each'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/lazy_load_hooks.rb:25:in `on_load'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activerecord-3.2.17/lib/active_record/railtie.rb:80
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/initializable.rb:30:in `instance_exec'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/initializable.rb:30:in `run'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/initializable.rb:55:in `run_initializers'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/initializable.rb:54:in `each'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/initializable.rb:54:in `run_initializers'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/application.rb:136:in `initialize!'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/railtie/configurable.rb:30:in `send'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/railtie/configurable.rb:30:in `method_missing'
from $HOME/src/app/config/environment.rb:48
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:251:in `require'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:251:in `require'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:236:in `load_dependency'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:251:in `require'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/application.rb:103:in `require_environment!'
from $HOME/.rvm/gems/ruby-1.8.7-p374/gems/railties-3.2.17/lib/rails/commands.rb:40
from script/rails:6:in `require'
from script/rails:6
This is on my development box. Any idea why this is happening?
EDIT: Below is the environment.rb. The error is happening on the last line App::Application.initialize!
# Load the rails application
require File.expand_path('../application', __FILE__)
module ActiveSupport
class BufferedLogger
def self.current_user
Thread.current[:user]
end
def self.current_user=(user)
Thread.current[:user] = user
end
def add(severity, message = nil, progname = nil, &block)
return if #level > severity
message = (message || (block && block.call) || progname).to_s
level = {
0 => "DEBUG",
1 => "INFO ",
2 => "WARN ",
3 => "ERROR",
4 => "FATAL"
}[severity] || "UNKNOWN"
user=BufferedLogger.current_user
if(!user.nil?)
idstr = "uid:#{user.id}"
if !user.current_app_user.nil?
idstr.concat(", acid: #{user.current_app_user.account_id}")
end
else
idstr=""
end
message = "[%s: %s #{idstr}] %s" %
["#{level} pid: #{$$}", Time.now.strftime("%y-%m-%d %H:%M:%S"), message]
message = "#{message}\n" unless message[-1] == ?\n
buffer << message
auto_flush
message
end
end
end
# Initialize the rails application
App::Application.initialize!
The error is happening inside ActiveSupport::BufferedLogger#add method call and you appear to be monkey-patching this class. The internals of ActiveSupport::BufferedLogger have likely changing between the versions of Rails you're using.
Try removing all of the ActiveSupport::BufferedLogger code from your environment.rb to work past this error and get your app running. Then, if you still need the monkeypatch (don't know why), you'd have to rewrite it on top of the newer version of the class.

Undefined method each in Ruby Regexp task

I have series of zip files under #workingdir, and am trying to unzip the files that match #Regexp, and print the lines from them.
require 'zip/zip'
#workingdir = '/my/dir/structure/*.zip'
#Regexp = '/yup:maybe.*nope/i'
Dir.glob(#workingdir) do |zips|
Zip::ZipFile.open(zips) do |file|
file.each do |search|
tempFile = file.read(search)
tempFile.each do |line|
if (line =~ #Regexp ) then
p line
end
end
end
end
end
Below is the error message from IRB:
NoMethodError: undefined method `each' for #<String:0x0000000168bf40>
from (irb):70:in `block (3 levels) in irb_binding'
from /var/lib/gems/1.9.1/gems/rubyzip2-2.0.2/lib/zip/zip.rb:1122:in `each'
from /var/lib/gems/1.9.1/gems/rubyzip2-2.0.2/lib/zip/zip.rb:1122:in `each'
from /var/lib/gems/1.9.1/gems/rubyzip2-2.0.2/lib/zip/zip.rb:1265:in `each'
from (irb):68:in `block (2 levels) in irb_binding'
from /var/lib/gems/1.9.1/gems/rubyzip2-2.0.2/lib/zip/zip.rb:1381:in `open'
from (irb):67:in `block in irb_binding'
from (irb):66:in `glob'
from (irb):66
from /usr/bin/irb:12:in `<main>'
I tried tempFile.grep, and received the same error, except that grep was an undefined method. I believe I need to define a class.
Turns out my code had two problems. 1) My regular expression was being processed as a string (I should not have used the quotes). 2) Seeing as it runs fine otherwise on Ruby 1.8.7, I suspect the is a difference in how 1.8.7 and 1.9.1 process the 'each' method. If anyone has additional insights, I'm more than happy to hear them. The code below works fine on 1.8.7:
require 'zip/zip'
#workingdir = '/my/dir/structure/*.zip'
#Regexp = /regexp/i
Dir.glob(#workingdir) do |zips|
Zip::ZipFile.open(zips) do |file|
file.each do |search|
tempFile = file.read(search)
tempFile.each do |line|
if (line =~ #Regexp) then
puts zips + ': ' + line.chomp
end
end
end
end
end
Thanks again everyone!

Having trouble with Nokogiri and Rails

Been at this for a while. If i tell Category to just create, everything works fine. If I tell it to find_or_create I get errors.
These work:
puts topic.at_xpath("#topicid")
puts topic.at_xpath("#topicname")
and
Category.create!(:topic_id => topic.at_xpath("#topicid"), :name => topic.at_xpath("#topicname"))
But these don't:
Category.find_by_name(topic.at_xpath("#topicname"))
or
Category.find_or_create_by_topic_id_and_name(topic.at_xpath("#topicid"), topic.at_xpath("#topicname"))
Where am I messing up?
class FeedEntry < ActiveRecord::Base
require 'nokogiri'
require 'open-uri'
has_many :category_feeds
has_many :categories, :through => :category_feeds
accepts_nested_attributes_for :categories, :allow_destroy => true, :reject_if => proc { |obj| obj.blank? }
def self.nokogiri_get_feed
url = "http://some_feed.com/atom_feed"
doc = Nokogiri::HTML(open(url))
doc.remove_namespaces!
doc.search('feed entry').each do |item|
unless exists? :guid => item.css('id').text
create!(:name => item.css('title').text, :summary => item.css('title').text, :url => item.at_css("link")[:href], :published_at => item.css('updated').text, :guid => item.css('id').text)
item.xpath('content').each do |i|
i.css('topic').each do |topic|
id = topic.at_xpath("#topicid")
name = topic.at_xpath("#topicname")
update_attributes!(:categories=>[Category.find_or_create_by_topic_id_and_name(id, name)])
end
end
end
end
end
end
errors are:
ruby-1.9.2-p180 :001 > FeedEntry.nokogiri_get_feed
TypeError: Cannot visit Nokogiri::XML::Attr
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:21:in `rescue in visit'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:15:in `visit'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:260:in `visit_Arel_Nodes_Equality'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:15:in `visit'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:120:in `visit_Arel_Nodes_Grouping'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:15:in `visit'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:91:in `block in visit_Arel_Nodes_SelectCore'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:91:in `map'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:91:in `visit_Arel_Nodes_SelectCore'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:77:in `block in visit_Arel_Nodes_SelectStatement'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:77:in `map'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:77:in `visit_Arel_Nodes_SelectStatement'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/sqlite.rb:7:in `visit_Arel_Nodes_SelectStatement'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:15:in `visit'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/visitor.rb:5:in `accept'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/arel-2.0.9/lib/arel/visitors/to_sql.rb:19:in `block in accept' ... 11 levels...
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/nokogiri-1.4.4/lib/nokogiri/xml/node_set.rb:238:in `each'
from /Users/pca/projects/cdapp/cdrails/app/models/feed_entry.rb:35:in `block (2 levels) in nokogiri_get_feed'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/nokogiri-1.4.4/lib/nokogiri/xml/node_set.rb:239:in `block in each'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/nokogiri-1.4.4/lib/nokogiri/xml/node_set.rb:238:in `upto'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/nokogiri-1.4.4/lib/nokogiri/xml/node_set.rb:238:in `each'
from /Users/pca/projects/cdapp/cdrails/app/models/feed_entry.rb:33:in `block in nokogiri_get_feed'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/nokogiri-1.4.4/lib/nokogiri/xml/node_set.rb:239:in `block in each'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/nokogiri-1.4.4/lib/nokogiri/xml/node_set.rb:238:in `upto'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/nokogiri-1.4.4/lib/nokogiri/xml/node_set.rb:238:in `each'
from /Users/pca/projects/cdapp/cdrails/app/models/feed_entry.rb:30:in `nokogiri_get_feed'
from (irb):1
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.0.3/lib/rails/commands/console.rb:44:in `start'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.0.3/lib/rails/commands/console.rb:8:in `start'
from /Users/pca/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.0.3/lib/rails/commands.rb:23:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'
Summary
Instead of these lines:
id = topic.at_xpath("#topicid")
name = topic.at_xpath("#topicname")
Use these instead:
id = topic['topicid']
name = topic['topicname']
Explanation
Let's look at a simple test case:
require 'nokogiri'
xml = Nokogiri::XML("<root foo='bar' />")
foo = xml.root.at_xpath('#foo')
puts foo
#=> bar
p foo
#=> #<Nokogiri::XML::Attr:0x15c1d64 name="foo" value="bar">
p foo.text
#=> "bar"
p xml.root['foo']
#=> "bar"
As you can see from the above, selecting an attribute via XPath actually gives you an Attr node, which is not the same as the string value of that attribute. (Using puts causes the to_s method of the Attr to show you only the value, but that doesn't mean that it's actually a string.)
As shown above, you need to use the text method (or value or content) on the Attr nodes to get the string value back that you really wanted:
id = topic.at_xpath("#topicid").text
name = topic.at_xpath("#topicname").text
Alternatively (and more simply) use the Element#[] method to fetch the value of an attribute off of an element directly:
id = topic['topicid']
name = topic['topicname']

Resources