I am creating a basic scraper that gets the total relief amount rewarded to each state and then displays it, but I'm receiving an error I don't understand. Can you help me fix my program please?
require 'rubygems'
require 'crack'
require 'open-uri'
URL = 'http://www.recovery.gov/pages/GetXmlData.aspx?data=recipientHomeMap'
Crack::XML.parse(open(URL).read)['totals']['state'].each do |state|
puts ['id', 'awarded', 'received', 'jobs'].map{|f| state[f]}.join(', ')
end
rb:7:in ' : undefined method '[]' for nil:NilClass(NoMethodError)
Check what Crack::XML.parse(open(URL).read) return
You aren't getting anything back from Crack::XML.parse(open(URL).read)
You are trying to access values from nil hence the undefined method '[]' for nil:NilClass
Make sure you are actually getting the file first.
Related
While writing a ruby script to decode rack session cookies and load the object using the ruby function Marshal load, I am getting the following error:
Error: undefined class/module Rack::Session::SessionId
This is my ruby script:
require 'pp'
require 'base64'
require 'uri'
require 'rack' # solves the error: Error: undefined class/module Rack::
c = gets
cookie, signature = c.split("--")
decoded = Base64.decode64(URI.decode(cookie))
begin
object = Marshal.load(decoded)
pp object
rescue ArgumentError => e
puts "Error: " + e.to_s
end
How do I fix this? I tried looking for solutions on stackoverflow but couldn't figure it out.
Thanks to Louis Nyffenegger (#snyff) who helped me with this
https://twitter.com/snyff/status/1407054131697758211?s=20
This error can be solved by creating a method stub
require 'pp'
require 'base64'
require 'uri'
require 'rack'
class Rack::Session::SessionId
end
c = gets
cookie, signature = c.split("--")
decoded = Base64.decode64(URI.decode(cookie))
begin
object = Marshal.load(decoded)
pp object
rescue ArgumentError => e
puts "Error: " + e.to_s
end
Try clearing all of the cookies for the site.
I think this works because something is causing an exception while loading the Rack::Session::SessionId class.
This type of error happens occasionally when working on Rails applications. A SyntaxError (or any other exception) inside a class definition (e.g. in a model) can then cause an undefined class/module error instead of the real error.
Am trying to go through a series of links with a css class title and click those links and then get the product title. But i keep getting the error undefined method each for #<Mechanize::Page::Link:0x007fbfe2524410> (NoMethodError)? I Don't understand what am doning wrong?
heres my code:
require 'mechanize'
file = File.new("outputscrape.txt", 'w')
agent = Mechanize.new { |agent|
agent.user_agent_alias = 'Windows Chrome'}
page = agent.get('http://www.amazon.com/s/ref=sr_nr_n_0?rh=n%3A283155%2Cn%3A%211000%2Cn%3A5%2Cn%3A15377001%2Cn%3A6133979011%2Cn%3A6133980011&bbn=6133979011&ie=UTF8&qid=1412193262&rnid=6133979011')
title_link = page.link_with(:dom_class => "title")
title_link.each do |link|
link.click
file.write(link.at('#productTitle').text.strip)
end
From the mechanize docs:
link_with(criteria)
Find a single link matching criteria.
You need to use:
links_with(criteria)
Find all links matching criteria.
The object mentioned in your error message, Page::Link:
undefined method each for #<Mechanize::Page::Link:0x007fbfe2524410>
(NoMethodError)
doesn't sound like more than one thing, does it? More than one thing would be more like Page::Links, or Page::Link::Group, or Page::LinkSet. You are doing the equivalent of:
10.each do |number|
puts number
end
However, numbers do not have an each() method, so that produces the error:
undefined method `each' for 10:Fixnum (NoMethodError)
Compare that to your error:
undefined method each for #<Mechanize::Page::Link:0x007fbfe2524410>
On the other hand an Array does have an each() method, so you can do this:
[10, 20, 30].each do |number|
puts number
end
I have this problem:
uninitialized constant Couch::Couchbase (NameError)
./features/step_definitions/lib/couchbase.rb:6:in `get'
./features/step_definitions/StepsLib.rb:130:in `/^I get couch$/'
features/test.feature:4:in `Then I get couch
The code is:
require 'rubygems'
require 'couchbase'
class Couch
def get
client = Couchbase.connect(:bucket => "user", :hostname => "192.168.1.50")
user = client.get("COMMENT-FO-1103")
return user
client.disconnect
end
end
I've been loocking all over, and no clue, I'm no ruby expert.
Thanks.
Try connecting through IRB and see if you get the same error.
Open up an IRB instance and type:
> require 'Couchbase'
You'll get a statement back saying: => true
Then connect as follows:
> c = Couchbase.new("http://localhost:8091/pools/default/buckets/MyBucket")
This should connect you directly to the bucket you wish to operate on.
Then try:
> c.set("mykey", "Some Value")
And you should get a confirmation that the object has been set in the Bucket.
Then use:
> c.get("mykey")
And you should print the value of the object you just set.
Regarding your code above, I'm not sure why exactly you're trying to wrap this call up in a Class? What is your Use case?
I notice on the github page that methods like get aren't run on client itself, but instead are used in a block argument to run like:
client.run { |conn| conn.get("COMMENT-FO-1103") }
That's really all I can think of there. Hope it helps.
I do notice that return user will prevent the line client.disconnect from ever running, as return kicks you out of the method entirely.
Dear stackoverflow community,
Beginner's question:
Why do I get the following error?
scraper_sample_2.rb:7:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError)
>Exit code: 1
Here's my code (copied from a ruby's intro guide):
require "rubygems"
require "crack"
require "open-uri"
URL = "http://www.recovery.gov/pages/GetXmlData.aspx?data=recipientHomeMap"
Crack::XML.parse(open(URL).read)["totals"]["state"].each do |state|
puts ["id", "awarded", "received", "jobs"].map{|f| state[f]}.join(",")
end
Because Crack::XML.parse(open(URL).read)["totals"] is nil. Try to split the call you do on line 7 on several lines and debug each call separately. Maybe the answer you get is not what you expect.
Given the format of the xml returned from your source, Crack::XML.parse(open(URL).read)["totals"] will, as Ivaylo said, return nil. The format of the xml must have changed, as totals are now within /map/view.
To get the expected output, change your code to:
Crack::XML.parse(open(URL).read)["map"]["view"]["totals"]["state"].each do |state|
puts ["id", "awarded", "received", "jobs"].map{|f| state[f]}.join(",")
end
When I run the following script to retrieve the first page of google results
#!/usr/bin/env ruby
require 'rubygems'
require 'nokogiri'
require 'open-uri'
doc = Nokogiri::HTML(open('http://www.google.co.uk/search?q=stackoverflow'))
doc.css('div.vsc').each do |element|
puts element.at_css("h3.r a.l").content
end
I get a undefined methodcontent' for nil:NilClass (NoMethodError)`
How could I solve that? Or at least how could avoid it showing when executing?
As Dave Newton already pointed out in his comment, the result of at_css("h3.r a.l") is nil in your case. Neither the NilClass nor the object nil have a method content.
Workaround:
doc.css('div.vsc').each do |element|
next unless elem = element.at_css("h3.r a.l")
puts elem.content
end