I'm trying to use domainatrix with nokogiri and am coming up with a holdup. Being relatively new to ruby, I've tried every syntax variation on the Domainatrix.parse function I can to get the a href's to parse properly. They do print during the "puts" command but when I uncomment the domainatrix code problems start:
require 'rubygems'
require 'domainatrix'
require 'anemone'
require 'open-uri'
require 'nokogiri'
doc = Nokogiri::HTML(open("http://www.cnn.com"))
doc.xpath('//a/#href').each do |node|
linkage = node.text
puts linkage
url = Domainatrix.parse(linkage)
print url.domain
print url.public_suffix
end
Anyone have any ideas on this? I think it is just a syntax issue or perhaps I cannot use the Domainatrix function where I'm using it?
It was getting snagged on some improperly formatted URLs.
Related
I am trying to verify text on a web page.
The line of code using is: #browser.test.include?('Favorites').should==true
When the script executes this line I get the error: undefined method 'should' for false:FalseClass (NoMethodError)
Here is the full code:
require 'watir-webdriver'
require 'rspec'
#browser=Watir::Browser.new :ff
#browser.goto('http://enoteca.demo.episerver.com/en-US/')
#browser.link(:text=>'Register').wait_until_present
#browser.text.include?("Favorites").should==true
You need to define an example group that contains one or more examples. For instance:
require 'watir-webdriver'
require 'rspec'
describe "an example group" do
it "is an example" do
browser = Watir::Browser.new
browser.goto('www.example.org')
browser.text.include?('Domain').should==true
end
end
If you place the above in a file ending with _spec.rb (e.g. foo_spec.rb), you can run it at command line or terminal: rspec foo_spec.rb. After completion, rspec will return a status:
Finished in X.XX seconds (files took X.XXXX seconds to load)
1 example, 0 failures
Also, the should method has been deprecated in rspec3 (although it will still work for now). In fact, rspec3 will return a deprecation warning if you use should.
Using should from rspec-expectations' old :should syntax without explicitly
enabling the syntax is deprecated. Use the new :expect syntax or explicitly enable :should with config.expect_with(:rspec) { |c| c.syntax = :should } instead.
Here's the same spec as above but using expect instead of should:
describe "an example group" do
it "is an example" do
browser = Watir::Browser.new
browser.goto('www.example.org')
expect(browser.text).to include 'Domain' # expect method instead of should
end
end
While I would think you should be using an example group (as #orde mentioned), it is not technically required. You can use the expectations outside an example group by including the RSpec::Expectations:
require 'watir-webdriver'
require 'rspec'
# Include the expectations so that they do not need an example group
include RSpec::Expectations
#browser=Watir::Browser.new :ff
#browser.goto('http://enoteca.demo.episerver.com/en-US/')
#browser.link(:text=>'Register').wait_until_present
#browser.text.include?("Favorites").should==true
I'm doing a test with "require" under ruby 2.0.0p576 (2014-09-19 revision 47628) [x86_64-darwin13.4.0] it doesn't work in many ways.
There are two files in ruby directory as shown below:
string_extensions.rb
class String
def vowels
self.scan(/[aeiou]/i)
end
end
vowels_test.rb
require 'string_extensions'
puts "This is a test".vowels.join('-')
fire up IRB
Snailwalkers-MacBook-Pro:ruby snailwalker$ ruby vowels_test.rb
returs : `require': cannot load such file -- string_extensions (LoadError)
I tried to change require 'string_extensions' to " require_relative 'string_extensions' ; require './string_extensions.rb' . They all didn't work.
both return error : vowels_test.rb:1:in require_relative': /Users/snailwalker/Ruby/string_extensions.rb:1: class/module name must be CONSTANT (SyntaxError)
Your help will be greatly appreciated!
You can use require_relative instead:
require_relative 'string_extensions'
puts "This is a test".vowels.join('-')
Or even require './string_extensions'.
Use:
ruby -I. vowels_test.rb
Automatic inclusion of the current directory in the load paths was removed in Ruby 2.
I'm trying to edit my Ruby file with Pry. There are few variables that are set in it, and for whatever reason I can't seem to cd into them because they aren't being defined even after I 'load' the file.
Here is the code:
require 'nokogiri'
require 'open-uri'
doc = Nokogiri.XML('<foo><bar /><foo>', nil, 'UTF-8')
url = "http://superbook.eventmarketer.com/category/agencies/"
puts "Finished!"
In Pry I do:
load "./AgencyListingScraper.rb"
and then this is the output:
7] pry(main)> load './AgencyListingScraper.rb'
Finished!
=> true
[8] pry(main)>
Then when I try to do something like:
[8] pry(main)> url
NameError: undefined local variable or method `url' for main:Object
from (pry):6:in `__pry__'
[9] pry(main)> cd url
Error: Bad object path: url. Failed trying to resolve: url. #<NameError: undefined local
variable or method `url' for main:Object>
[10] pry(main)>
This is what I get.
I think I'm not loading the file correctly although I've been searching for hours and I can't figure out how to properly do this. I was doing it right months ago when I had made a scraper with Ruby, but this time I'm having trouble just getting started because of this bit.
Thanks for your help in advance!
Try it this way:
In your file include Pry and do a binding.pry:
require 'nokogiri'
require 'open-uri'
require 'pry'
doc = Nokogiri.XML('<foo><bar /><foo>', nil, 'UTF-8')
url = "http://superbook.eventmarketer.com/category/agencies/"
binding.pry
puts "Finished!"
Then run the file by executing:
ruby AgencyListingScraper.rb
That should drop you into a Pry session where you can use commands like ls to see all of the variables.
Both the way you used Pry, and this way, work. However, the reason that load may not be working in your case is that local variables don't get carried over across files, like when you require one file from another.
Try loading this file:
#test.rb
y = "i dont get carried over cause i am a local variable"
b= "i dont get carried over cause i am a local variable"
AAA= "i am a constant so i carry over"
#per = "i am an instance var so i get carried over as well"
When you load it in Pry using load "test.rb" you can see that you can't get access to the local variables from that file.
I found this question googling but the proposed solution did not work for me because the file I wanted to load was not a class nor a script but a complex ruby config file, so I was not able to inject pry in the code.
But I also found an answer in Reddit linked to this gist that was exactly what I was looking for.
Doing a
Pry.toplevel_binding.eval File.read("stuff.rb")
Will effectively execute the ruby code of the file stuff.rb in the current pry session, leaving the resulting objects for inspecting.
I am trying to read a gzipped file using Zlib:GzipReader. This works as expected using ruby 1.9.3 but I am getting a method_missing error for each_line when using Rubinius.
Is there any way to read a gzipped file using Rubinius?
require 'zlib'
Zlib::GzipReader.open("lines.txt.gz").each_line { |line|
puts "#{line}"
}
Kernel(Zlib::GzipReader)#each_line (method_missing) at kernel/delta/kernel.rb:81
I believe this is a bug in Rubinius, you should consider opening an issue for it with the project. However, this workaround should get you going:
require 'zlib'
require 'stringio'
file = File.read("lines.txt.gz")
lines = Zlib::GzipReader.new(StringIO.new(file)).read
I want to get the content off this* page. Everything I've looked up gives the solution of parsing CSS elements; but, that page has none.
Here's the only code that I found that looked like it should work:
file = File.open('http://hiscore.runescape.com/index_lite.ws?player=zezima', "r")
contents = file.read
puts contents
Error:
tracker.rb:1:in 'initialize': Invalid argument - http://hiscore.runescape.com/index_lite.ws?player=zezima (Errno::EINVAL)
from tracker.rb:1:in 'open'
from tracker.rb:1
*http://hiscore.runescape.com/index_lite.ws?player=zezima
If you try to format this as a link in the post it doesn't recognize the underscore (_) in the URL for some reason.
You really want to use open() provided by the Kernel class which can read from URIs you just need to require the OpenURI library first:
require 'open-uri'
Used like so:
require 'open-uri'
file = open('http://hiscore.runescape.com/index_lite.ws?player=zezima')
contents = file.read
puts contents
This related SO thread covers the same question:
Open an IO stream from a local file or url
The appropriate way to fetch the content of a website is through the NET::HTTP module in Ruby:
require 'uri'
require 'net/http'
url = "http://hiscore.runescape.com/index_lite.ws?player=zezima"
r = Net::HTTP.get_response(URI.parse(url).host, URI.parse(url).path)
File.open() does not support URIs.
Best wishes,
Fabian
Please use open-uri, its support both uri and local files
require 'open-uri'
contents = open('http://www.google.com') {|f| f.read }