Crawling list of URLs and bypass those with no DNS - ruby

Question

I am crawling a large list of URLS with Ruby but all the URLS I have are not active and not associated with a DNS. When I hit that url my crawler errors.
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'net/http'
require 'colorize'
URL_LIST = [
'http://website.com',
'http://website.net'
]
URL_LIST.each do |url|
item = "#{url}"
resp = Net::HTTP.get_response(URI.parse(item))
case resp.code.to_i
when 200
puts "Success: #{url}".green
when 301..303
new_url = resp['location']
puts "Redirect #{url} => #{new_url}".yellow
else
resp.code
end
end
When I run this script and hit a bad url I receive an error like this:
/Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http.rb:879:in `initialize': getaddrinfo: nodename nor servname provided, or not known (SocketError)
from /Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http.rb:879:in `open'
from /Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http.rb:879:in `block in connect'
from /Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/timeout.rb:76:in `timeout'
from /Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http.rb:878:in `connect'
from /Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
from /Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http.rb:852:in `start'
from /Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http.rb:583:in `start'
from /Users/<name>/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http.rb:478:in `get_response'
from spider.rb:808:in `block in <main>'
from spider.rb:806:in `each'
from spider.rb:806:in `<main>'

Answer 1

Use a begin/rescue block to rescue the error and output error info in red:
URL_LIST = [
'http://website.com',
'http://sdfasdfwqeasdfasdfr.com',
'http://website.net'
]
URL_LIST.each do |url|
item = "#{url}"
begin
resp = Net::HTTP.get_response(URI.parse(item))
case resp.code.to_i
when 200
puts "Success: #{url}".green
when 301..303
new_url = resp['location']
puts "Redirect #{url} => #{new_url}".yellow
else
resp.code
end
rescue SocketError => e
puts "Error: #{url} - #{e}".red
end
end
The output will look like:
Redirect http://website.com => http://www.website.com/
Error: http://sdfasdfwqeasdfasdfr.com - getaddrinfo: nodename nor servname provided, or not known
Success: http://website.net