Skipping unresponsive host using net/http - ruby

Question

I am using net/http to send a bunch of request to some internal IP addresses.
Here's a snippet of the code:
File.open("internalcorpIPs", "r") do |f|
f.each_line do |line|
puts line
res = Net::HTTP.get_response(URI.parse(line))
getCode = res.code
end
end
I'm strictly just making a request to http://IP and https://IP but it seems like this method only works if every single IP/line address is live. How do I skip IP addresses with no webserver (or 80/443 ports)?
Is it possible to make it read the line, and move on to the next if no response code was returned?
Thanks!

Answer 1

You could simply wrap your request in begin/rescue block like this:
File.open("internalcorpIPs", "r") do |f|
f.each_line do |line|
puts line
begin
# strip and encode uri from the file
uri = URI.parse(URI.encode(line.strip))
res = Net::HTTP.get_response(uri)
getCode = res.code
rescue Timeout::Error => e
puts e
false
end
end
end
But you will wait for 60 seconds at least before going in timeout, so I suggest to decrease the timeout. Furthermore, you could introduce an additional guard clause to check if the uri contains the scheme http:// or https://, otherwise raise an exception (or something else).
require 'net/http'
File.open("internalcorpIPs", "r") do |f|
f.each do |line|
puts line
begin
# strip and encode uri from the file
uri = URI.parse(URI.encode(line.strip))
# if uri misses the schema (http:// or https://) -> raise error
raise URI::Error, "uri #{uri} miss the scheme" unless uri.scheme
http = Net::HTTP.new(uri.host, uri.port)
http.open_timeout = 2 # seconds
http.read_timeout = 2 # seconds
http.start do |conn|
response = conn.request_get(path = '/')
puts response.code
end
rescue Timeout::Error, URI::Error, SocketError => e
puts e
false
end
end
end
Additional notes:
Open Timeout
Number of seconds to wait for the connection to open. Any number may be used, including Floats for fractional seconds. If the HTTP object cannot open a connection in this many seconds, it raises a Net::OpenTimeout exception. The default value is 60 seconds.
Read Timeout
Number of seconds to wait for one block to be read (via one read(2) call). Any number may be used, including Floats for fractional seconds. If the HTTP object cannot read data in this many seconds, it raises a Net::ReadTimeout exception. The default value is 60 seconds.
URI Scheme
Difference between generic uri (URI::Generic) and http uri (URI::HTTP).
uri = URI.parse('1.1.1.1')
=> #<URI::Generic 1.1.1.1>
uri.scheme
=> nil
uri.host
=> nil
uri.port
=> nil
uri.path
=> "1.1.1.1"
uri = URI.parse('http://1.1.1.1')
=> #<URI::HTTP http://1.1.1.1>
uri.scheme
=> "http"
uri.host
=> "1.1.1.1"
uri.port
=> 80
uri.path
=> ""
references:
Net::HTTP Api
URI Module
hope it helps!
UPDATE
URI.parse accepts a string as argument and automatically set the port if not specified:
❯ irb
2.2.0 :001 > require 'net/http'
=> true
2.2.0 :002 > uri = URI.parse('http://1.1.1.1')
=> #<URI::HTTP http://1.1.1.1>
2.2.0 :003 > uri.host
=> "1.1.1.1"
2.2.0 :004 > uri.port
=> 80
2.2.0 :005 > uri2 = URI.parse('http://mydomain')
=> #<URI::HTTP http://mydomain>
2.2.0 :006 > uri2.host
=> "mydomain"
2.2.0 :007 > uri2.port
=> 80
2.2.0 :008 > uri3 = URI.parse('https://mydomain')
=> #<URI::HTTPS https://mydomain>
2.2.0 :009 > uri3.host
=> "mydomain"
2.2.0 :010 > uri3.port
=> 443