Unable to rescue from Redis connection refusal - ruby

I am attempting to write a function which tries to connect to Redis using default TCP settings, and if that fails, tries to connect to Redis through a unix socket. My intent is to have a single connection script that works on all my systems, some of which use TCP and others which use sockets.
However, I can't seem to rescue from the failed TCP connection. Here is my test script.
require "redis"
def r
begin
$redis ||= Redis.new
rescue
$redis = Redis.new(:path => "/tmp/redis.sock")
end
end
puts "You have #{r.keys.count} redis keys"
The rescue block never gets executed and instead an exception is raised. Here is the output of this script.
/usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/client.rb:236:in `rescue in establish_connection': Connection refused - Unable to connect to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED)
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/client.rb:222:in `establish_connection'
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/client.rb:23:in `connect'
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/client.rb:247:in `ensure_connected'
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/client.rb:137:in `block in process'
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/client.rb:206:in `logging'
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/client.rb:136:in `process'
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/client.rb:46:in `call'
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis.rb:246:in `block in keys'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
from /usr/local/rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis.rb:245:in `keys'
from scripts/redis.rb:11:in `<main>'
I have verified that Redis.new(:path => "/tmp/redis.sock") works as expected. I have tried to be more specific with my rescue block by using rescue Errno::ECONNREFUSED to no avail. I'm not sure why I cannot catch this exception.
Any ideas?

It turns out the exception is not being thrown when calling Redis.new. The exception doesn't get thrown until certain methods (in this case, Redis#keys) on the connection object are called. This revised connection function appears to do the trick.
require "redis"
def r
begin
$redis ||= Redis.new
$redis.inspect # needed to know if connection failed
rescue
$redis = Redis.new(:path => "/tmp/redis.sock")
end
$redis
end

I found that $redis.inspect didn't actually exercise the REDIS connection. I replaced it with $redis.keys and that correctly threw the exception. Note, am running on Heroko and it passes in the environment variable REDISTOGO_URL. I then have a constant REDIS that I use throughout the application.
In my config/initializers/redis.rb:
uri = URI.parse(ENV['REDISTOGO_URL'])
begin
redis ||= Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
redis.keys # needed to know if connection failed
REDIS = redis
rescue
puts("Redis not loaded on #{uri.port}")
REDIS = nil
end

Related

Ruby, net/ssh with proxy connection to router, channel hangs

My current network setup is like this:
LocalMachine <---SSH---> Server <---Telnet---> Router.
I am trying to connect to the Router from LocalMachine via Server (direct connection is not possible, Server acts as sort of proxy). Ruby's net/ssh gem is used for the ssh connection to the Server.
require 'net/ssh'
#hostname = "10.16.96.40"
#username = "server_username"
#password = "server_password"
Net::SSH::start(#hostname, #username, :password => #password) do |session|
session.open_channel do |channel|
channel.send_channel_request "shell" do |ch, success|
if success
puts "user shell started successfully"
else
puts "could not start user shell"
end
end
channel.send_data("telnet 192.168.11.201\n")
channel.send_data("router_username\n")
channel.send_data("router_password\n")
channel.send_data("sh ip int brief\n")
channel.on_data do |ch, data|
print data
end
end
session.loop
end
The script gets connected to the router. It gets the printout. But then the session hangs when expected it to be closed.
When I stop script execution manually with Ctrl+C I get the following message:
^C/usr/local/lib/ruby/gems/2.2.0/gems/net-ssh-3.2.0/lib/net/ssh/ruby_compat.rb:25:in `select': Interrupt
from /usr/local/lib/ruby/gems/2.2.0/gems/net-ssh-3.2.0/lib/net/ssh/ruby_compat.rb:25:in `io_select'
from /usr/local/lib/ruby/gems/2.2.0/gems/net-ssh-3.2.0/lib/net/ssh/connection/session.rb:210:in `process'
from /usr/local/lib/ruby/gems/2.2.0/gems/net-ssh-3.2.0/lib/net/ssh/connection/session.rb:170:in `block in loop'
from /usr/local/lib/ruby/gems/2.2.0/gems/net-ssh-3.2.0/lib/net/ssh/connection/session.rb:170:in `loop'
from /usr/local/lib/ruby/gems/2.2.0/gems/net-ssh-3.2.0/lib/net/ssh/connection/session.rb:170:in `loop'
from 25_net_ssh_channel_shell.rb:33:in `block in <main>'
from /usr/local/lib/ruby/gems/2.2.0/gems/net-ssh-3.2.0/lib/net/ssh.rb:240:in `start'
from 25_net_ssh_channel_shell.rb:9:in `<main>'
Any idea how to close the channel/session and get rid of the blinking cursor?

Ruby gem 'tcp_timeout' failed to suppress raising error

I wrote a script using 'socket' that connects to a host and port and because socket.timeout doesn't really work I tried using the 'tcp_timeout' gem that works properly but I can't seem to suppress the error raised when connect/read/write timeout happens. Any idea where am I wrong?
begin
socket = TCPTimeout::TCPSocket.new(server, port, connect_timeout: 6, read_timeout: 6)
unless socket.read(12) =~ /^SMTH\n$/
puts "[!] #{server} banner error"
exit(1)
end
rescue TCPTimeout::SocketTimeout => err
puts "[!] #{server} Timeout"
exit(1)
end
The error raised, as expected is a read timeout error:
/usr/local/rvm/gems/ruby-1.9.3-p551/gems/tcp_timeout-0.1.1/lib/tcp_timeout.rb:160:in `select_timeout': read timeout (TCPTimeout::SocketTimeout)
from /usr/local/rvm/gems/ruby-1.9.3-p551/gems/tcp_timeout-0.1.1/lib/tcp_timeout.rb:108:in `block in read'
from /usr/local/rvm/gems/ruby-1.9.3-p551/gems/tcp_timeout-0.1.1/lib/tcp_timeout.rb:107:in `loop'
from /usr/local/rvm/gems/ruby-1.9.3-p551/gems/tcp_timeout-0.1.1/lib/tcp_timeout.rb:107:in `read'
from ./myhost.rb:67:in `<main>'
I tried even:
rescue TCPTimeout::SocketTimeout, StandardError, Timeout::Error => err
Same thing happens.
Author of tcp_timeout here; your code looks correct. This snippet works as expected (for me):
require 'tcp_timeout'
begin
socket = TCPTimeout::TCPSocket.new('stackoverflow.com', 80, read_timeout: 1)
socket.read(100)
rescue TCPTimeout::SocketTimeout => e
puts 'Rescued!', e
end
If you can find a snippet that fails reliably against a public server please file a bug: https://github.com/lann/tcp-timeout-ruby/issues

How is Ruby TCPSocket timeout defined?

$ irb
1.9.3-p448 :001 > require 'socket'
=> true
1.9.3-p448 :002 > TCPSocket.new('www.example.com', 111)
gives
Errno::ETIMEDOUT: Operation timed out - connect(2)
Questions:
How can I define the timeout value for TCPSocket.new?
How can I properly catch the timeout (or, in general, socket) exception(s)?
At least since 2.0 one can simply use Socket::tcp:
Socket.tcp("www.ruby-lang.org", 10567, connect_timeout: 5) {}
Note the block at the end of the expression, which is used to get connection closed in case such is established.
For older versions #falstru answer appears to be best.
Use begin .. rescue Errno::ETIMEDOUT to catch the timeout:
require 'socket'
begin
TCPSocket.new('www.example.com', 111)
rescue Errno::ETIMEDOUT
p 'timeout'
end
To catch any socket exceptions, use SystemCallError instead.
According to the SystemCallError documentation:
SystemCallError is the base class for all low-level platform-dependent errors.
The errors available on the current platform are subclasses of
SystemCallError and are defined in the Errno module.
TCPSocket.new does not support timeout directly.
Use Socket::connect_non_blocking and IO::select to set timeout.
require 'socket'
def connect(host, port, timeout = 5)
# Convert the passed host into structures the non-blocking calls
# can deal with
addr = Socket.getaddrinfo(host, nil)
sockaddr = Socket.pack_sockaddr_in(port, addr[0][4])
Socket.new(Socket.const_get(addr[0][0]), Socket::SOCK_STREAM, 0).tap do |socket|
socket.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1)
begin
# Initiate the socket connection in the background. If it doesn't fail
# immediatelyit will raise an IO::WaitWritable (Errno::EINPROGRESS)
# indicating the connection is in progress.
socket.connect_nonblock(sockaddr)
rescue IO::WaitWritable
# IO.select will block until the socket is writable or the timeout
# is exceeded - whichever comes first.
if IO.select(nil, [socket], nil, timeout)
begin
# Verify there is now a good connection
socket.connect_nonblock(sockaddr)
rescue Errno::EISCONN
# Good news everybody, the socket is connected!
rescue
# An unexpected exception was raised - the connection is no good.
socket.close
raise
end
else
# IO.select returns nil when the socket is not ready before timeout
# seconds have elapsed
socket.close
raise "Connection timeout"
end
end
end
end
connect('www.example.com', 111, 2)
The above code comes from "Setting a Socket Connection Timeout in Ruby".
If you like the idea of avoiding the pitfalls of Timeout, but prefer to avoid having to deal with your own implementation of the *_nonblock+select implementation, you can use the tcp_timeout gem.
The tcp_timeout gem monkey-patches TCPSocket#connect, #read, and #write so that they use non-blocking I/O and have timeouts that you can enable.
You to make a timeout you can use ruby's Timeout module:
reqiure 'socket'
reqiure 'timeout'
begin
Timeout.timeout(10) do
begin
TCPSocket.new('www.example.com', 111)
rescue Errno::ENETUNREACH
retry # or do something on network timeout
end
end
rescue Timeout::Error
puts "timed out"
# do something on timeout
end
and you'll get after 10 seconds:
# timed out
# => nil
NOTE: Some people may think that it is dangerous solution, well, this opinion has right to exist, but there were no real investigations proceeded, so, that opinion is just a hypothesis. And currently it is better to use internal ruby's timeout engine in Socket class like the following:
Socket.tcp("www.ruby-lang.org", 80, connect_timeout: 80) do |sock|
...
end

How to handle timeout with net/http in ruby while sending requests to IPs from given IP range and skip IPs with timeout and move to next ones?

I want to handle timeout for IP range taken from console for which I make requests to IPs within taken range and getting timeout error.
I want to make requests to all IPs and get responses from them.
For IP that time out , want to skip it and move to next one. How to handle this so loop dont get exception and script sends request to all IPs that can give response handling timed out ones.
Attaching code here:
require 'net/http'
require 'uri'
require 'ipaddr'
puts "Origin IP:"
originip = gets()
(IPAddr.new("209.85.175.121")..IPAddr.new("209.85.175.150")).each do |address|
req = Net::HTTP.get(URI.parse("http://#{address.to_s}"))
puts req
end
Error:
C:/Ruby187/lib/ruby/1.8/net/http.rb:560:in `initialize': A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. - connect(2) (Errno::ETIMEDOUT)
from C:/Ruby187/lib/ruby/1.8/net/http.rb:560:in `open'
from C:/Ruby187/lib/ruby/1.8/net/http.rb:560:in `connect'
from C:/Ruby187/lib/ruby/1.8/timeout.rb:53:in `timeout'
from C:/Ruby187/lib/ruby/1.8/timeout.rb:101:in `timeout'
from C:/Ruby187/lib/ruby/1.8/net/http.rb:560:in `connect'
from C:/Ruby187/lib/ruby/1.8/net/http.rb:553:in `do_start'
from C:/Ruby187/lib/ruby/1.8/net/http.rb:542:in `start'
from C:/Ruby187/lib/ruby/1.8/net/http.rb:379:in `get_response'
from C:/Ruby187/lib/ruby/1.8/net/http.rb:356:in `get'
from IP Range 2.rb:9
from IP Range 2.rb:8:in `each'
Just like Marc says. You should rescue the exception. Like so:
begin
response = Net::HTTP.get(...)
rescue Errno::ECONNREFUSED => e
# Do what you think needs to be done
end
Also, what you get back from the call to get() is a response, not a request.
Catch the exception using timeout,
require 'timeout'
(IPAddr.new("209.85.175.121")..IPAddr.new("209.85.175.150")).each do |address|
begin
req = Net::HTTP.get(URI.parse("http://#{address.to_s}"))
puts req
rescue Timeout::Error => exc
puts "ERROR: #{exc.message}"
rescue Errno::ETIMEDOUT => exc
puts "ERROR: #{exc.message}"
# uncomment the following two lines, if you are not able to track the exception type.
#rescue Exception => exc
# puts "ERROR: #{exc.message}"
end
end
Edit: When we rescue Timeout::Error, only those exceptions which belongs to Timeout::Error class will be caught. We need to catch the raised exception using their error class, updated the code accordingly.

how to test open-uri url exist before processing any data

I'm trying to process content from a list of links using "open-uri" in ruby (1.8.6), but the bad thing happens when I'm getting an error when one link is broken or requires authentication:
open-uri.rb:277:in `open_http': 404 Not Found (OpenURI::HTTPError)
from C:/tools/Ruby/lib/ruby/1.8/open-uri.rb:616:in `buffer_open'
from C:/tools/Ruby/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
from C:/tools/Ruby/lib/ruby/1.8/open-uri.rb:162:in `catch'
or
C:/tools/Ruby/lib/ruby/1.8/net/http.rb:560:in `initialize': getaddrinfo: no address associated with hostname. (SocketError)
from C:/tools/Ruby/lib/ruby/1.8/net/http.rb:560:in `open'
from C:/tools/Ruby/lib/ruby/1.8/net/http.rb:560:in `connect'
from C:/tools/Ruby/lib/ruby/1.8/timeout.rb:53:in `timeout'
or
C:/tools/Ruby/lib/ruby/1.8/net/protocol.rb:133:in `sysread': An existing connection was forcibly closed by the remote host. (Errno::ECONNRESET)
from C:/tools/Ruby/lib/ruby/1.8/net/protocol.rb:133:in `rbuf_fill'
from C:/tools/Ruby/lib/ruby/1.8/timeout.rb:62:in `timeout'
from C:/tools/Ruby/lib/ruby/1.8/timeout.rb:93:in `timeout'
is there a way to test the response (url) before processing any data?
the code is:
require 'open-uri'
smth.css.each do |item|
open('item[:name]', 'wb') do |file|
file << open('item[:href]').read
end
end
Many thanks
You could try something along the lines of
require 'open-uri'
smth.css.each do |item|
begin
open('item[:name]', 'wb') do |file|
file << open('item[:href]').read
end
rescue => e
case e
when OpenURI::HTTPError
# do something
when SocketError
# do something else
else
raise e
end
rescue SystemCallError => e
if e === Errno::ECONNRESET
# do something else
else
raise e
end
end
end
I don't know of any way of testing the connection without opening it and trying, so rescuing these errors would be the only way I can think of. The thing to be aware of is that OpenURI::HTTPError and SocketError are both subclasses of StandardError, whereas Errno::ECONNRESET is a subclass of SystemCallError. So rescue => e won't catch Errno::ECONNRESET.
I was able to solve this problem by using a conditional if/else statement to check the return value of the action for "failure":
def controller_action
url = "some_API"
response = open(url).read
data = JSON.parse(response)["data"]
if response["status"] == "failure"
redirect_to :action => "home"
else
do_something_else
end
end

Resources