I have the following code:
task :wait_for_vault_ssl_up do
Timeout::timeout(15) do
begin
TCPSocket.new('localhost', 8200).close
puts "Vault ssl up and ready!"
true
rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, SocketError
puts "Vault ssl not ready... retrying"
retry
false
end
end
rescue Timeout::Error
false
end
In this, i set a timeout of 15 seconds and try to connect to the localhost:8200 my question is, if its not up, i currently do a retry in the rescue. is that necessary or will it automatically continue trying to connect for 15 seconds?
The use of retry will retry the execution of the whole block being rescued. So in this case, retry will start execution over again at your call to TCPSocket.new. This will happen immediately after the exception is rescued and there will not be a delay.
As Stefan has mentioned in the comments, you should use Kernel#sleep before calling retry. This will continue retrying until interrupted, so if you found yourself in a situation where the SSL connection will never be ready, the code will loop forever. You can either keep track of a retry count, or continue utilizing Timeout::timeout, using a longer timeout length, at which point the loop should give up entirely.
I think the clearer and simpler solution would be to use a retry counter, and give up after a predefined number of retries.
Related
I have a series of scripts that I have developed using Ruby and the Watir gem. Those are wrapped by Spinach, but that is beside what I am about to ask.
The intent of those scripts is to do some functional spot check or simply alleviate some very repetitive tasks.
They have been running well for a while, but lately, I've started to see a lot of failure due to Timeouts between the Chromedriver / Geckodriver (tried both browsers) and the scripts. Of course, I could simply restart the script, but when the success rate goes below 70 % it really starts to be aggravating.
What I ended up doing is wrap up all of my calls to Watir in a Proc with a Begin, rescue that would do a retry in case of a timeout.
This is ugly and violates so many rules that I am nearly ashamed to had to resort to this solution, but at least using this my scripts are now completing.
here is how I worked around the issue:
# takes a proc and wraps it around a series of rescue
def execute_block_and_rety_if_needed
yield
rescue Net::ReadTimeout
puts 'Read Timeout detected, retrying operation'
retry
rescue Net::HTTPRequestTimeOut
puts 'Http Request Timeout detected, retrying operation'
retry
rescue Errno::ETIMEDOUT
puts 'Errno::ETIMEDOUT detected, retrying operation'
retry
end
a sample use would look like this:
execute_block_and_rety_if_needed { #browser.link(name: 'OK').wait_until_present.click } # click the 'OK' button
As you can see, this clearly violates the DRY principle as I need to call this proc every single time.
My question is: how can I move this as a module / feature of Watir so that it picks it up automatically. (ideally I would add a maximum number of retry to prevent an infinite loop).
Version information:
- Chromedriver => 2.29.461585
- GeckoDriver => 0.16.1
- Firefox => ESR 52
- Chrome => 58
- Watir => 6.2.1
As far as the DRY comment, I referred to the fact that I had to wrap ALL of my Watir calls with the proc, sorry if this wasn't clear.
execute_block_and_rety_if_needed { #browser.link(name: 'User').wait_until_present.click } # click the 'Edit' button
execute_block_and_rety_if_needed { #browser.link(name: 'Cancel').wait_until_present.click } # click the 'Cancel' button
execute_block_and_rety_if_needed { #browser.link(name: 'OK').wait_until_present.click } # click the 'OK' button
The above is just an example that has to happen if I want to use the retry mechanism.
Given that you want to retry every command sent to the browser, you might want to consider addressing the issue in the underlying Selenium-WebDriver rather than Watir. Watir commands get sent to Selenium-WebDriver, which in turn sends them to the browser/driver.
Each command (or at least most) is currently sent through Selenium::WebDriver::Remote::Http:Default#request. You could patch the method to wrap it in a retry. Not only would your clicks retry for timeouts, but so would every other command - eg navigation, setting fields, getting values, etc.
# Patch to retry timeouts during requests
require 'watir'
module Selenium
module WebDriver
module Remote
module Http
module DefaultExt
def request(*args)
tries ||= 3
super
rescue Net::ReadTimeout, Net::HTTPRequestTimeOut, Errno::ETIMEDOUT => ex
puts "#{ex.class} detected, retrying operation"
(tries -= 1).zero? ? raise : retry
end
end
end
end
end
end
Selenium::WebDriver::Remote::Http::Default.prepend(Selenium::WebDriver::Remote::Http::DefaultExt)
# Then you can use Watir as usual
browser = Watir::Browser.new :chrome # this will retry timeouts
browser.goto('http://www.example.com') # this will also retry timeouts
browser.link.click # this will also retry timeouts
You shouldn't need to use a block for this. You can implement a method that does something like:
def ensure_click(element, retries = 3)
#retries ||= retries
element.click
rescue Net::ReadTimeout, Net::HTTPRequestTimeOut, Errno::ETIMEDOUT => ex
raise unless #retries > 0
#retries = #retries - 1
puts "#{ex.class} detected, retrying"
retry
end
...
ensure_click(#browser.link(name: 'User'))
...
That being said, those exceptions are not typically driver errors, but network issues of some sort. The are not normal.
I woul like to get unseen mails "as soon as possible", using a Ruby (2.1) script to implement IMAP IDLE ("push notify") feature.
With the help of some guys (see also: Support for IMAP IDLE in ruby), I wrote the script here:
https://gist.github.com/solyaris/b993283667f15effa579
def idle_loop(imap, search_condition, folder)
# https://stackoverflow.com/questions/4611716/how-imap-idle-works
loop do
begin
imap.select folder
imap.idle do |resp|
#trap_shutdown
# You'll get all the things from the server.
#For new emails you're only interested in EXISTS ones
if resp.kind_of?(Net::IMAP::UntaggedResponse) and resp.name == "EXISTS"
# Got something. Send DONE. This breaks you out of the blocking call
imap.idle_done
end
end
# We're out, which means there are some emails ready for us.
# Go do a search for UNSEEN and fetch them.
retrieve_emails(imap, search_condition, folder) { |mail| process_email mail}
#rescue Net::IMAP::Error => imap_err
# Socket probably timed out
# puts "IMAP IDLE socket probably timed out.".red
rescue SignalException => e
# https://stackoverflow.com/questions/2089421/capturing-ctrl-c-in-ruby
puts "Signal received at #{time_now}: #{e.class} #{e.message}".red
shutdown imap
rescue Exception => e
puts "Something went wrong at #{time_now}: #{e.class} #{e.message}".red
imap.noop
end
end
end
Now, all run smootly at first glance, BUT I have the exception
Something went wrong: SSL_write: bad write retry
at this line in code:
https://gist.github.com/solyaris/b993283667f15effa579#file-idle-rb-L189
The error happen when I leave the script running for more than... say more than 30 minutes.
BTW, the server is imap.gmail.com (arghh...), and I presume is something related to IMAP IDLE reconnection socket (I din't read yet the ruby UMAP library code) but I do not understand the reason of the exception;
Any idea for the reason if the exception ? Just trap the exception to fix the issue ?
thanks
giorgio
UPDATE
I modified a bit the exception handling (see gist code: https://gist.github.com/solyaris/b993283667f15effa579)
Now I got a Net::IMAP::Error connection closed I just restart the IMAP connection and it seems working...
Sorry for confusing, anyway in general any comments on code I wrote, IDLE protocol correct management, are welcome.
The IMAP IDLE RFC says to stop IDLE after at most 29 minutes and reissue a new IDLE command. IMAP servers are permitted to assume that the client is dead and has gone away after 31 minutes of inactivity.
You may also find that some NAT middleboxes silently sabotage your connection long before the half-hour is up, I've seen timeouts as short as about two minutes. (Every time I see something like that I scream "vivat ipv6!") I don't think there's any good solution for those middleboxes, except maybe to infect them with a vile trojan, but the bad solutions include adjusting your idle timeout if you get the SSL exception before a half-hour is up.
I am writing a Ruby client which will open tcp socket and stream data.
If I could not able to open socket within 20 secs I will trigger the Timeout error.
begin
Timeout::timeout(20) { socket = open_socket(host, port) }
rescue Errno::ECONNREFUSED
puts "Failed to connect to server"
rescue Timeout::Error
puts "Timeout error occurred while connecting to the server"
end
My open_socket method is given below.
def open_socket(host,port)
TCPSocket.new(host,port)
end
Code works fine. My question is
What is the standard timeout in secs in socket programming?
Does the timeout in secs can be setup according to our need?
I found 2 articles that seem to confirm the timeout as 20 seconds:
In Windows
In Linux (not ruby-specific)
The second article seems to imply that the timeout period is defined by the OS.
I do not have a answer for your second question.
That's exactly how you do it.The default value of timeout is 10 seconds.
timeout( sec) { ...}
Executes the block and returns true if the block execution terminates successfully prior to elapsing of the timeout period,
otherwise immediately terminates execution of the block and raises a TimeoutError exception.
require 'timeout'
status = timeout(5) {
# something that may take time
}
On Linux, the send/recv timeout can be accessed using setsockopt/getsocktopt.
Do man 7 socket and look for the SO_RCVTIMEO and SO_SNDTIMEO options. setsockopt/getsockopt is available on socket objects in Ruby.
I'm implementing a ruby server for handling sockets being created from GPRS modules. The thing is that when the module powers down, there's no indication that the socket closed.
I'm doing threads to handle multiple sockets with the same server. What I'm asking is this: Is there a way to use a timer inside a thread, reset it after every socket input, and that if it hits the timeout, closes the thread? Where can I find more information about this?
EDIT: Code example that doesn't detect the socket closing
require 'socket'
server = TCPServer.open(41000)
loop do
Thread.start(server.accept) do |client|
puts "Client connected"
begin
loop do
line = client.readline
open('log.txt', 'a') { |f|
f.puts line.strip
}
end
rescue
puts "Client disconnected"
end
end
end
I think you need a heartbeat mechanism.
At a guess, your sockets are inexplably closing because you're not catching exceptions that are raised when they are closed by the remote end.
you need to wrap the connection handler in an exception catching block.
Without knowing what module/model you're using I will just fudge it and say you have a process_connection routine. So you need to do something like this:
def process_connection(conn)
begin
# do stuff
rescue Exception => e
STDERR.print "Caught exception #{e}: #{e.message}\n#{e.backtrace}\n"
ensure
conn.close
end
end
This will catch all exceptions and dump them to stderr with a stack trace. From there you can see what is causing them, and possibly handle them more gracefully elsewhere.
Just check the standar API Timeout:
require 'timeout'
status = Timeout::timeout(3){sleep(1)}
puts status.inspect
status = Timeout::timeout(1){sleep(2)}
I read this snippet, and I am trying to understand how I can use retry, and I am unable to think of a use. How are others using it?
#!/usr/bin/ruby
for i in 1..5
retry if i > 2
puts "Value of local variable is #{i}"
end
There are several use cases. Here's one from the Programming Ruby book
#esmtp = true
begin
# First try an extended login. If it fails because the
# server doesn't support it, fall back to a normal login
if #esmtp then
#command.ehlo(helodom)
else
#command.helo(helodom)
end
rescue ProtocolError
if #esmtp then
#esmtp = false
retry
else
raise
end
end
An other common case is the email delivery. You might want to retry the SMTP delivery for N times adding a sleep between each retry to avoid temporary issues caused by network connectivity.
I am using it for a module that makes an api call to a 3rd party web api, and so if it fails, I retry 2 more times.