I am getting the following server response error while trying to scrape SERP results:
/Users/*********/.rvm/gems/ruby-2.3.0/gems/mechanize-2.7.5/lib/mechanize/http/agent.rb:323:in `fetch': 503 => Net::HTTPServiceUnavailable for http://******.*****.com/sorry/index?continue=http://www.********.com/search%3Fq%3D<term1>%2B<term2> -- unhandled response (Mechanize::ResponseCodeError)
I am trying to figure out how to escape the error / exception, so that the program will continue to run instead of automatically exiting.
Like anything in Ruby it probably boils down to rescue and recover:
loop do
begin
Mechanize.do_stuff!
# Success!
break
rescue Mechanize::ResponseCodeError
# Server-side failure, so let's try again after a quick break
sleep(10)
end
end
Note the sleep(10) is there to avoid slamming the server furiously and making it malfunction even harder.
Related
I'm running an external command using the Open3.popen2e function. The external command fails spectacularly halfway through. Unfortunately this also kills my ruby process that is using popen2e. What is the reason, and how can I avoid?
begin
Open3.popen2e("node mynode.js") do |i, oe|
oe.each do |ln|
puts ln.chomp
end
end
rescue => exception
puts exception.message
end
Using ruby 2.5.1 on Ubuntu
Not sure about the reason, but a non-StandardError may be raised. So, what about changing your rescue to:
rescue Exception => exception
That may give you more clues as to what is happening.
I am seeing this pattern in someone else's Rakefile:
begin
sh "..."
rescue
abort
end
What's the author doing here? Why rescue if I'm going to abort anyway? Do I understand correctly that this mutes the potential shell error?
I am new to ruby and rake. As far as I understand, the sh is rake's shorthand for FileUtils.sh() (ref). And FileUtils.sh() may raise a RuntimeError (ref).
Does this mean that we are effectively throwing away the information/message in the RuntimeError and exiting rake with an error status, but without a specific error message? Is this a rake/ruby pattern? Any recommended reading?
Some of my tests pass one time then fail with "end of file reached(EOFError)". Can't figure out what causes this consistency issue. Sometimes it fails when filling out a form. Other times it fails when clicking a button.
Using the following:
OSX 10.9.3
Watir-webdriver 0.6.10
Ruby 1.9.3
Chrome 35.0
Chromedriver ChromeDriver v2.10
Not sure what the issue is but a simple work around to get rid of this error is to use a Begin/Rescue statement around the code that is causing this error (check what line number the terminal output says is causing the error).
For example:
browser.close #This is the line giving the EOFError
Do the following:
begin
browser.close #if there is an error: jump to the rescue statement
rescue
#don't put any code in the rescue statement (ignore the error)
end
#rest of code
The way the begin/rescue statement works is if the code in the begin statement causes an error, it will run the code in the rescue statement. In this case, since there isn't any code in the rescue statement, it will merely ignore the error and continue with the rest of the code.
Is there a Perl equivalent END block in Ruby? In Perl, if I specify an END block, the code in that block will get executed no matter where the program bails out. It is great functionality for closing open file handles. Does Ruby support similar functionality? I tried Ruby's "END{}" block but that doesnt seem to get called if I had an exit in the code due to an error.
Thanks!
Use at_exit, which will run regardless of whether an exception was raised or not:
at_exit { puts 'exited!' }
raise
prints "exited" as expected.
You should only consider this if you cannot use an ensure, as at_exit causes logic to reside far away from where the actual exit occurs.
Yes. A block may have an 'ensure' clause. Here's an example:
begin
# This will cause a divide by zero exception
puts 3 / 0
rescue Exception => e
puts "An error occurred: #{e}"
ensure
puts "I get run anyway"
end
Running this produces:
An error occurred: divided by 0
I get run anyway
I'm working with the Twitter Gem and I've created a long running ruby task. I would like it to be able to handle common errors so I'm looking to build a list of those I should consider to protect against (for example the fail whale 500)
Here is the begin/end loop my code functions in:
Begin
# My (omitted) very long ruby task
# filled with Twitter API requests
rescue Errno::ENOENT
sleep(5)
logger.info "ENOENT error - attempting to retry"
retry
rescue Errno::ETIMEDOUT
sleep(5)
logger.info " Operation timed out - attempting to retry"
retry
rescue Errno::ECONNRESET
sleep(5)
logger.info "Connection reset by peer - attempting to retry"
retry
end
Can you think of any other Errors to protect and retry against? Is this a well structured way to handle errors? What are some design implementations I should consider?
Consider having a catch-all exception handler at the end that logs what kind of exception was encountered and re-raises it. Your script may fail the first time, but at least you'll find out why.
begin
# My (omitted) very long ruby task
# filled with Twitter API requests
rescue Errno::ENOENT
sleep(5)
logger.info "ENOENT error - attempting to retry"
retry
rescue Errno::ETIMEDOUT
sleep(5)
logger.info " Operation timed out - attempting to retry"
retry
rescue Errno::ECONNRESET
sleep(5)
logger.info "Connection reset by peer - attempting to retry"
retry
rescue # This rescues StandardError and its children
sleep(5)
# The next line is somewhat pseudocode, because I don't use logger
logger.this_is_somewhat_bad "Somewhat bad exception #{$!.class} #{$!} happened - I'm giving up"
raise
rescue Exception
sleep(5)
# The next line is somewhat pseudocode, because I don't use logger
logger.omg_wtf_bbq "Really bad exception #{$!.class} #{$!} happened - I'm giving up"
raise
end
I also catch the Twitter::Forbidden error in my code that uses the Twitter gem.
Alternatively, you could try rescue SystemCallError, since all Errno errors are subclasses of that.