I'm writing a Ruby program using the mysql2 gem. This program iterates over large sets of data and makes lots of individual database queries using the mysql2 .query method. Occasionally, one of them will fail (usually because it's a duplicate of a previous row), ending the program.
I'd like to change this, so that when a query fails, the information is logged to a log file, the query is skipped, and the program continues to run. But I have no idea how I would go about doing something like this. Can anyone let me know where to start reading, or what I should be doing?
Put the Code that might throw an error in an begin ... rescue ... end block:
begin
# code that might throw the database error
rescue ActiveRecord::StatementInvalid => e
puts "An error occurred: #{e.message}"
end
Not that ActiveRecord::StatementInvalid will only invalid queries (like duplicate entries), but not other errors like when the db connection died. It is good practice to catch only the most specific errors you want to handle and not all possible errors.
I am using Mechanize in Ruby to post forms to a site to obtain data. Occasionally, I will get the error
too many connection resets (due to Operation timed out - Errno::ETIMEDOUT) after 13 requests on 34234234234242, last used 20.518373 seconds ago
This will crash the server. I would like to catch this error and handle it(by retrying later on).
I tried
begin
postForm(form)
rescue Errno::ETIMEDOUT
puts "=====>TimeOut ERROR!:"
end
But it is not catching the error.
What am I missing?
Thanks.
It looks like Mechanize caught the Errno::ETIMEDOUT error and threw a new exception containing the error message. Therefore, the exception class name is different, and can't be caught with Errno::ETIMEDOUT.
To determine the type of error, try this:
begin
postForm(form)
rescue => e
puts e.class
end
That will give you the exception class name, and you can update the rescue clause.
If you read the entire error message, you see that you failed because of a 'too many connection resets' : The timeout is only what caused the connection to reset 13 times, and has already been rescued.
So if you read the source of Net-Http-persistent >here<, you will see that the error you need to catch is :
Net::HTTP::Persistent::Error
I am trying to make calls to an external API. I handle four or more exceptions for the call.
If I make multiple calls, the code increases very quick. Should I be writing a wrapper for each such call which handles the exceptions and returns data?
Here is an example of such code (this is not mine). The call to user_search is followed by the exception handling.
Note: I am not using Rails
begin
#twitter = Twitter.user_search(name)
rescue Twitter::Unauthorized
puts "Not authorized. Please check the Twitter credentials at the top of the script."
break
rescue Twitter::BadRequest => e
puts "Hit rate limit. Continuing scraping at #{e.ratelimit_reset}"
sleep e.retry_after
retry
rescue Exception => e
puts "Something else went wrong:"
puts e.message
end
I've changed the title of the question. I think the issue is more how to handle long exception handling code. In the example code suppose I have multiple calls to the twitter API followed by exception handling, it seems like the exception handling code disrupts reading the code which does the actual work.
Write your exception handler around a chunk of code so that, if the first line in the block fails, you're happy skipping all the code up until the last line in the block.
If an exception invalidates the whole rest of the method that the handler appears in, consider letting the exception bubble up to the next layer. Not everything necessarily has to be caught by your method.
Often I find myself writing exception handlers around single lines of code (with suitable recovery code) but it's not a rule.
Ryan Davis’s Ruby QuickRef says (without explanation):
Don’t rescue Exception. EVER. or I will stab you.
Why not? What’s the right thing to do?
TL;DR: Use StandardError instead for general exception catching. When the original exception is re-raised (e.g. when rescuing to log the exception only), rescuing Exception is probably okay.
Exception is the root of Ruby's exception hierarchy, so when you rescue Exception you rescue from everything, including subclasses such as SyntaxError, LoadError, and Interrupt.
Rescuing Interrupt prevents the user from using CTRLC to exit the program.
Rescuing SignalException prevents the program from responding correctly to signals. It will be unkillable except by kill -9.
Rescuing SyntaxError means that evals that fail will do so silently.
All of these can be shown by running this program, and trying to CTRLC or kill it:
loop do
begin
sleep 1
eval "djsakru3924r9eiuorwju3498 += 5u84fior8u8t4ruyf8ihiure"
rescue Exception
puts "I refuse to fail or be stopped!"
end
end
Rescuing from Exception isn't even the default. Doing
begin
# iceberg!
rescue
# lifeboats
end
does not rescue from Exception, it rescues from StandardError. You should generally specify something more specific than the default StandardError, but rescuing from Exception broadens the scope rather than narrowing it, and can have catastrophic results and make bug-hunting extremely difficult.
If you have a situation where you do want to rescue from StandardError and you need a variable with the exception, you can use this form:
begin
# iceberg!
rescue => e
# lifeboats
end
which is equivalent to:
begin
# iceberg!
rescue StandardError => e
# lifeboats
end
One of the few common cases where it’s sane to rescue from Exception is for logging/reporting purposes, in which case you should immediately re-raise the exception:
begin
# iceberg?
rescue Exception => e
# do some logging
raise # not enough lifeboats ;)
end
The real rule is: Don't throw away exceptions. The objectivity of the author of your quote is questionable, as evidenced by the fact that it ends with
or I will stab you
Of course, be aware that signals (by default) throw exceptions, and normally long-running processes are terminated through a signal, so catching Exception and not terminating on signal exceptions will make your program very hard to stop. So don't do this:
#! /usr/bin/ruby
while true do
begin
line = STDIN.gets
# heavy processing
rescue Exception => e
puts "caught exception #{e}! ohnoes!"
end
end
No, really, don't do it. Don't even run that to see if it works.
However, say you have a threaded server and you want all exceptions to not:
be ignored (the default)
stop the server (which happens if you say thread.abort_on_exception = true).
Then this is perfectly acceptable in your connection handling thread:
begin
# do stuff
rescue Exception => e
myLogger.error("uncaught #{e} exception while handling connection: #{e.message}")
myLogger.error("Stack trace: #{backtrace.map {|l| " #{l}\n"}.join}")
end
The above works out to a variation of Ruby's default exception handler, with the advantage that it doesn't also kill your program. Rails does this in its request handler.
Signal exceptions are raised in the main thread. Background threads won't get them, so there is no point in trying to catch them there.
This is particularly useful in a production environment, where you do not want your program to simply stop whenever something goes wrong. Then you can take the stack dumps in your logs and add to your code to deal with specific exception further down the call chain and in a more graceful manner.
Note also that there is another Ruby idiom which has much the same effect:
a = do_something rescue "something else"
In this line, if do_something raises an exception, it is caught by Ruby, thrown away, and a is assigned "something else".
Generally, don't do that, except in special cases where you know you don't need to worry. One example:
debugger rescue nil
The debugger function is a rather nice way to set a breakpoint in your code, but if running outside a debugger, and Rails, it raises an exception. Now theoretically you shouldn't be leaving debug code lying around in your program (pff! nobody does that!) but you might want to keep it there for a while for some reason, but not continually run your debugger.
Note:
If you've run someone else's program that catches signal exceptions and ignores them, (say the code above) then:
in Linux, in a shell, type pgrep ruby, or ps | grep ruby, look for your offending program's PID, and then run kill -9 <PID>.
in Windows, use the Task Manager (CTRL-SHIFT-ESC), go to the "processes" tab, find your process, right click it and select "End process".
If you are working with someone else's program which is, for whatever reason, peppered with these ignore-exception blocks, then putting this at the top of the mainline is one possible cop-out:
%W/INT QUIT TERM/.each { |sig| trap sig,"SYSTEM_DEFAULT" }
This causes the program to respond to the normal termination signals by immediately terminating, bypassing exception handlers, with no cleanup. So it could cause data loss or similar. Be careful!
If you need to do this:
begin
do_something
rescue Exception => e
critical_cleanup
raise
end
you can actually do this:
begin
do_something
ensure
critical_cleanup
end
In the second case, critical cleanup will be called every time, whether or not an exception is thrown.
TL;DR
Don't rescue Exception => e (and not re-raise the exception) - or you might drive off a bridge.
Let's say you are in a car (running Ruby). You recently installed a new steering wheel with the over-the-air upgrade system (which uses eval), but you didn't know one of the programmers messed up on syntax.
You are on a bridge, and realize you are going a bit towards the railing, so you turn left.
def turn_left
self.turn left:
end
oops! That's probably Not Good™, luckily, Ruby raises a SyntaxError.
The car should stop immediately - right?
Nope.
begin
#...
eval self.steering_wheel
#...
rescue Exception => e
self.beep
self.log "Caught #{e}.", :warn
self.log "Logged Error - Continuing Process.", :info
end
beep beep
Warning: Caught SyntaxError Exception.
Info: Logged Error - Continuing Process.
You notice something is wrong, and you slam on the emergency breaks (^C: Interrupt)
beep beep
Warning: Caught Interrupt Exception.
Info: Logged Error - Continuing Process.
Yeah - that didn't help much. You're pretty close to the rail, so you put the car in park (killing: SignalException).
beep beep
Warning: Caught SignalException Exception.
Info: Logged Error - Continuing Process.
At the last second, you pull out the keys (kill -9), and the car stops, you slam forward into the steering wheel (the airbag can't inflate because you didn't gracefully stop the program - you terminated it), and the computer in the back of your car slams into the seat in front of it. A half-full can of Coke spills over the papers. The groceries in the back are crushed, and most are covered in egg yolk and milk. The car needs serious repair and cleaning. (Data Loss)
Hopefully you have insurance (Backups). Oh yeah - because the airbag didn't inflate, you're probably hurt (getting fired, etc).
But wait! There's more reasons why you might want to use rescue Exception => e!
Let's say you're that car, and you want to make sure the airbag inflates if the car is exceeding its safe stopping momentum.
begin
# do driving stuff
rescue Exception => e
self.airbags.inflate if self.exceeding_safe_stopping_momentum?
raise
end
Here's the exception to the rule: You can catch Exception only if you re-raise the exception. So, a better rule is to never swallow Exception, and always re-raise the error.
But adding rescue is both easy to forget in a language like Ruby, and putting a rescue statement right before re-raising an issue feels a little non-DRY. And you do not want to forget the raise statement. And if you do, good luck trying to find that error.
Thankfully, Ruby is awesome, you can just use the ensure keyword, which makes sure the code runs. The ensure keyword will run the code no matter what - if an exception is thrown, if one isn't, the only exception being if the world ends (or other unlikely events).
begin
# do driving stuff
ensure
self.airbags.inflate if self.exceeding_safe_stopping_momentum?
end
Boom! And that code should run anyways. The only reason you should use rescue Exception => e is if you need access to the exception, or if you only want code to run on an exception. And remember to re-raise the error. Every time.
Note: As #Niall pointed out, ensure always runs. This is good because sometimes your program can lie to you and not throw exceptions, even when issues occur. With critical tasks, like inflating airbags, you need to make sure it happens no matter what. Because of this, checking every time the car stops, whether an exception is thrown or not, is a good idea. Even though inflating airbags is a bit of an uncommon task in most programming contexts, this is actually pretty common with most cleanup tasks.
Because this captures all exceptions. It's unlikely that your program can recover from any of them.
You should handle only exceptions that you know how to recover from. If you don't anticipate a certain kind of exception, don't handle it, crash loudly (write details to the log), then diagnose logs and fix code.
Swallowing exceptions is bad, don't do this.
That's a specific case of the rule that you shouldn't catch any exception you don't know how to handle. If you don't know how to handle it, it's always better to let some other part of the system catch and handle it.
This blog post explains it perfectly:
Ruby's Exception vs StandardError: What's the difference?
Why you shouldn't rescue Exception
The problem with rescuing Exception
is that it actually rescues every exception that inherits from
Exception. Which is....all of them!
That's a problem because there are some exceptions that are used
internally by Ruby. They don't have anything to do with your app, and
swallowing them will cause bad things to happen.
Here are a few of the big ones:
SignalException::Interrupt - If you rescue this, you can't exit your
app by hitting control-c.
ScriptError::SyntaxError - Swallowing syntax errors means that things
like puts("Forgot something) will fail silently.
NoMemoryError - Wanna know what happens when your program keeps
running after it uses up all the RAM? Me neither.
begin
do_something()
rescue Exception => e
# Don't do this. This will swallow every single exception. Nothing gets past it.
end
I'm guessing that you don't really want to swallow any of these
system-level exceptions. You only want to catch all of your
application level errors. The exceptions caused YOUR code.
Luckily, there's an easy way to to this.
Rescue StandardError Instead
All of the exceptions that you should care about inherit from StandardError. These are our old friends:
NoMethodError - raised when you try to invoke a method that doesn't exist
TypeError - caused by things like 1 + ""
RuntimeError - who could forget good old RuntimeError?
To rescue errors like these, you'll want to rescue StandardError. You COULD do it by writing something like this:
begin
do_something()
rescue StandardError => e
# Only your app's exceptions are swallowed. Things like SyntaxErrror are left alone.
end
But Ruby has made it much easier for use.
When you don't specify an exception class at all, ruby assumes you mean StandardError. So the code below is identical to the above code:
begin
do_something()
rescue => e
# This is the same as rescuing StandardError
end
I have an EventMachine server that I am monitoring with monit. Sometimes it crashes, and I am trying to figure out why, but it is unclear to me how I can just log all top level failures. I tried code like this:
begin
EventMachine::run do
EventMachine::start_server('0.0.0.0', PORT, MyServer)
end
rescue Exception => e
puts "FAILURE: #{e.class}: #{e}"
end
but that does not seem to ever catch errors. I suspect it might be something like running out of memory, which I am tracking separately, but still I would like this server to log its proximate cause of failure if possible.
If you want a catch-all error handler, try EM.error_handler. Example from the docs:
EM.error_handler{ |e|
puts "Error raised during event loop: #{e.message}"
}
You may also want more fine-grained error handling, in which case you can use the errback mechanism (see Deferrable). So for example you could have in your reactor loop:
EventMachine::run do
server = EventMachine::start_server('0.0.0.0', PORT, MyServer)
server.errback { # handle error thrown by server here }
end
For this to work, include Deferrable in your MyServer, then whenever you want to raise an error, call fail.