I am using a RubyGem (DeathByCaptcha) that makes HTTP calls to deathbycaptcha.com. Every so often the HTTP request times out or fails for some other unknown reason, and my Ruby scripts exits with an exception. I am trying to automate repeated instances of this method ("decode") and I am trying to determine if there is a way to prevent an error in this method from exiting the whole script.
EDIT: Since I'm bound to get flamed on here, I will mention upfront that the purpose of this is to determine the effectiveness of different captcha options on my website's registration page with common captcha-breakers, because I have had problems with spam signups.
Here is how to prevent the exception from exiting the script.
tries = 0
begin
# risky failing code
rescue
sleep(1) # sleep n seconds
tries += 1
retry if tries <= 3 # retry the risky code again
end
You would need to catch the exception that is raised and somehow handle it.
You are looking for something like
begin
# Send HTTP request
rescue WhateverExceptionClassYouGet > error
# Do something with the error
end
def perform
refund_log = {
success: refund_retry.success?,
amount: refund_amount,
action: "refund"
}
if refund_retry.success?
refund_log[:reference] = refund_retry.transaction.id
refund_log[:message] = refund_retry.transaction.status
else
refund_log[:message] = refund_retry.message
refund_log[:params] = {}
refund_retry.errors.each do |error|
refund_log[:params][error.code] = error.message
end
order_transaction.message = refund_log[:params].values.join('|')
raise "delayed RefundJob has failed"
end
end
When I raise "delayed RefundJob has failed" in the else statement, it creates an Airbrake. I want to run the job again if it ends up in the else section.
Is there any way to re-queue the job without raising an exception? And prevent creating an airbrake?
I am using delayed_job version 1.
The cleanest way would be to re-queue, i.e. create a new job and enqueue it, and then exit the method normally.
To elaborate on #Roman's response, you can create a new job, with a retry parameter in it, and enqueue it.
If you maintain the retry parameter (increment it each time you re-enqueue a job), you can track how many retries you made, and thus avoid an endless retry loop.
DelayedJob expects a job to raise an error to requeued, by definition.
From there you can either :
Ignore your execpetion on airbrake side, see https://github.com/airbrake/airbrake#filtering so it still gets queued again without filling your logs
Dive into DelayedJob code where you can see on https://github.com/tobi/delayed_job/blob/master/lib/delayed/job.rb#L65 that a method named reschedule is available and used by run_with_lock ( https://github.com/tobi/delayed_job/blob/master/lib/delayed/job.rb#L99 ). From there you can call reschedule it manually, instead of raising your exception.
About the later solution, I advise adding some mechanism that still fill an airbrake report on the third or later try, you can still detect that something is wrong without the hassle of having your logs filled by the attempts.
Ryan Davis’s Ruby QuickRef says (without explanation):
Don’t rescue Exception. EVER. or I will stab you.
Why not? What’s the right thing to do?
TL;DR: Use StandardError instead for general exception catching. When the original exception is re-raised (e.g. when rescuing to log the exception only), rescuing Exception is probably okay.
Exception is the root of Ruby's exception hierarchy, so when you rescue Exception you rescue from everything, including subclasses such as SyntaxError, LoadError, and Interrupt.
Rescuing Interrupt prevents the user from using CTRLC to exit the program.
Rescuing SignalException prevents the program from responding correctly to signals. It will be unkillable except by kill -9.
Rescuing SyntaxError means that evals that fail will do so silently.
All of these can be shown by running this program, and trying to CTRLC or kill it:
loop do
begin
sleep 1
eval "djsakru3924r9eiuorwju3498 += 5u84fior8u8t4ruyf8ihiure"
rescue Exception
puts "I refuse to fail or be stopped!"
end
end
Rescuing from Exception isn't even the default. Doing
begin
# iceberg!
rescue
# lifeboats
end
does not rescue from Exception, it rescues from StandardError. You should generally specify something more specific than the default StandardError, but rescuing from Exception broadens the scope rather than narrowing it, and can have catastrophic results and make bug-hunting extremely difficult.
If you have a situation where you do want to rescue from StandardError and you need a variable with the exception, you can use this form:
begin
# iceberg!
rescue => e
# lifeboats
end
which is equivalent to:
begin
# iceberg!
rescue StandardError => e
# lifeboats
end
One of the few common cases where it’s sane to rescue from Exception is for logging/reporting purposes, in which case you should immediately re-raise the exception:
begin
# iceberg?
rescue Exception => e
# do some logging
raise # not enough lifeboats ;)
end
The real rule is: Don't throw away exceptions. The objectivity of the author of your quote is questionable, as evidenced by the fact that it ends with
or I will stab you
Of course, be aware that signals (by default) throw exceptions, and normally long-running processes are terminated through a signal, so catching Exception and not terminating on signal exceptions will make your program very hard to stop. So don't do this:
#! /usr/bin/ruby
while true do
begin
line = STDIN.gets
# heavy processing
rescue Exception => e
puts "caught exception #{e}! ohnoes!"
end
end
No, really, don't do it. Don't even run that to see if it works.
However, say you have a threaded server and you want all exceptions to not:
be ignored (the default)
stop the server (which happens if you say thread.abort_on_exception = true).
Then this is perfectly acceptable in your connection handling thread:
begin
# do stuff
rescue Exception => e
myLogger.error("uncaught #{e} exception while handling connection: #{e.message}")
myLogger.error("Stack trace: #{backtrace.map {|l| " #{l}\n"}.join}")
end
The above works out to a variation of Ruby's default exception handler, with the advantage that it doesn't also kill your program. Rails does this in its request handler.
Signal exceptions are raised in the main thread. Background threads won't get them, so there is no point in trying to catch them there.
This is particularly useful in a production environment, where you do not want your program to simply stop whenever something goes wrong. Then you can take the stack dumps in your logs and add to your code to deal with specific exception further down the call chain and in a more graceful manner.
Note also that there is another Ruby idiom which has much the same effect:
a = do_something rescue "something else"
In this line, if do_something raises an exception, it is caught by Ruby, thrown away, and a is assigned "something else".
Generally, don't do that, except in special cases where you know you don't need to worry. One example:
debugger rescue nil
The debugger function is a rather nice way to set a breakpoint in your code, but if running outside a debugger, and Rails, it raises an exception. Now theoretically you shouldn't be leaving debug code lying around in your program (pff! nobody does that!) but you might want to keep it there for a while for some reason, but not continually run your debugger.
Note:
If you've run someone else's program that catches signal exceptions and ignores them, (say the code above) then:
in Linux, in a shell, type pgrep ruby, or ps | grep ruby, look for your offending program's PID, and then run kill -9 <PID>.
in Windows, use the Task Manager (CTRL-SHIFT-ESC), go to the "processes" tab, find your process, right click it and select "End process".
If you are working with someone else's program which is, for whatever reason, peppered with these ignore-exception blocks, then putting this at the top of the mainline is one possible cop-out:
%W/INT QUIT TERM/.each { |sig| trap sig,"SYSTEM_DEFAULT" }
This causes the program to respond to the normal termination signals by immediately terminating, bypassing exception handlers, with no cleanup. So it could cause data loss or similar. Be careful!
If you need to do this:
begin
do_something
rescue Exception => e
critical_cleanup
raise
end
you can actually do this:
begin
do_something
ensure
critical_cleanup
end
In the second case, critical cleanup will be called every time, whether or not an exception is thrown.
TL;DR
Don't rescue Exception => e (and not re-raise the exception) - or you might drive off a bridge.
Let's say you are in a car (running Ruby). You recently installed a new steering wheel with the over-the-air upgrade system (which uses eval), but you didn't know one of the programmers messed up on syntax.
You are on a bridge, and realize you are going a bit towards the railing, so you turn left.
def turn_left
self.turn left:
end
oops! That's probably Not Good™, luckily, Ruby raises a SyntaxError.
The car should stop immediately - right?
Nope.
begin
#...
eval self.steering_wheel
#...
rescue Exception => e
self.beep
self.log "Caught #{e}.", :warn
self.log "Logged Error - Continuing Process.", :info
end
beep beep
Warning: Caught SyntaxError Exception.
Info: Logged Error - Continuing Process.
You notice something is wrong, and you slam on the emergency breaks (^C: Interrupt)
beep beep
Warning: Caught Interrupt Exception.
Info: Logged Error - Continuing Process.
Yeah - that didn't help much. You're pretty close to the rail, so you put the car in park (killing: SignalException).
beep beep
Warning: Caught SignalException Exception.
Info: Logged Error - Continuing Process.
At the last second, you pull out the keys (kill -9), and the car stops, you slam forward into the steering wheel (the airbag can't inflate because you didn't gracefully stop the program - you terminated it), and the computer in the back of your car slams into the seat in front of it. A half-full can of Coke spills over the papers. The groceries in the back are crushed, and most are covered in egg yolk and milk. The car needs serious repair and cleaning. (Data Loss)
Hopefully you have insurance (Backups). Oh yeah - because the airbag didn't inflate, you're probably hurt (getting fired, etc).
But wait! There's more reasons why you might want to use rescue Exception => e!
Let's say you're that car, and you want to make sure the airbag inflates if the car is exceeding its safe stopping momentum.
begin
# do driving stuff
rescue Exception => e
self.airbags.inflate if self.exceeding_safe_stopping_momentum?
raise
end
Here's the exception to the rule: You can catch Exception only if you re-raise the exception. So, a better rule is to never swallow Exception, and always re-raise the error.
But adding rescue is both easy to forget in a language like Ruby, and putting a rescue statement right before re-raising an issue feels a little non-DRY. And you do not want to forget the raise statement. And if you do, good luck trying to find that error.
Thankfully, Ruby is awesome, you can just use the ensure keyword, which makes sure the code runs. The ensure keyword will run the code no matter what - if an exception is thrown, if one isn't, the only exception being if the world ends (or other unlikely events).
begin
# do driving stuff
ensure
self.airbags.inflate if self.exceeding_safe_stopping_momentum?
end
Boom! And that code should run anyways. The only reason you should use rescue Exception => e is if you need access to the exception, or if you only want code to run on an exception. And remember to re-raise the error. Every time.
Note: As #Niall pointed out, ensure always runs. This is good because sometimes your program can lie to you and not throw exceptions, even when issues occur. With critical tasks, like inflating airbags, you need to make sure it happens no matter what. Because of this, checking every time the car stops, whether an exception is thrown or not, is a good idea. Even though inflating airbags is a bit of an uncommon task in most programming contexts, this is actually pretty common with most cleanup tasks.
Because this captures all exceptions. It's unlikely that your program can recover from any of them.
You should handle only exceptions that you know how to recover from. If you don't anticipate a certain kind of exception, don't handle it, crash loudly (write details to the log), then diagnose logs and fix code.
Swallowing exceptions is bad, don't do this.
That's a specific case of the rule that you shouldn't catch any exception you don't know how to handle. If you don't know how to handle it, it's always better to let some other part of the system catch and handle it.
This blog post explains it perfectly:
Ruby's Exception vs StandardError: What's the difference?
Why you shouldn't rescue Exception
The problem with rescuing Exception
is that it actually rescues every exception that inherits from
Exception. Which is....all of them!
That's a problem because there are some exceptions that are used
internally by Ruby. They don't have anything to do with your app, and
swallowing them will cause bad things to happen.
Here are a few of the big ones:
SignalException::Interrupt - If you rescue this, you can't exit your
app by hitting control-c.
ScriptError::SyntaxError - Swallowing syntax errors means that things
like puts("Forgot something) will fail silently.
NoMemoryError - Wanna know what happens when your program keeps
running after it uses up all the RAM? Me neither.
begin
do_something()
rescue Exception => e
# Don't do this. This will swallow every single exception. Nothing gets past it.
end
I'm guessing that you don't really want to swallow any of these
system-level exceptions. You only want to catch all of your
application level errors. The exceptions caused YOUR code.
Luckily, there's an easy way to to this.
Rescue StandardError Instead
All of the exceptions that you should care about inherit from StandardError. These are our old friends:
NoMethodError - raised when you try to invoke a method that doesn't exist
TypeError - caused by things like 1 + ""
RuntimeError - who could forget good old RuntimeError?
To rescue errors like these, you'll want to rescue StandardError. You COULD do it by writing something like this:
begin
do_something()
rescue StandardError => e
# Only your app's exceptions are swallowed. Things like SyntaxErrror are left alone.
end
But Ruby has made it much easier for use.
When you don't specify an exception class at all, ruby assumes you mean StandardError. So the code below is identical to the above code:
begin
do_something()
rescue => e
# This is the same as rescuing StandardError
end
This is more of an opinion oriented question. When handling exceptions in nested codes such as:
Assuming you have a class that initialize another class to run a job. The job returns a value, which is then processed by the class which initially called it.
Where would you put the exception and error logging? Would you define it on the initialization of the job class in the calling class, which will handle then exception in the job execution or on both levels ?
if the job handles exceptions then you don't need to wrap the call to the job in a try catch.
but the class that initializes and runs the job could throw exceptions, so you should handle exceptions at that level as well.
here is an example:
def some_job
begin
# a bunch of logic
rescue
# handle exception
# log it
end
end
it wouldn't make sense then to do this:
def some_manager
begin
some_job
rescue
# log
end
end
but something like this makes more sense:
def some_manager
begin
# a bunch of logic
some_job
# some more logic
rescue
# handle exception
# log
end
end
and of course you would want to catch specific exceptions.
Probably the best answer, in general, for handling Exceptions in Ruby is reading Exceptional Ruby. It may change your perspective on error handling.
Having said that, your specific case. When I hear "job" in hear "background process", so I'll base my answer on that.
Your job will want to report status while it's doing it's thing. This could be states like "in queue", "running", "finished", but it also could be more informative (user facing) information: "processing first 100 out of 1000 records".
So, if an error happens in your background process, my suggestion is two-fold:
Make sure you catch exceptions before you exit the job. Your background job processor might not like a random exception coming from your code. I, personally, like the idea of catching the exception and saving it to the database, for easy retrieval later. Then again, depending on your background job processor, maybe it handles error reporting for you. (I think reque does, for example).
On the front end, use AJAX (or something) to occasionally check in to how the job is doing. Say every 10 seconds or something. In additional to getting the status of the job, also make sure you return this additional information to the user (if appropriate).
I write a simple bot using "SimpleMUCClient". But got error: app.rb:73:in stop': deadlock detected (fatal)
from app.rb:73:in'. How to fix it?
Most likely the code you're running is executed in another thread. That particular thread is then joined (meaning Ruby waits for it to finish upon exiting the script) using Thread.join(). Calling Thread.stop() while also calling .join() is most likely the cause of the deadlock. Having said that you should following the guides of StackOverflow regarding how to ask questions properly, since you haven't done so I've down voted your question.
Joining a thread while still calling Thread.stop can be done as following:
th = Thread.new do
Thread.stop
end
if th.status === 'sleep'
th.run
else
th.join
end
It's not the cleanest way but it works. Also, if you want to actually terminate a thread you'll have to call Thread.exit instead.