I'm using rspotify to gather a list of data from albums names. All the while I've reached Spotify's api rate limit and would now like to create a few fallbacks to wait until the I can search and retry the search in order to not lose the (x) amount of data I've already retrieved.
The gem uses RestClient but the unfortunately when I reach the rate limit I don't get the amount of time needed to wait until I can make another call:
.rvm/gems/ruby-2.5.1/gems/rest-client 2.0.2/lib/restclient/abstract_response.rb:223:in 'exception_with_response': 429 Too Many Requests (RestClient::TooManyRequests)
The above is all I'm given. The begin/rescue statement below doesn't work as when the code fails, it fails entirely without retrying. What am I doing wrong here?
begin
search = RSpotify::Album.search(album[:title])
rescue RestClient::ExceptionWithResponse, RestClient::TooManyRequests, Exception => e
puts e
retry
rescue e
puts e
retry
end
Here is how they suggest error handling:
https://github.com/rest-client/rest-client#response-callbacks-error-handling
I was thinking of maybe throttling, so inside the exception to use
RSpotify::authenticate("id", "token") with some of the multiple spotify accounts that I have
and then retry
So something like that all put together
begin
album = RSpotify::Album.find(track.first.album.id)
rescue RestClient::ExceptionWithResponse, RestClient::TooManyRequests, Exception => e
RSpotify::authenticate("id2", "token2")
retry
end
Related
I've created a program that pulls websites off of google and then strips them down to their basic url: example http://google.com/search/owie/weikw => http://google.com. It then saves these to a file.
After that it runs a .each_line on the file then runs a whois command, what I want to do is if the command doesn't respond in a certain amount of time, skip that line of the file and go to the next one, is there a way I can do this?
Use the Timeout Module
If your scraper or whois doesn't support timeout natively, you can use Timeout::timeout to set an upper bound in seconds. For example:
require 'timeout'
MAX_SECONDS = 10
begin
Timeout::timeout(MAX_SECONDS) do
# run your whois
end
rescue Timeout::Error
# handle the exception
end
By default, this will raise a Timeout::Error exception if the block exceeds the time limit, but you can have the method raise other exceptions if you prefer. How you handle the exceptions is then up to you.
Bugsnag reports that from time to time IO::EAGAINWaitReadable exception is raised in production.
IO::EAGAINWaitReadable: Resource temporarily unavailable - read would block
The exception is raised on HTTP request via HTTParty, ultimately leading to net/protocol.rb:153:in read_nonblock in Ruby 2.1.3.
Why do I get IO::EAGAINWaitReadable? Why are sometimes HTTP requests blocking? And why not let them block, what's the deal?
The most general way to handle IO::EAGAINWaitReadable is:
begin
result = io.read_nonblock(maxlen)
rescue IO::EAGAINWaitReadable
IO.select([io])
retry
end
So can do it without selection a port, but better with a selection as it is shewn in the example. Also you can look at the SO answer on how to trap the WaitReadable additionally to the specified.
I am using a RubyGem (DeathByCaptcha) that makes HTTP calls to deathbycaptcha.com. Every so often the HTTP request times out or fails for some other unknown reason, and my Ruby scripts exits with an exception. I am trying to automate repeated instances of this method ("decode") and I am trying to determine if there is a way to prevent an error in this method from exiting the whole script.
EDIT: Since I'm bound to get flamed on here, I will mention upfront that the purpose of this is to determine the effectiveness of different captcha options on my website's registration page with common captcha-breakers, because I have had problems with spam signups.
Here is how to prevent the exception from exiting the script.
tries = 0
begin
# risky failing code
rescue
sleep(1) # sleep n seconds
tries += 1
retry if tries <= 3 # retry the risky code again
end
You would need to catch the exception that is raised and somehow handle it.
You are looking for something like
begin
# Send HTTP request
rescue WhateverExceptionClassYouGet > error
# Do something with the error
end
I am trying to make calls to an external API. I handle four or more exceptions for the call.
If I make multiple calls, the code increases very quick. Should I be writing a wrapper for each such call which handles the exceptions and returns data?
Here is an example of such code (this is not mine). The call to user_search is followed by the exception handling.
Note: I am not using Rails
begin
#twitter = Twitter.user_search(name)
rescue Twitter::Unauthorized
puts "Not authorized. Please check the Twitter credentials at the top of the script."
break
rescue Twitter::BadRequest => e
puts "Hit rate limit. Continuing scraping at #{e.ratelimit_reset}"
sleep e.retry_after
retry
rescue Exception => e
puts "Something else went wrong:"
puts e.message
end
I've changed the title of the question. I think the issue is more how to handle long exception handling code. In the example code suppose I have multiple calls to the twitter API followed by exception handling, it seems like the exception handling code disrupts reading the code which does the actual work.
Write your exception handler around a chunk of code so that, if the first line in the block fails, you're happy skipping all the code up until the last line in the block.
If an exception invalidates the whole rest of the method that the handler appears in, consider letting the exception bubble up to the next layer. Not everything necessarily has to be caught by your method.
Often I find myself writing exception handlers around single lines of code (with suitable recovery code) but it's not a rule.
I read this snippet, and I am trying to understand how I can use retry, and I am unable to think of a use. How are others using it?
#!/usr/bin/ruby
for i in 1..5
retry if i > 2
puts "Value of local variable is #{i}"
end
There are several use cases. Here's one from the Programming Ruby book
#esmtp = true
begin
# First try an extended login. If it fails because the
# server doesn't support it, fall back to a normal login
if #esmtp then
#command.ehlo(helodom)
else
#command.helo(helodom)
end
rescue ProtocolError
if #esmtp then
#esmtp = false
retry
else
raise
end
end
An other common case is the email delivery. You might want to retry the SMTP delivery for N times adding a sleep between each retry to avoid temporary issues caused by network connectivity.
I am using it for a module that makes an api call to a 3rd party web api, and so if it fails, I retry 2 more times.