I'd like to do some HTTP REST requests in Ruby, using rest-client gem,
Following readme.md at https://github.com/rest-client/rest-client
I wrote this simple command line script, trying to catch exceptions in case of response codes differents from 2xx:
RestClient.get('http://thisurldoesnotexist/resource') { |response, request, result, &block|
case response.code
when 200
p "It worked !"
response
else
response.return!(request, result, &block)
end
}
Hi got this on stdout output:
/home/*****/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `initialize': getaddrinfo: Name or service not known (SocketError)
from /home/solyaris/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `open'
from /home/solyaris/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `block in connect'
from /home/solyaris/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/timeout.rb:52:in `timeout'
from /home/solyaris/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:877:in `connect'
from /home/solyaris/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in `do_start'
from /home/solyaris/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in `start'
from /home/solyaris/.rvm/gems/ruby-2.0.0-p247/gems/rest-client-1.6.7/lib/restclient/request.rb:172:in `transmit'
from /home/solyaris/.rvm/gems/ruby-2.0.0-p247/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
from /home/solyaris/.rvm/gems/ruby-2.0.0-p247/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
from /home/solyaris/.rvm/gems/ruby-2.0.0-p247/gems/rest-client-1.6.7/lib/restclient.rb:68:in `get'
from prova_rest.rb:3:in `<main>'
How can i catch SocketError ?
where I'm wrong ?
thanks
giorgio
The callback block is executed only when receiving some response from the server. In this case, the name resolving is failed so RestClient.get just throws an exception without entering the block. Thus just wrap your code within a begin...end construct.
begin
RestClient.get('http://thisurldoesnotexist/resource') { |response, request, result, &block|
case response.code
when 200
p "It worked !"
response
else
response.return!(request, result, &block)
end
}
rescue SocketError => e
# Handle your error here
end
Related
I have is a deadlock, but I am not using any threads in my program. Plus, the error only happens about once every 1000 to 1500 function calls, making it very difficult to pinpoint and correct.
Here is the complete error message when the issue occurs:
/usr/lib/ruby/2.3.0/timeout.rb:95:in `join': No live threads left. Deadlock? (fatal)
from /usr/lib/ruby/2.3.0/timeout.rb:95:in `ensure in block in timeout'
from /usr/lib/ruby/2.3.0/timeout.rb:95:in `block in timeout'
from /usr/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
from /usr/lib/ruby/2.3.0/net/http.rb:878:in `connect'
from /usr/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
from /usr/lib/ruby/2.3.0/net/http.rb:852:in `start'
from /usr/lib/ruby/2.3.0/open-uri.rb:319:in `open_http'
from /usr/lib/ruby/2.3.0/open-uri.rb:737:in `buffer_open'
from /usr/lib/ruby/2.3.0/open-uri.rb:212:in `block in open_loop'
from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `catch'
from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `open_loop'
from /usr/lib/ruby/2.3.0/open-uri.rb:151:in `open_uri'
from /usr/lib/ruby/2.3.0/open-uri.rb:717:in `open'
from /usr/lib/ruby/2.3.0/open-uri.rb:35:in `open'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/utils.rb:85:in `get_pic'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:87:in `page_link'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:116:in `chapter_link'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:142:in `chapter'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:57:in `block in MF_manga_missing_chapters'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:45:in `reverse_each'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:45:in `MF_manga_missing_chapters'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:80:in `MF_update'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:5:in `update_manga'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:15:in `block in update_all'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:14:in `each'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:14:in `update_all'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:22:in `update'
from ./MangaScrap.rb:28:in `<main>'
The link to the complete program is https://github.com/Hellfire01/MangaScrap
The issue happens to the three different methods that use open. Here is the one that crashed this time:
# conect to link and download picture
def get_pic(link)
safe_link = link.gsub(/[\[\]]/) { '%%%s' % $&.ord.to_s(16) }
tries ||= 20
begin
page = open(safe_link, "User-Agent" => "Ruby/#{RUBY_VERSION}")
rescue URI::InvalidURIError => error
puts "Warning : bad url"
puts link
puts "message is : " + error.message
return nil
rescue => error
if tries > 0
tries -= 1
sleep(0.2)
retry
else
puts 'could not get picture ' + safe_link + ' after ' + $nb_tries.to_s + ' tries'
puts "message is : " + error.message
return nil
end
end
sleep(0.2)
return page
end
Here is the link to the file: https://github.com/Hellfire01/MangaScrap/blob/master/sources/utils.rb
I would like to know:
How can I fix this error?
If I can not fix this error, are there alternatives to OpenUri that I can use?
You're not catching all exceptions here. When nothing is specified after rescue, it means that you're catching StandardError which is not at the root of Exceptions' hierarchy.
If you want to make sure you're catching all exceptions and retry opening a URL (or whatever behavior you'd like), what you want to do is:
rescue Exception => error
I have a method that should raise a custom error with a message. When I catch the error and raise my own custom error, it is still raising and printing the backtrace of the original error. I just want the custom error and message. Code below.
Method:
def load(configs)
begin
opts = {access_token: configs['token'],
api_endpoint: configs['endpoint'],
web_endpoint: configs['site'],
auto_paginate: configs['pagination']}
client = Octokit::Client.new(opts)
repos = client.org_repos(configs['org'])
repos.each do |r|
Project.create(name: r.name)
end
rescue Octokit::Unauthorized
raise GitConfigError, "boom"
end
#rescue Octokit::Unauthorized
end
class GitConfigError < StandardError
end
My test (which is failling):
context 'with incorrect git configs' do
before do
allow(loader).to receive(:load).and_raise Octokit::Unauthorized
end
it { expect{loader.load(configs)}.to raise_error(GitConfigError, "boom" ) }
end
Test Output:
GitProjectLoader#load with incorrect git configs should raise GitConfigError with "boom"
Failure/Error: it { expect{loader.load(configs)}.to raise_error(GitConfigError, "boom" ) }
expected GitConfigError with "boom", got #<Octokit::Unauthorized: Octokit::Unauthorized> with backtrace:
# ./spec/lib/git_project_loader_spec.rb:24:in `block (5 levels) in <top (required)>'
# ./spec/lib/git_project_loader_spec.rb:24:in `block (4 levels) in <top (required)>'
# ./spec/lib/git_project_loader_spec.rb:24:in `block (4 levels) in <top (required)>'
If you intend to test the handling of the Octokit::Unauthorized error, then raise the error anywhere before the rescue kicks in. Preferably, someplace where it would actually be raised.
Something like this, for example:
before do
allow(Octokit::Client).to receive(:new).and_raise(Octokit::Unauthorized)
end
And then:
expect{ loader.load(configs) }.to raise_error(GitConfigError, "boom" )
As a side note, I would discourage enclosing all lines of your method in a begin;rescue;end structure; you should enclose only the lines from which you are expecting errors.
You are not testing your code as you think. You have mocked it out.
The line
allow(loader).to receive(:load).and_raise Octokit::Unauthorized
replaces the load method on loader with a stub which just raises the named error.
Remove your before block, and it should test your code as intended. Note as written it will make a real request via Octokit, unless you mock that out instead.
Can anyone explain why I might see this stack (caused by an HTTParty::post request) when the call to the method looks like this:
begin
response = HTTParty::post(url, options)
rescue
logger.warn("Could not post to #{url}")
rescue Timeout::Error
logger.warn("Could not post to #{url}: timeout")
end
The stack:
/usr/local/lib/ruby/1.8/timeout.rb:64:in `timeout'
/usr/local/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
/usr/local/lib/ruby/1.8/net/protocol.rb:104:in `read_all'
/usr/local/lib/ruby/1.8/net/http.rb:2228:in `read_body_0'
/usr/local/lib/ruby/1.8/net/http.rb:2181:in `read_body'
/usr/local/lib/ruby/1.8/net/http.rb:2206:in `body'
/usr/local/lib/ruby/1.8/net/http.rb:2145:in `reading_body'
/usr/local/lib/ruby/1.8/net/http.rb:1053:in `request_without_newrelic_trace'
[GEM_ROOT]/gems/newrelic_rpm-3.1.1/lib/new_relic/agent/instrumentation/net.rb:20:in `request'
[GEM_ROOT]/gems/newrelic_rpm-3.1.1/lib/new_relic/agent/method_tracer.rb:242:in `trace_execution_scoped'
[GEM_ROOT]/gems/newrelic_rpm-3.1.1/lib/new_relic/agent/instrumentation/net.rb:19:in `request'
/usr/local/lib/ruby/1.8/net/http.rb:1037:in `request_without_newrelic_trace'
/usr/local/lib/ruby/1.8/net/http.rb:543:in `start'
/usr/local/lib/ruby/1.8/net/http.rb:1035:in `request_without_newrelic_trace'
[GEM_ROOT]/gems/newrelic_rpm-3.1.1/lib/new_relic/agent/instrumentation/net.rb:20:in `request'
[GEM_ROOT]/gems/newrelic_rpm-3.1.1/lib/new_relic/agent/method_tracer.rb:242:in `trace_execution_scoped'
[GEM_ROOT]/gems/newrelic_rpm-3.1.1/lib/new_relic/agent/instrumentation/net.rb:19:in `request'
[GEM_ROOT]/gems/httparty-0.7.8/lib/httparty/request.rb:69:in `perform'
[GEM_ROOT]/gems/httparty-0.7.8/lib/httparty.rb:390:in `perform_request'
[GEM_ROOT]/gems/httparty-0.7.8/lib/httparty.rb:358:in `post'
[GEM_ROOT]/gems/httparty-0.7.8/lib/httparty.rb:426:in `post'
As you can see, I am handling the Timeout::Error exception. This is in Ruby 1.8.7. I am well aware that in Ruby 1.8.7, StandardException and TimeoutException have different inheritance trees, so I handle both, but it does not seem to make a difference.
When you omit the exception class in rescue, it will capture any StandardError. Since Timeout::Error is a subclass of StandardError, it will be captured by the first rescue statement. If you want to separately capture it, you have to place that before the omitted one:
begin
response = HTTParty::post(url, options)
rescue Timeout::Error
logger.warn("Could not post to #{url}: timeout")
rescue
logger.warn("Could not post to #{url}")
end
I need to rescue a Timeout::Error raised from a the Redis library but i'm running into a problem, rescuing that specific class doesn't seem to work.
begin
Redis.new( { :host => "127.0.0.X" } )
rescue Timeout::Error => ex
end
=> Timeout::Error: Timeout::Error from /Users/me/.rvm/gems/ree-1.8.7-2011.03#gowalla/gems/redis-2.2.0/lib/redis/connection/hiredis.rb:23:in `connect'
When i try to rescue Exception it still doesn't work
begin
Redis.new( { :host => "127.0.0.X" } )
rescue Exception => ex
end
=> Timeout::Error: Timeout::Error from /Users/me/.rvm/gems/ree-1.8.7-2011.03#gowalla/gems/redis-2.2.0/lib/redis/connection/hiredis.rb:23:in `connect'
If i try to raise the exception manually, i can rescue it but don't know why i can't rescue it when it's called from within the Redis Gem (2.2.0).
begin
raise Timeout::Error
rescue Timeout::Error => ex
puts ex
end
Timeout::Error
=> nil
Any clue how to rescue this exception?
You ran this code in irb, right? The exception you are getting is not actually being raised by Redis.new. It is being raised by the inspect method, which irb calls to show you the value of the expression you just typed.
Just look at the stack trace (I shortened the paths to make it legible):
ruby-1.8.7-p330 :009 > Redis.new(:host => "google.com")
Timeout::Error: time's up!
from /.../SystemTimer-1.2.3/lib/system_timer/concurrent_timer_pool.rb:63:in `trigger_next_expired_timer_at'
from /.../SystemTimer-1.2.3/lib/system_timer/concurrent_timer_pool.rb:68:in `trigger_next_expired_timer'
from /.../SystemTimer-1.2.3/lib/system_timer.rb:85:in `install_ruby_sigalrm_handler'
from /..../lib/ruby/1.8/monitor.rb:242:in `synchronize'
from /.../SystemTimer-1.2.3/lib/system_timer.rb:83:in `install_ruby_sigalrm_handler'
from /.../redis-2.2.2/lib/redis/connection/ruby.rb:26:in `call'
from /.../redis-2.2.2/lib/redis/connection/ruby.rb:26:in `initialize'
from /.../redis-2.2.2/lib/redis/connection/ruby.rb:26:in `new'
from /.../redis-2.2.2/lib/redis/connection/ruby.rb:26:in `connect'
from /.../SystemTimer-1.2.3/lib/system_timer.rb:60:in `timeout_after'
from /.../redis-2.2.2/lib/redis/connection/ruby.rb:115:in `with_timeout'
from /.../redis-2.2.2/lib/redis/connection/ruby.rb:25:in `connect'
from /.../redis-2.2.2/lib/redis/client.rb:227:in `establish_connection'
from /.../redis-2.2.2/lib/redis/client.rb:23:in `connect'
from /.../redis-2.2.2/lib/redis/client.rb:247:in `ensure_connected'
from /.../redis-2.2.2/lib/redis/client.rb:137:in `process'
... 2 levels...
from /.../redis-2.2.2/lib/redis/client.rb:46:in `call'
from /.../redis-2.2.2/lib/redis.rb:90:in `info'
from /..../lib/ruby/1.8/monitor.rb:242:in `synchronize'
from /.../redis-2.2.2/lib/redis.rb:89:in `info'
from /.../redis-2.2.2/lib/redis.rb:1075:in `inspect'
from /..../lib/ruby/1.8/monitor.rb:242:in `synchronize'
from /.../redis-2.2.2/lib/redis.rb:1074:in `inspect'
from /..../lib/ruby/1.8/irb.rb:310:in `output_value'
from /..../lib/ruby/1.8/irb.rb:159:in `eval_input'
from /..../lib/ruby/1.8/irb.rb:271:in `signal_status'
from /..../lib/ruby/1.8/irb.rb:155:in `eval_input'
from /..../lib/ruby/1.8/irb.rb:154:in `eval_input'
from /..../lib/ruby/1.8/irb.rb:71:in `start'
from /..../lib/ruby/1.8/irb.rb:70:in `catch'
from /..../lib/ruby/1.8/irb.rb:70:in `start'
from /..../bin/irb:17
As you can see above, the exception occurs inside inspect, not Redis.new. When you call inspect on a Redis object, instead of just printing out its state it actually does a lot of things. In this case, inspect attempts to connect to the server and throws an exception when that times out. This seems like a very bad design to me and maybe we should file a bug report to the maintainers of the Redis gem.
This leads to some interesting behavior in IRB:
Typing Redis.new(:host => "google.com") results in an exception as shown above
Typing Redis.new(:host => "google.com"); 'hello' results in '=> "hello"'
If you want to catch this exception, try calling ensure_connected inside your begin/rescue/end block.
I'm trying to process content from a list of links using "open-uri" in ruby (1.8.6), but the bad thing happens when I'm getting an error when one link is broken or requires authentication:
open-uri.rb:277:in `open_http': 404 Not Found (OpenURI::HTTPError)
from C:/tools/Ruby/lib/ruby/1.8/open-uri.rb:616:in `buffer_open'
from C:/tools/Ruby/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
from C:/tools/Ruby/lib/ruby/1.8/open-uri.rb:162:in `catch'
or
C:/tools/Ruby/lib/ruby/1.8/net/http.rb:560:in `initialize': getaddrinfo: no address associated with hostname. (SocketError)
from C:/tools/Ruby/lib/ruby/1.8/net/http.rb:560:in `open'
from C:/tools/Ruby/lib/ruby/1.8/net/http.rb:560:in `connect'
from C:/tools/Ruby/lib/ruby/1.8/timeout.rb:53:in `timeout'
or
C:/tools/Ruby/lib/ruby/1.8/net/protocol.rb:133:in `sysread': An existing connection was forcibly closed by the remote host. (Errno::ECONNRESET)
from C:/tools/Ruby/lib/ruby/1.8/net/protocol.rb:133:in `rbuf_fill'
from C:/tools/Ruby/lib/ruby/1.8/timeout.rb:62:in `timeout'
from C:/tools/Ruby/lib/ruby/1.8/timeout.rb:93:in `timeout'
is there a way to test the response (url) before processing any data?
the code is:
require 'open-uri'
smth.css.each do |item|
open('item[:name]', 'wb') do |file|
file << open('item[:href]').read
end
end
Many thanks
You could try something along the lines of
require 'open-uri'
smth.css.each do |item|
begin
open('item[:name]', 'wb') do |file|
file << open('item[:href]').read
end
rescue => e
case e
when OpenURI::HTTPError
# do something
when SocketError
# do something else
else
raise e
end
rescue SystemCallError => e
if e === Errno::ECONNRESET
# do something else
else
raise e
end
end
end
I don't know of any way of testing the connection without opening it and trying, so rescuing these errors would be the only way I can think of. The thing to be aware of is that OpenURI::HTTPError and SocketError are both subclasses of StandardError, whereas Errno::ECONNRESET is a subclass of SystemCallError. So rescue => e won't catch Errno::ECONNRESET.
I was able to solve this problem by using a conditional if/else statement to check the return value of the action for "failure":
def controller_action
url = "some_API"
response = open(url).read
data = JSON.parse(response)["data"]
if response["status"] == "failure"
redirect_to :action => "home"
else
do_something_else
end
end