How to fix a deadlock caused by open - ruby

Question

I have is a deadlock, but I am not using any threads in my program. Plus, the error only happens about once every 1000 to 1500 function calls, making it very difficult to pinpoint and correct.
Here is the complete error message when the issue occurs:
/usr/lib/ruby/2.3.0/timeout.rb:95:in `join': No live threads left. Deadlock? (fatal)
from /usr/lib/ruby/2.3.0/timeout.rb:95:in `ensure in block in timeout'
from /usr/lib/ruby/2.3.0/timeout.rb:95:in `block in timeout'
from /usr/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
from /usr/lib/ruby/2.3.0/net/http.rb:878:in `connect'
from /usr/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
from /usr/lib/ruby/2.3.0/net/http.rb:852:in `start'
from /usr/lib/ruby/2.3.0/open-uri.rb:319:in `open_http'
from /usr/lib/ruby/2.3.0/open-uri.rb:737:in `buffer_open'
from /usr/lib/ruby/2.3.0/open-uri.rb:212:in `block in open_loop'
from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `catch'
from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `open_loop'
from /usr/lib/ruby/2.3.0/open-uri.rb:151:in `open_uri'
from /usr/lib/ruby/2.3.0/open-uri.rb:717:in `open'
from /usr/lib/ruby/2.3.0/open-uri.rb:35:in `open'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/utils.rb:85:in `get_pic'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:87:in `page_link'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:116:in `chapter_link'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:142:in `chapter'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:57:in `block in MF_manga_missing_chapters'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:45:in `reverse_each'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:45:in `MF_manga_missing_chapters'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:80:in `MF_update'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:5:in `update_manga'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:15:in `block in update_all'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:14:in `each'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:14:in `update_all'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:22:in `update'
from ./MangaScrap.rb:28:in `<main>'
The link to the complete program is https://github.com/Hellfire01/MangaScrap
The issue happens to the three different methods that use open. Here is the one that crashed this time:
# conect to link and download picture
def get_pic(link)
safe_link = link.gsub(/[\[\]]/) { '%%%s' % $&.ord.to_s(16) }
tries ||= 20
begin
page = open(safe_link, "User-Agent" => "Ruby/#{RUBY_VERSION}")
rescue URI::InvalidURIError => error
puts "Warning : bad url"
puts link
puts "message is : " + error.message
return nil
rescue => error
if tries > 0
tries -= 1
sleep(0.2)
retry
else
puts 'could not get picture ' + safe_link + ' after ' + $nb_tries.to_s + ' tries'
puts "message is : " + error.message
return nil
end
end
sleep(0.2)
return page
end
Here is the link to the file: https://github.com/Hellfire01/MangaScrap/blob/master/sources/utils.rb
I would like to know:
How can I fix this error?
If I can not fix this error, are there alternatives to OpenUri that I can use?

Answer 1

You're not catching all exceptions here. When nothing is specified after rescue, it means that you're catching StandardError which is not at the root of Exceptions' hierarchy.
If you want to make sure you're catching all exceptions and retry opening a URL (or whatever behavior you'd like), what you want to do is:
rescue Exception => error