Ruby 2.0 Open Uri always uses proxy, even with proxy = nil flag - ruby

Question

The following call is made in our code base:
require 'open-uri'
open(url, :proxy => nil)
However, when the call is made, Open uri uses the http_proxy environment variable to make the call, which is effectively blocked by our firewall. According to the docs when passing nil or false to the proxy option, the environment variables should be ignored as well, but this doesn't seem to be the case.
If we either switch to Ruby version 1.8.7 or unset the environment variable, the call is made successfully without the proxy. The problem is that we need to use a proxy for calls to the internet, but we cannot use them for internal calls (which is the case here) due to our firewall configuration.
Any help would be greatly appreciated!

Answer 1

I can reproduce what you described
http_proxy=http://127.0.0.1 ruby -e "require 'open-uri'; open('http://google.com', proxy: nil)"
~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `open'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `block in connect'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/timeout.rb:52:in `timeout'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:877:in `connect'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in `do_start'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in `start'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:313:in `open_http'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:708:in `buffer_open'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:210:in `block in open_loop'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:208:in `catch'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:208:in `open_loop'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:149:in `open_uri'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:688:in `open'
from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:34:in `open'
from -e:1:in `<main>'
I've digged the code, and I think I've found the problem.
So far open-uri manage correctly the proxy. For instance, in OpenURI.open_loop, it correctly considers that the proxy is set to nil (see open-uri.rb#195
case opt_proxy
when true
find_proxy = lambda {|u| pxy = u.find_proxy; pxy ? [pxy, nil, nil] : nil}
when nil, false
find_proxy = lambda {|u| nil} # <-- THIS LINE
But when it creates a Net::HTTP instance into OpenURI.open_http
http = klass.new(target_host, target_port)
it doesn't pass p_addr = nil. According the source code and doc for HTTP.new
def HTTP.new(address, port = nil, p_addr = :ENV, p_port = nil, p_user = nil, p_pass = nil)
http = super address, port
if proxy_class? then # from Net::HTTP::Proxy()
http.proxy_from_env = #proxy_from_env
http.proxy_address = #proxy_address
http.proxy_port = #proxy_port
http.proxy_user = #proxy_user
http.proxy_pass = #proxy_pass
elsif p_addr == :ENV then # <---- THIS LINE
http.proxy_from_env = true
else
http.proxy_address = p_addr
http.proxy_port = p_port || default_port
http.proxy_user = p_user
http.proxy_pass = p_pass
end
http
end
Above the doc says
To disable proxy detection set +p_addr+ to nil.
I've patched open-uri.rb by replacing the line 291 by
http = proxy ? klass.new(target_host, target_port) : klass.new(target_host, target_port, nil)
Now, it behaves as expected.
just filled an issue in Ruby bug tracker about that.