Installing gems from behind a corporate firewall - ruby

Question

I suspect that the corporate firewall is preventing gems from getting installed. I have HTTP_PROXY defined and I'm able to view remote gems via the following command:
jruby -S gem list -r
But when I go to install a gem, I get a 404:
jruby -S gem install rails
Is there a good workaround for resolving this issue other than maintaining an internal gem repository?

Answer 1

for ruby gems, placing this in my gem.bat this works for me
#"%~dp0ruby.exe" "%~dpn0" %* --http-proxy http://domainname.ccc.com:8080
for jruby gems this works
#"%~dp0jruby.exe" "%~dpn0" %* -p http://domainname.ccc.com:8080
Also setting the environment variable works like
set http-proxy=http://domainname.ccc.com:8080
or if your proxyserver needs authentication
set http-proxy=http://user:password#host:port)
EDIT: for folks who have a very restrictive firewall or no internet accesss (eg on a server) you can do the following: install the gem(s) on a pc who has free access to internet, afterward you check the folder C:\Ruby193\lib\ruby\gems\1.9.1\cache and copy all the gems with a date last modified after the moment you did your install. On the target pc you copy them in the same folder or if you clean up afterward in your bin folder and start the install there with
c:\ruby193\bin\gem.bat install --local gemname-x.x.x.gem
gemname-x.x.x.gem being the gem filename of the base gem you want to install. The gem with all its dependencies should install. This is for windows, but other OS's can use the same technique, just adapt a few things.
Success

Answer 2

Look in Internet Explorer proxy settings to find the name of your proxy server.
Use the name of your proxy server in the -p option to the gem command.
gem update rails -p http://mylocalproxy.mycompany.com
If this doesn't work, you can always set your own proxy server up on the internet somewhere.

Answer 3

I'm not an expert with Ruby. None the less I might give the following "generic" advices (useful for lots of situations with installers) :
Check which protocol is the installation running over. Some installer download package from mirror instead of main site, and some mirrors might be using FTP or something else instead of HTTP. Thus you might need to update your settings accordingly, and check with your admin that the corporate proxy is able to retrieve data from FTP sites.
Check which user the installer is running as (some of the installer have to escalate privileges to "root". And therefore inherit a different environment with another HTTP_PROXY defined).
Check what the installer program is using to download the packages (lots of distribution package managers rely on curl/libcurl which stores its proxy parameters in a rc file). Same warning as 2 regarding the users whose homedir should contain the rc file.
Transparent proxying : it is possible to transfer on-the-fly HTTP/FTP requests initially targeting the web to a proxy. Either the corporate proxy (if it can work with such an installation) or some mini local proxy which will then cascade to the corporate one.
Setup a quick VPN (either Tunnel other PPP) over SSH (see SSH's man page) between your machine and a special machine within the corporation which is allowed to see the web, then update your routing tables to route traffic through the VPN.
You can also use corkscrew to connect through SSH (and thus be able to install a VPN over SSH like 5) with a machine outside, on the other side of the corporate firewall, like your own at home. (this uses the HTTPS "CONNECT" mode of the proxy to access a SSH outside). Just make sure that the admins are- or at least one person in charge is- kept in loop and don't panic.
Desperate measure : create a tunnel between two machines running each http tunnel, over a proxy which only allows HTTP requests. Then use the tunnel to establish SSH VPN.
These solutions aren't specific to Ruby but can help you through any desperate solution behind an over-restrictive firewall.
1 to 4 should work in most situation. You'll have to resort to 5 to 7 if confronted with some bizarre and/or paranoid firewalls. 7 is rather extreme but always works, although sometimes with catastrophic performance.

Answer 4

Simple thing first: are you in the same shell session when you tried both the list and the install? Maybe you forget to set the env second time around?
Try setting http_proxy instead of HTTP_PROXY.
Make sure the proxy setting is a valid URL (see below)
Do you have multiple proxy options? Open IE and type wpad in the address bar. You might see some other possibilities.
Is your proxy authenticated? If so, do you have your credentials in the URL. Mine is like this (on windows):
set http_proxy=http://myuserid:mypassword#internetproxy:3128
Does your your password have any punctuation characters? Try it with just numbers and letters, as : and # (and possibly others) are significant in the URL string itself.

Answer 5

You can use this tool:
http://ntlmaps.sourceforge.net/
gems don't support proxy authentication, so going through a secondary proxy (like, say, YOUR machine) that does the authentication for you and lets gems think there is no authentication happening should make your life easier.

Answer 6

Another stupid question, but what's the corporate proxy software ? Some Windows-based software tend to used weird authentication standards (NTLM hashes, etc.) which aren't supported by all clients (wget doesn't support it, for example), but are supported by lots of browsers.
Thus even if you tried writing the username and password into the proxy URL, it won't work when you try to download and install a packages, although it works pretty well when you try to display a page in FireFox. I've had similar problems with some corporate network and my distro's package manager.
In these situation, you might use something like ntlmaps. You use it as a local proxy which will then cascade the requests to the corporate proxy. The good thing is that ntlmaps will be able to authenticate the weird NTLM password with the corporate proxy, and all your applications will be able to connect to ntlmaps, even those which don't support NTLM.

Answer 7

I've retested this since the gemcutter.org folks made the mode to Amazon's Cloudfront and am able to download gems again through my employer's firewall.

Answer 8

I described a procedure to get around any restrictions of a corporate firewall in:
How do I update Ruby Gems from behind a Proxy (ISA-NTLM)
It basically uses an ssh tunnel and the programm tsocks.

Answer 9

1.GO to the drive in which you installed ruby.
2.you will find a folder named "ruby200-x64" or something similar to it based on which version you installed.
3.inside the folder open ...bin/
4. find gem.bat, open it with any text editor you will find a bunch of settings.
5. after #"%~dp0ruby.exe" "%~dpn0" %* place you proxy server address and port.
**
#"%~dp0ruby.exe" "%~dpn0" %* --http-proxy
http://domainname.ccc.com:8080
**