How do you get the IP address of a domain and the server information of that domain, to tell if the server info is Apache, nginix, etc..?
I have a pretty good idea on how to get the IP, but for some reason it won't output the correct information:
require 'socket'
IPSocket::getaddress('http://www.prairiegraphicdesign.com/phpStuff/week2Shopping/cartAdd.php?action=add&id=4')
#<= irb(main):001:0> require 'socket'
# => true
# irb(main):002:0>IPSocket::getaddress('http://www.prairiegraphicdesign.com/phpStuff/week2Shopping/cartAdd.php?action=add&id=4')
# SocketError: getaddrinfo: No such host is known.
# from (irb):2:in `getaddress'
# from (irb):2
# from C:/Ruby22/bin/irb:11:in `<main>'
Is this because I need to strip the URL down to the original URL?
http://www.prairiegraphicdesign.com
That's a URL, and getaddress has no idea what to do with those. All it can handle is fully-qualified domain names (FQDN) like "example.com".
Instead you need to break it out:
require 'uri'
uri = URI.parse('http://www.prairiegraphicdesign.com/phpStuff/week2Shopping/cartAdd.php?action=add&id=4')
uri.host
#=> "www.prairiegraphicdesign.com"
You can resolve that to an address if you like.
As for what server software it's running, you need to make an HTTP request to the server and extract the right header from the response:
require 'net/http'
res = Net::HTTP.get_response(uri)
res['server']
#=> "Microsoft-IIS/6.0"
Related
I'm new to Ruby and to the BC API so there may be something obvious that I'm missing. When I'm running the following code with the proper details replaced with my store's legacy API credentials:
require 'bigcommerce'
Bigcommerce.configure do |config|
config.auth = 'legacy'
# You will get this url when registering for an API key
config.url = ENV['BC_API_ENDPOINT_LEGACY']
config.username = ENV['BC_USERNAME']
config.api_key = ENV['BC_API_KEY']
end
puts Bigcommerce::System.time
I get the following error:
.../lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect': Failed to open TCP connection to :80 (Connection refused - connect(2) for nil port 80) (Faraday::ConnectionFailed)
I appreciate any pointers.
tldr: you're environment variables aren't actually being set :p
Sounds like your environment variables aren't actually being set.
I say this because the error Failed to open TCP connection to :80, does not list any URL, but only the port.
To confirm, within your code, can you simply run:
puts ENV['BC_API_ENDPOINT_LEGACY'], and see if anything is printed?
An alternative is to just hardcode your credentials, and not rely on environment variables.
Bigcommerce.configure do |config|
config.auth = 'legacy'
config.url = 'https://url.com'
config.username = 'api_username'
config.api_key = 'api_key'
end
FYI, the way you set an environment variable would generally be (from the command line):
export BC_API_ENDPOINT_LEGACY="https://url.com"
And then confirm it is set by echo $BC_API_ENDPOINT_LEGACY
You can view this link for other options on setting environment variables for your RoR app.
I am trying to run a ruby script from my computer and I would like to have the script use a proxy IP address / server that I have setup, as opposed to the default IP address associated with my local machine.
I have been able to get my web browsers to use this proxy IP address by making changes inside network settings. But When I run the ruby script from textmate, it doesn't seem to use the proxy IP address I have put into my network settings. Instead it defaults back to the base ip address of my local machine.
Is there anything I can do in textmate or in the script itself to specify a proxy IP address it should route through?
My script looks like the following:
require "open-uri"
url = "some-url"
pattern = "<img"
page = open(url).read
tags = page.scan(pattern)
puts "The site #{url} has #{tags.length} img tags"
Thanks for your help!
Use :proxy option to let open-uri know your proxy server:
page = open(url, :proxy => "http://#{proxy_host}:#{proxy_port}/").read
You can also set environment variable http_proxy instead. If you do so, give :proxy => true for option.
page = open(url, :proxy => true).read
[ADDED]
If you want to use proxy with basic authentication, you can give :proxy_http_basic_authentication option instead of :proxy as follows:
:proxy_http_basic_authentication => ["http://#{proxy_host}:#{proxy_port}/", login, password]
Note that :proxy_http_basic_authentication can be used in ruby 1.9.2 or later.
I recommend using mechanize, and css instead of regex:
require "mechanize"
url = "http://www.google.com/"
#agent = Mechanize.new{|a| a.set_proxy 'localhost', 8888}
page = #agent.get url
tags = page.search('img')
puts "The site #{url} has #{tags.length} img tags"
I have an application that initiates multiple HTTP connections and I would like to add a proxy to all connections.
The application is using net/HTTP, TCP sockets and open-uri so ideally I would like to be able to patch all connections initiated from those libraries instead of adding it manually to each and every location in the code that initiates a connection.
Is there a way to accomplish that (on Ruby 1.9.2)?
Open URI uses the HTTP_PROXY environment variable
Here is an article on how to use it on both windows and unix variants.
http://kaamka.blogspot.com/2009/06/httpproxy-environment-variable.html
you can also set it directly in ruby using the ENV hash
ENV['HTTP_PROXY'] = 'http://username:password#hostname:port'
the net/http documentation says not to rely on the environment and set it each time
require 'net/http'
require 'uri'
proxy_host = 'your.proxy.host'
proxy_port = 8080
uri = URI.parse(ENV['http_proxy'])
proxy_user, proxy_pass = uri.userinfo.split(/:/) if uri.userinfo
Net::HTTP::Proxy(proxy_host, proxy_port,
proxy_user, proxy_pass).start('www.example.com') {|http|
# always connect to your.proxy.addr:8080 using specified username and password
:
}
from http://ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html
Yes and mechanize does too (this is for the 1.0.0 verison)
require 'mechanize'
url = 'http://www.example.com'
agent = Mechanize.new
agent.user_agent_alias = 'Mac Safari'
agent.set_proxy('127.0.0.1', '3128')
#page = agent.get(:url => url)
I'm trying to get Ruby's Net::HTTP implementation to work with SNI.
Both mail.google.com and gmail.com live on the same IP address, so when connecting via SSL, the Google server needs to know which certificate to use. By default, it returns the mail.google.com certificate, which is a problem if you're trying to implement WebFinger.
WebFinger requires you to retrieve https://gmail.com/.well-known/host-meta to get the LRDD information, however, for security reasons, it's critical to verify the SSL certificate information.
Since Google serves up the default mail.google.com certificate in this case, the SSL post_connection_check fails. The correct solution here would be to enable Server Name Indication for Net::HTTP, but it's not clear to me how to get that working with the Ruby bindings for OpenSSL. Anyone else have an idea?
You should be able to see the problem by running:
require 'open-uri'
open('https://gmail.com/.well-known/host-meta') { |f| f.read }
I've also created a gist that exhibits the problem using an earlier version of curl and OpenSSL:
https://gist.github.com/7936ef38787092a22897
For SNI support, you need a newer OpenSSL release (0.9.8f with --enable-tlsext or 0.9.8j or later) and call OpenSSL::SSL::SSLSocket#hostname = 'hostname' before SSLSocket#connect. Net::HTTPS does not support SNI yet, and open-uri doesn't.
Checking out httpclient development repository should support SNI.
https://github.com/nahi/httpclient
Let me know if you need released gem real soon now...
Ruby 2.0 will address the TLS SNI (Server Name Indication) issue:
from net/http..
# ...
# s.session = #ssl_session if #ssl_session
# # Server Name Indication (SNI) RFC 3546
# s.hostname = #address if s.respond_to? :hostname=
# Timeout.timeout(#open_timeout, Net::OpenTimeout) { s.connect }
# if #ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE
# s.post_connection_check(#address)
# end
# ...
To make this work in 1.9.2 (or higher )
apply similar patch to net/http
# ...
# BEGIN: SNI PATCH http://bugs.ruby-lang.org/issues/4351
# s.hostname = #address if s.respond_to? :hostname=
# END: SNI PATCH http://bugs.ruby-lang.org/issues/4351
# timeout(#open_timeout) { s.connect }
# if #ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE
# s.post_connection_check(#address)
# end
# ...
see also:
http://bugs.ruby-lang.org/issues/4351
http://en.wikipedia.org/wiki/Server_Name_Indication
I am trying to access an HTTPS web service that uses SSL cert authentication using Ruby EventMachine but I am not getting it to work.
I have written the following simple code block to test it end-to-end:
require 'rubygems'
require 'em-http'
EventMachine.run do
url = 'https://foobar.com/'
ssl_opts = {:private_key_file => '/tmp/private.key',
:cert_chain_file => '/tmp/ca.pem',
:verify_peer => false}
http = EventMachine::HttpRequest.new(url).get :ssl => ssl_opts
http.callback do
p http.response_header.status
p http.response_header
p http.response
EventMachine.stop
end
http.errback do
EventMachine.stop
fail "Request failed"
end
end
Running the above outputs <SSL_incomp> followed by the raised RuntimeError message. I have tried running with :verify_peer set to both true and false and it gives me the same error. Running EventMachine::HttpRequest#get without the :ssl option does the same.
I have also tried sending the request to GMail (https://mail.google.com) without the :ssl option (i.e. plain HTTPS without cert) and that works, outputting status code 200, the headers and the body.
I have tried doing the same request to the web service with curl and that works:
curl --silent --cert /tmp/private.key --cacert /tmp/ca.pem https://foobar.com/
I am thinking that I am either using the em-http-request gem or EventMachine incorrectly or that the SSL files are in a format that works with curl but not EventMachine.
I someone knows how to solve the example above or provide a similar example using EventMachine directly would be much appreciated!
The file passed to curl's --cert contains both the cert and the key (unless you pass in a --key separately). Just use /tmp/private.key as the argument to both :private_key_file and :cert_chain_file
See http://github.com/eventmachine/eventmachine/issues/#issue/115 for more details about the issue and a patch that exposes the underlying error (instead of just printing out SSL_incomp).