I am following this website for connecting to Microsoft Graph to get the access token using ADAL gem.
I have written a Rake task to perform this. This works fine in the open network but fails in the Corporate Network.
I have set the proxy credentials in the bash profile and in the bashrc file in my terminal
export http_proxy=http://username:pwd#proxy.corporate.com:8080/
export HTTP_PROXY=http://username:pwd#proxy.corporate.com:8080/
export https_proxy=http://username:pwd#proxy.corporate.com:8080/
export HTTPS_PROXY=http://username:pwd#proxy.corporate.com:8080/
I am receiving
SocketError: Failed to open TCP connection to login.microsoftonline.com:443 (Hostname not known: login.microsoftonline.com)
Solution by OP.
Issue has been resolved after tweeking the user_credential.rb file in ADAL gem
File to edit: /azure-activedirectory-library-for-ruby-24e1b7f0dc37/lib/adal/user_credential.rb
def realm_discovery_response
#realm_discovery_response ||=
JSON.parse(Net::HTTP.get(realm_discovery_uri))
end
Replace the above code with the below as Net::HTTP.get doesnot take the system proxy use and only Net::HTTP.Get.new works
def realm_discovery_response
uri = URI.parse(realm_discovery_uri.to_s)
puts "URI #{uri}"
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true if uri.scheme == 'https'
req = Net::HTTP::Get.new(uri.request_uri)
res = http.request(req)
#realm_discovery_response ||= JSON.parse(res.body)
end
Related
I want to use Ruby httpclient to make requests. I tried to implement this code:
def submit!
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = HTTPClient.new(url)
request.basic_auth('username', 'pass')
request['content-type'] = 'application/xml'
request['cache-control'] = 'no-cache'
request.body = request_body
response = http.request(request).body
response = Response.new(response)
check_for_approved_response(response)
response
end
But I get error:
unsupported proxy https://staging.gate.some_address.net/process/3ab20f2ddbe78ab8be9f5a9645c1010330dc868f
Can you recommend some solution how to fix it?
make sure ruby can see and read /etc/resolv.conf and that you don't have HTTP_PROXY env variable set. Also check that your ssl isn't using a non-standard ssl port.
Run echo $HTTP_PROXY on your terminal. If you see any value, its not good. You'll have to unset that variable.
See - https://github.com/savonrb/savon/issues/123#issuecomment-620825
https://www.rubydoc.info/gems/httpclient/2.1.5.2/HTTPClient
Accessing resources through HTTP proxy. You can use environment variable 'http_proxy' or 'HTTP_PROXY' instead.
clnt = HTTPClient.new('http://myproxy:8080')
So in your code request = HTTPClient.new(url) is putting target url as proxy of HTTPClient. (It is indeed a little confusing when you come from net/http)
I have the following method in our Ruby 1.8.7 project:
def self.ping_server
request_data = get_request_data
uri = 'another_server.our_company.com'
https = Net::HTTP.new(uri)
https.use_ssl = true
path = "/our_service"
data = request_data.to_json
response = https.post(path, data, {'Content-Type' => 'application/json'})
return response
end
Whenever I run this method, I get the following time out error:
Completed 500 Internal Server Error in 128936ms
Errno::ETIMEDOUT (Connection timed out - connect(2)):
lib/my_class.rb:51:in `ping_our_server'
I checked with a coworker who has access to the logs for another_server.our_company.com. My request isn't arriving at the other server.
What should I do to get my request to work?
EDIT: On further inspection, this is what I think is happening (but I'm not completely sure): Our other server will only accept HTTPS requests, but it looks like my request is being sent over HTTP for some reason. Is there something I need to add to make sure my request is sent over HTTPS?
According to this website, this is how you send an HTTPS request:
require "net/https"
require "uri"
uri = URI.parse("https://secure.com/")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Get.new(uri.request_uri)
response = http.request(request)
response.body
response.status
response["header-here"] # All headers are lowercase
According to this website (which is linked from my first link), you should also do this to close up a vulnerability in the net/https library:
To get going, you need a local CA certificates bundle, the official
curl site maintains an up to date cacert.pem / ca-bundle.crt file
containing all of the major certificates if you need one.
Next, after a gem install always_verify_ssl_certificates, you can be
up and running with a test as simply as:
require 'always_verify_ssl_certificates'
AlwaysVerifySSLCertificates.ca_file = "/path/path/path/cacert.pem"
http= Net::HTTP.new('https://some.ssl.site', 443)
http.use_ssl = true
req = Net::HTTP::Get.new('/')
response = http.request(req)
If the site
has a bad certificate an error will be raised at this point. If not, a
legitimate HTTP response object will be returned.
After looking a lot, I've found some solutions that seem working, but not for me...
For example, I have this script:
require 'net/http'
require "net/https"
#http=Net::HTTP.new('www.xxxxxxx.net', 443)
#http.use_ssl = true
#http.verify_mode = OpenSSL::SSL::VERIFY_NONE
#http.start() {|http|
req = Net::HTTP::Get.new('/gb/PastSetupsXLS.asp?SR=31,6')
req.basic_auth 'my_user', 'my_password'
response = http.request(req)
print response.body
}
When I run it, it gives me a page that requests for authentication, but if I write the following URL in the browser, I get into the website without problems:
https://my_user:my_password#www.xxxxxxx.net/gb/PastSetupsXLS.asp?SR=31,6
I have also tried with open-uri:
module OpenSSL
module SSL
remove_const :VERIFY_PEER
end
end
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE
def download(full_url, to_here)
writeOut = open(to_here, "wb")
writeOut.write(open(full_url, :http_basic_authentication=>["my_user", "my_password"]).read)
writeOut.close
end
download('https://www.xxxxxxx.net/gb/PastSetupsXLS.asp?SR=31,6', "target_file.html")
But the result is the same, the site is asking for user authentication.
Any tips of what am I doing wrong?. Must I encode the password in Base 64?
I wrote a piece of code based on examples given in the Net::HTTP docs and tested it on my local WAMP server - it works fine. Here's what I have:
require 'net/http'
require 'openssl'
uri = URI('https://localhost/')
Net::HTTP.start(uri.host, uri.port,
:use_ssl => uri.scheme == 'https',
:verify_mode => OpenSSL::SSL::VERIFY_NONE) do |http|
request = Net::HTTP::Get.new uri.request_uri
request.basic_auth 'matt', 'secret'
response = http.request request # Net::HTTPResponse object
puts response
puts response.body
end
And my .htaccess file looks like this:
AuthName "Authorization required"
AuthUserFile c:/wamp/www/ssl/.htpasswd
AuthType basic
Require valid-user
My .htpasswd is just a one liner generated with htpasswd -c .htpasswd matt for password "secret". When I run my code I get "200 OK" and contents of index.html. If I remove the request.basic_auth line, I get 401 error.
UPDATE:
As indicated by #stereoscott in the comments, the :verify_mode value I used in the example (OpenSSL::SSL::VERIFY_NONE) is not safe for production.
All available options listed in the OpenSSL::SSL::SSLContext docs are: VERIFY_NONE, VERIFY_PEER, VERIFY_CLIENT_ONCE, VERIFY_FAIL_IF_NO_PEER_CERT, out of which (according to the OpenSSL docs) only the first two ones are used in the client mode.
So VERIFY_PEER should be used on production, which is the default btw, so you can skip it entirely.
The following is what ended up working for me:
require "uri"
require "net/http"
url = URI("https://localhost/")
https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true
request = Net::HTTP::Get.new(url)
request["Authorization"] = "Basic " + Base64::encode64("my_user:my_password")
response = https.request(request)
puts response.read_body
I came up with this by building a new HTTP Request in Postman, specifying the URL, choosing an Authorization Type of "Basic Auth," and inputting the credentials.
Clicking the Code icon (</>) and selecting "Ruby - Net::HTTP" will then generate a code snippet, giving you the output above.
Postman took care of encoding the credentials, but this answer helped me to dynamically set these values. You also can likely omit the "cookie" key as part of the request.
For a project I need to make an HTTPS call through a Socks5 proxy so I can access the remote API of a customer.
The original code I'm using is this:
url = "/proxyValidate?service=#{migration_station_verification_url}&ticket=#{service_ticket}"
if Rails.env == 'development'
uri = URI.parse(soe_cas_url + url)
Net::HTTP.SOCKSProxy('127.0.0.1', 1080).start(uri.host, uri.port) do |http|
res = http.get(uri.path)
doc = Nokogiri::XML(res)
doc.remove_namespaces!
doc.xpath('//sessionId').first.content
end
else
http = Net::HTTP.new(soe_cas_url)
http.use_ssl = true
req = Net::HTTP::Get.new(url)
res = http.request(req)
doc = Nokogiri::XML(res)
doc.remove_namespaces!
doc.xpath('//sessionId').first.content
end
The goal is to use the proxy only in development environment. When I make the call using the proxy I receive the error:
wrong status line: "<!DOCTYPE
Googling this shows I need to set the use_ssl flag to true, which works outside the development cycle. When using the socksify there is no such mechanism. If I call this inside the do block, I receive an error that the session is already started. Calling it outside the block returns an exception that no such function exists.
Anyone can tell me how to run SSL/HTTPS through the socksify proxy?
EDIT:
I'm currently playing around with pure sockets to get what I want, using the following code:
socket = Socket::TCPSocket.open('127.0.0.1', 1080)
ssl_context = OpenSSL::SSL::SSLContext.new
ssl_context.verify_mode = OpenSSL::SSL::VERIFY_NONE
ssl_context.ca_file = File.join(Rails.root, 'ssl', 'cacert.pem')
ssl_socket = OpenSSL::SSL::SSLSocket.new(socket, ssl_context)
ssl_socket.sync_close = true
ssl_socket.connect
request = "GET #{url} HTTP/1.1\r\nAccept: */*\r\nHost: #{soe_cas_url}"
ssl_socket.puts(request)
ssl_socket.puts("")
response = ""
while (line = ssl_socket.gets)
response << line
end
This however fails at the ssl_socket.connect with the following error:
ssl_connect syscall returned=5 errno=0 state=sslv2/v3 read server hello a
I've tried using self-signed certificates, no certificates etc but nothing seems to work. I do know the endpoint relies on self-signed certificates on staging environmnet.
Excuse the tabs. I'm trying to download a file from remote to local and I keep getting one back that is exactly 310 bytes (regardless of what file I choose to download). I tried setting the timeout to 0, but this isn't working. What am I doing wrong? Thanks!
#downloadUrl = 'https://username:password#api.net'
Net::HTTP.start(#downloadUrl) do |http|
response = http.get('/file.ext')
open('/Users/me/file.ext', "wb", :read_timeout=>0) do |file|
file.write(response.body)
end
end
EDIT: I don't want to use httpclient, I want to use standard net/http. I am almost there, but I keep getting initialize': getaddrinfo: nodename nor servname provided, or not known (SocketError) thrown at Net::HTTP.start(url.path). But when I remote "https", I get ECONNREFUSED. ...Getting closer?
url = URI.parse('https://api.net/file.ext')
#request = Net::HTTP.start(url.path)
#request.basic_auth 'username', 'password'
sock = Net::HTTP.new(url.host, 443)
sock.use_ssl = true
sock.ssl_version='SSLv3'
sock.start do |http|
response = http.get(#request)
open('/Users/me/file.ext', "wb", :read_timeout=>0) do |file|
file.write(response.body)
end
end
Using httpclient is much simpler when accessing via SSL.
gem install httpclient
I haven't tried this out, but this should work for you. Here is the rdoc.
require 'httpclient'
domain = "https://api.net/file.ext"
username = "username"
password = "password"
clnt = HTTPClient.new
clnt.set_auth(domain, username, password)
res = clnt.get_content(https_url)
You can refer to the "writing a binary file in ruby" question for saving your response to a file.
Updated Using net/http
You should be doing Net::HTTP.start(url.host, url.port) instead of Net:HTTP.start(url.path). See RDoc for Net::HTTP.start.
url = URI.parse("https://api.net/file.ext")
download_file = opne("/Users/me/file.ext", "wb")
request = Net::HTTP.start(url.host, url.port)
# .. set basic auth, verify peer etc
begin
request.request_get(url.path) do |resp|
resp.read_body { |segment| download_file.write(segment) }
end
ensure
download_file.close
end