Problem requesting a HTTPS with TCL - https

I'm trying to do the following request using TCL (OpenACS)
http::register https 443 tls::socket
set url "https://encrypted.google.com"
set token [http::geturl $url -timeout 30000]
set status [http::status $token]
set answer [http::data $token]
http::cleanup $token
http::unregister https
The problem is that when I read the $status variable I get "eof" and the $answer variable becomes empty. I tried enabling tls V.1
http::register https 443 [list tls::socket -tls1 1]
and it works only for the site https://www.galileo.edu, but not for https://encrypted.google.com.
The site what I'm trying to connect is https://graph.facebook.com/me/feed?access_token=...
but it doesn't work.
I used curl to retrieve the contents of the pages in HTTPS and it works, I have installed OpenSSL, so I can't see the problem, there is another way to do HTTPS connections with TCL?.
I can't see if this is a problem of coding (maybe I'm registered wrong the https protocol) or maybe It is a bad configuration of my server. Hope somebody helps!! Thanks!

Hmm, I can't reproduce that. Which patchlevel of Tcl are you using, and which version of the tls package?
I did this:
package require http
package require tls
# This is your code, cut-n-pasted with blank lines removed
http::register https 443 tls::socket
set url "https://encrypted.google.com"
set token [http::geturl $url -timeout 30000]
set status [http::status $token]
set answer [http::data $token]
http::cleanup $token
http::unregister https
puts $status
And it produces “ok” as output, with $status's contents looking likely too (but too long to paste here). This is with Tcl 8.5.2 (I know I need to upgrade), http 2.7 and tls 1.6.

Related

Failure/Error: $driver.navigate.to ENV['URL'] in ruby

While running the test case in the terminal with headless = true. I am getting the below error message.
Error message
Please find the details below:
I am using chromedriver version 86 with chrome browswer version 86 and same has been updated in .env file.
Also, I am running the scripts with Ubuntu in windows.
The error, ERR_NAME_NOT_RESOLVED means that the URL could not be resolved into an IP address. Most likely, ENV['URL'] is not returning a valid URL, there is a typo in the hostname, or it is referencing a private host that the script's DNS cannot resolve. Consider temporarily adding some debugging information, such as
url = ENV['URL']
STDERR.puts "Navigating to [#{url}]"
$driver.navigate.to url
or using Pry to pause your code, and check to make sure things are as you expect:
require 'pry'; binding.pry
$driver.navigate.to ENV['URL']
From the Pry shell, you can see what ENV['URL'] evaluates to, and also manually drive the $driver to see if it works as you expect.
If the URL in fact seems correct, then the issue is likely related to a private host and DNS issue. Ensure you can navigate to the url by other means on that same server (if it's a server, consider trying to ping the hostname, or use curl or wget for a quick check).

ruby rest-client ipv6 request failure with Apache 2.2.31

I've been stuck with this for about two days...
I use ruby(version 2.3.3p222) gem rest-client(v2.0.0) to send a GET request with a ipv6 url to the server (Apache/2.2.31):
url = 'https://[fd36:4928:8040:dc10:0000:0000:0000:0160]:8080/resources/1'
resource = RestClient::Resource.new(url, :ssl_version => 'TLSv1', :verify_ssl => false, :headers => {'Authorization' => 'Basic cm9vdDAbCdEfwYXNzMSE='})
resource.get
I got a 400 bad response and the body says:"Your browser sent a request that this server could not understand. Additionally, a 400 Bad Request error was encountered while trying to use an ErrorDocument to handle the request"
However I can use curl command with the same parameters and get the right response, so I suspect maybe it's something wrong with the header of my rest-client request.
PS: I also tested with adding the 'host'
header: {'Authorization' => 'Basic cm9vdDAbCdEfwYXNzMSE=', 'host' => '[fd36:4928:8040:dc10:0000:0000:0000:0160]:8080' }
It still failed with the same bad response.
I just noticed the appache error for this request, it says:
"httpd[29124]: [error] Hostname fd36:4928:8040:dc10:0000:0000:0000:0160 provided via SNI and hostname fd36:4928:8040:dc10:0000:0000:0000 provided via HTTP are different
"
The curl command you supplied would translate --user admin:password into the following header:
Authorization: Basic YWRtaW46cGFzc3dvcmQ=
However, you're sending
Basic: cm9vdDAbCdEfwYXNzMSE=
which is not the same thing... so the server is probably complaining about not getting the correct auth...
After debugging and googling for another day, this problem seems to be clear:
From a similar bug report to chrome https://bugs.chromium.org/p/chromium/issues/detail?id=500981, "SNI is only hostnames, and should never contain IPs.". However ruby does use ip as hostname (in this case rest-client is nothing to blame since it just delegate everything down to ruby lib). You can find evidence in Net::HTTP#connect (around line 922):
# Server Name Indication (SNI) RFC 3546
s.hostname = #address if s.respond_to? :hostname=
Just comment out the last line it will work (to workaround this you have to do a monkey patch). Additionally, as pointed out by #alberge, host header does not contain brackets, the final request host header is like this: "FD36:4928:8040:DC10::162", no "[ ]" around.
Also on Apache side, it does something wrong since it just strips off everything from the last colon to get the host name without any extra check- this still exists in version 2.4.10, not sure if it is fixed or not.
This appears to be either a bug in rest-client or a regression in Ruby Net::HTTP.
https://github.com/rest-client/rest-client/issues/583
What version of Ruby are you using? Have you tried using Ruby 2.1 to see if it works there?
EDIT:
This is Ruby Bug #12642. The Host header for an IPv6 address is sent with no enclosing [ ].
Ruby in 2.1.6 - 2.1.10 doesn't have the bug, but versions >= 2.2.0 are affected.
And worse still, there's a bug with setting an explicit IPv6 Host header so you get an exception URI::InvalidComponentError: bad component(expected host component): [

Unable to determine SOCKS version from socks

Using proxy connection (HTTP Proxy : 10.3.100.207, Port 8080).
Using python's request module's get function, getting following error:
"Unable to determine SOCKS version from socks://10.3.100.207:8080/"
Try export all_proxy="socks5://10.3.100.207:8080" if you want to use socks proxy.
Else export all_proxy="" for no proxy.
Hope This works. :D
I resolved this problem by removing "socks:" in_all_proxy.
Try this:
unset all_proxy && unset ALL_PROXY

Bower calls blocked by corporate proxy

I'm trying to use Bower for a web app, but find myself hitting some sort of proxy issues:
D:\>bower search jquery
bower retry Request to https://bower.herokuapp.com/packages/search/jquery failed with ECONNRESET, retrying in 1.2s
bower retry Request to https://bower.herokuapp.com/packages/search/jquery failed with ECONNRESET, retrying in 2.5s
bower retry Request to https://bower.herokuapp.com/packages/search/jquery failed with ECONNRESET, retrying in 6.8s
bower retry Request to https://bower.herokuapp.com/packages/search/jquery failed with ECONNRESET, retrying in 15.1s
bower retry Request to https://bower.herokuapp.com/packages/search/jquery failed with ECONNRESET, retrying in 20.3s
bower ECONNRESET Request to https://bower.herokuapp.com/packages/search/jquery failed: tunneling socket could not be established, cause=Parse Error
Relevant points:
I can browse to https://bower.herokuapp.com/packages/search/jquery and it returns a full json response.
I can use git to clone, both using the git:// protocol and http(s).
I can use NPM directly without these issues
I've tried using Fiddler to determine what's being blocked, but it doesn't detect any calls from the Bower command. I can see calls from NPM commands in Fiddler.
I've searched the Bower issues list, seen similar issues, but they either have no solution or it doesn't seem quite the same as mine.
Any ideas?
Thanks #user3259967
This did the job.
I would like to add that if you are behind a proxy that needs to be authenticated, you can add the username/password to your .bowerrc file.
{
"directory": "library",
"registry": "http://bower.herokuapp.com",
"proxy":"http://<USERNAME>:<PASSWORD>#<PROXY_IP>:<PROXY_PORT>/",
"https-proxy":"http://<USERNAME>:<PASSWORD>#<PROXY_IP>:<PROXY_PORT>/"
}
NOTICE the use of http:// in https-proxy
The solution for me is this config .bowerrc
{
"directory": "vendor",
"registry": "http://bower.herokuapp.com",
"proxy": "http://<user>:<pwd>#proxy.host.br:8080",
"https-proxy": "http://<user>:<pwd>#proxy.host.br:8080",
"strict-ssl": false
}
Using the http protocol in https-proxy plus registry entry with http protocol.
Remember to change 8080 port number to whatever is yours proxy server port.
Are you behind a proxy?
Have you set up environment variables HTTP_PROXY and HTTPS_PROXY?
SET HTTP_PROXY=http://yourproxyserver:yourproxyport
SET HTTPS_PROXY=http://yourproxyserver:yourproxyport
Try changing the registry value in your .bowerrc:
{
"registry": "http://bower.herokuapp.com"
}
I did not have a .bowerrc file to configure my bower settings. I found the settings living in a file called defaults.js. found under "C:\...\bower\node_modules\bower-config\lib\util\defaults.js"
I hope this helps others:
var defaults = {
'cwd': process.cwd(),
'directory': 'bower_components',
'registry': 'http://bower.herokuapp.com',
'shorthand-resolver': 'git://github.com/{{owner}}/{{package}}.git',
'tmp': paths.tmp,
'proxy': '<<http://user:pass#proxy:port>>', // change proxy here or at the top
'https-proxy': '<<http://user:pass#proxy:port>>', // change proxy here or at the top
'timeout': 30000,
'ca': { search: [] },
'strict-ssl': false,
'user-agent': userAgent,
'color': true,
'interactive': null,
'storage': {
packages: path.join(paths.cache, 'packages'),
links: path.join(paths.data, 'links'),
completion: path.join(paths.data, 'completion'),
registry: path.join(paths.cache, 'registry'),
empty: path.join(paths.data, 'empty') // Empty dir, used in GIT_TEMPLATE_DIR among others
}
};
module.exports = defaults;
you can try suggest #thebignet into same issue on GitHub
Set proxy, https-proxy and strict-ssl configuration into .bowerrc File :
{
"proxy" : "http://<host>:<port>",
"https-proxy" : "http://<host>:<port>",
"strict-ssl" : false
}
But you must run Command from terminal:
git config --global url."https://".insteadOf git://
"strict-ssl": false
in .bowerrc did for me
For Win 7.
What worked for me , are below steps as suggested at this link - read #nanowizard answer.
In .bowerrc file, remove any http_proxy / https_proxy settings that
you might have done earlier. This is important.
So final content of this file should look like :-
{
"directory": "app/bower_components"
}
Set environment variables in your pc - http_proxy and https_proxy to your corporate firewall proxy. In case, your corporate proxy requires authentication and if your password contains special characters, convert it to hex form as suggested by this link. As in my case escaping of characters with '\' did not help. Also I had to restart my system.
Note :
http_proxy and https_proxy should contain same proxy address as shown below
http_proxy = http://<user>:<password>#<your company proxy>:<port>
https_proxy= http://<user>:<password>#<your company proxy>:<port> ->Note no 's' in http://...
I am behind corporate firewall and I have to specify domain name too.
None of these answers worked for me. Here is what I did -
Downloaded CNTLM from http://cntlm.sourceforge.net/
Obviously installed it.
Open up cntml.ini and change the following
Domain your_domain_name
Username your_domain_username
Password your_domain_passowrd
PassLM 1AD35398BE6565DDB5C4EF70C0593492 (uncomment this)
PassNT 77B9081511704EE852F94227CF48A793 (uncomment this too)
Proxy http://localhost:8888
Go to services.msc and start the CNTLM Authentication service.
Download Fiddler 4/2 (whatever they call it).
Install this too. This will run in http://localhost:8888
Now whatever program you’re running forward(proxy) it to http://locahost:3128 ( that’s what CNTLM is running.)
In this case specify http.proxy and https.proxy as http://localhost:8888
This will work for other client programs. Just specify proxy as http://localhost:8888
its work for me to change in .bowerrc file
{
"directory": "client/lib",
"registry": "http://bower.herokuapp.com",
"proxy":"http://192.168.1.205:3228",
"https-proxy":"http://192.168.1.205:3228"
}
where client/lib is installation directory where do you want to install
and http://192.168.1.205:3228 is your proxy ip with port. corporate proxy can be different according to oraganization.
In addition to setting the below proxy in .bowerrc:
{
"directory": "app/bower_components",
"proxy":"http://<user>:<password>#proxy.company.com:<proxy-port>",
"https-proxy":"http://<user>:<password>#proxy.company.com:<proxy-port>",
"http-proxy":"http://<user>:<password>#proxy.company.com:<proxy-port>",
"strict-ssl": false,
"registry": "http://bower.herokuapp.com"
}
I am required to run the following commands to fix the issue:
npm cache clean
bower cache clean
bower install
The registry used in the the other answers is now deprecated. Please update it!
{
"proxy":"http://<user>:<password>#proxy.company.com:<proxy-port>",
"https-proxy":"http://<user>:<password>#proxy.company.com:<proxy-port>",
"registry": "https://registry.bower.io"
}
{
"directory": "library",
"registry": "http://bower.herokuapp.com",
"proxy":"http://<USERNAME>:<PASSWORD>#<PROXY_IP>:<PROXY_PORT>/",
"https-proxy":"http://<USERNAME>:<PASSWORD>#<PROXY_IP>:<PROXY_PORT>/"
}
This code worked for me. I am using Win 7 and chrome and also git bash. Here few things need to be cleared. This takes me huge time to find the actual data regarding the user name, password, proxy IP and Port. I will describe it step by step so that every learners can easily grasp the message:
Create a file in the notepad named .bowerrc in the login folder; You can go there by typing at Start>Run>%UserProfile% and press OK.
Type above code in the .bowerrc file with the following changes:
Replace <USERNAME> with your internet connection user ID or login ID
Replace <PASSWORD> with your internet connection password or login password.
Replace <PROXY_IP> and <PROXY_PORT> with the working proxy IP address and its port number.
**Note: There should be no angle brackets.**
Proxy IP should be different than your own IP.
Before using any proxy IP and port you should check it is working by changing your proxy IP and port.
You can go through this link to know the details of proxy settings at description here
From this proxy settings you will get Proxy IP and Port.
Recheck all the input so that all are correct and save and close the file.
Open git bash and change directory to the project file and type command and hit enter, in my case, git bash command:
a#a-PC MINGW32 /d/conFusion
$ bower install
It worked like magic.
In case it helps someone, I had a 'bower blocked by group policy' error.
Solution was to make an exception in CryptoPrevent, a application installed on our company computers to prevent crypto lockers.
For info, in your .bowerrc file you can add a no-proxy attribute. I don't know since when it is supported but it works on bower 1.7.4 and it solve the issue of bower behind a corporate proxy with an internal repository
.bowerrc :
{
"directory": "bower_components",
"proxy": "http://yourProxy:yourPort",
"https-proxy":"http://yourProxy:yourPort",
"no-proxy":"myserver.mydomain.com"
}
Regards
Please make sure there are no special characters in your proxy password. Convert it to hex. It works for me.

Server Name Indication support in Net::HTTP?

I'm trying to get Ruby's Net::HTTP implementation to work with SNI.
Both mail.google.com and gmail.com live on the same IP address, so when connecting via SSL, the Google server needs to know which certificate to use. By default, it returns the mail.google.com certificate, which is a problem if you're trying to implement WebFinger.
WebFinger requires you to retrieve https://gmail.com/.well-known/host-meta to get the LRDD information, however, for security reasons, it's critical to verify the SSL certificate information.
Since Google serves up the default mail.google.com certificate in this case, the SSL post_connection_check fails. The correct solution here would be to enable Server Name Indication for Net::HTTP, but it's not clear to me how to get that working with the Ruby bindings for OpenSSL. Anyone else have an idea?
You should be able to see the problem by running:
require 'open-uri'
open('https://gmail.com/.well-known/host-meta') { |f| f.read }
I've also created a gist that exhibits the problem using an earlier version of curl and OpenSSL:
https://gist.github.com/7936ef38787092a22897
For SNI support, you need a newer OpenSSL release (0.9.8f with --enable-tlsext or 0.9.8j or later) and call OpenSSL::SSL::SSLSocket#hostname = 'hostname' before SSLSocket#connect. Net::HTTPS does not support SNI yet, and open-uri doesn't.
Checking out httpclient development repository should support SNI.
https://github.com/nahi/httpclient
Let me know if you need released gem real soon now...
Ruby 2.0 will address the TLS SNI (Server Name Indication) issue:
from net/http..
# ...
# s.session = #ssl_session if #ssl_session
# # Server Name Indication (SNI) RFC 3546
# s.hostname = #address if s.respond_to? :hostname=
# Timeout.timeout(#open_timeout, Net::OpenTimeout) { s.connect }
# if #ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE
# s.post_connection_check(#address)
# end
# ...
To make this work in 1.9.2 (or higher )
apply similar patch to net/http
# ...
# BEGIN: SNI PATCH http://bugs.ruby-lang.org/issues/4351
# s.hostname = #address if s.respond_to? :hostname=
# END: SNI PATCH http://bugs.ruby-lang.org/issues/4351
# timeout(#open_timeout) { s.connect }
# if #ssl_context.verify_mode != OpenSSL::SSL::VERIFY_NONE
# s.post_connection_check(#address)
# end
# ...
see also:
http://bugs.ruby-lang.org/issues/4351
http://en.wikipedia.org/wiki/Server_Name_Indication

Resources