Accessing a specific website return always a 403 Forbidden error - indy

I try to read the website https://www.eroids.com/reviews with Indy and always get a 403.
This website seems only to load when I set the ssl version to sslvTLSv1_1. If I do that, this website loads fine, but other websites not. Most other seems to use sslvTLSv1_2.
As long I only add [sslvTLSv1_1] to the sslversion, it works, but when I add [sslvTLSv1_1, sslvTLSv1_2], the mentioned site does not load anymore (again 403), but any other site does.
My question is: How can I determine what sslversion a website need? Do I need to try to access the site with each ssl version until I get a 200 back or is there something to me unknown integrated into indy to automatically do that?

How can I determine what sslversion a website need?
In general, you can't. However, most servers support version negotiation during the TLS handshake, so that clients supporting multiple/different TLS versions can negotiate with the server for which TLS version to use. But, it sounds like maybe this particular server does not support that.
However, the fact that you are even getting an HTTP 403 response at all for an HTTPS url means a TLS session is being created fine, so the issue is something else.
Unless the server is ignoring all TLS errors during the handshake and creating a simple TLS session, THEN is sending an HTTP error in reply to an earlier TLS error. Which is rare, but not unheard of.
Do I need to try to access the site with each ssl version until I get a 200 back
In this particular situation, probably so, yes. Start with [sslvTLSv1_1, sslvTLSv1_2], and if that fails then retry with [sslvTLSv1_2], then retry with [sslvTLSv1_1], and so on.
is there something to me unknown integrated into indy to automatically do that?
Indy does not have that capability at this time, no.

Related

RabbitMQ: configuring ssl of rabbitmq_management, fail_if_no_peer_cert and fail_if_no_peer_cert parameters

general questions about the using of *fail_if_no_peer_cert *and *fail_if_no_peer_cert *params in *rabbitmq_management *on windows
if a client calls management API through https, the requests are secured by the certificate which is installed on the server and trusted on the client. actually, it means, this certificate shouldn't be rabbitmq compatible, it can be just http-certificate... is it correct?
in case i want to validate clients as well, i must have these params verify and true? what is the best practice? because i see these params explained in AMPQ settings and never in management
actually my motivation for this questions is just to understand if i need to deal with this issue at all. because setting *fail_if_no_peer_cert *as true makes a lot of things much more complicated. foe example you just can simply open management api plugin in browser, you need deal with client certificate.

Websocket connection - client attempts to TLS handshake for one address but not the other

I have a setup which involves devices connecting to a server via web sockets. I'm experiencing a strange problem where they can connect to one test server without issues, but cannot connect to a different server (hosted on Azure).
I've installed Wireshark on both of them, and can watch both the successful connection and the unsuccessful connection. It appears that the unsuccessful one attempts to initiate an SSL handshake.
Here is the successful connection:
Here is the unsuccessful connection:
It seems like the client in the successful connection is simply setting up an HTTP web socket, but in the unsuccessful setup it's try to set up a secure connection.
Why would the client be setting up different connections depending upon the server address?
The client code to create the websocket is just javascript, invoking new Websocket(address), and in both cases the address begins with the 'ws' prefix.
I have done some further investigation and found another weird behavior. As it happens, there are two domain names pointing to the same server.
If I used the domain name with the top level domain "com" (XXXX.australiaeast.cloudapp.azure.com), then the connection works.
If I used the domain name with the top level domain "dev" (comutername.mydomainname.dev) then the connection fails, with the weird TLSV1 packet.
Both works fine if I run the same client code on the Microsoft Edge browser.
This appears to be a defect in Chrome's implementation of The WebScoket API
I have posted a defect here, let's see how it goes. https://bugs.chromium.org/p/chromium/issues/detail?id=1067076
The issue in play here is HTTP Strict Transport Security (HSTS)
A browser can be configured so that certain domains have an HSTS policy attached, meaning any insecure link will automatically be converted to a secure link.
This policy will be applied if a request returns a Strict-Transport-Security header. But, and here's the significant bit, the Chrome and Firefox browsers automatically apply the policy to all dev domains.
It appears that up until recently, the Chrome browser and Chrome OS was not observing the policy with regard to WebSocket connections. This has changed, and now WebSockets will observe the HSTS policy.
The upshot is, if you have a websocket using the ws protocol and not the wss protocol, and it's on a .dev domain, your Chrome and Firefox browser will not be able to connect to it.

Microsoft Internet Transfer Control failing over https

I have an old VB app that uses the Microsoft Internet Transfer Control (or Inet) to read info from a web page over HTTPS. It is installed on a number of Windows 10 machines and it works fine on all of them except for one. On this machine, when the call is made over HTTPS, the response comes back blank. The request never makes it to the web server because there is no entry for it in the IIS logs. Calls over HTTP to the same URL work just fine, only the HTTPS call has this problem.
I suspect the problem is with TLS 1.0. That protocol is disabled on the web server. I'm aware that older browsers, including old versions of IE, require TLS 1.0. Is there a setting that controls whether Inet can support TLS 1.1+? I did check Internet Options and "Use TLS 1.1" and "Use TLS 1.2" are already checked, so maybe these settings don't apply to Inet and I need to look elsewhere. Or is the problem something else?
Here is the code that uses Inet to make the HTTPS call. It's pretty straightforward.
response = Inet1.OpenURL("https://my_site/some_page")
' response is blank
I had the same issue. Experimenting with internet properties I found that unchecking "Use HTTP 1.1", leave "HTTP 1.1 through proxy connections" checked, fixes the problem. Have to close your program and restart it if you make the change while it's running. Don't have to reboot your PC. Hope this helps
if you disabled the TLS 1.0 in the WebServer then it will not work in the machines those are supported till TLS 1.0.
VB browser uses IE7 by default. If the HTTPS link works on the machine regular browser then you need to check for document mode settings otherwise you need to enable TLS 1.0 in the webserver.

gevent-socketio - Error during WebSocket handshake: Unexpected response code: 401

I am working on a pyramid realtime app that would benefit from utilizing websockets. I found that the gevent-socketio library would probably be best for my purposes.
When testing locally on my PC, the websocket transport is used and runs just fine. However, when testing on a server, the handshake fails reporting a 401 response. It then falls back to a long polling transport, which works, but I would really benefit from the speed of websocket.
This makes me believe that there is some kind of authorization issue happening since I am not connecting to the local machine anymore, but I am at a complete loss as to where to look for help.
EDIT:
Now the issue was narrowed down to the fact that we were using mod_proxy, a lighttpd module, to instance the server for development and production. Is there any work around for proxying or do we have to a abandon the idea of server instancing? Ideally we would like to have some way of authorizing users to access our development server, which is how we had it set up with mod_proxy.

lighttpd/mod_websocket mqtt handshake fail (no subproto)

I have set up lighttpd with mod_websocket as discussed in Dom Bramley's blog entry (except that I am using a BeagleBone Black with Debian Wheezy instead of an rPi.)
https://www.ibm.com/developerworks/community/blogs/B-Fool/entry/setting_up_an_mqtt_websocket_gateway_for_raspberry_pi?lang=en
[During the lighttpd/mod_websocket build process I was asked if I wanted to patch the server and I said yes.]
I have the mosquito MQTT broker running on the same host and publishing on various topics.
When I try to connect to the broker with a browser client via the web socket, I can see that everything works okay in terms of the http upgrade to websocket and forwarding the connection request to mosquitto. Mosquitto gets the connection request and accepts it. However, the response that gets back to the browser does not include the Sec-Websocket-Protocol header echoing the subproto specifier mqttv3.1 that was in the original upgrade request. The client correctly rejects this answer and the connection is shut down.
The javascript error from mqttws31.js:912 is "Sent non-empty Sec-Websocket-Protocol header but no response is received." With Wireshark, I can see that this is true; the 101 Switching Protocols response has headers Upgrade, Connection, and Sec-Websocket-Accept, but nothing else.
My mod_websocket config file defines host, port, type, and subproto the same as Dom's example, and I can see from various debug statements that the request gets all the way to Mosquitto correctly.
Can anyone suggest how to get the Websocket-Protocol header to be included in the response? It must work, Dom wrote a blog post describing how he did it!
I think recent versions of mod_websocket broke/removed subprotocol support, but can't confirm it right now. You could try an earlier version, or use a dedicated websocket to tcp gateway like WSS
https://github.com/stylpen/WSS/
The mod_websocket author (Norio Kobota) quickly and effectively resolved this issue for me by making an update to mod_websocket. The fix is currently in a development branch, and available on github. Our discussion is part of the thread for mod_websocket issue 28.
Briefly, the use case that I have (pre-written client library and existing TCP backend) is much less flexible than a roll-your-own client and server combination with respect to connect-time protocol negotiation. However, in my case I don't really need any flexibility or negotiation with the backend, and so mod_websocket can just echo the configuration it has been given without having to dive into the details of the subprotocol.
The updated mod_websocket echoes the subproto entry from its config file during websocket handshake which satisfies the MQTT client library.
So now I have two solutions for adapters between websocket clients and TCP backends! Thanks all for your help.
Doug Johnson

Resources