I understand from various sources that the HTTPS handshake is the heaviest part of using HTTPS. I'm using POSTs internally between my servers to communicate information and would like to use HTTPS for it. I wondered how long the actual HTTPS handshake lasts/"stays open"? Is it re-done for each POST I'm sending to a server, or what is the lifetime?
The SSL handshake is only done at the beginning of a session and is mainly about generating a shared session key that is used to encrypt all later traffic.
You can find a very good description of the handshake here.
I believe the handshake occurs on connection (ie, as part of the SSL negotiation). It you use HTTP keep-alive connections then the handshake only occurs once as long as the connection is active.
I don't know the particulars, but I'm sure the handshake is supposed to occur only when the session is started. It would be too expensive otherwise.
Edit: Here's a nice description of the process.
Related
If it matters, I am asking for the HMR (hot module replacement) use case.
I have already read this. My understanding is that WebSocket uses the same connection that was used during the handshake. My question is after that:
Can the client (browser) still send requests to the same server and same port after the WebSocket protocol starts?
If it can, will it be a different connection?
The short answer is YES and YES.
TL;DR
After reading this and this, I think I can answer my own question now.
A client can build as many connections to a server (to the limit that they can afford). There are always two ports involved in a connection - one at the server, and another at the client.
The server-side port is specified (and is known to the client). Clients open a temporary or dynamic port that lasts only as long as the connection is made.
I've read a lot of related topics in the net, but I still don't have an answer to my question.
Is it possible to implement flow described below?
Proxy receive request.
If request is encrypted and proxy cert is trusted then intercept.
If request is not encrypted, then intercept.
If request is encrypted and proxy cert is NOT trusted then pass it through without interception.
This behaviour should be default for all traffic going through the proxy.
It'd be also really nice to be able to get all possible info for passing encrypted requests (src and dst ip addresses etc.). Basically the same info which I can get with fiddler.
Not really. The main problem is that mitmproxy can not know if proxy cert is trusted by the client or not.
In the SSL/TLS protocol client starts with the CLIENT_HELLO and in response the server (in this case motmproxy) sends back the SERVER_HELLO message containing the generated server certificate.
The client now checks if the received server certificate is trusted. If not the connection is terminated. As far as I know the SSL/TLS spec does not define how to do so. Sems clients end back an SSL_ALERT message, other simply drop the connection, and a third group continues the SSL/TLS handshake but have certain internal values set in a way that always let the handshake fail.
There is a mitmproxy script that tries to identify connections that were not successful and then if the client asks for the same domain a second time bypasses interception.
Of course this requires that the client resends requests which is not always the case.
https://github.com/sociam/x-ray/blob/master/mitmproxy/examples/tls_passthrough.py
I am trying to capture an old application that didn't honour the system's proxy setting. The only config I can change is the server IP address.
Capturing the packets with Wireshark. Without the Charles reverse proxy, I can see requests after the first three handshake requests.
With the reverse proxy, the connection stuck after the handshake requests.
I notice that when Charles received a request and connecting to somewhere but it will just stuck there:
Following is the config of the reverse proxy (Remote host removed):
Any help, solution and workarounds would be appreciated!
First of all, your app uses neither HTTP nor HTTPS. Studying screen shot of successful connection gives some details on protocol used:
the first message after handhsake is originated by server contrary to common client-server approach, where client is responsible for sending query. This fact is enough to cross out HTTP and HTTPS.
payload data isn't human-readable, so it's a binary protocol.
based on PUSH flags, protocol is much more likely to be message-based rather than stream-based
So client establishes connection, immediately gets some command from server and replies it. Then communication continues. I can't guess exact protocol. Port number might be irrelevant, but even if it's not, there are only few protocols using 4321 port by default. Anyway, it can always be custom private protocol.
I'm not familiar with Charles, but forwarding arbitrary TCP stream is probably covered by its port forwarding feature rather than reverse proxy. However, I don't really see any benefits in sending traffic through Charles in this case, capturing data on your PC should be enough to study details.
If you are looking for traffic manipulation, for arbitrary TCP stream it's not an easy task, but it must be possible. I'm not aware of suitable tools, quick googling shows lots of utils, but some of them looks applicable to text based stream only, so deeper study is required.
Reason for Failure
It may be because you are requesting a local IP address from a remote scope, which Charles proxy doesn't applies. For POS(Proof Of Statement), please refer to the below link
https://www.charlesproxy.com/documentation/faqs/localhost-traffic-doesnt-appear-in-charles/
Solution
So In order to solve the problem for the current scenario, use
http://192.168.86.22.charlesproxy.com/
Note: The url that you request will only be proxied properly by Charles not any other proxy services.
I want to take a look at the direct encrypted https traffic of my requests. I got a server and 2 clients. With client 1 there are no problems. client 2 gets problems if the request exceeds a certain text-size. I was able to decrypt the traffic and found that client 1 extends my SOAP request every 3996 signs by an extra "f9c"-pattern. Client 2 is not doing this which is probably causing the problem on the serverside. But that is not all. If I use fiddler to decrypt the https request the server also throws an exception. So my guess is that the client is probably adding something on the https encryption too. That is why I want to take a look at this but I cannot figure out how to force fiddler to show this to me. I only get the http traffic if I disable https-decryption that shows my the handshake between client and server. So what can I do here?
problem solved.
the "f9c"-pattern is just the hex value for 3996-bytes. The problem was that client 1 was correctly chunking the http-request. Client 2 on the contrary was sending a single block and just setting the content-length of the http-request. Therefore I do not need to take a look anymore on the encrypted request.
I'm trying to write a server for a webSocket connection. I've read the spec (76, not 75) carefully. I'm using minefield as the browser.
When I try to create a WebSocket from javascript in the browser:
var ws = new WebSocket("ws://localhost:8766/hoho");
The browser responds with
"Firefox can't establish a connection to the server at ws://localhost:8766/hoho."
My server is getting a valid client handshake request, it sends back the response and then boom.
I've run every example handshake example I can find through my server and I match the given responses exactly in every instance. I'm pretty confident that the return byte stream is correct. I don't need help debugging my code, it's doing what I mean it to do. I need help debugging my use of the handshake protocol since when I give minefield what I think is a correct response it laughs at me.
My question is this: How can I debug this thing? I can think of two possibilities.
Is there any way to get minefield to tell me WHY it's rejecting my handshake?
Is there a working, public, webSocket server service on the web? If there is, I can proxy it, watch the byte streams in both direction and figure out where mine is different.
Does anyone have any ideas in these directions or any other ideas?
Thanks for any help.
I'm in the process of debugging a similar situation, and the tool I'm relying on most is netcat, with some additional use of openssl. Shut down your websocket server and run nc -l 8766. That lets you record exactly what headers are being sent. Turn the websocket server back on and use nc 8766 to paste in those same headers and see the result. openssl s_client -connect localhost:443 will let you make the request with ssl, if that is in your mix.
From there, make sure the responses conform completely to the websocket handshake protocol. For instance, my problem right now is that my responses have Connection: close, which is no good.
About the 2nd possibility.
Yes, there is a websocket server out there.
the jWebSocket demo server at http://jwebsocket.org/demos/chat/chat.htm
hope this helps
Added: Echo socket server at: http://www.websocket.org/echo.html
Here's a jsfiddle that I made from http://www.websocket.org echo websocket server which works in Chrome but not in Firefox 6: http://jsfiddle.net/awDLc/
It is adjusted to use MozWebSocket rather than WebSocket, but perhaps that isn't enough?