What additional changes are required to make this simple HTTP header to speak to a HTTPS enabled server.
GET /index.php HTTP/1.1
Host: localhost
[CR]
[CR]
EDIT
To add some context, all I'm trying to do is open a TCP port (443) and read the index page but the server seems to return a 400 - Bad request along with a message that goes "You're speaking plain HTTP to an SSL-enabled server port." I thought this probably meant altering the header in some fashion.
HTTP runs on top of secured channel. No adjustments are needed at all on HTTP level. You need to encrypt the whole traffic going to the socket (after it leaves HTTP client code) and decrypt the traffic coming from the socket before it reaches HTTP client.
You encrypt the payload with the information from the server to encrypt. This is done via handshake on a server by server basis, so you can't just fake it once have it work everywhere.
The payload includes the query string, cookies, form, etc.
Related
Is there a way for Squid Proxy to intercept and filter HTTPS payload from request made before it is being sent? Just like the way mitmproxy does.
I found out it's possible to log the payload with ssl-bumping, I had some success also altering response data with sample ecap. Only if I knew some C programming.
Also, iptables has a function to filter payload by string but only with http traffic. I don't know if it's possible to strip the SSL traffic first before feeding it to iptables and send the request outside before sending back (mitm proxy basically).
The only drawback with mitmproxy right now is unable for daemonization (you need to hack it), and I have my doubt using it in a large load.
Any one way or another?
I want to take a look at the direct encrypted https traffic of my requests. I got a server and 2 clients. With client 1 there are no problems. client 2 gets problems if the request exceeds a certain text-size. I was able to decrypt the traffic and found that client 1 extends my SOAP request every 3996 signs by an extra "f9c"-pattern. Client 2 is not doing this which is probably causing the problem on the serverside. But that is not all. If I use fiddler to decrypt the https request the server also throws an exception. So my guess is that the client is probably adding something on the https encryption too. That is why I want to take a look at this but I cannot figure out how to force fiddler to show this to me. I only get the http traffic if I disable https-decryption that shows my the handshake between client and server. So what can I do here?
problem solved.
the "f9c"-pattern is just the hex value for 3996-bytes. The problem was that client 1 was correctly chunking the http-request. Client 2 on the contrary was sending a single block and just setting the content-length of the http-request. Therefore I do not need to take a look anymore on the encrypted request.
I am serving content locally, accessible through http://0.0.0.0:4000. That works ok, I get a correct webpage, which contains the following line inside a script:
var socket = io('http://example.com');
i.e. I am referencing an external server. Now my browser shows the followoing error:
GET http://example.com:4000/socket.io/?EIO=3&transport=polling&t=1417447089410-1 net::ERR_CONNECTION_REFUSED
That is, the browser is trying to connect using the same port that it used to get the original page.
Everything works fine when both the SocketIO server and the web server listen on the same port.
Am I missing something? Is this a bug? Is there a workaround? Thank you.
You can read here about how a plain webSocket is initially set up. It all starts with a somewhat standard HTTP GET request, but one that has some special headers set:
GET /chat HTTP/1.1
Host: example.com:8000
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
The interchange may also allow the host to enforce requests only from web pages on certain origins. While this header can be spoofed from non-web-browser agents (so the server has to be prepared for that), it will likely be correct when the OP is using a real browser (assuming no proxy is modifying it).
If the server accepts the incoming request, it will then return an HTTP response that looks something like this:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
At this point, the socket which used to be an HTTP socket is now a webSocket and both endpoints have agreed that they're going to use the webSocket data format from now on. This initial connection may be followed by some form of authentication or new or existing cookies can also be used in the authentication during the initial HTTP portion of the connection.
socket.io adds some enhancements on top of this by initially requesting a particular path of /socket.io and adding some parameters to the URL. This allows socket.io to negotiate whether it's going to use long polling or a webSocket so there are some exchanges between client/server with socket.io before the above webSocket is initialized.
So, back to your question. The socket.io server simply spies at all incoming web requests on the normal web port (and looks for both it's special path and for special headers to indicate a webSocket initiation rather than a classic HTTP request). So, it runs over the same port as the web server. This is done for a bunch of reasons, all of which provide convenience to the server and server infrastructure since they don't have to configure their network to accept anything other than the usual port 80 they were already accepting (or whatever port they were already using for web requests).
By default in socket.io, the domain and port will default to the same domain and port as the web page you are on. So, if you don't specify one or the other in your connect call, it will use the domain or port from the web page you are on. If you want to use both a different domain and port, then you must specify both of them.
Our web application has a button that is supposed to send data to a server on the local network that in turn prints something on a printer.
So far it was easy: The button triggered an AJAX POST request to http://printerserver/print.php with a token, that page connected to the web application to verify the token and get the data to print and then printed.
However, we are now delivering our web application via HTTPs (and I would rather not go back to HTTP for this) and newer versions of Chrome and Firefox don't make the request to the HTTP address anymore, they don't even send the request to check CORS headers.
Now, what is a modern alternative to the cross-protocol XHR? Do Websockets suffer from the same problem? (A Google search did not make clear what is the current state here.) Can I use TCP Sockets already? I would rather not switch to GET requests either, because the action is not idempotent and it might have practical implications with preloading and caching.
I can change the application on the printerserver in any way (so I could replace it with NodeJS or something) but I cannot change the users' browsers (to trust a self-signed certificate for printerserver for example).
You could store the print requests on the webserver in a queue and make the printserver periodically poll for requests to print.
If that isn't possible I would setup a tunnel or VPN between the webserver and printserver networks. That way you can make the print request from the webserver on the server-side instead of the client. If you use curl, there are flags to ignore invalid SSL certificates etc. (I still suspect it's nicer to introduce a queue anyway, so the print requests aren't blocking).
If the webserver can make an ssh connection to something on the network where the printserver is on, you could do something like: ssh params user#host some curl command here.
Third option I can think of, if printserver can bind to for example a subdomain of the webserver domain, like: print.somedomain.com, you may be able to make it trusted by the somedomain.com certificate, IIRC you have to create a CSR (Certificate Signing Request) from the printserver certificate, and sign it with the somedomain.com certificate. Perhaps it doesn't even need to be a subdomain for this per se, but maybe that's a requirement for the browser to do it client-side.
The easiest way is to add a route to the webapp that does nothing more than relay the request to the print server. So make your AJAX POST request to https://myapp.com/print, and the server-side code powering that makes a request to http://printerserver/print.php, with the exact same POST content it received itself. As #dnozay said, this is commonly called a reverse proxy. Yes, to do that you'll have to reconfigure your printserver to accept (authenticated) requests from the webserver.
Alternatively, you could switch the printserver to https and directly call it from the client.
Note that an insecure (http) web-socket connection on a secure (https) page probably won't work either. And for good reason: generally it's a bad idea to mislead people by making insecure connections from what appears to them to be a secure page.
The server hosting the https webapp can reverse proxy the print server,
but since the printer is local to the user, this may not work.
The print server should have the correct CORS headers
Access-Control-Allow-Origin: *
or:
Access-Control-Allow-Origin: https://www.example.com
However there are pitfalls with using the wildcard.
From what I understand from the question, printserver is not accessible from the web application so the reverse proxy solution won't work here.
You are restricted from making requests from the browser to the printserver by cross-origin-policy.
If wish to communicate with the printserver from an HTTPS page you will need the printserver to expose print.php as HTTPS too.
You could create a DNS A record as a subdomain of your web application that resolves to the internal address of your printserver.
With those steps in place you should be able to update your printserver page to respond with permissive CORS headers which the browser should then respect. I don't think the browser will even issue CORS requests across different protocol schemes (HTTPS vs HTTP) or to internal domains, without a TLD.
I'm building a WebClient library. Now I'm implementing a proxy feature, so I am making some research and I saw some code using the CONNECT method to request a URL.
But checking it within my web browser, it doesn't use the CONNECT method but calls the GET method instead.
So I'm confused. When I should use both methods?
TL;DR a web client uses CONNECT only when it knows it talks to a proxy and the final URI begins with https://.
When a browser says:
CONNECT www.google.com:443 HTTP/1.1
it means:
Hi proxy, please open a raw TCP connection to google; any following
bytes I write, you just repeat over that connection without any
interpretation. Oh, and one more thing. Do that only if you talk to
Google directly, but if you use another proxy yourself, instead you
just tell them the same CONNECT.
Note how this says nothing about TLS (https). In fact CONNECT is orthogonal to TLS; you can have only one, you can have other, or you can have both of them.
That being said, the intent of CONNECT is to allow end-to-end encrypted TLS session, so the data is unreadable to a proxy (or a whole proxy chain). It works even if a proxy doesn't understand TLS at all, because CONNECT can be issued inside plain HTTP and requires from the proxy nothing more than copying raw bytes around.
But the connection to the first proxy can be TLS (https) although it means a double encryption of traffic between you and the first proxy.
Obviously, it makes no sense to CONNECT when talking directly to the final server. You just start talking TLS and then issue HTTP GET. The end servers normally disable CONNECT altogether.
To a proxy, CONNECT support adds security risks. Any data can be passed through CONNECT, even ssh hacking attempt to a server on 192.168.1.*, even SMTP sending spam. Outside world sees these attacks as regular TCP connections initiated by a proxy. They don't care what is the reason, they cannot check whether HTTP CONNECT is to blame. Hence it's up to proxies to secure themselves against misuse.
A CONNECT request urges your proxy to establish an HTTP tunnel to the remote end-point.
Usually is it used for SSL connections, though it can be used with HTTP as well (used for the purposes of proxy-chaining and tunneling)
CONNECT www.google.com:443
The above line opens a connection from your proxy to www.google.com on port 443.
After this, content that is sent by the client is forwarded by the proxy to www.google.com:443.
If a user tries to retrieve a page http://www.google.com, the proxy can send the exact same request and retrieve response for him, on his behalf.
With SSL(HTTPS), only the two remote end-points understand the requests, and the proxy cannot decipher them. Hence, all it does is open that tunnel using CONNECT, and lets the two end-points (webserver and client) talk to each other directly.
Proxy Chaining:
If you are chaining 2 proxy servers, this is the sequence of requests to be issued.
GET1 is the original GET request (HTTP URL)
CONNECT1 is the original CONNECT request (SSL/HTTPS URL or Another Proxy)
User Request ==CONNECT1==> (Your_Primary_Proxy ==CONNECT==> AnotherProxy-1 ... ==CONNECT==> AnotherProxy-n) ==GET1(IF is http)/CONNECT1(IF is https)==> Destination_URL
As a rule of thumb GET is used for plain HTTP and CONNECT for HTTPS
There are more details though so you probably want to read the relevant RFC-s
http://www.ietf.org/rfc/rfc2068.txt
http://www.ietf.org/rfc/rfc2817.txt
The CONNECT method converts the request connection to a transparent TCP/IP tunnel, usually to facilitate SSL-encrypted communication (HTTPS) through an unencrypted HTTP proxy.