When you GET
https://encrypted.google.com/search?q=%s
Is the %s query encrypted? Or just the response?
If it is not, why should Google serve its public content also with encryption?
The entire request is encrypted, including the URL, and even the command (GET). The only thing an intervening party such as a proxy server can glean is the destination address and port.
Note, however, that the Client Hello packet of a TLS handshake can advertise the fully qualified domain name in plaintext via the SNI extension (thanks #hafichuk), which is used by all modern mainstream browsers, though some only on newer OSes.
EDIT: (Since this just got me a "Good Answer" badge, I guess I should answer the entire question…)
The entire response is also encrypted; proxies cannot intercept any part of it.
Google serves searches and other content over https because not all of it is public, and you might also want to hide some of the public content from a MITM. In any event, it's best to let Google answer for themselves.
The URL itself is encrypted, so the parameters in the query string do not travel in plain across the wire.
However, keep in mind that URLs including the GET data are often logged by the webserver, whereas POST data seldom is. So if you're planning to do something like /login/?username=john&password=doe, then don't; use a POST instead.
HTTPS Establishes an underlying SSL conenction before any HTTP data is
transferred. This ensures that all URL data (with the exception of
hostname, which is used to establish the connection) is carried solely
within this encrypted connection and is protected from
man-in-the-middle attacks in the same way that any HTTPS data is.
The above is a part of a VERY comprehensive answer from Google Answers located here:
http://answers.google.com/answers/threadview/id/758002.html#answer
The portion of the URL after the host name is sent securely.
For example,
https://somewhere.com/index.php?NAME=FIELD
The /index.php?NAME=FIELD part is encrypted. The somewhere.com is not.
Everything is encrypted, but you need to remember, that your query will stay in server's logs and will be accessible to various log analysers etc (which is usually not the case with POST request).
The connection gets encrypted before the request is transmitted. So yes, the request is encrypted as well, including the query string.
I just connected via HTTPS to a website and passed a bunch of GET parameters. I then used wireshark to sniff the network. Using HTTP, the URL is sent unencrypted, which means I can easily see all the GET parameters in the URL. Using HTTPS, everything is encrypted and I can't even see which packet is the GET command, let alone its contents!
Yes, it is secure. SSL encrypts everything.
Excerpt from POST request:
POST /foo HTTP/1.1
... some other headers
Excerpt from GET request:
GET /foo?a=b HTTP/1.1
... some other headers
In both cases whatever is sent on the socket is encrypted. The fact that the client sees parameters in his browser during a GET request doesn't mean that a man in the middle would see the same.
The SSL takes place before the header parsing, this means:
Client creates Request
Request gets encrypted
Encrypted request gets transmitted to the Server
Server decrypts the Request
Request gets parsed
A Request looks something like this (can't remember the exact syntax, but this should be close enough):
GET /search?q=qwerty HTTP/1.1
Host: www.google.de
This is also why having different SSL Certificates for several hosts on the same IP are problematic, the requested Hostname is not known until decryption.
The GET request is encrypted when using HTTPS - in fact this is why secured websites need to have a unique IP address - there's no way to get the intended hostname (or virtual directory) from the request until after it's been decrypted.
There is a little confusion above:
despite what is said, it is NOT required, anymore, to have unique IP address since the SNI field tells the server which cert to use
everything is encrypted EXCEPT the SNI request which is VERY early. This tells the server which Name and therefore which certificate to use. That is, the Server Name Indicator IS sent unencrypted so that the server and the client can establish and then use a secure connection for everything else.
So, in answer to the original question. Everything EXCEPT the host name (and I guess port) is secured in BOTH directions.
Related
Our web application has a button that is supposed to send data to a server on the local network that in turn prints something on a printer.
So far it was easy: The button triggered an AJAX POST request to http://printerserver/print.php with a token, that page connected to the web application to verify the token and get the data to print and then printed.
However, we are now delivering our web application via HTTPs (and I would rather not go back to HTTP for this) and newer versions of Chrome and Firefox don't make the request to the HTTP address anymore, they don't even send the request to check CORS headers.
Now, what is a modern alternative to the cross-protocol XHR? Do Websockets suffer from the same problem? (A Google search did not make clear what is the current state here.) Can I use TCP Sockets already? I would rather not switch to GET requests either, because the action is not idempotent and it might have practical implications with preloading and caching.
I can change the application on the printerserver in any way (so I could replace it with NodeJS or something) but I cannot change the users' browsers (to trust a self-signed certificate for printerserver for example).
You could store the print requests on the webserver in a queue and make the printserver periodically poll for requests to print.
If that isn't possible I would setup a tunnel or VPN between the webserver and printserver networks. That way you can make the print request from the webserver on the server-side instead of the client. If you use curl, there are flags to ignore invalid SSL certificates etc. (I still suspect it's nicer to introduce a queue anyway, so the print requests aren't blocking).
If the webserver can make an ssh connection to something on the network where the printserver is on, you could do something like: ssh params user#host some curl command here.
Third option I can think of, if printserver can bind to for example a subdomain of the webserver domain, like: print.somedomain.com, you may be able to make it trusted by the somedomain.com certificate, IIRC you have to create a CSR (Certificate Signing Request) from the printserver certificate, and sign it with the somedomain.com certificate. Perhaps it doesn't even need to be a subdomain for this per se, but maybe that's a requirement for the browser to do it client-side.
The easiest way is to add a route to the webapp that does nothing more than relay the request to the print server. So make your AJAX POST request to https://myapp.com/print, and the server-side code powering that makes a request to http://printerserver/print.php, with the exact same POST content it received itself. As #dnozay said, this is commonly called a reverse proxy. Yes, to do that you'll have to reconfigure your printserver to accept (authenticated) requests from the webserver.
Alternatively, you could switch the printserver to https and directly call it from the client.
Note that an insecure (http) web-socket connection on a secure (https) page probably won't work either. And for good reason: generally it's a bad idea to mislead people by making insecure connections from what appears to them to be a secure page.
The server hosting the https webapp can reverse proxy the print server,
but since the printer is local to the user, this may not work.
The print server should have the correct CORS headers
Access-Control-Allow-Origin: *
or:
Access-Control-Allow-Origin: https://www.example.com
However there are pitfalls with using the wildcard.
From what I understand from the question, printserver is not accessible from the web application so the reverse proxy solution won't work here.
You are restricted from making requests from the browser to the printserver by cross-origin-policy.
If wish to communicate with the printserver from an HTTPS page you will need the printserver to expose print.php as HTTPS too.
You could create a DNS A record as a subdomain of your web application that resolves to the internal address of your printserver.
With those steps in place you should be able to update your printserver page to respond with permissive CORS headers which the browser should then respect. I don't think the browser will even issue CORS requests across different protocol schemes (HTTPS vs HTTP) or to internal domains, without a TLD.
I've seen several examples of writing an HTTP proxy in Ruby, e.g. this gist by Torsten Becker, but how would I extend it to handle HTTPS, aka for a "man in the middle" SSL proxy?
I'm looking for a simple source code framework which I can extend for my own logging and testing needs.
update
I already use Charles, a nifty HTTPS proxy app similar to Fiddler and it is essentially what I want except that it's packaged up in an app. I want to write my own because I have specific needs for filtering and presentation.
update II
Having poked around, I understand the terminology a little better. I'm NOT after a full "Man in the Middle" SSL proxy. Instead, it will run locally on my machine and so I can honor whatever SSL cert it offers. However, I need to see the decrypted contents of packets of my requests and the decrypted contents of the responses.
Just for background information, a normal HTTP proxy handles HTTPS requests via the CONNECT method: it reads the host name and port, establishes a TCP connection to this target server on this port, returns 200 OK and then merely tunnels that TCP connection to the initial client (the fact that SSL/TLS is exchanged on top of that TCP connection is barely relevant).
This is what the do_CONNECT method if WEBrick::HTTPProxyServer.
If you want a MITM proxy, i.e. if you want to be able to look inside the SSL/TLS traffic, you can certainly use WEBrick::HTTPProxyServer, but you'll need to change do_CONNECT completely:
Firstly, your proxy server will need to embed a mini CA, capable of generating certificates on the fly (failing that, you might be able to use self-signed certificates, if you're willing to bypass warning messages in the browser). You would then import that CA certificate into the browser.
When you get the CONNECT request, you'll need to generate a certificate valid for that host name (preferable with a Suject Alt. Name for that host name, or in the Subject DN's Common Name), and upgrade the socket into an SSL/TLS server socket (using that certificate). If the browser accepts to trust that certificate, what you get from thereon on this SSL/TLS socket is the plain text traffic.
You would then have to handle the requests (get the request line, headers and entity) and take it to use it via a normal HTTPS client library. You might be able to send that traffic to a second instance of WEBrick::HTTPProxyServer, but it would have to be tweaked to make outgoing HTTPS requests instead of plain HTTP requests.
Webrick can proxy ssl:
require 'webrick'
require 'webrick/httpproxy'
WEBrick::HTTPProxyServer.new(:Port => 8080).start
from my experience HTTPS is nowhere near "simple". Do you need a proxy that would catch traffic from your own machine? There are several applications, like Fiddler. Or google for alternatives. Comes with everything you need to debug the web traffic.
That blog is no way to write a proxy. It's very easy: you just accept a connection, read one line which tells you what to connect to, attempt the upstream connection, if it fails send the appropriate response and close the socket, otherwise just start copying bytes in both directions, simultaneously, until EOS has occurred in both directions. The only difference HTTPS makes is that you have to speak SSL instead of plaintext.
Is the data secure if posted programmatically (not through a browser) to an https endpoint?
My understanding is browser encrypts data and sends it to https endpoint. How can a Ruby or Node.js or any other program do the same?
Yes. If you connect to an https endpoint with curl, wget, or whatever library, the transfer is secure from the source of the connection to the destination. That source could be a server (your webserver) or the client browser.
However, if it's done in client side JS or other browser scripting language, you have to make sure the initial request from client to your site is secure as well if first passing secure data to the client for it to pass to the destination https server.
I checked node.js request library as well as Ruby HTTParty libraries. Both these support SSL encryption based on proper options (port: 443 etc.). In general if we use any well supported library that enables HTTP gets and posts, we should be covered in terms of transmitting data securely to the https endpoint.
I think I understand what you mean and that question has been answered. However, I would just point out that HTTPS does not make your data secure, only the connection and even that is only encrypted from eavesdropping which is not really secure.
There is, of course, lots more to think about and do to make your data secure end-to-end.
This question tries to look into whether doing HTTPS log in is very important for any website.
Is it true that for many websites, if the login is done through HTTP but not HTTPS, then anybody can pretty much see the userID and password easily along the internet highway (or by looking between a router and the internet connection in an Internet Cafe)?
If so... do popular frameworks actually use HTTPS by default (or at least as an option), such as Rails 2.3.5 or Django, CakePHP, or .Net?
Yes, any machine on the pathway (that the packets pass through) can just examine the contents of the those packets. All it takes is a capturing proxy or a promiscuous mode network card with something like WireShark. Assuming that the passwords aren't encrypted in some other way (at a higher level), they will be visible.
I can't answer the second part of your question since I have no knowledge of those particular products but I would say that the inability to use secure sockets would pretty much make them useless.
Pax is right about passwords that aren't otherwise encrypted being visible.
Still, most sites don't use SSL still, and it does put the users at a certain degree of risk when accessing sites from public wifi.
HTTPS isn't a framework level option, it would be something you'd do when you set up the webserver. If you were to use an apache configuration for instance, you would open it up to a properly configured https, close http and install a certification. The framework wouldn't have a direct influence on that portion of the release.
If the user credentials are submitted via an HTML webform without HTTPS, then it is unsecure, the data is submitted in plain text. However, if the website uses HTTP authentication instead, then the server can send back a 401 reply (or 407 for proxies) to any request that does not provide valid credentials. 401/407 is the server's way to ask for credentials, and the reply provides a list of authentication schemes (Digest, NTLM, Negotiate, etc) that the server supports, which are usually more secure by themselves. The client/browser sends the same request again with the necessariy credentials in one of the schemes, then the server either sends the requested data, or sends another 401/407 reply if the credentials are rejected.
I am writing a little app similar to omegle. I have a http server written in Java and a client which is a html document. The main way of communication is by http requests (long polling).
I've implemented some sort of security by using the https protocol and I have a securityid for every client that connects to the server. When the client connects, the server gives it a securityid which the client must always send back when it wants a request.
I am afraid of the man in the middle attack here, do you have any suggestions how I could protect the app from such an attack.
Note that this app is build for theoretical purposes, it won't be ever used for practical reasons so your solutions don't have to be necessarily practical.
HTTPS does not only do encryption, but also authentication of the server. When a client connects, the server shows it has a valid and trustable certificate for its domain. This certificate can not simply be spoofed or replayed by a man-in-the-middle.
Simply enabling HTTPS is not good enough because the web brings too many complications.
For one thing, make sure you set the secure flag on the cookies, or else they can be stolen.
It's also a good idea to ensure users only access the site via typing https://<yourdomain> in the address bar, this is the only way to ensure an HTTPS session is made with a valid certificate. When you type https://<yourdomain>, the browser will refuse to let you on the site unless the server provides a valid certificate for <yourdomain>.
If you just type <yourdomain> without https:// in front, the browser wont care what happens. This has two implications I can think of off the top of my head:
The attacker redirects to some unicode domain with a similar name (ie: looks the same but has a different binary string and is thus a different domain) and then the attacker provides a valid certificate for that domain (since he owns it), the user probably wouldn't notice this...
The attacker could emulate the server but without HTTPS, he would make his own secured connection to the real server and become a cleartext proxy between you and the server, he can now capture all your traffic and do anything he wants because he owns your session.