Is the data secure if posted programmatically (not through a browser) to an https endpoint?
My understanding is browser encrypts data and sends it to https endpoint. How can a Ruby or Node.js or any other program do the same?
Yes. If you connect to an https endpoint with curl, wget, or whatever library, the transfer is secure from the source of the connection to the destination. That source could be a server (your webserver) or the client browser.
However, if it's done in client side JS or other browser scripting language, you have to make sure the initial request from client to your site is secure as well if first passing secure data to the client for it to pass to the destination https server.
I checked node.js request library as well as Ruby HTTParty libraries. Both these support SSL encryption based on proper options (port: 443 etc.). In general if we use any well supported library that enables HTTP gets and posts, we should be covered in terms of transmitting data securely to the https endpoint.
I think I understand what you mean and that question has been answered. However, I would just point out that HTTPS does not make your data secure, only the connection and even that is only encrypted from eavesdropping which is not really secure.
There is, of course, lots more to think about and do to make your data secure end-to-end.
Related
I have an https site that needs data from an API that is only available in http.
To get around the mixed content warning, I changed it so the JS requests a path on the server, which then makes the http request and returns the data.
Is this bad? If it is bad, why?
My understanding of what you're doing :
You are providing a HTTPS url on your server which is essentially acting as a proxy, making a backend connection between your server and the API provider over HTTP.
If my understanding of what you're doing is correct, then what you're doing is better than just serving everything over HTTP...
You are providing security between the client and your server. Most security threats that would take advantage of a plain HTTP connection are in the local environment of the client - such as on a shared local network. Dodgy wifi in a cafe. School lans. etc.
The connection between your server and the API provider is unencrypted but apparently they only provide that unencrypted anyway. This is really the best you can do unless your API provider starts providing an HTTPS interface.
It's more secure than doing nothing and should eliminate the browser errors.
If there is a real need for security (PCI compliance, HIPAA etc) however, you should stop using that API. However it seems unlikely considering the circumstantial evidence in your question.
Our web application has a button that is supposed to send data to a server on the local network that in turn prints something on a printer.
So far it was easy: The button triggered an AJAX POST request to http://printerserver/print.php with a token, that page connected to the web application to verify the token and get the data to print and then printed.
However, we are now delivering our web application via HTTPs (and I would rather not go back to HTTP for this) and newer versions of Chrome and Firefox don't make the request to the HTTP address anymore, they don't even send the request to check CORS headers.
Now, what is a modern alternative to the cross-protocol XHR? Do Websockets suffer from the same problem? (A Google search did not make clear what is the current state here.) Can I use TCP Sockets already? I would rather not switch to GET requests either, because the action is not idempotent and it might have practical implications with preloading and caching.
I can change the application on the printerserver in any way (so I could replace it with NodeJS or something) but I cannot change the users' browsers (to trust a self-signed certificate for printerserver for example).
You could store the print requests on the webserver in a queue and make the printserver periodically poll for requests to print.
If that isn't possible I would setup a tunnel or VPN between the webserver and printserver networks. That way you can make the print request from the webserver on the server-side instead of the client. If you use curl, there are flags to ignore invalid SSL certificates etc. (I still suspect it's nicer to introduce a queue anyway, so the print requests aren't blocking).
If the webserver can make an ssh connection to something on the network where the printserver is on, you could do something like: ssh params user#host some curl command here.
Third option I can think of, if printserver can bind to for example a subdomain of the webserver domain, like: print.somedomain.com, you may be able to make it trusted by the somedomain.com certificate, IIRC you have to create a CSR (Certificate Signing Request) from the printserver certificate, and sign it with the somedomain.com certificate. Perhaps it doesn't even need to be a subdomain for this per se, but maybe that's a requirement for the browser to do it client-side.
The easiest way is to add a route to the webapp that does nothing more than relay the request to the print server. So make your AJAX POST request to https://myapp.com/print, and the server-side code powering that makes a request to http://printerserver/print.php, with the exact same POST content it received itself. As #dnozay said, this is commonly called a reverse proxy. Yes, to do that you'll have to reconfigure your printserver to accept (authenticated) requests from the webserver.
Alternatively, you could switch the printserver to https and directly call it from the client.
Note that an insecure (http) web-socket connection on a secure (https) page probably won't work either. And for good reason: generally it's a bad idea to mislead people by making insecure connections from what appears to them to be a secure page.
The server hosting the https webapp can reverse proxy the print server,
but since the printer is local to the user, this may not work.
The print server should have the correct CORS headers
Access-Control-Allow-Origin: *
or:
Access-Control-Allow-Origin: https://www.example.com
However there are pitfalls with using the wildcard.
From what I understand from the question, printserver is not accessible from the web application so the reverse proxy solution won't work here.
You are restricted from making requests from the browser to the printserver by cross-origin-policy.
If wish to communicate with the printserver from an HTTPS page you will need the printserver to expose print.php as HTTPS too.
You could create a DNS A record as a subdomain of your web application that resolves to the internal address of your printserver.
With those steps in place you should be able to update your printserver page to respond with permissive CORS headers which the browser should then respect. I don't think the browser will even issue CORS requests across different protocol schemes (HTTPS vs HTTP) or to internal domains, without a TLD.
I've seen several examples of writing an HTTP proxy in Ruby, e.g. this gist by Torsten Becker, but how would I extend it to handle HTTPS, aka for a "man in the middle" SSL proxy?
I'm looking for a simple source code framework which I can extend for my own logging and testing needs.
update
I already use Charles, a nifty HTTPS proxy app similar to Fiddler and it is essentially what I want except that it's packaged up in an app. I want to write my own because I have specific needs for filtering and presentation.
update II
Having poked around, I understand the terminology a little better. I'm NOT after a full "Man in the Middle" SSL proxy. Instead, it will run locally on my machine and so I can honor whatever SSL cert it offers. However, I need to see the decrypted contents of packets of my requests and the decrypted contents of the responses.
Just for background information, a normal HTTP proxy handles HTTPS requests via the CONNECT method: it reads the host name and port, establishes a TCP connection to this target server on this port, returns 200 OK and then merely tunnels that TCP connection to the initial client (the fact that SSL/TLS is exchanged on top of that TCP connection is barely relevant).
This is what the do_CONNECT method if WEBrick::HTTPProxyServer.
If you want a MITM proxy, i.e. if you want to be able to look inside the SSL/TLS traffic, you can certainly use WEBrick::HTTPProxyServer, but you'll need to change do_CONNECT completely:
Firstly, your proxy server will need to embed a mini CA, capable of generating certificates on the fly (failing that, you might be able to use self-signed certificates, if you're willing to bypass warning messages in the browser). You would then import that CA certificate into the browser.
When you get the CONNECT request, you'll need to generate a certificate valid for that host name (preferable with a Suject Alt. Name for that host name, or in the Subject DN's Common Name), and upgrade the socket into an SSL/TLS server socket (using that certificate). If the browser accepts to trust that certificate, what you get from thereon on this SSL/TLS socket is the plain text traffic.
You would then have to handle the requests (get the request line, headers and entity) and take it to use it via a normal HTTPS client library. You might be able to send that traffic to a second instance of WEBrick::HTTPProxyServer, but it would have to be tweaked to make outgoing HTTPS requests instead of plain HTTP requests.
Webrick can proxy ssl:
require 'webrick'
require 'webrick/httpproxy'
WEBrick::HTTPProxyServer.new(:Port => 8080).start
from my experience HTTPS is nowhere near "simple". Do you need a proxy that would catch traffic from your own machine? There are several applications, like Fiddler. Or google for alternatives. Comes with everything you need to debug the web traffic.
That blog is no way to write a proxy. It's very easy: you just accept a connection, read one line which tells you what to connect to, attempt the upstream connection, if it fails send the appropriate response and close the socket, otherwise just start copying bytes in both directions, simultaneously, until EOS has occurred in both directions. The only difference HTTPS makes is that you have to speak SSL instead of plaintext.
I want a bit of clarity on whether HTTPS is stateful or stateless? This is with regards to a RESTful API I built. We were initially using HTTP. Since HTTP essentially works over TCP/IP which is stateless hence HTTP is stateless, but when I switched to HTTPS my API became stateful. I wanted to know whether my conclusion that HTTPS is stateful. is correct or not?
I created my API using a middleware tool called webMethods.
Thanks
TLS/SSL is stateful. The web server and the client (browser) cache the session including the cryptographic keys to improve performance and do not perform key exchange for every request.
HTTP 1 is not stateful. HTTP/2 however defines many stateful components, but the "application layer" still remains stateless.
TL;DR: The transport pipe (TLS) is stateful, original HTTP is not.
Additional note: Cookies and other stateful mechanisms are later additions defined in separate RFC's. They are not part of the original HTTP/1.0 specification, although other stateful mechanisms like caching and HTTP auth are defined HTTP 1.1 RFC and RFC 2617. HTTP 1 is said to be stateless although in practice we use standardized stateful mechanisms. HTTP/2 defines stateful components in its standard and is therefore stateful. A particular HTTP/2 application can use a subset of HTTP/2 features to maintain statelessness.
Theory aside, in practice you use HTTP statefully in your everyday life.
The S in HTTPS is concerned with the transport, not the protocol. The semantics of the HTTP protocol remain the same for HTTPS. As the article about HTTPS on Wikipedia states,
Strictly speaking, HTTPS is not a separate protocol, but refers to use of ordinary HTTP over an encrypted SSL/TLS connection.
And the HTTP protocol is stateless by design, not because it is used most frequently over TCP/IP (nothing stops you to use HTTP over UDP for example).
HTTPS is HTTP over a secure connection.
HTTP is a higher level than a connection.
When connecting to a web server, your connection is (maybe always?) of type TCP/IP. So, in case you are visiting a website via HTTPS, your TCP/IP connection is encrypted.
The data the server and/or client send has not been encrypted by the server and/or client. It is just sent, as it is usually via HTTP, but this time using a connection via TCP/IP that is secured via encryption.
If data were vehicles, and the connexion the highway, then:
- using HTTP would be like the vehicles going on the highway, and everyone can see them;
- using HTTPS would be like the same, but the vehicles go through a tunnel or anything that prevents people not on the highway from seeing them. You can determine there is trafic, but you cannot identify the vehicles, except on both ends of the tunnel.
I believe this is an image close to what happens behind the scene. But I'm no expert. I just hope it helps.
HTTP and HTTPS both are stateless protocols. The S in HTTPS stands for Secure and it refers to use of ordinary HTTP over an encrypted SSL/TLS connection.
Use of JWT tokens or the traditional way of establishing sessions using cookies help us to overcome the problem of HTTP being a stateless protocol, as it enables the server to authenticate the identity of the client, so that you don't need to login every time you click a link to navigate on the web-page.
So For example, when you log in to the website of your bank, it only asks you to enter your login details once. Once you are signed in, you don't need to re-enter them when you navigate to the account settings page, this is because the bank site is able to authenticate your identity using JWT tokens.
JWT tokens are only used on HTTPS and not in HTTP, because the connection is encrypted in HTTPS, so it cannot be intercepted by anyone.
Thus, HTTP and HTTPS both are stateless protocols, but JWT Tokens provides a workaround for it.
I believe HTTPS is a stateful protocol as it contains Session identifier field.This generated by server initially to identify a session with the chosen client.
This question tries to look into whether doing HTTPS log in is very important for any website.
Is it true that for many websites, if the login is done through HTTP but not HTTPS, then anybody can pretty much see the userID and password easily along the internet highway (or by looking between a router and the internet connection in an Internet Cafe)?
If so... do popular frameworks actually use HTTPS by default (or at least as an option), such as Rails 2.3.5 or Django, CakePHP, or .Net?
Yes, any machine on the pathway (that the packets pass through) can just examine the contents of the those packets. All it takes is a capturing proxy or a promiscuous mode network card with something like WireShark. Assuming that the passwords aren't encrypted in some other way (at a higher level), they will be visible.
I can't answer the second part of your question since I have no knowledge of those particular products but I would say that the inability to use secure sockets would pretty much make them useless.
Pax is right about passwords that aren't otherwise encrypted being visible.
Still, most sites don't use SSL still, and it does put the users at a certain degree of risk when accessing sites from public wifi.
HTTPS isn't a framework level option, it would be something you'd do when you set up the webserver. If you were to use an apache configuration for instance, you would open it up to a properly configured https, close http and install a certification. The framework wouldn't have a direct influence on that portion of the release.
If the user credentials are submitted via an HTML webform without HTTPS, then it is unsecure, the data is submitted in plain text. However, if the website uses HTTP authentication instead, then the server can send back a 401 reply (or 407 for proxies) to any request that does not provide valid credentials. 401/407 is the server's way to ask for credentials, and the reply provides a list of authentication schemes (Digest, NTLM, Negotiate, etc) that the server supports, which are usually more secure by themselves. The client/browser sends the same request again with the necessariy credentials in one of the schemes, then the server either sends the requested data, or sends another 401/407 reply if the credentials are rejected.