i would like to replace my node-http-proxy module with nginx proxy_pass module. Is it possible with new released nginx version, as i have read, that it supports HTTP/1.1 out of the box. I saw some threads struggeling with that problem, that websockets are not supported by nginx.
In my case im running several node projects in background and want to route my websocket connections from port 80 to 8000-8100, depending on domain. Is there a native way to do websocket proxy/reverse proxy without using the tcp_module addon?
I tried to setup an upstream in nginx.conf with proxy_passing to it, but if i try to connect to port 80 over websocket, i get an 502 Gateway error.
Anyone facing the same problem?
Does anyone have a working example for nginx + spcket.io, proxying over port 80?
No, this is not yet possible; nginx 1.2 incorporates stuff from the 1.1.x development branch which indeed includes HTTP/1.1 reverse proxying. Websocket connections are established using the HTTP/1.1 "Upgrade" header, but the fact that nginx now supports this kind of headers does not mean it supports websockets (websockets are a different protocol, not HTTP).
(I tried this myself using the 1.1.x branch (which I found to be stable enough for my purpose) and it doesn't work without the tcp_module)
Websockets will probably be supported in 1.3.x ( http://trac.nginx.org/nginx/roadmap ).
Your alternatives are:
keep using node-http-proxy
use nginx without tcp module; socket.io won't use websockets but something else (e.g. long polling)
nginx with tcp module: in this case I think you need an additional port for this module (never tried this myself)
put something else in front as a reverse proxy: I use HAProxy (which supports websockets) in front of nginx and node. Nginx now simply acts as a static fileserver, not a proxy. Varnish is another option, if you want additional caching.
In relation to NginX with TCP module there are few problems I have encountered.
But the most tricky one is trying to run your websockets with nginx on port 80 on EC2 instance.
I described whole configuration here
Related
We have two applications running on ibm cloud cloud foundry (appA and appB).
appA is accessing appB over a container-to-container networking while appB is also available externally over a Gorouter route.
The thing is that while it is http-8080 our app exposes - all is good.
Now we have to do container-to-container networking over https.
We configured the app to expose https-8080. 8080 is used as https://docs.cloudfoundry.org/devguide/custom-ports.html states that:
By default, apps only receive requests on port 8080 for both HTTP and TCP routing,
and so must be configured, or hardcoded, to listen on this port
container-to-container networking works as expected now using https.
But we are no longer able to use the appB over the external Gorouter route.
What is the best way to have it all up and running as we expect?
There isn't a good answer to this question, at least at the time I write this.
You do have a couple options though:
Manually set up HTTPS for the internal route. To do this, you would need to use the instructions for your application/server of choice to configure HTTPS. Then use whatever functionality your buildpack provides to inject this confirmation into the application container. This would also require you to bundle and push TLS certs with your application. The platform isn't going to provide you TLS certs if you take this option.
The trick to making both the internal and public route work is that you need your application to listen on both port 8080 and the port you choose for your HTTPS traffic. As long as you continue taking HTTP traffic on port 8080, then your public routes should keep working.
If you want a quick, but not ideal solution you can use port 61001. For newer versions of Cloud Foundry, this port is used by Envoy to accept traffic to your app over HTTPS. Envoy then proxies the request to your app via HTTP over port 8080. You can use this port for your container to container traffic as well, however the configured subject name on the TLS cert won't match your route.
Here's an example of what the subject name will look like.
subject: OU=organization:639f74aa-5d97-4a47-a6b3-e9c2613729d8 + OU=space:10180e2b-33b9-44ee-9f8f-da96da17ac1a + OU=app:10a4752e-be17-41f5-bfb2-d858d49165f2; CN=b7520259-6428-4a52-60d4-5f25
Because it's using this format, you would need to have your clients ignore certificate subject name match errors (not ideal as that weakens HTTPS), or perhaps create a custom hostname matcher.
For what it's worth, I don't think you want or need to change the port. That is typically used if your application is not flexible and unable to listen on port 8080. It changes the port for inbound traffic. Since you're only using C2C networking, you don't need that option.
What you want, from what I understand, is that you want HTTPS for C2C traffic. In that case, the public traffic doesn't matter. It can still go through Gorouter to port 8080. For your container-to-container traffic, you can pick any port you want. You just need to make sure the port you choose has network policy set to allow that traffic (by default all C2C traffic is blocked). Once the network policy is set, you can connect directly over whatever port you designate.
I want to achieve L7 load balancing of grpc traffic using HAProxy 1.8 . Since HAProxy only supports L4 routing for grpc traffic, can we read the grpc request as http2 request (as grpc runs overs http2) in HAproxy and route the request based on some custom header of the request.
Or is there any alternate way to achieve L7 load balancing for grpc traffic. I am aware of the envoyby lyft, but don't plan on using it.
Just a quick update on this, HAProxy supports L7 with H2 on both sides since version 1.9, and gRPC works since 1.9.2 (trailers support was required).
For HAProxy to work as a L7 balancer, HAProxy needs speak http/2 to backends. Last time when I asked about this, I was told that http/2 support isn't on the immediate road-map for HAProxy (but incoming http/2 support is)
Shall I use WebSocket on non-80 ports? Does it ruin the whole purpose of using existing web/HTTP infrastructures? And I think it no longer fits the name WebSocket on non-80 ports.
If I use WebSocket over other ports, why not just use TCP directly? Or is there any special benefits in the WebSocket protocol itself?
And since current WebSocket handshake is in the form of a HTTP UPGRADE request, does it mean I have to enable HTTP protocol on the port so that WebSocket handshake can be accomplished?
Shall I use WebSocket on non-80 ports? Does it ruin the whole purpose
of using existing web/HTTP infrastructures? And I think it no longer
fits the name WebSocket on non-80 ports.
You can run a webSocket server on any port that your host OS allows and that your client will be allowed to connect to.
However, there are a number of advantages to running it on port 80 (or 443).
Networking infrastructure is generally already deployed and open on port 80 for outbound connections from the places that clients live (like desktop computers, mobile devices, etc...) to the places that servers live (like data centers). So, new holes in the firewall or router configurations, etc... are usually not required in order to deploy a webSocket app on port 80. Configuration changes may be required to run on different ports. For example, many large corporate networks are very picky about what ports outbound connections can be made on and are configured only for certain standard and expected behaviors. Picking a non-standard port for a webSocket connection may not be allowed from some corporate networks. This is the BIG reason to use port 80 (maximum interoperability from private networks that have locked down configurations).
Many webSocket apps running from the browser wish to leverage existing security/login/auth infrastructure already being used on port 80 for the host web page. Using that exact same infrastructure to check authentication of a webSocket connection may be simpler if everything is on the same port.
Some server infrastructures for webSockets (such as socket.io in node.js) use a combined server infrastructure (single process, one listener) to support both HTTP requests and webSockets. This is simpler if both are on the same port.
If I use WebSocket over other ports, why not just use TCP directly? Or
is there any special benefits in the WebSocket protocol itself?
The webSocket protocol was originally defined to work from a browser to a server. There is no generic TCP access from a browser so if you want a persistent socket without custom browser add-ons, then a webSocket is what is offered. As compared to a plain TCP connection, the webSocket protocol offers the ability to leverage HTTP authentication and cookies, a standard way of doing app-level and end-to-end keep-alive ping/pong (TCP offers hop-level keep-alive, but not end-to-end), a built in framing protocol (you'd have to design your own packet formats in TCP) and a lot of libraries that support these higher level features. Basically, webSocket works at a higher level than TCP (using TCP under the covers) and offers more built-in features that most people find useful. For example, if using TCP, one of the first things you have to do is get or design a protocol (a means of expressing your data). This is already built-in with webSocket.
And since current WebSocket handshake is in the form of a HTTP UPGRADE
request, does it mean I have to enable HTTP protocol on the port so
that WebSocket handshake can be accomplished?
You MUST have an HTTP server running on the port that you wish to use webSocket on because all webSocket requests start with an HTTP request. It wouldn't have to be heavily featured HTTP server, but it does have to handle the initial HTTP request.
Yes - Use 443 (ie, the HTTPS port) instead.
There's little reason these days to use port 80 (HTTP) for anything other than a redirection to port 443 (HTTPS), as certification (via services like LetsEncrypt) are easy and free to set up.
The only possible exceptions to this rule are local development, and non-internet facing services.
Should I use a non-standard port?
I suspect this is the intent of your question. To this, I'd argue that doing so adds an unnecessary layer of complication with no obvious benefits. It doesn't add security, and it doesn't make anything easier.
But it does mean that specific firewall exceptions need to be made to host and connect to your websocket server. This means that people accessing your services from a corporate/school/locked down environment are probably not going to be able to use it, unless they can somehow convince management that it is mandatory. I doubt there are many good reasons to exclude your userbase in this way.
But there's nothing stopping you from doing it either...
In my opinion, yes you can. 80 is the default port, but you can change it to any as you like.
We're working on a Ruby on Rails app that needs to take advantage of html5 websockets. At the moment, we have two separate "servers" so to speak: our main app running on nginx+passenger, and a separate server using Pratik Naik's Cramp framework (which is running on Thin) to handle the websocket connections.
Ideally, when it comes time for deployment, we'd have the rails app running on nginx+passenger, and the websocket server would be proxied behind nginx, so we wouldn't need to have the websocket server running on a different port.
Problem is, in this setup it seems that nginx is closing the connections to Thin too early. The connection is successfully established to the Thin server, then immediately closed with a 200 response code. Our guess is that nginx doesn't realize that the client is trying to establish a long-running connection for websocket traffic.
Admittedly, I'm not all that savvy with nginx config, so, is it even possible to configure nginx to act as a reverse proxy for a websocket server? Or do I have to wait for nginx to offer support for the new websocket handshake stuff? Assuming that having both the app server and the websocket server listening on port 80 is a requirement, might that mean I have to have Thin running on a separate server without nginx in front for now?
Thanks in advance for any advice or suggestions. :)
-John
You can't use nginx for this currently[it's not true anymore], but I would suggest looking at HAProxy. I have used it for exactly this purpose.
The trick is to set long timeouts so that the socket connections are not closed. Something like:
timeout client 86400000 # In the frontend
timeout server 86400000 # In the backend
If you want to serve say a rails and cramp application on the same port you can use ACL rules to detect a websocket connection and use a different backend. So your haproxy frontend config would look something like
frontend all 0.0.0.0:80
timeout client 86400000
default_backend rails_backend
acl websocket hdr(Upgrade) -i WebSocket
use_backend cramp_backend if websocket
For completeness the backend would look like
backend cramp_backend
timeout server 86400000
server cramp1 localhost:8090 maxconn 200 check
How about use my nginx_tcp_proxy_module module?
This module is designed for general TCP proxy with Nginx. I think it's also suitable for websocket. And I just add tcp_ssl_module in the development branch.
nginx (>= 1.3.13) now supports reverse proxying websockets.
# the upstream server doesn't need a prefix!
# no need for wss:// or http:// because nginx will upgrade to http1.1 in the config below
upstream app_server {
server localhost:3000;
}
server {
# ...
location / {
proxy_pass http://app_server;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
}
}
Out of the box (i.e. official sources) Nginx can establish only HTTP 1.0 connections to an upstream (=backend), which means no keepalive is possibe: Nginx will select an upstream server, open connection to it, proxy, cache (if you want) and close the connection. That's it.
This is the fundamental reason frameworks requiring persistent connections to the backend would not work through Nginx (no HTTP/1.1 = no keepalive and no websockets I guess). Despite having this disadvantage there is an evident benefit: Nginx can choose out of several upstreams (load balance) and failover to alive one in case some of them failed.
Edit: Nginx supports HTTP 1.1 to backends & keepalive since version 1.1.4. "fastcgi" and "proxy" upstreams are supported. Here it is the docs
For anyone that wondering about the same problem, nginx now officially supports HTTP 1.1 upstream. See nginx documentation for "keepalive" and "proxy_http_version 1.1".
How about Nginx with the new HTTP Push module: http://pushmodule.slact.net/. It takes care of the connection juggling (so to speak) that one might have to worry about with a reverse proxy. It is certainly a viable alternative to Websockets which are not fully in the mix yet. I know developer of the HTTP Push module is still working on a fully stable version, but it is in active development. There are versions of it being used in production codebases. To quote the author, "A useful tool with a boring name."
I use nginx to reverse proxy to a comet style server with long polling connections and it works great. Make sure you configure proxy_send_timeout and proxy_read_timeout to appropriate values. Also make sure your back-end server that nginx is proxying to supports http 1.0 because I don't think nginx's proxy module does http 1.1 yet.
Just to clear up some confusion in a few of the answers: Keepalive allows a client to reuse a connection to send another HTTP request. It does not have anything to do with long polling or holding connections open until an event occurs which is what the original question was asking about. So it doesn't matter than nginx's proxy module only supports HTTP 1.0 which does not have keepalive.
I have an event-machine websocket application (using the em-websocket gem) and it runs fine. The problem is that I need to deploy it using port 80 through nginx (can't compile it with tcp proxy module). Is it possible to use a simple nginx proxy_pass pointing to a thin server and have the thin server to pass the requests to my websocket server?
From what I understand you can't proxy websocket traffic with a proxy_pass.
Since web sockets are done over HTTP 1.1 connections (where the handshake and upgrade are completed), your backend needs to support HTTP 1.1, and from what I have researched, they break the HTTP 1.0 spec...
I've seen some people try to do the same thing with socket.io and HAProxy (see links). I would guess that you could try to swap out the socket.io with em-websockets and expect similar results.
1: http://www.letseehere.com/reverse-proxy-web-sockets
2: HAProxy + WebSocket Disconnection