My application consists of two pieces: WebSocket server - which is hosted on OpenShift DIY cartridge; WebSocket client - which connects to my server from home PC. WebSocket server is written using embedded Jetty and its library for WebSockets. Client side is written using JAVA and Tyrus library. It works pretty well except for one glitch that I cannot explain.
When running WebSocket server on OpenShift DIY cartridge, WebSocket connection gets dropped every 2 min. Connection drops happen quite precisely so obviously it is not related to potential network outages. Besides I have tested exactly the same application on Heroku and there were no connection drop. Moreover onClose(...) method receives NORMAL_CLOSURE close code.
I am almost sure that OpenShift Apache layer closes idle WebSocket connections every 2 min. even though WebSocket client sends Ping messages and receives Pong messages from the server. Has anyone experienced this type of WebSocket connection drops? Are there are parameters I can use to prevent connection drops?
Thank you in advance.
Update: I added a dedicated thread on the server side to send Pong messages to the client (Jetty does not support Pong handlers yet so I cannot use Ping messages) and drops disappeared. It seems like OpenShift Apache layer started treating connection as "alive" and does not close it. Then I noticed one more strange behavior: someone ping my server side application via HTTPS every hour. HTTP headers look like this:
HTTP/1.1 HEAD /
Accept: /
User-Agent: Ruby
X-Forwarded-Proto: https
X-Forwarded-Host: ....rhcloud.com
Connection: keep-alive
X-Request-Start: t=1409771442217677
X-Forwarded-For: 10.158.21.225
Host: wsproxy-gimes4dieni.rhcloud.com
X-Forwarded-Port: 443
X-Client-IP: 10.158.21.225
X-Forwarded-SSL-Client-Cert: (null)
X-Forwarded-Server: localhost
I do not use Ruby, I am using only HTTP and IP address is different from my regular requests. Does anybody has a clue whether this is some sort of OpenShift "service" of this is coming from the Internet?
SSH into your project, open ~/haproxy/conf/haproxy.cfg with a text editor such as vi and edit timeout queue,timeout client, and timeout server to whatever you want. I set mine to 5m, which is 5 minutes. After you have made the changes, exit and run
~/haproxy/bin/control restart
Now your websocket timeout should be set.
Related
I want to make the client of proxy server keepAlive. Thus, I don't want the proxy client to make a tcp close handshake everytime.
Please look at this example in netty.
Adding the keepAlive option to this example doesn't seem to work properly. Because it makes a client and connect everytime the server get request and close the client when the response is arrived.
Then how can I make my proxy client keepAlive? Is there any reference/example for it?
Using SO_KEEPALIVE socket option doesn't mean that the server (or the other peer in the connection) should ignore an explicit request to close the connection. It helps in cases like
Idle sessions timing-out/getting killed by the other end due to non-activity
Idle or long-running requests being disconnected by a firewall in-between after a certain time passes (e.g. 1 hour, for resource clean-up purposes).
If the client's logic is not to re-use the same socket for different requests (i.e. if its application logic uses a new socket for each request), there's nothing you can do about that on your proxy.
The same argument is valid for the "back-end" side of your proxy as well. If the server you're proxying to doesn't allow the socket to be re-used, and explicitly closes a connection after a request is served, that wouldn't work as you wanted either.
If you are not closing the connection on your side then the proxy is. Different proxy servers will behave in different ways.
Try sending Connection: Keep-Alive as a header.
If that doesn't work, try also sending Proxy-Connection: Keep-Alive as a header.
We have a web server and a client, both written in go, that interact with each other. We want HAProxy to load balance requests between several instance of the server, but it's not working. The client will always connect to the same server while it's still up.
If I look at the output of "netstat -anp", I can see that there is a persistent connection that was established between the client and the sever through HAProxy. I tried setting the Connection Header in the response to "close", but that didn't work at all.
Needless to say, I'm completely confused by this. My first question is, is this a problem with the client, server, or HAProxy? How does one force the client to disconnect? Am I missing something regarding this? Curl works fine, so I know that HAProxy does load balance, but curl also completely shuts down when finished, hence why I'm suspecting it's the persistent connection that's causing me issues since the client and server are long running.
Just as an FYI, I'm using go-martini on the server.
Thanks.
HTTP/1.1 uses KeepAlive by default. Since the connections aren't closed, HAProxy can't balance the requests between different backends.
You have a couple options:
Force the connection to close after each request in your code. Setting Request.Close = true on either the client or the server will send a Connection: close header, telling both sides to close the tcp connection.
Alternatively you could have HAPoxy alter the requests by setting http-server-close so the backend is closed after each request, or http-closeto shutdown both sides after each request.
http-server-close is usually the best option, since that still maintains persistent connections for the client, while proxying each request individually.
According to the Wikipedia article: http://en.wikipedia.org/wiki/WebSocket,
The server sends back this response to the client during handshake:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=
Sec-WebSocket-Protocol: chat
Does this close the connection (as HTTP responses usually do) or it is kept open throughout the entire handshake and it can start sending WebSocket frames straight away (assuming that it succeeds)?
An HTTP socket going through the handshake process to be upgraded to the webSocket protocol is not closed during that process. The same open socket goes through the whole process and then becomes the socket used for the webSocket protocol. As soon as the upgrade is complete, that very socket is ready for messages to be sent per the webSocket protocol.
It is this use of the exact same socket that enables a webSocket connection to run on the same port as an HTTP request (no extra port is needed) because it literally starts out as an HTTP request (with some extra headers attached) and then when those headers are recognized and both sides agree, the socket from that original HTTP request on the original web port (often port 80) is then switched to use the webSocket protocol. No additional connection on some new port is needed.
I actually find it a relatively elegant design because it makes for easy coexistence with a web server which was an important design parameter. And, a slight extra bit of connection overhead (protocol upgrade negotiation) is generally not an issue because webSocket connections by their very nature are designed to be long running sockets which you open once and use over an extended period of time so a little extra overhead to open them doesn't generally bother their use.
If, for any reason, the upgrade is not completed (both sides don't agree on the upgrade to webSocket), then the socket would remain an HTTP socket and would behave as HTTP sockets normally do (likely getting closed right away, but subject to normal HTTP interactions).
You can see this answer for more details on the back and forth during an upgrade to webSocket: SocketIO tries to connect using same port as the browser used to get web page
I am serving content locally, accessible through http://0.0.0.0:4000. That works ok, I get a correct webpage, which contains the following line inside a script:
var socket = io('http://example.com');
i.e. I am referencing an external server. Now my browser shows the followoing error:
GET http://example.com:4000/socket.io/?EIO=3&transport=polling&t=1417447089410-1 net::ERR_CONNECTION_REFUSED
That is, the browser is trying to connect using the same port that it used to get the original page.
Everything works fine when both the SocketIO server and the web server listen on the same port.
Am I missing something? Is this a bug? Is there a workaround? Thank you.
You can read here about how a plain webSocket is initially set up. It all starts with a somewhat standard HTTP GET request, but one that has some special headers set:
GET /chat HTTP/1.1
Host: example.com:8000
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
The interchange may also allow the host to enforce requests only from web pages on certain origins. While this header can be spoofed from non-web-browser agents (so the server has to be prepared for that), it will likely be correct when the OP is using a real browser (assuming no proxy is modifying it).
If the server accepts the incoming request, it will then return an HTTP response that looks something like this:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
At this point, the socket which used to be an HTTP socket is now a webSocket and both endpoints have agreed that they're going to use the webSocket data format from now on. This initial connection may be followed by some form of authentication or new or existing cookies can also be used in the authentication during the initial HTTP portion of the connection.
socket.io adds some enhancements on top of this by initially requesting a particular path of /socket.io and adding some parameters to the URL. This allows socket.io to negotiate whether it's going to use long polling or a webSocket so there are some exchanges between client/server with socket.io before the above webSocket is initialized.
So, back to your question. The socket.io server simply spies at all incoming web requests on the normal web port (and looks for both it's special path and for special headers to indicate a webSocket initiation rather than a classic HTTP request). So, it runs over the same port as the web server. This is done for a bunch of reasons, all of which provide convenience to the server and server infrastructure since they don't have to configure their network to accept anything other than the usual port 80 they were already accepting (or whatever port they were already using for web requests).
By default in socket.io, the domain and port will default to the same domain and port as the web page you are on. So, if you don't specify one or the other in your connect call, it will use the domain or port from the web page you are on. If you want to use both a different domain and port, then you must specify both of them.
Case:
A WebSocket connection have been established between the client and server endpoint.
Now I have the network connection go down (for example the ADSL dies), after 10 min I recover the network, I find that the client and server are still able to communicate with each
other. Why?
Note:
The client was developed with Java-WebSocket framework, and the client did with ws4py.
1 - If they did not try to exchange any data and only the connection (not the endpoints) between them is down, this is normal behaviour.
2 - If the websocket connection ended, Browser may have re-established it without you knowing about it. I just checked that this is not normal behaviour. But maybe there is some parameter somewhere :-)