GKE Ingress dropping websocket connections when using https - websocket

Question

I have an Ingress (the default GKE one) which is handling all the SSL before my services. One of my services is a WebSocket service (python autobahn). When I am exposing the service using LoadBalancer and not passing throw the ingress, using ws:// everything us working good. When instead I am exposing it using NodePort and passing through the ingress I am constantly seeing connections that are dropping even when no client is connecting. Here are the autobahnlogs:
WARNING:autobahn.asyncio.websocket.WebSocketServerProtocol:dropping connection to peer tcp:10.156.0.58:36868 with abort=False: None
When I connect using a client with wss:// the connection is successful but
a disconnection happens every few seconds (could not get a consistent number).
Although I do not think it is related I changed the timeout of the related backend service in GCE to 3600 sec and also tried to give it session affinity using both clientIP and cookie but none seems to stop the dropping connections.
Here is my ingress definition:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: {{ .Values.ingressName }}-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: {{ .Values.staticIpName }}-static-ip
labels:
oriient-app: "rest-api"
oriient-system: "IPS"
spec:
tls:
- secretName: sslcerts
rules:
- host: {{ .Values.restApiHost }}
http:
paths:
- backend:
serviceName: rest-api-internal-service
servicePort: 80
- host: {{ .Values.dashboardHost }}
http:
paths:
- backend:
serviceName: dashboard-internal-service
servicePort: 80
- host: {{ .Values.monitorHost }}
http:
paths:
- backend:
serviceName: monitor-internal-service
servicePort: 80
- host: {{ .Values.ipsHost }}
http:
paths:
- backend:
serviceName: server-internal-ws-service
servicePort: 80
The ws service is the "server-internal-ws-service".
Any suggestions?

Answer 1

I did not solve the issue, but I did walk around it by exposing my wss with a LoadBalancer service and I implemented the secure layer of WebSocket by myself.
I saved the certificate (the private key and full chain public key - pem format) as a secret and mounted it as volume and then in the python used the SSLContex to and passed it to the asyncio loop create server.
For creating the certificate secret create a yaml:
apiVersion: v1
kind: Secret
type: tls
metadata:
name: sslcerts
data:
# this is base64 of your pem fullchain and private key
tls.crt: XXX
tls.key: YYY
and then
kubectl apply -f [path to the yaml above]
In your server deployment mount the secret:
apiVersion: apps/v1beta2
kind: Deployment
metadata:
labels:
...
name: server
spec:
replicas: {{ .Values.replicas }}
selector:
matchLabels:
...
template:
metadata:
labels:
...
spec:
volumes:
- name: wss-ssl-certificate
secret:
secretName: sslcerts
containers:
- image: ...
imagePullPolicy: Always
name: server
volumeMounts:
- name: wss-ssl-certificate
mountPath: /etc/wss
And in the python code:
sslcontext = ssl.SSLContext()
sslcontext.load_cert_chain(/etc/wss/tls.crt, /etc/wss/tls.key)
wssIpsClientsFactory = WebSocketServerFactory()
...
loop = asyncio.get_event_loop()
coro = loop.create_server(wssIpsClientsFactory, '0.0.0.0', 9000, ssl=sslcontext)
server = loop.run_until_complete(coro)
Hope it helps someone