How to set up TLS connection for GKE Ingress - https

I want to enable https connections for my application in Google Cloud. I followed several tutorials, yet nothing seems to work.
I've deployed application on GKE using Ingress. Here is ingress.yml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: myapp-ip
spec:
backend:
serviceName: myapp-service
servicePort: 80
Accessing it with ip address works just fine, site is accessible from the internet.
I've purchased the domain with Google domains and configured DNS Zones by adding A record pointing to my external IP address and CNAME record. Now app is perfectly accessible with this domain.
Then I created SSL certificate and Kubernate secret by running commands:
openssl genrsa -out myapp-ingress-1.key 2048
openssl req -new -key myapp-ingress-1.key -out myapp-ingress-1.csr -subj "/CN=myapp.co.uk"
openssl x509 -req -days 365 -in myapp-ingress-1.csr -signkey myapp-ingress-1.key -out myapp-ingress-1.crt
kubectl create secret tls myapp-tls-secret-1 --cert myapp-ingress-1.crt --key myapp-ingress-1.key
And updated ingress.yml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: myapp-ip
spec:
tls:
- secretName: myapp-tls-secret-1
rules:
- host: myapp.co.uk
http:
paths:
- backend:
serviceName: myapp-service
servicePort: 80
I tried several ingress.yml files following different examples on internet, but none of them worked.
Can anyone advice me?

You are missing a line in your yaml file that specifies what host you are securing. Since you are not using external-dns you will have to create records on your domain provider side manually to connect the IP address to your domain (you seem to have already done this). Looks like both your ingress and your tls secret are in the default namespace, so you can use the following ingress.yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
kubernetes.io/tls-acme: "true"
kubernetes.io/ingress.global-static-ip-name: myapp-ip
spec:
tls:
- hosts:
- "myapp.co.uk"
secretName: myapp-tls-secret-1
rules:
- host: "myapp.co.uk"
http:
paths:
- path: /
backend:
serviceName: myapp-service
servicePort: 80

Related

Deploying Spring Boot with http/2 enabled, in kubernetes with ingress and namesheap certificate

I want to deploy kubernetes with http/2 enabled in kubernetes cluster with namesheap certificate, but i have always this error :
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 474554202f66617669636f6e2e69636f20485454502f312e310d0a486f73743a206261636b2d61646d696e2d68747470322e6361707461696e776f726b732e636f6d0d0a582d526571756573742d49443a2032393838326665623234633864333063346565373063363238623033666464390d0a582d5265616c2d49503a2035312e38332e3130362e3234380d0a582d466f727761726465642d466f723a2035312e38332e3130362e3234380d0a582d466f727761726465642d486f73743a206261636b2d61646d696e2d68747470322e6361707461696e776f726b732e636f6d0d0a582d466f727761726465642d506f72743a203434330d0a582d466f727761726465642d50726f746f3a2068747470730d0a582d466f727761726465642d536368656d653a2068747470730d0a582d536368656d653a2068747470730d0a7365632d63682d75613a2022476f6f676c65204368726f6d65223b763d223933222c2022204e6f743b41204272616e64223b763d223939222c20224368726f6d69756d223b763d223933220d0a7365632d63682d75612d6d6f62696c653a203f300d0a557365722d4167656e743a204d6f7a696c6c612f352e30202857696e646f7773204e542031302e303b2057696e36343b2078363429204170706c655765624b69742f3533372e333620284b48544d4c2c206c696b65204765636b6f29204368726f6d652f39332e302e343537372e3832205361666172692f3533372e33360d0a7365632d63682d75612d706c6174666f726d3a202257696e646f7773220d0a4163636570743a20696d6167652f617669662c696d6167652f776562702c696d6167652f61706e672c696d6167652f7376672b786d6c2c696d6167652f2a2c2a2f2a3b713d302e380d0a5365632d46657463682d536974653a2073616d652d6f726967696e0d0a5365632d46657463682d4d6f64653a206e6f2d636f72730d0a5365632d46657463682d446573743a20696d6167650d0a526566657265723a2068747470733a2f2f6261636b2d61646d696e2d68747470322e6361707461696e776f726b732e636f6d2f676574416c6c44696374696f6e6172790d0a4163636570742d456e636f64696e673a20677a69702c206465666c6174652c2062720d0a4163636570742d4c616e67756167653a2066722d46522c66723b713d302e390d0a0d0a
My configs are :
-application.properties :
server.port=8443
server.http2.enabled=true
server.ssl.enabled=true
server.ssl.key-store=classpath:keystore/cert.p12
server.ssl.key-store-type=PKCS12
server.ssl.key-store-password=password
-Dockerfile :
FROM openjdk:11.0.8-slim
VOLUME /tmp
ARG DEPENDENCY=target/dependency
COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY ${DEPENDENCY}/META-INF /app/META-INF
COPY ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","com.package.app"]
-ingress :
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
name: example
namespace: http2
spec:
rules:
- host: domain.com
http:
paths:
- backend:
serviceName: service-http2
servicePort: 8443
path: /
tls:
- hosts:
- domain.com
secretName: secret-tls
-cert : i have two files :
exemple.cert
exemple.ca-bundle
i used this command to convert my cert to .p12 :
OpenSSL pkcs12 -export -in cert.crt -inkey key.key -out cert.p12
-my.yaml file :
apiVersion: v1
kind: Service
metadata:
name: back-http2
namespace: http2
labels:
app: back-http2
spec:
type: ClusterIP
ports:
- name: http
protocol: TCP
port: 8080
- name: https
protocol: TCP
port: 8443
selector:
app: back-http2
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: back-http2-deployment
namespace: http2
labels:
app: back-http2
spec:
replicas: 1
selector:
matchLabels:
app: back-http2
template:
metadata:
labels:
app: back-http2
spec:
containers:
- name: back-dev
image: docker/registry:back-http2
imagePullPolicy: Always
ports:
- name: http
protocol: TCP
containerPort: 8080
- name: https
protocol: TCP
containerPort: 8443
imagePullSecrets:
- name: secret
-versions:
spring boot : 2.4.2
kubernetes : 1.20.2
nginx ingress controller deployed with helm : ingress-nginx-4.0.3
Any help would be greatly appreciated! Thank you!
You need to configure TLS passthrough in the nginx ingress definition
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
Make sure that the ingress itself is started supporting this flag, f.e.
args:
- --enable-ssl-passthrough
The reason behind this is that HTTP2 requires TLS by default.

Go API deployed on GKE throws SSL Error: Unable to verify the first certificate . Am I missing something?

Following is my Kubernetes configuration. The API deployed using this config works as expected when SSL verification is disabled by the client or when HTTP is used instead of HTTPS. But on enabling, it throws SSL Error: Unable to verify the first certificate. The SSL certificate files are added as Kubernetes secret and the API is exposed on port 8080.
---
apiVersion: "v1"
kind: "ConfigMap"
metadata:
name: "test-config"
namespace: "default"
labels:
app: "test"
data:
ENV: "DEV"
---
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
name: "test"
namespace: "default"
labels:
app: "test"
spec:
replicas: 1
selector:
matchLabels:
app: "test"
template:
metadata:
labels:
app: "test"
spec:
containers:
- name: "test"
image: "gcr.io/test-project/test:latest"
env:
- name: "ENV"
valueFrom:
configMapKeyRef:
key: "ENV"
name: "test-config"
---
apiVersion: "extensions/v1beta1"
kind: "Ingress"
metadata:
name: "test-ingress"
annotations:
kubernetes.io/ingress.global-static-ip-name: "test-static-ip"
labels:
app: "test"
spec:
tls:
- hosts:
- "test.myhost.com"
secretName: "test-ssl-certificate"
backend:
serviceName: "test-service-nodeport"
servicePort: 8080
rules:
- host: "test.myhost.com"
http:
paths:
- path: "/*"
backend:
serviceName: "test-service-nodeport"
servicePort: 8080
---
kind: "Service"
apiVersion: "v1"
metadata:
name: "test-service-nodeport"
spec:
selector:
app: "test"
ports:
- protocol: "TCP"
port: 8080
targetPort: 8080
type: "NodePort"
Go server code
http.HandleFunc("/hello", HelloServer)
err := http.ListenAndServeTLS(":8080", "server.crt", "server.key", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
A potential cause for this could be intermediate certificates were not installed on the server properly which caused a breakdown in the certificate chain. Even if it works in your browser, it may not be including all the public certificates in the chain needed for a cache-empty client to verify. Here are some preliminary troubleshooting steps:
Verify that your certificate chain is complete [https://certificatechain.io/ ]
Verify your server’s configuration [https://www.ssllabs.com/ssltest/ or https://www.sslshopper.com/ssl-checker.html ]
Look for this error:
This server's certificate chain is incomplete.
And this:
Chain issues.........Incomplete
If you encounter these issues, it means that the web server you are connecting to is misconfigured and did omit the intermediate certificate in the certificate chain it sent to you. Your server needs to serve not just the certificate for your domain, but also the intermediate certificates too.
Intermediate certificate should be installed on the server, along with the server certificate. Root certificates are embedded into the software applications, browsers and operating systems. The application serving the certificate has to send the complete chain, this means the server certificate itself and all the intermediates.
Refer stack post and combine the server certificate and intermediate certificate into a chained certificate for information.

Ingress rewrite rule in aks agic gives 502

I'm trying to create HTTPS ingress for my node.js authentication (auth) REST service in AKS, but I'm getting a 502 Bad Gateway response.
Here's my deployment and service definitions:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth
namespace: auth
labels:
app: auth
spec:
selector:
matchLabels:
app: auth
replicas: 1
template:
metadata:
labels:
app: auth
spec:
imagePullSecrets:
- name: docker-hub-creds
containers:
- name: auth
image: ***image***
ports:
- containerPort: 80
name: auth
---
apiVersion: v1
kind: Service
metadata:
name: auth
namespace: auth
spec:
selector:
app: auth
ports:
- protocol: TCP
port: 80
targetPort: 80
I think that's all pretty basic and it seems to work ok. I can see the service running and if I expose a node-port then I can access it with no problems. The service responds to well-formed POST requests on the /auth path with a JWT.
I have configured an Azure Application Gateway following Microsoft's instructions, and following the troubleshooting guide leads me to believe that the installation has worked ok. I have also checked through the web-ui and there appear to be no errors. Finally, I worked through the support options and the automated analysis of my cluster found no major configuration issues.
Next, I tried to create an HTTPS ingress route for my service, and this is where it goes wrong. This is made more complicated by the dynamic generation of certificates for TLS.
The ingress definition looks like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: auth-in
namespace: auth
annotations:
kubernetes.io/ingress.class: azure/application-gateway
cert-manager.io/cluster-issuer: letsencrypt-staging
cert-manager.io/acme-challenge-type: http01
ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
tls:
- hosts:
- ***hostname***
secretName: ***secret***
rules:
- host: ***same hostname***
http:
paths:
- backend:
serviceName: auth
servicePort: 80
path: /api/(auth/.*)
I have two rewrite-targets in there because I can't determine which one this ingress controller uses. All the example from the web use the nginx. prefix so I added it in desperation, despite thinking that it's probably not necessary.
Accessing the service through: ***hostname***/api/auth results in a Bad Gateway error.
I have checked through the portal and I can see the route is registered, listeners and rules are there, and my service is listed in the backend pools, but there is nothing in the 'rewrite' tabs. I expected to see something in the rewrite tabs.
I've tooled my service to log all access, and the logs show this, repeatedly:
{"level":30,"time":1611739355140,"pid":17,"hostname":"auth-6c7757bb89-d72td","msg":"Req-URL: /api/(auth/.*)"}
Describing the ingress gives me this:
Name: auth-in
Namespace: auth
Address: **redacted***
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
***redacted cert name** terminates **hostname***
Rules:
Host Path Backends
---- ---- --------
***hostname***
/api/(auth/.*) auth:80 10.0.0.69:80)
Annotations: cert-manager.io/acme-challenge-type: http01
cert-manager.io/cluster-issuer: letsencrypt-staging
ingress.kubernetes.io/rewrite-target: /$1
kubernetes.io/ingress.class: azure/application-gateway
nginx.ingress.kubernetes.io/rewrite-target: /$1
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CreateCertificate 43m cert-manager Successfully created Certificate "***cert-name***"
Two things to note. 1st that the logs show that the URL isn't being rewritten -- it's being passed exactly as the path shows, including the regex part. 2nd, that the Default Backend entry in the ingress description shows an error. I'm not sure that the 2nd one matters, but the 1st is clearly wrong.
I am keen to discover how to diagnose the problem and then fix it.
Since you are using AGIC you can include Backend Path Prefix annotation appgw.ingress.kubernetes.io/backend-path-prefix: "/"
The Ingress will be like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: auth-in
namespace: auth
annotations:
kubernetes.io/ingress.class: azure/application-gateway
cert-manager.io/cluster-issuer: letsencrypt-staging
cert-manager.io/acme-challenge-type: http01
appgw.ingress.kubernetes.io/backend-path-prefix: "/"
spec:
tls:
- hosts:
- ***hostname***
secretName: ***secret***
rules:
- host: ***same hostname***
http:
paths:
- backend:
serviceName: auth
servicePort: 80
path: /api/auth/*
AGIC on Nov 12 '21 has also included a rewrite-rule-set as part of this PR. For rewrite-rule, you can use the rewrite-rule annotation.
appgw.ingress.kubernetes.io/rewrite-rule-set: <rewrite rule set>

Istio Gateway Fail To Connect Via HTTPS

Deployments in a GKE cluster with Istio is working correctly via HTTP. But when i tried to secure it with cert-manager with following resources, HTTPS request fails state like so on curl
`Immediate connect fail for 64:ff9b::2247:fd8a: Network is unreachable
* connect to 34.71.253.138 port 443 failed: Connection refused`.
What should i do to make it work with HTTPS as well.
ClusterIssuer with following configuration
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
namespace: istio-system
spec:
acme:
# The ACME server URL
server: https://acme-staging-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: iprocureservers#iprocu.re
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-staging
solvers:
# ACME DNS-01 provider configurations
- dns01:
# Google Cloud DNS
clouddns:
# Secret from the google service account key
serviceAccountSecretRef:
name: cert-manager-credentials
key: gcp-dns-admin.json
# The project in which to update the DNS zone
project: iprocure-server
Certificate configuration like so, which made a certifiate in a Ready:True state
apiVersion: cert-manager.io/v1alpha3
kind: Certificate
metadata:
name: letsencrypt-staging
namespace: istio-system
spec:
secretName: letsencrypt-staging
commonName: "*.iprocure.tk"
dnsNames:
- '*.iprocure.tk'
issuerRef:
name: letsencrypt-staging
kind: ClusterIssuer
And lastly a Gateway
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: iprocure-gateway
namespace: default
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
tls:
httpsRedirect: false
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- "*"
tls:
mode: SIMPLE
credentialName: letsencrypt-staging
If i do, kubectl describe certificate -n istio-system
Name: letsencrypt-staging
Namespace: istio-system
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1
Kind: Certificate
Metadata:
Creation Timestamp: 2020-10-13T13:32:37Z
Generation: 1
Resource Version: 28030994
Self Link: /apis/cert-manager.io/v1/namespaces/istio-system/certificates/letsencrypt-staging
UID: ad838d28-5349-4aaa-a618-cc3bfc316e6e
Spec:
Common Name: *.iprocure.tk
Dns Names:
*.iprocure.tk
Issuer Ref:
Kind: ClusterIssuer
Name: letsencrypt-staging-clusterissuer
Secret Name: letsencrypt-staging-cert-secret
Status:
Conditions:
Last Transition Time: 2020-10-13T13:35:05Z
Message: Certificate is up to date and has not expired
Reason: Ready
Status: True
Type: Ready
Not After: 2021-01-11T12:35:05Z
Not Before: 2020-10-13T12:35:05Z
Renewal Time: 2020-12-12T12:35:05Z
Revision: 1
Events: <none>
Running kubectl get certificates -o wide -n istio-system, yields
NAME READY SECRET ISSUER STATUS AGE
letsencrypt-staging True letsencrypt-staging-cert-secret letsencrypt-staging-clusterissuer Certificate is up to date and has not expired 17h
Issue
I assume that https wasn't working because of the requirements which have to be enabled if you want to use cert-menager with istio in older versions.
Solution
As #Yunus Einsteinium mentioned in comments
Thank you for guiding me in the right direction. Using the OSS Istio, not the GKE one, is the way to go! I managed to make HTTPS work!
So the solution here was to use OOS istio installed with istioctl instead of the older istio gke addon.

Kubernetes with rewrite-target and kube-lego

I am trying to create redirect rule to GC buckets with my own certs. I have such configuration:
kind: Service
apiVersion: v1
metadata:
name: proxy-to-gcs
spec:
ports:
- protocol: TCP
port: 80
targetPort: 80
type: ExternalName
externalName: storage.googleapis.com
----
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: proxy-to-gcs
annotations:
kubernetes.io/tls-acme: "true"
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/rewrite-target: bucket_name/public
kubernetes.io/ingress.class: nginx
spec:
tls:
- hosts:
- www.example.com
secretName: secret-name-tls
rules:
- host: www.example.com
http:
paths:
- path: /
backend:
serviceName: proxy-to-gcs
servicePort: 80
When I want to see www.example.com/.well-known/acme-challenge/ as kube-lego endpoint, I see google storage bucket 404 page. There is a problem in that rewrite-target, which doesn't consider existence of kube-lego. Any suggestions? Thanks.
If you want just to host a static website from a bucket, you can use the official doc as a how-to
For Ingress, you can use HTTP(S) Load Balancer - internal google cloud loadbalancer.
You can route your traffic from 2 URL to one bucket and have HTTPS on both.

Resources