I installed NGINX Ingress in kubernetes cluster. When i am trying to access the micro service end via Ingress Controller its not working as expected
I have deployed two spring boot application
Ingress Rules
Path 1 -> /customer
Path 2 -> /prac
When i am trying to access one of the service ex.
http://test.practice.com/prac/practice/getprac , it does not work
but when i try to access without Ingress path http://test.practice.com/practice/getprac, it works
I am not able to understand why with Ingress path its not working and same happens for other service
Micro service 1 (Port 9090)
apiVersion: apps/v1
kind: Deployment
metadata:
name: customer
namespace: practice
labels:
app: customer
spec:
replicas: 5
selector:
matchLabels:
app: customer
template:
metadata:
labels:
app: customer
spec:
imagePullSecrets:
- name: testkuldeepsecret
containers:
- name: customer
image: kuldeep99/customer:v1
ports:
- containerPort: 9090
hostPort: 9090
---
apiVersion: v1
kind: Service
metadata:
name: customer-service
namespace: practice
labels:
spec:
ports:
- port: 9090
targetPort: 9090
protocol: TCP
name: http
selector:
app: customer
Micro service 2 (port 8000)
apiVersion: apps/v1
kind: Deployment
metadata:
name: prac
namespace: practice
labels:
app: prac
spec:
replicas: 4
selector:
matchLabels:
app: prac
template:
metadata:
labels:
app: prac
spec:
imagePullSecrets:
- name: testkuldeepsecret
containers:
- name: prac
image: kuldeep99/practice:v1
ports:
- containerPort: 8000
hostPort: 8000
---
apiVersion: v1
kind: Service
metadata:
name: prac-service
namespace: practice
labels:
spec:
ports:
- port: 8000
targetPort: 8000
protocol: TCP
name: http
selector:
app: prac
Service (customer-service and prac-service)
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
customer-service ClusterIP 10.97.203.19 <none> 9090/TCP 39m
ngtest ClusterIP 10.98.74.149 <none> 80/TCP 21h
prac-service ClusterIP 10.96.164.210 <none> 8000/TCP 15m
some-mysql ClusterIP None <none> 3306/TCP 2d16h
Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: practice-ingress
namespace: practice
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: practice.example.com
http:
paths:
- backend:
serviceName: customer-service
servicePort: 9090
path: /customer
- backend:
serviceName: prac-service
servicePort: 8000
path: /prac
You have installed this nginx ingress
nginx.ingress.kubernetes.io/rewrite-target: / annotation to work properly you need to install this nginx ingress.
Alternative way to solve this issue is to configure contextPath to /prac in the spring application
On top the discussion, i observed one thing. We should not confuse with
apiVersion: networking.k8s.io/v1
kind: Ingress
And
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
First ensure which Ingress controller we are using and based on that decide apiVersion. I'm using "ingress-nginx" (not "nginx-ingress"). This one supports "apiVersion: networking.k8s.io/v1beta1" and works charm as per "Arsene" comment.
This Ingress yaml file WORKS with "ingress-nginx" Ingress controller
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: k8-exercise-03-two-app-ingress
spec:
rules:
- host: ex03.k8.sb.two.app.ingress.com
http:
paths:
- backend:
serviceName: k8-excercise-01-app-service
servicePort: 8080
path: /one(/|$)(.*)
- backend:
serviceName: k8-exercise-03-ms-service
servicePort: 8081
path: /two(/|$)(.*)
But, this Ingress yaml file NOT WORKING with "ingress-nginx" Ingress controller
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: k8-exercise-03-two-app-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
# nginx.ingress.kubernetes.io/use-regex: "true"
ingress.kubernetes.io/rewrite-target: /$2
spec:
# ingressClassName: nginx
rules:
#192.168.1.5 ex03.k8.sb.com is mapped in host file. 192.168.1.5 is Host machine IP
- host: ex03.k8.sb.two.app.ingress.com
http:
paths:
- backend:
service:
name: k8-excercise-01-app-service
port:
number: 8080
path: /one(/|$)(.*)
pathType: Prefix
- pathType: Prefix
path: /two(/|$)(.*)
backend:
service:
name: k8-exercise-03-ms-service
port:
number: 8081
I can access the Spring Boot API Calls as like:
For App-1:
http://ex03.k8.sb.two.app.ingress.com/one/
Result: App One - Root
http://ex03.k8.sb.two.app.ingress.com/one/one
Result: App One - One API
http://ex03.k8.sb.two.app.ingress.com/one/api/v1/hello
Result: App One - Hello API
App-2:
http://ex03.k8.sb.two.app.ingress.com/two/message/James%20Bond
Result: App Two- Hi James Bond API
Finally If any one knows how to change "apiVersion: networking.k8s.io/v1" yaml to support "ingress-nginx" Controller, will be appreciate. Thank you. Sorry for long content
I spend literally a day with this problem. The problem was simply the wrong nginx installed. I used helm found here to install nginx-ingress
Install it, please use helm version 3:
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx
Once run, in the logs you shall see a snippet that illustrates how your ingress should look like. In case you want to do the above, you can the annotation suggested above and henceforth, you can follow tutorials here to achieve more such as rewrite.
My cluster is deployed on GCP using GKE
when done, this is the output log:
NAME: ingress-nginx
LAST DEPLOYED: Sat Apr 24 07:56:11 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w ingress-nginx-controller'
An example Ingress that makes use of the controller:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: example
namespace: foo
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
serviceName: exampleService
servicePort: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
This is how it looks like now after installing it:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: example
# namespace: foo
spec:
rules:
- host: [your ip address].sslip.io
http:
paths:
- backend:
serviceName: registry-app-server
servicePort: 8761
path: /eureka/(.*)
- backend:
serviceName: api-gateway-server
servicePort: 7000
path: /api(/|$)(.*)
As you can see I am deploying spring micro-services using kubernetes(gke).
There are a lot of benefits of using nginx-ingress over built-in gke ingress, and it is more popular than its counterparts
Related
If possible-
I have a question:
this is my kibana:
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
name: quickstart
spec:
version: 7.6.2
count: 1
elasticsearchRef:
name: cdbridgerpayelasticsearch
http:
service:
spec:
type: LoadBalancer
the kibana ran well (the LB too)
and this is my Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
cert-manager.io/cluster-issuer: "letsencrypt-http"
name: bcd-ingress-kibana-bcd
spec:
rules:
- host: kibana.some.net
http:
paths:
- backend:
serviceName: quickstart
servicePort: 5601
path: /
tls:
- hosts:
- kibana.some.net
when I ran: kubectl get ingress
I got:
$ kubectl get ingress
W0614 15:48:48.425600 1675 gcp.go:120] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.25+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
NAME CLASS HOSTS ADDRESS PORTS AGE
kibana-ingress-bcd kibana.some.net 80 5m24s
and when I tried browsing with this host-
the browsre didnt recognize this hostname.
If anyone knows what the problem is - it will help me a lot×¥
thanks
Frida
I am running Kubernetes cluster on my windows PC via Docker desktop. I am trying to create a very basic pod with a simple ingress configuration, but it doesn't seem to work. I thought the backend pod + service + ingress is a very basic setup, however I don't find a lot of help online. Please advise what I am doing wrong here.
My deployment.yaml file
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
service.yaml
apiVersion: v1
kind: Service
metadata:
name: test-cluster-ip
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 1234
targetPort: 80
ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /testpath
pathType: Exact
backend:
service:
name: test-cluster-ip
port:
number: 1234
This is what I see when I access localhost from the browser
Also, I would like to ask if it is uncommon to run Kubernetes on windows even for testing (especially with ingress). I don't seem to find a lot of examples in the internet.
I thought the backend pod + service + ingress is a very basic setup, however I don't find a lot of help online. Please advise what I am doing wrong here.
It is indeed a very basic setup. And your k8s deployment/service/ingress yaml files are correct.
First, check if you installed NGINX ingress controller. If not, run:
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml
After that, you will be able to reach the k8s cluster using the following URL:
http://kubernetes.docker.internal/
But deploying ingress like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /testpath
pathType: Exact
backend:
service:
name: test-cluster-ip
port:
number: 1234
you are configuring the ingress to rewrite /testpath to the /. And requesting url without /testpath will return 404 status code.
See more rewrite examples here.
So, if you use the following URL, you will get the Nginx webpage from k8s deployment.
http://kubernetes.docker.internal/testpath
I am trying to deploy a springboot application running on 8080 port. My target is to have https protocol for custom subdomain with google managed-certificates.
here are my yamls.
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-deployment
namespace: my-namespace
spec:
replicas: 1
selector:
matchLabels:
app: my-deployment
namespace: my-namespace
template:
metadata:
labels:
app: my-deployment
namespace: my-namespace
spec:
containers:
- name: app
image: gcr.io/PROJECT_ID/IMAGE:TAG
imagePullPolicy: Always
ports:
- containerPort: 8080
resources:
requests:
memory: "256Mi"
ephemeral-storage: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
ephemeral-storage: "512Mi"
cpu: "250m"
2.service.yaml
apiVersion: v1
kind: Service
metadata:
name: my-service
namespace: my-namespace
annotations:
cloud.google.com/backend-config: '{"default": "my-http-health-check"}'
spec:
selector:
app: my-deployment
namespace: my-namespace
type: NodePort
ports:
- port: 80
name: http
targetPort: http
protocol: TCP
ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
namespace: my-name-space
annotations:
kubernetes.io/ingress.global-static-ip-name: my-ip
networking.gke.io/managed-certificates: my-cert
kubernetes.io/ingress.class: "gce"
labels:
app: my-ingress
spec:
rules:
- host: my-domain.com
http:
paths:
- pathType: ImplementationSpecific
backend:
service:
name: my-service
port:
name: http
I followed various documentation, most of them could help to make http work but, couldn't make https work and ends with error ERR_SSL_VERSION_OR_CIPHER_MISMATCH. Looks like there is issue with "Global forwarding rule". Ports shows 443-443. What is the correct way to terminate the HTTPS traffic at loadbalancer and route it to backend app with http?
From the information provided, I can see that the "ManagedCertificate" object is missing, you need to create a yaml file with the following structure:
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
name: my-cert
spec:
domains:
- <your-domain-name1>
- <your-domain-name2>
And then apply it with the command: kubectl apply -f file-name.yaml
Provisioning of the Google-managed certificate can take up to 60 minutes; you can check the status of the certificate using the following command: kubectl describe managedcertificate my-cert, wait for the status to be as "Active".
A few prerequisites you need to be aware, though:
You must own the domain name. The domain name must be no longer than
63 characters. You can use Google Domains or another registrar.
The cluster must have the HttpLoadBalancing add-on enabled.
Your "kubernetes.io/ingress.class" must be "gce".
You must apply Ingress and ManagedCertificate resources in the same
project and namespace.
Create a reserved (static) external IP address. Reserving a static IP
address guarantees that it remains yours, even if you delete the
Ingress. If you do not reserve an IP address, it might change,
requiring you to reconfigure your domain's DNS records.
Finally, you can take a look at the complete Google's guide on Creating an Ingress with a Google-managed certificate.
I am looking to deploy a SaaS platform I have built on kubernetes but I have hit a barrier when it comes to setting up the deployment correctly. You can find the setup I am going for
here. I have tried setting up the ingress controller, and it is, in fact forwarding requests to the GraphQL gateway, the problem is that the GraphQL Gateway itself is uncapable of connecting to the other services due to gRPC connection errors. Below you will find my Deployment.yml which contains the graphql gateway, an ingress controller and one service called general
What I am asking basically is how can I connect my gRPC based services to the main GraphQL gateway
apiVersion: v1
kind: Service
metadata:
name: graphql-gateway
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 8080
selector:
app: gateway
---
apiVersion: v1
kind: Service
metadata:
name: general-service
spec:
ports:
- protocol: TCP
port: 443
targetPort: 600
selector:
app: general
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: gateway
spec:
replicas: 2
selector:
matchLabels:
app: gateway
template:
metadata:
labels:
app: gateway
spec:
containers:
- name: gateway
image: myregistry/gateway:latest
imagePullPolicy: "Always"
ports:
- containerPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: general
spec:
replicas: 2
selector:
matchLabels:
app: general
template:
metadata:
labels:
app: general
spec:
containers:
- name: general
image: myregistry/general:latest
imagePullPolicy: "Always"
ports:
- containerPort: 600
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gateway-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/grpc-backend: "true"
spec:
rules:
- host: "playground.mydomain.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: graphql-gateway
port:
number: 80
I'm trying to deploy a laravel application in kubernetes at Google Cloud Platform.
I followed couple of tutorials and was successful trying them locally on a docker VM.
https://learnk8s.io/blog/kubernetes-deploy-laravel-the-easy-way
https://blog.cloud66.com/deploying-your-laravel-php-applications-with-cloud-66/
But when tried to deploy in kubernetes using an ingress to assign a domain name to the application. I keep getting the 502 bad gateway page.
I'm using a nginx ingress controller with image k8s.gcr.io/nginx-ingress-controller:0.8.3 and my ingress is as following
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- domainname.com
secretName: sslcertificate
rules:
- host: domain.com
http:
paths:
- backend:
serviceName: service
servicePort: 80
path: /
this is my application service
apiVersion: v1
kind: Service
metadata:
name: service
labels:
name: demo
version: v1
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
selector:
name: demo
type: NodePort
this is my ingress controller
apiVersion: v1
kind: Service
metadata:
name: default-http-backend
labels:
k8s-app: default-http-backend
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
k8s-app: default-http-backend
---
apiVersion: v1
kind: ReplicationController
metadata:
name: default-http-backend
spec:
replicas: 1
selector:
k8s-app: default-http-backend
template:
metadata:
labels:
k8s-app: default-http-backend
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
# Any image is permissable as long as:
# 1. It serves a 404 page at /
# 2. It serves 200 on a /healthz endpoint
image: gcr.io/google_containers/defaultbackend:1.0
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
---
apiVersion: v1
kind: ReplicationController
metadata:
name: nginx-ingress-controller
labels:
k8s-app: nginx-ingress-lb
spec:
replicas: 1
selector:
k8s-app: nginx-ingress-lb
template:
metadata:
labels:
k8s-app: nginx-ingress-lb
name: nginx-ingress-lb
spec:
terminationGracePeriodSeconds: 60
containers:
- image: gcr.io/google_containers/nginx-ingress-controller:0.8.3
name: nginx-ingress-lb
imagePullPolicy: Always
readinessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
timeoutSeconds: 1
# use downward API
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- containerPort: 80
hostPort: 80
- containerPort: 443
hostPort: 443
# we expose 18080 to access nginx stats in url /nginx-status
# this is optional
- containerPort: 18080
hostPort: 18080
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
and here is my laravel application deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: demo-rc
labels:
name: demo
version: v1
spec:
strategy:
type: Recreate
template:
metadata:
labels:
name: demo
version: v1
spec:
containers:
- image: gcr.io/projectname/laravelapp:vx
name: app-pod
ports:
- containerPort: 8080
I tried to add the domain entry to the hosts file but with no luck !!
is there a specific configurations I have to add to the configmap.yaml file for the nginx ingress controller?
In short, to be able to reach your application via external domain name (singapore.smartlabplatform.com), you need to create a A DNS record for GCP L4 Load Balancer's external IP address (this is in other words EXTERNAL-IP of your default nginx-ingress-controller's Service), here seen as pending:
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP
nginx-ingress-controller LoadBalancer 10.7.248.226 pending
nginx-ingress-default-backend ClusterIP 10.7.245.75 none
how to do this? it's explained on the GKE tutorials page here.
In the current state of your environment you can only reach your application in two ways:
From outside, via Load Balancer EXTERNAL-IP:
From inside, your Kubernetes cluster using laravel-kubernetes-demo service dns name:
$ curl laravel-kubernetes-demo.default.svc.cluster.local
<title>Laravel Kubernetes Demo :: LearnK8s</title>
If you want all that magic, like the automatic creation of DNS records, happen along with appearance of host: domain.com in your ingress resource spec, you should use external-dns (makes Kubernetes resources discoverable via public DNS servers), and here is the tutorial on how to set it up specifically for GKE.