Problem Statement:
I have deployed a spring boot app in gke under a namespace
when the app starts it uses a default gce sa credentials to authenticate.
what i did is created a gke service account and used iam policy binding to bind with a google service account and added workload identity user role
then annotated the gke sa by executing below 2 commands
issue is still my spring boot uses default gce sa credentials
Can someOne Please help me in resolving this.
I can see serviceAccountName is changed to new gke k8 SA and secret is also getting created and mounted.But app deployed are not using this Gke SA
Note: I am using Helsm chart for deployment
gcloud iam service-accounts add-iam-policy-binding \
--member serviceAccount:{projectID}.svc.id.goog[default/{k8sServiceAccount}] \
--role roles/iam.workloadIdentityUser \
{googleServiceAccount}
kubectl annotate serviceaccount \
--namespace default \
{k8sServiceAccount} \
iam.gke.io/gcp-service-account={googleServiceAccount}
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: helloworld
appVersion: {{ .Values.appVersion }}
name: helloworld
spec:
replicas: 1
selector:
matchLabels:
app: helloworld
template:
metadata:
labels:
app: helloworld
environment: {{ .Values.environment }}
spec:
serviceAccountName: {{ .Values.serviceAccountName }}
containers:
- name: helloworld
image: {{ .Values.imageSha }}
imagePullPolicy: Always
securityContext:
allowPrivilegeEscalation: false
runAsUser: 1000
ports:
- containerPort: 8080
env:
- name: SPRING_CONFIG_LOCATION
value: "/app/deployments/config/"
volumeMounts:
- name: application-config
mountPath: "/app/deployments/config"
readOnly: true
volumes:
- name: application-config
configMap:
name: {{ .Values.configMapName }}
items:
- key: application.properties
path: application.properties
When you create a pod, if you do not specify a service account, it is automatically assigned the default service account in the same namespace. If you get the raw json or yaml for a pod you have created (for example, kubectl get pods/<podname> -o yaml), you can see the spec.serviceAccountName field has been automatically set.
You can access the API from inside a pod using automatically mounted service account credentials, as described in Accessing the Cluster [1]. The API permissions of the service account depend on the authorization plugin and policy [2] in use.
In version 1.6+, you can opt out of automounting API credentials for a service account by setting automountServiceAccountToken: false on the service account:
apiVersion: v1
kind: ServiceAccount
metadata:
name: build-robot
automountServiceAccountToken: false
...
In version 1.6+, you can also opt out of automounting API credentials for a particular pod:
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
serviceAccountName: build-robot
automountServiceAccountToken: false
...
The pod spec takes precedence over the service account if both specify a automountServiceAccountToken value.
[1] https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/
[2] https://kubernetes.io/docs/reference/access-authn-authz/authorization/#authorization-modules
Related
I have spring boot application with embedded hazelcast that I am trying to deploy on a shared Kubernetes platform. I want to use kubernetes API strategy for auto discovery. Can I do this without creating Cluster Roles and Cluster Role Bindings and have just Role and Role Binding created under my namespace. If yes, what would the rbac.yaml look like ?
Tried creating the following roles and role bindings but no auto discovery so far.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: hazelcast-role
namespace: dev
rules:
- apiGroups:
- ""
resources:
- endpoints
- pods
- nodes
- services
verbs:
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: hazelcast-role-binding
namespace: dev
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: hazelcast-role
subjects:
- kind: ServiceAccount
name: default
namespace: dev
I am working on Springboot and Kubernetes and I have really simple application that connects to Postgres database. I want to get the value of datasource from configmap and password from secrets as mount file.
Configmap file :
apiVersion: v1
kind: ConfigMap
metadata:
name: customer-config
data:
application.properties: |
server.forward-headers-strategy=framework
spring.datasource.url=jdbc:postgresql://test/customer
spring.datasource.username=postgres
Secrets File :
apiVersion: v1
kind: Secret
metadata:
name: secret-demo
data:
spring.datasource.password: cG9zdGdyZXM=
deployment file :
spec:
containers:
- name: customerc
image: localhost:8080/customer
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8282
volumeMounts:
- mountPath: /workspace/config/default
name: config-volume
- mountPath: /workspace/secret/default
name: secret-volume
volumes:
- name: config-volume
configMap:
name: customer-config
- name: secret-volume
secret:
secretName: secret-demo
items:
- key: spring.datasource.password
path: password
If I move spring.datasource.password prop from secret to configmap then it works fine or If I populate its value as env variable then also work fine.
But as we know both are not secure way to do so, can someone tell me what's wrong with file mounting for secrets.
Spring Boot 2.4 added support for importing a config tree. This support can be used to consume configuration from a volume mounted by Kubernetes.
As an example, let’s imagine that Kubernetes has mounted the following volume:
etc/
config/
myapp/
username
password
The contents of the username file would be a config value, and the contents of password would be a secret.
To import these properties, you can add the following to your application.properties file:
spring.config.import=optional:configtree:/etc/config/
This will result in the properties myapp.username and myapp.password being set . Their values will be the contents of /etc/config/myapp/username and /etc/config/myapp/password respectively.
By default, consuming secrets through the API is not enabled for security reasons.Spring Cloud Kubernetes requires access to Kubernetes API in order to be able to retrieve a list of addresses of pods running for a single service. The simplest way to do that when using Minikube is to create default ClusterRoleBinding with cluster-admin privilege.
Example on how to create one :-
$ kubectl create clusterrolebinding admin --clusterrole=cluster-admin --serviceaccount=default:default
You need to give secret type in manifest file. Hope it will work.
apiVersion: v1
kind: Secret
metadata:
name: secret-demo
type: Opaque
data:
spring.datasource.password: cG9zdGdyZXM
I created registry credits and when I apply on pod like this:
apiVersion: v1
kind: Pod
metadata:
name: private-reg
spec:
containers:
- name: private-reg-container
image: registry.io.io/simple-node
imagePullSecrets:
- name: regcred
it works succesfly pull image
But if I try to do this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: node123
namespace: node123
spec:
replicas: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 2
maxUnavailable: 0
selector:
matchLabels:
name: node123
template:
metadata:
labels:
name: node123
spec:
containers:
- name: node123
image: registry.io.io/simple-node
ports:
- containerPort: 3000
imagePullSecrets:
- name: regcred
On pod will get error: ImagePullBackOff
when I describe it getting
Failed to pull image "registry.io.io/simple-node": rpc error: code =
Unknown desc = Error response from daemon: Get
https://registry.io.io/v2/simple-node/manifests/latest: no basic auth
credentials
Anyone know how to solve this issue?
We are always running images from private registry. And this checklist might help you :
Put your params in env variable in your terminal to have single source of truth:
export DOCKER_HOST=registry.io.io
export DOCKER_USER=<your-user>
export DOCKER_PASS=<your-pass>
Make sure that you can authenticate & the image really exist
echo $DOCKER_PASS | docker login -u$DOCKER_USER --password-stdin $DOCKER_HOST
docker pull ${DOCKER_HOST}/simple-node
Make sure that you created the Dockerconfig secret in the same namespace of pod/deployment;
namespace=mynamespace # default
kubectl -n ${namespace} create secret docker-registry regcred \
--docker-server=${DOCKER_HOST} \
--docker-username=${DOCKER_USER} \
--docker-password=${DOCKER_PASS} \
--docker-email=anything#will.work.com
Patch the service account used by the Pod with the secret
namespace=mynamespace
kubectl -n ${namespace} patch serviceaccount default \
-p '{"imagePullSecrets": [{"name": "regcred"}]}'
# if the pod use another service account,
# replace "default" by the relevant service account
or
Add imagePullSecrets in the pod :
imagePullSecrets:
- name: regcred
containers:
- ....
I use:
Spring Boot
Microservices (containerized)
Docker
Kubernetes
My case is as follows:
I have to generate link:
https://dev-myapp.com OR https://qa-myapp.com
depending on the environment in which my service is running (DEV, QA)
Depending on the environment (DEV, QA). I have one Spring profile BUT under this profile my app can run in kubernetes on 2 types of environment: DEV or QA. I want to generate proper link - read it from my properties file:
#Value("${email.body}")
private String emailBody;
application.yaml:
email:
body: Click on the following URL: ${ENVIRONMENT_URL:}/edge/invitation?code={0}&email={1}
DEVOPS(Kubernetes):
Manifest in workloads folder (DEV branch, the same for qa branch nut this time with https://qa-myapp.com):
apiVersion: v1
kind: Service
...
...
apiVersion: apps/v1
kind: Deployment
...
...
containers:
env:
- name: ENVIRONMENT_URL
value: https://dev-myapp.com
So is i possible to read that value from kubernetes container in my Spring properties file? I want to get email.body property depending on the container my service is running on.
Yes this is possible and have corrected the syntax of the yaml
apiVersion: v1
kind: Service
...
...
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
command: ["/bin/sh", "-c", "env | grep ENVIRONMENT_URL"]
env:
- name: ENVIRONMENT_URL
value: https://myapp.com. #Indedntation Changed
ports:
- containerPort: 80
Hi I am playing around with Kubernetes secrets.
My deployment file is :
---
apiVersion: v1
kind: Secret
metadata:
name: my-secrets
labels:
app: my-app
data:
username: dXNlcm5hbWU=
password: cGFzc3dvcmQ=
I am able to create secrets and I am mounting them in my deployments as below:
---
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app: my-service
ports:
- protocol: TCP
port: 80
targetPort: 8080
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-service
labels:
app: spring-service
spec:
replicas: 1
selector:
matchLabels:
app: spring-service
template:
metadata:
labels:
app: spring-service
spec:
containers:
- name: spring-service
image: my-image:tag
imagePullPolicy: Always
ports:
- containerPort: 8080
volumeMounts:
- name: my-secret-vol
mountPath: "/app/secrets/my-secret"
readOnly: true
volumes:
- name: my-secret-vol
secret:
secretName: my-secrets
My question is how can I access username and password I created in secret in spring-boot app?
I have tried loading in with ${my-secrets.username} and ${username}, but it fails to find values.
I also tried adding secrets as enviroment variables as below in deployment.yml:
env:
- name: username
valueFrom:
secretKeyRef:
name: my-secrets
key: username
- name: password
valueFrom:
secretKeyRef:
name: my-secrets
key: password
In this case, values are loaded from secrets and when I change values of secrets in minikube dashboard, it does not reflect the changes.
Please help me to understand how this works.
I am using minikube and docker as containers
You don't inject the secret into properties.yml. Instead, you use the content of the secret as properties.yml. The process is look like the following:
Create a properties.yml with the sensitive data (e.g. password)
Base64 encode this file (e.g. base64 properties.yml).
Take the base64 encoded value and put that in the secret under the key properties.yaml.
You should end up with a secret in the following format:
apiVersion: v1
kind: Secret
metadata:
name: my-secrets
labels:
app: my-app
data:
properties.yml: dXNlcm5hbWU=
Now when you mount this secret on your pod, Kubernetes will decrypt the secret and put the value under the relevant path and you can just mount it.
The pattern is to have 2 configuration files - one with non-sensitive configurations that is stored with the code, and the second (which includes sensitive configurations) stored as a secret. I don't know if that possible to load multiple config files using Spring Boot.
And one final comment - this process is cumbersome and error-prone. Each change to the configuration file requires decoding the original secret and repeating this manual process. Also, it's very hard to understand what changed - all you see is the entire content has changed. For that reason, we build Kamus. It let you encrypt only the sensitive value instead of the entire file. Let me know if that could be relevant for you :)
For the first approach you'll find the values on:
- /app/secrets/my-secret/username
- /app/secrets/my-secret/password
and for the second approach you can't change the value of env vars during runtime, you need to restart or redeploy the pod