I have a mount volume has a file urls.txt with database source url, like
databasesource: mysql://xxxx
and in my springboot application which will be running as a container in a openshift pod, and in the application I need to change the SPRING_DATASOURCE_URL as mentioned in the file above, here is what I want to achieve in my template file
env:
- name: SPRING_DATASOURCE_URL
valueFrom:
mount:
name: my-volume
key: databasesource
volumeMounts:
- name: my-volume
mountPath: /someDir
I know we can valueFrom configMap or secret, but I want to achieve via a volumeMount
if you can use below format in
urls.txt
databasesource=mysql://xxxx
as part of your container start you run
source /somedir/urls.txt
which will load the key & values in env. which can be further used.
The problem is resolved by a Springboot2.0 feature: https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.external-config.files.importing-extensionless
Related
I want to overwrite the file on the pod container. Right now I have elasticsearch.yml at location /usr/share/elasticsearch/config.
I was trying to achieve that with initContainer at kubernetes deployment file, so I added something like:
- name: disabled-the-xpack-security
image: busybox
command:
- /bin/sh
- -c
- |
sleep 20
rm /usr/share/elasticsearch/config/elasticsearch.yml
cp /home/x/IdeaProjects/BD/infra/istio/kube/elasticsearch.yml /usr/share/elasticsearch/config/
securityContext:
privileged: true
But this doesn't work, error looks like:
rm: can't remove '/usr/share/elasticsearch/config/elasticsearch.yml': No such file or directory
cp: can't stat '/home/x/IdeaProjects/BD/infra/istio/kube/elasticsearch.yml': No such file or directory
I was trying to use some echo "some yaml config" >> elasticsearch.yml, but this kind of workarounds doesn't work, because I was able to keep proper yaml formatting.
Do you have any suggestions, how can I do this?
As stated by Arman in the comments, you can create a ConfigMap with the contents of /home/x/IdeaProjects/BD/infra/istio/kube/elasticsearch.ymland mount it as a volume in the deployment.
To create the config map from your file you can run:
kubectl create configmap my-es-config --from-file=/home/x/IdeaProjects/BD/infra/istio/kube/elasticsearch.yml
This will create a ConfigMap inside your kubernetes cluster with the yaml file.
You can then use that and add the volume mount to your deployment as:
containers:
- name: elasticsearch
image: k8s.gcr.io/busybox
.
.
.
volumeMounts:
- name: config-volume
mountPath: /usr/share/elasticsearch/config/
volumes:
- name: config-volume
configMap:
name: my-es-config
Notes
It is recommended to create your ConfigMap as yaml as well. More information here
Mounting a configmap directly on /usr/share/elasticsearch/config/, will replace everything inside that path and place the config file from the configmap. If that causes an issue, you might want to mount it at another location and then copy it.
Note if you don't want to override everything in the mounted directory you could mount the file only using "subPath" in whatever directory you want.
https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath
For the life of Bryan, how do I do this?
Terraform is used to create an SQL Server instance in GCP.
Root password and user passwords are randomly generated, then put into the Google Secret Manager.
The DB's IP is exposed via private DNS zone.
How can I now get the username and password to access the DB into my K8s cluster? Running a Spring Boot app here.
This was one option I thought of:
In my deployment I add an initContainer:
- name: secrets
image: gcr.io/google.com/cloudsdktool/cloud-sdk
args:
- echo "DB_PASSWORD=$(gcloud secrets versions access latest --secret=\"$NAME_OF_SECRET\")" >> super_secret.env
Okay, what now? How do I get it into my application container from here?
There are also options like bitnami/sealed-secrets, which I don't like since the setup is using Terraform already and saving the secrets in GCP. When using sealed-secrets I could skip using the secrets manager. Same with Vault IMO.
On top of the other answers and suggestion in the comments I would like to suggest two tools that you might find interesting.
First one is secret-init:
secrets-init is a minimalistic init system designed to run as PID 1
inside container environments and it`s integrated with
multiple secrets manager services, e.x. Google Secret Manager
Second one is kube-secrets-init:
The kube-secrets-init is a Kubernetes mutating admission webhook,
that mutates any K8s Pod that is using specially prefixed environment
variables, directly or from Kubernetes as Secret or ConfigMap.
It`s also support integration with Google Secret Manager:
User can put Google secret name (prefixed with gcp:secretmanager:) as environment variable value. The secrets-init will resolve any environment value, using specified name, to referenced secret value.
Here`s a good article about how it works.
How do I get it into my application container from here?
You could use a volume to store the secret and mount the same volume in both init container and main container to share the secret with the main container from the init container.
apiVersion: v1
kind: Pod
metadata:
name: my-app
spec:
containers:
- name: my-app
image: my-app:latest
volumeMounts:
- name: config-data
mountPath: /data
initContainers:
- name: secrets
image: gcr.io/google.com/cloudsdktool/cloud-sdk
args:
- echo "DB_PASSWORD=$(gcloud secrets versions access latest --secret=\"$NAME_OF_SECRET\")" >> super_secret.env
volumeMounts:
- name: config-data
mountPath: /data
volumes:
- name: config-data
emptyDir: {}
You can use spring-cloud-gcp-starter-secretmanager to load secrets from Spring application itself.
Documentation - https://cloud.spring.io/spring-cloud-gcp/reference/html/#secret-manager
Using volumes of emptyDir with medium: Memory to guarantee that the secret will not be persisted.
...
volumes:
- name: scratch
emptyDir:
medium: Memory
sizeLimit: "1Gi"
...
If one has control over the image, it's possible to change the entry point and use berglas.
Dockerfile:
FROM adoptopenjdk/openjdk8:jdk8u242-b08-ubuntu # or whatever you need
# Install berglas, see https://github.com/GoogleCloudPlatform/berglas
RUN mkdir -p /usr/local/bin/
ADD https://storage.googleapis.com/berglas/main/linux_amd64/berglas /usr/local/bin/berglas
RUN chmod +x /usr/local/bin/berglas
ENTRYPOINT ["/usr/local/bin/berglas", "exec", "--"]
Now we build the container and test it:
docker build -t image-with-berglas-and-your-app .
docker run \
-v /host/path/to/credentials_dir:/root/credentials \
--env GOOGLE_APPLICATION_CREDENTIALS=/root/credentials/your-service-account-that-can-access-the-secret.json \
--env SECRET_TO_RESOLVE=sm://your-google-project/your-secret \
-ti image-with-berglas-and-your-app env
This should print the environment variables with the sm:// substituted by the actual secret value.
In K8s we run it with Workload Identity, so the K8s service account on behalf of which the pod is scheduled needs to be bound to a Google service account that has the right to access the secret.
In the end your pod description would be something like this:
apiVersion: v1
kind: Pod
metadata:
name: your-app
spec:
containers:
- name: your-app
image: image-with-berglas-and-your-app
command: [start-sql-server]
env:
- name: AXIOMA_PASSWORD
value: sm://your-google-project/your-secret
I have configMap that are loading properties files for my spring boot application.
My configMap is mounted as a volume and my springboot app is reading from that volume.
my typical property files are:
application-dev1.yml has
integrations-queue-name=integration-dev1
search-queue-name=searchindex-dev1
application-dev2.yml
integrations-queue-name=integration-dev2
search-queue-name=searchindex-dev1
application-dev3.yml
integrations-queue-name=integration-dev3
search-queue-name=searchindex-dev1
My goal is to have 1 properties file
application-env.yml
integrations-queue-name=integration-{env}
search-queue-name=searchindex-{env}
I want to do parameter substitution of env with the profile that is active for my service.
Is it possible to do parameter substitution in configMaps from my spring boot application running in the pod? I am lookin for something similar to maven-resource-plugin that can be done run time.
If it's just those two, then likely you will get more mileage out of using the SPRING_APPLICATION_JSON environment variable, which should supersede anything in the configmap:
containers:
- name: my-spring-app
image: whatever
env:
- name: ENV_NAME
value: dev2
- name: SPRING_APPLICATION_JSON
value: |
{"integrations-queue-name": "integration-$(ENV_NAME)",
"search-queue-name": "searchindex-$(ENV_NAME)"}
materializes as:
$ kubectl exec my-spring-pod -- printenv
ENV_NAME=dev2
SPRING_APPLICATION_JSON={"integrations-queue-name": "integration-dev2",
"search-queue-name": "searchindex-dev2"}
We are setting up a Jenkins-based CI pipeline on our Kubernetes cluster (Rancher if that matters) and up to now we have used the official maven:3-jdk-11-slim image for experiments. Unfortunately it does not provide any built-in way of overriding the default settings.xml to use a mirror, which we need - preferably just by setting an environment variable. I am not very familar with kubernetes so I may be missing something simple.
Is there a simple way to add a file to the image? Should I use another image with this functionality built in?
pipeline {
agent {
kubernetes {
yaml """
kind: Pod
metadata:
name: kaniko
spec:
containers:
- name: maven
image: maven:3-jdk-11-slim
command:
- cat
tty: true
- name: kaniko
.... etc
Summary: you can mount your settings.xml file on the pod at some specific path and use that file with command mvn -s /my/path/to/settings.xml.
Crou's ConfigMap approach is one way to do it. However, since the settings.xml file usually contains credentials, I would treat it as Secrets.
You can create a Secret in Kubernetes with command:
$ kubectl create secret generic mvn-settings --from-file=settings.xml=./settings.xml
The pod definition will be something like this:
apiVersion: v1
kind: Pod
metadata:
name: kaniko
spec:
containers:
- name: maven
image: maven:3-jdk-11-slim
command:
- cat
tty: true
volumeMounts:
- name: mvn-settings-vol
mountPath: /my/path/to
volumes:
- name: mvn-settings-vol
secret:
secretName: mvn-settings
Advanced/Optional: If you practice "Infrastructure as Code", you might want to save the manifest file for that secret for recovery. This can be achieved by this command after secret already created:
$ kubectl get secrets mvn-settings -o yaml
You can keep secrets.yml file but do not check into any VCS/Github repo since this version of secrets.yml contains unencrypted data.
Some k8s administrators may have kubeseal installed. In that case, I'd recommend using kubeseal to get encrypted version of secrets.yml.
$ kubectl create secret generic mvn-settings --from-file=settings.xml=./settings.xml --dry-run -o json | kubeseal --controller-name=controller --controller-namespace=k8s-sealed-secrets --format=yaml >secrets.yml
# Actually create secrets
$ kubectl apply -f secrets.yml
The controller-name and controller-namespace should be obtained from k8s administrators.
This secrets.yml contains encrypted data of your settings.xml and can be safely checked into VCS/Github repo.
If you want to override a file inside pod you can use ConfigMap to store the changed file and mount it instead of previous one.
You can create the ConfigMap from a file using
kubectl create configmap settings-xml --from-file=settings.xml
Your pod definition might look like this:
apiVersion: v1
kind: Pod
metadata:
name: kaniko
spec:
containers:
- name: maven
image: maven:3-jdk-11-slim
command:
- cat
tty: true
volumeMounts:
- name: config-settings
mountPath: /usr/share/maven/ref/settings.xml
volumes:
- name: config-settings
configMap:
# Provide the name of the ConfigMap containing the files you want
# to add to the container
name: settings-xml
...
This worked for me:
Install Config File Provider Plugin
Go to Manage Jenkins > Config File Management > Add a new config and insert here your settings.xml
In your jenkinsfile just put your rtMavenRun inside a configFileProvider block, and put the same fileId of the jenkins config file you created before
stage('Build Maven') {
steps {
configFileProvider([configFile(fileId: 'MavenArtifactorySettingId', variable: 'MAVEN_SETTINGS_XML')]) {
retry(count: 3) {
rtMavenRun(
tool: "Maven 3.6.2", //id specified in Global Tool Configuration
pom: 'pom.xml',
goals: '-U -s $MAVEN_SETTINGS_XML clean install',
)
}
}
}
}
this is exactly the pipeline that I used if you want to see more: https://gist.github.com/robertobatts/42da9069e13b61a238f51c36754de97b
If you versioned the settings.xml of the project with the code, it makes sense to build with mvn install -s settings.xml using sh step. It what I did at work. If settings.xml is not versioned with the project, it indeed makes sens to mount the file with Crou's solution.
To answer your question "Should I use another image with this functionality built in?" I would recommend to avoid a maximum to build custom images because you will end up having to maintain them
I'm deploying a Spring Boot app in minikube that connects to a database running on the host. Following the 12 factor app recommendations I use environment variables for the necessary configuration:
SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.postgresql.Driver
SPRING_DATASOURCE_PASSWORD=...
SPRING_DATASOURCE_URL=jdbc:postgresql://<HOST_IP_FROM_K8S>:5432/myservice
SPRING_DATASOURCE_USERNAME=...
The kubernetes docs only show how to set environment variables in the service and deployment .yaml files which I don't want to do. Is there a way to pass environment variables on the command line for minikube or kubectl when I create the deployment? (In Docker I do this with -e.)
Note that the environment variables have to be set before starting the app or it crashes.
Following Ansil's comment above I used configmap and secret to pass the configuration like this:
kubectl create secret generic springdatasourcepassword --from-literal=SPRING_DATASOURCE_PASSWORD=postgres
kubectl create secret generic springdatasourceusername --from-literal=SPRING_DATASOURCE_USERNAME=postgres
kubectl create configmap springdatasourcedriverclassname --from-literal=SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.postgresql.Driver
kubectl create configmap springdatasourceurl --from-literal=SPRING_DATASOURCE_URL=jdbc:postgresql://172.18.0.1:5432/bookservice
These are referenced in the deployment.yaml file like this:
env:
- name: GET_HOSTS_FROM
value: dns
- name: SPRING_DATASOURCE_DRIVER_CLASS_NAME
valueFrom:
configMapKeyRef:
name: springdatasourcedriverclassname
key: SPRING_DATASOURCE_DRIVER_CLASS_NAME
- name: SPRING_DATASOURCE_URL
valueFrom:
configMapKeyRef:
name: springdatasourceurl
key: SPRING_DATASOURCE_URL
- name: SPRING_DATASOURCE_PASSWORD
valueFrom:
secretKeyRef:
name: springdatasourcepassword
key: SPRING_DATASOURCE_PASSWORD
- name: SPRING_DATASOURCE_USERNAME
valueFrom:
secretKeyRef:
name: springdatasourceusername
key: SPRING_DATASOURCE_USERNAME
A full explanation can be found here.
We have no direct option with create subcommand in kubectl to pass the environmental variables.
You may use Helm to customize deployment.
(Example: https://docs.bitnami.com/kubernetes/how-to/deploy-java-application-kubernetes-helm/)