I have a very simple containerized Spring Boot application that I am trying to deploy to Docker Desktop Kubernetes.
My application.yaml looks like:
spring:
application:
name: helloworld
datasource:
password: ${SPRING_DATASOURCE_PASSWORD:secretpassword}
url: ${SPRING_DATASOURCE_URL:jdbc:postgresql://localhost:5432/helloworld}
username: ${SPRING_DATASOURCE_USERNAME:helloworld}
My Dockerfile looks like:
FROM openjdk:17 AS build
WORKDIR /workspace/app
COPY ../mvnw .
COPY ../.mvn .mvn
COPY ../pom.xml .
COPY ../src src
RUN ./mvnw install -DskipTests
FROM openjdk:17
VOLUME /tmp
COPY --from=build /workspace/app/target/*.jar app.jar
ENTRYPOINT ["java", "-jar","/app.jar"]
My deployment.yml looks like:
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: helloworld
name: helloworld
spec:
replicas: 1
selector:
matchLabels:
app: helloworld
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: helloworld
spec:
containers:
- image: xxx/yyy
name: zzz
resources: {}
env:
- name: SPRING_DATASOURCE_URL
value: "jdbc:postgresql:host.docker.internal:5432/helloworld"
status: {}
When I run kubectl get all I can see the pod is in a "CrashLoopBackoff"
When I run (before the pod crashes)
kubectl exec POD_NAME -- printenv
I can see the environment variable has been created (or I get an error I assume is because of the pod crashing: error: unable to upgrade connection: container not found ("yyy"))
But when I check the logs, I see:
SQL State : 08001
Error Code : 0
Message : Connection to localhost:5432 refused.
So the Spring Boot app is not picking up the environment variable SPRING_DATASOURCE_URL. It is still trying to connect to localhost:5432
If I add the env variable to the docker-compose (or IntelliJ runtime config) and run outside of K8s, the Spring Boot application picks up the environment variable.
Any idea what I am doing wrong?
The value of the env variable was causing the issue. It should be:
env:
- name: SPRING_DATASOURCE_URL
value: jdbc:postgresql://host.docker.internal:5432/helloworld
Because I was missing the // it kept referring trying to connect to localhost. Which was very confusing.
Related
I'm trying to deploy my spring boot application to local kubernates cluster.
I have performed the below steps
1.created the application build the image and pushed to docker hub registry
2.created the secret and configured
3.created the deployment file and executed
But Pod was not getting created as I'm getting containerCreationError in the status .when I checked the logs I can see that image has been successfully pulled but the below error occurs
kublet Error: Error response from daemon: No command specified
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-app-deployment
labels:
name: auth-app-deployment
spec:
replicas: 1
selector:
matchLabels:
name: auth-app
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
name: auth-app
spec:
imagePullSecrets:
- name: regcred
containers:
- name: auth-app
image: vishnusnair1995/vsn-tec:auth
imagePullPolicy: Always
ports:
- containerPort: 9999
env:
- name: SPRING_PROFILES_ACTIVE
value: prod
livenessProbe:
httpGet:
path: /actuator/health
port: 9999
initialDelaySeconds: 30
periodSeconds: 40
readinessProbe:
httpGet:
path: /actuator/health
port: 9999
initialDelaySeconds: 30
periodSeconds: 40
restartPolicy: Always
FROM openjdk:8
COPY build/libs/auth-0.0.1-SNAPSHOT.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
DOCKERFILE
From my understanding the issue is only when we are pulling the image from remote registry,
because what I observed is that when I build the docker image in local and change the imagePullPolicy : IfNotPresent
pod was getting created with out any issues
This kind of error: Error response from daemon: No command specified happens when masterConfig or workerConfig doesn't have containerCommand.
If the user doesn't specify any workerConfig, workerConfig inherits the same config as masterConfig and the issue won't happen. However if the user specified anything in workerConfig, masterConfig will have containerCommand added but workerConfig won't, then the issue will happen.
Try Below possible Workarounds:
Workaround 1 :
The workaround is to not specify any workerConfig explicitly which makes the workers inherit all configs from master.
Workaround 2 :
You might need to update your Dockerfile with the following:
FROM python
WORKDIR /build
ENV PYTHONUNBUFFERED=1
ENV PYTHONIOENCODING=UTF-8
COPY test.py /build
RUN chmod 755 /build/test.py
CMD ["python", "test.py"]
Then build and push the docker image and recreate the pod.
Also refer to similar SO for more information.
I’m trying to set some environment variables in k8s deployment and use them within the application.properties in my spring boot application, but it looks like I'm doing something wrong because spring is not reading those variables, although when checking the env vars on the pod, all the vars are set correctly.
The error log from the container:
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port...
Any help will be appreciated.
application.properties:
spring.datasource.url=jdbc:postgresql://${DB_URL}:${DB_PORT}/${DB_NAME}
spring.datasource.username=${DB_USER_NAME}
spring.datasource.password=${DB_PASSWORD}
DockerFile
FROM openjdk:11-jre-slim-buster
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-deployment
labels:
app: api
spec:
replicas: 1
selector:
matchLabels:
app: api
template:
metadata:
labels:
app: api
spec:
containers:
- name: api
image: .../api
resources:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 80
env:
- name: DB_URL
value: "posgres"
- name: DB_NAME
valueFrom:
configMapKeyRef:
name: postgres-config
key: dbName
- name: DB_USER_NAME
valueFrom:
secretKeyRef:
name: db-secret
key: dbUserName
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: dbPassword
The DockerFile was wrong.
Everything is working fine after changing the DockerFile to this:
FROM maven:3.6.3-openjdk-11-slim as builder
WORKDIR /app
COPY pom.xml .
COPY src/ /app/src/
RUN mvn install -DskipTests=true
FROM adoptopenjdk/openjdk11:jre-11.0.8_10-alpine
COPY --from=builder /app/target/*.jar /app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]
You miss the env: in your deployment.yaml, see here : https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/
if you have kubectl installed you can check how env vars must be declared by doing an explain like follow : kubectl explain deployment.spec.template.spec.containers
I use:
Spring Boot
Microservices (containerized)
Docker
Kubernetes
My case is as follows:
I have to generate link:
https://dev-myapp.com OR https://qa-myapp.com
depending on the environment in which my service is running (DEV, QA)
Depending on the environment (DEV, QA). I have one Spring profile BUT under this profile my app can run in kubernetes on 2 types of environment: DEV or QA. I want to generate proper link - read it from my properties file:
#Value("${email.body}")
private String emailBody;
application.yaml:
email:
body: Click on the following URL: ${ENVIRONMENT_URL:}/edge/invitation?code={0}&email={1}
DEVOPS(Kubernetes):
Manifest in workloads folder (DEV branch, the same for qa branch nut this time with https://qa-myapp.com):
apiVersion: v1
kind: Service
...
...
apiVersion: apps/v1
kind: Deployment
...
...
containers:
env:
- name: ENVIRONMENT_URL
value: https://dev-myapp.com
So is i possible to read that value from kubernetes container in my Spring properties file? I want to get email.body property depending on the container my service is running on.
Yes this is possible and have corrected the syntax of the yaml
apiVersion: v1
kind: Service
...
...
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
command: ["/bin/sh", "-c", "env | grep ENVIRONMENT_URL"]
env:
- name: ENVIRONMENT_URL
value: https://myapp.com. #Indedntation Changed
ports:
- containerPort: 80
I am new to Kubernetes and kubectl. I am basically running a GRPC server in my localhost. I would like to use this endpoint in a spring boot app running on kubernetes using kubectl on my mac. If I set the following config in application.yml and run in kubernetes, it doesn't work. The same config works if I run in IDE.
grpc:
client:
local-server:
address: static://localhost:6565
negotiationType: PLAINTEXT
I see some people suggesting port-forward, but it's the other way round (It works when I want to use a port that is already in kubernetes from localhost just like the tomcat server running in kubernetes from a browser on localhost)
apiVersion: apps/v1
kind: Deployment
metadata:
name: testspringconfigvol
labels:
app: testspring
spec:
replicas: 1
selector:
matchLabels:
app: testspringconfigvol
template:
metadata:
labels:
app: testspringconfigvol
spec:
initContainers:
# taken from https://gist.github.com/tallclair/849601a16cebeee581ef2be50c351841
# This container clones the desired git repo to the EmptyDir volume.
- name: git-config
image: alpine/git # Any image with git will do
args:
- clone
- --single-branch
- --
- https://github.com/username/fakeconfig
- /repo # Put it in the volume
securityContext:
runAsUser: 1 # Any non-root user will do. Match to the workload.
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
volumeMounts:
- mountPath: /repo
name: git-config
containers:
- name: testspringconfigvol-cont
image: username/testspring
ports:
- containerPort: 8080
volumeMounts:
- mountPath: /usr/local/lib/config/
name: git-config
volumes:
- name: git-config
emptyDir: {}
What I need in simple terms:
Ports having some server in my localhost:
localhost:6565, localhost:6566, I need to access these ports some how in my kubernetes. Then what should I set it in application.yml config? Will it be the same localhost:6565, localhost:6566 or how-to-get-this-ip:6565, how-to-get-this-ip:6566.
We can get the vmware host ip using minikube with this command minikube ssh "route -n | grep ^0.0.0.0 | awk '{ print \$2 }'". For me it's 10.0.2.2 on Mac. If using Kubernetes on Docker for mac, it's host.docker.internal.
By using these commands, I managed to connect to the services running on host machine from kubernetes.
1) Inside your application.properties define
server.port=8000
2) Create Dockerfile
# Start with a base image containing Java runtime (mine java 8)
FROM openjdk:8u212-jdk-slim
# Add Maintainer Info
LABEL maintainer="vaquar.khan#gmail.com"
# Add a volume pointing to /tmp
VOLUME /tmp
# Make port 8080 available to the world outside this container
EXPOSE 8080
# The application's jar file (when packaged)
ARG JAR_FILE=target/codestatebkend-0.0.1-SNAPSHOT.jar
# Add the application's jar to the container
ADD ${JAR_FILE} codestatebkend.jar
# Run the jar file
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/codestatebkend.jar"]
3) Make sure docker is working fine
docker run --rm -p 8080:8080
4)
https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
use following command to find the pod name
kubectl get pods
then
kubectl port-forward <pod-name> 8080:8080
Useful links :
https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/#forward-a-local-port-to-a-port-on-the-pod
https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-services/#manually-constructing-apiserver-proxy-urls
https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
https://developer.okta.com/blog/2019/05/22/java-microservices-spring-boot-spring-cloud
I have a spring boot application with two profiles, dev and prod, my docker file is:
FROM openjdk:8-jdk-alpine
VOLUME /tmp
ARG DEPENDENCY=target/dependency
COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY ${DEPENDENCY}/META-INF /app/META-INF
COPY ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-Dspring.profiles.active=dev","-cp","app:app/lib/*","com.my.Application"]
please not that, when building the image, I specify the entrypoint as command line argument.
This is the containers section of my kubernetes deployment where I use this image:
containers:
- name: myapp
image: myregistry.azurecr.io/myapp:0.1.7
imagePullPolicy: "Always"
ports:
- containerPort: 8080
name: myapp
readinessProbe:
httpGet:
path: /actuator/health
port: 8080
timeoutSeconds: 3
periodSeconds: 20
failureThreshold: 3
It works but has a major flaw: how can I now switch to the production environment without rebuilding the image?
The best would be to remove that ENTRYPOINT in my docker file and give this configuration in my kubernetes yml so that I could always use the same image...is this possible?
edit: I saw that there is a lifecycle istruction but note that I have a readiness probe based on the spring boot's actuator. It would always fail if I used this construct.
You can override an image's ENTRYPOINT by using the command property of a Kubernetes Pod spec. Likewise, you could override CMD by using the args property (also see the documentation):
containers:
- name: myapp
image: myregistry.azurecr.io/myapp:0.1.7
imagePullPolicy: "Always"
command: ["java","-Dspring.profiles.active=prod","-cp","app:app/lib/*","com.my.Application"]
ports:
- containerPort: 8080
name: myapp
Alternatively, to provide a higher level of abstraction, you might write your own entrypoint script that reads the application profile from an environment variable:
#!/bin/sh
PROFILE="${APPLICATION_CONTEXT:-dev}"
exec java "-Dspring.profiles.active=$PROFILE" -cp app:app/lib/* com.my.Application
Then, you could simply pass that environment variable into your pod:
containers:
- name: myapp
image: myregistry.azurecr.io/myapp:0.1.7
imagePullPolicy: "Always"
env:
- name: APPLICATION_CONTEXT
value: prod
ports:
- containerPort: 8080
name: myapp
Rather than putting spring.profiles.active in dockerfile in the entrypoint.
Make use of configmaps and application.properties.
Your ENTRYPOINT in dockerfile should look like:
ENTRYPOINT ["java","-cp","app:app/lib/*","com.my.Application","--spring.config.additional-location=/config/application-dev.properties"]
Create a configmap that acts as application.properties for your springboot application
---
apiVersion: v1
kind: ConfigMap
metadata:
name: myapp-config
namespace: flow
data:
application-dev.properties: |
spring.application.name=myapp
server.port=8080
spring.profiles.active=dev
NOTE: Here we have specified spring.profiles.active.
In containers section of my kubernetes deployment mount the configmap inside container that will act as application.properties.
containers:
- name: myapp
image: myregistry.azurecr.io/myapp:0.1.7
imagePullPolicy: "Always"
command: ["java","-cp","app:app/lib/*","com.my.Application","--spring.config.additional-location=/config/application-dev.properties"]
ports:
- containerPort: 8080
name: myapp
volumeMounts:
- name: myapp-application-config
mountPath: "/config"
readOnly: true
volumes:
- name: myapp-application-config
configMap:
name: myapp-config
items:
- key: application-dev.properties
path: application-dev.properties
readinessProbe:
httpGet:
path: /actuator/health
port: 8080
timeoutSeconds: 3
periodSeconds: 20
failureThreshold: 3
NOTE: --spring.config.additional-location points to location of application.properties that we created in configmaps.
So making use of configmaps and application.properties one can override any configuration of your application wihtout rebuilding the image.
If you want to add a new config or update value of existing config, just make appropriate changes in configmap and kubectl apply it. Then scale down and scale up your application pod, to bring the new config in action.
Hope this helps.
There are many many ways to set Spring configuration values. With some rules, you can use ordinary environment variables to specify individual property values. You might see if you can use this instead of having a separate Spring profile control.
Using environment variables has two advantages here: it means you (or your DevOps team) can change deploy-time settings without recompiling the application; and if you're using a deployment manager like Helm where some details like hostnames are intrinsically unpredictable, this lets you specify values that can't be known until deploy time.
For example, let's say you have a Redis dependency:
cache:
redis:
url: redis://localhost:6379/0
You could override this at deploy time by setting
containers:
- name: myapp
env:
- name: CACHE_REDIS_URL
value: "redis://myapp-redis.default.svc.cluster.local:6379/0"
One way to do this is using spring cloud Kubernetes as described here
https://docs.spring.io/spring-cloud-kubernetes/docs/current/reference/html/index.html#configmap-propertysource
You can define your profiles in a configmap like below
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
application.yml: |-
greeting:
message: Say Hello to the World
farewell:
message: Say Goodbye
---
spring:
profiles: development
greeting:
message: Say Hello to the Developers
farewell:
message: Say Goodbye to the Developers
---
spring:
profiles: production
greeting:
message: Say Hello to the Ops
And can then select the desired profile by passing an environment variable in your Kubernetes deployment manifest
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment-name
labels:
app: deployment-name
spec:
replicas: 1
selector:
matchLabels:
app: deployment-name
template:
metadata:
labels:
app: deployment-name
spec:
containers:
- name: container-name
image: your-image
env:
- name: SPRING_PROFILES_ACTIVE
value: "development"