This is a fairly simple question I think, however I haven't seen too many examples or really anything that explains the connection between using docker configs (v 3.3+) and loading that config into Spring Boot for reference.
sample docker-stack.yml
version: '3.3'
services:
test-service:
image: myrepo/test-service:1.0.0
configs:
- service-config
networks:
- test-network
configs:
service-config:
external:true
networks:
test-network:
sample swarm "service-config".
I entered this as a new "configs" entry in Portainer.
services:
test-service:
key1: sample value
key2: sample two
What I'm trying to do it load this config into Spring such that I can reference the values from this config within a component.
Either via #ConfigurationProperties
#ConfigurationProperties("services.test-service")
public MyBeanName myBean() { return new MyBeanName(); }
or via the #Value:
#Value("${services.test-service.key1}")
private String key1;
How can I get this docker "configs" configuration loaded into Spring. This has got to be simple enough.. lol. Thanks!
Sorry for the delay in responding to this question, or at least posting the solution...
It took a bit more digging into how configs work with docker, however as it turns out you need to specify a target for the "config" held within your "configs" entries in your swarm cluster, and then map it into your container, and load as an external config to your spring application. In my case, I didn't want to override the the application.yml in my spring boot app, just wanted to pick up additional configs. So I went with the setting:
--spring.config.additional-location=file:/configs/sample-config.yml
Lets say I create a docker configs entry with the name "sample-config" and having the following data:
Configs Entry => "sample-config"
services:
test-service:
key1: sample value
key2: sample two
Then on my compose/stack file I need to reference the "configs" entry, and provide a target file that corresponds to the file I specified in my "spring.config.additional-location" setting.
version: '3.3'
services:
test-service:
image: myrepo/test-service:1.0.0
configs:
- source: sample-config
target: /configs/sample-config.yml
networks:
- test-network
configs:
sample-config:
external:true
networks:
test-network:
In my Dockerfile then, I'd specify the following to essentially load the "sample-config" configs entry when starting the jar/app:
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar", "--spring.config.additional-location=file:/configs/sample-config.yml"]
This allowed me to access the configuration entries, loaded externally to my spring application, with the #Value and #ConfigurationProperties annotations.
Related
I have a springboot application which uses a postgresql database and a mongoDB database , I have been able to correctly configure them but now when I want to dockerize my application to later deploy it on a Kubernetes cluster, I am completely clueless. Most of the youtube tutorials and articles are on how to dockerize simple springboot applications or springboot applications that use only one database, thus any input on how I can proceed to dockerize my application would be really appreciated!
Edit:
I am following this tutorial -
https://www.section.io/engineering-education/running-a-multi-container-springboot-postgresql-application-with-docker-compose/
Here in the docker-compose.yml file-
version: '3.1'
services:
API:
image: 'blog-api-docker.jar'
ports:
- "8080:8080"
depends_on:
PostgreSQL:
condition: service_healthy
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://PostgreSQL:5432/postgres
- SPRING_DATASOURCE_USERNAME=postgres
- SPRING_DATASOURCE_PASSWORD=password
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
PostgreSQL:
image: postgres
ports:
- "5432:5432"
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_USER=postgres
- POSTGRES_DB=postgres
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
Only one postgreSQL datasource is defined ,in my project with a similar postgreSQL datasource as given in the tutorial I am also using a mongoDB database which is running on atlas.
I am also including my application.properties file for your reference-
spring.ds-psql.datasource.jdbcUrl=jdbc:postgresql://localhost:5432/devicestatspsql
spring.ds-psql.datasource.username=
spring.ds-psql.datasource.password=
spring.data.mongodb.users-mongo-atlas.uri=*mongodb database url here*
spring.jpa.generate-ddl=true
spring.jpa.show-sql=true
So I just need to know what changes are required in my docker-compose.yml file to accommodate this mongodb database in the docker image
You can use the Kubernetes Configmap and Secret as of now to store the configuration of your application.
Configmap and Secret are mostly for storing configurations like database connection strings, usernames, and passwords.
You can create different configuration maps as per requirement for Dev,Stag and Prod then inject the specific to your deployment so the application will get those values from either .env file or from OS environment.
Here's the reference article.
Hello I'm starting with docker and docker compose and I have the following problem:
I'm working in a spring micro services architecture where I have one configuration service, one discovery service, one gateway service and multiple resource services.
To run these services, I build jar files, which I place in separated folder per service with their config files (application.yml and bootstrap.yml):
e.g:
config-service/
config-service.jar
application.yml
discovery-service/
discovery-service.jar
bootstrap.yml
gateway-service/
gateway-service.jar
bootstrap.yml
crm-service/
crm-service.jar
bootstrap.yml
This works so far on my server.
Now I want to deploy my services in different environments as docker images (created with mvn build image and buildpack) using docker compose, where the configuration files vary depending on the environment. How can I deploy a service as a container using an existing image but with a different configuration file?
Thank you in advance!
There are a few possibilities when handling configuration in a containerized environment.
One of the options is that Spring boot allows you to use environment variables for each application property. For example, let's say you have a spring.datasource.url property, in that case you could also define that property by setting a SPRING_DATASOURCE_URL environment variable:
version: '3.8'
services:
my-spring-boot-app:
image: my-image:0.0.1
environment:
- SPRING_DATASOURCE_URL=jdbc:my-database-url
Alternatively, you could use volumes to put an external file on a specific location within a container:
version: '3.8'
services:
my-spring-boot-app:
image: my-image:0.0.1
volumes:
./my-app/bootstrap.yml:/etc/my-app/bootstrap.yml
In this example I'm copyingbootstrap.yml from a relative folder on my host machine, to /etc/my-app within the container. If you put these files within the same folder as your JAR file, you can override the configuration.
I am trying to read environment variable declared in deployment yaml of Kubernetes into springboot application.yaml
Below is sample in deployment.yaml
spec:
containers:
env:
- name: SECRET_IN
value: dev
Below is sample in application.yaml
innovation:
in: ${SECRET_IN:demo}
But on localhost when I try to print innovation.in (#Configuration is created correctly) I am not getting "dev" in output, it always prints demo, it appears the link between deployment, application yaml is not happening ,could someone please help.
You can store the whole application.YAML config file into the config map or secret and inject it with the deployment only
For example :
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
application.yaml: |-
pool:
size:
core: 1
max:16
if your application.properties is something like
example:
spring.datasource.url=jdbc:mysql://${MYSQL_HOST:localhost}:3306/dbname
spring.datasource.username=user
spring.datasource.password=password
You can replace it with
jdbc:mysql://${MYSQL_HOST:localhost}:3306/dbname
Deployment.yaml will be something like
spec:
containers:
- name: demowebapp
image: registry.gitlab.com/unicorn/unicornapp:1.0
ports:
- containerPort: 8080
imagePullPolicy: Always
env:
- name: MYSQL_HOST
value: mysql-prod
You can save more config into the config map & secret also based on the requirement.
Read more at : https://pushbuildtestdeploy.com/spring-boot-application.properties-in-kubernetes/
I think you did everything right, I have a similar working setup, although without a default 'demo'.
A couple of clarification from the spring boot's standpoint that might help.
application.yml can contain placeholders that can be resolved from the environment variables indeed.
Make sure that this application.yml is not "changed" (rewritten, filtered by maven whatever) during the compilation of the spring boot application artifact.
The most important: spring boot knows nothing about the k8s setup. If the environment variable exists - it will pick it. So the same could be checked even locally - define the env. variable on your local machine and run the spring boot application.
The chances are that somehow when the application runs (with the user/group) the environment variables are not accessible - check it by printing the environment variables (or this specific one) right before starting the spring boot application. Or you can do it in java in the main method:
Map<String, String> env = System.getenv();
env.entrySet().forEach(System.out::println);
I would like to configure my dockerized spring boot application using Docker Environments. The property is a map, which I configure it in application.yml like below.
spring:
kafka:
producer:
properties:
"schema.registry.url": http://schema-registry.com:8081
I tried the following, but it didn't worked,
environment:
- SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL=http://schema-registry.com:8081
How can I configure this schema.registry.url parameter from docker environment ?
Well, first of all, I would require a little bit more of information about how do you use that containerized application: Do you deploy it with docker-compose? Does it forms part of a Docker Swarm?
Depending of this, the possible solutions can vary.
Docker Swarm
For example, if you are using Docker Swarm, you can define your application.yml as a template:
application.yml.template
spring:
kafka:
producer:
properties:
"schema.registry.url": {{ env "schema_registry" }}
Then, you will have to parse that template. For that, I will suppose you have located your Spring Boot executable JAR under /usr/app in the container and that your image is named springboot-app.
docker-compose.yml
version: "3.8"
services:
springboot-app:
image: springboot-app:latest
environment:
SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL: 'http://schema-registry.com:8081'
configs:
- source: springboot-app.application.yml
target: /usr/app/config/application.yaml
mode: 0440
configs:
springboot-app.application.yml:
template_driver: golang
file: ./application.yml.template
So you can deploy now your Swarm with docker stack deploy -c docker-compose.yml springboot-app.
Or even better, if you are working in a production environment, you can separate the environment variables from the common configuration:
docker-compose.yml
version: "3.8"
services:
springboot-app:
image: springboot-app:latest
configs:
- source: springboot-app.application.yml
target: /usr/app/config/application.yaml
mode: 0440
configs:
springboot-app.application.yml:
template_driver: golang
file: ./application.yml.template
docker-compose.dev.yml
version: "3.8"
services:
springboot-app:
environment:
SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL: 'http://schema-registry.com:8081'
And deploy it as docker stack deploy -c docker-compose.yml -c docker-compose.dev.yml springboot-app.
Docker Compose
Since you mentioned in a lately comment that you are using docker-compose, the way of working with isn't the same.
First of all, not all the properties in Spring can be overridden in the Docker Compose file, only the ones that you can pass to Maven at the time of building or starting the application.
Also, it seems you have wrongly defined the environment Property, since normally all those that you provide you should change the dots '.' by underscores '_', but anyway, since normally the configuration of a Kafka Producer goes further than just defining an URL, I would use the profiles feature of Spring.
You can create several profiles with the configuration combinations that you want, and inform Spring via Compose which one you want to use. Let's see an example.
application.yml
spring:
config:
activate:
on-profile: "development"
kafka:
producer:
properties:
"schema.registry.url": https://kafka-dev-endpoint.com
---
spring:
config:
activate:
on-profile: "production"
kafka:
producer:
properties:
"schema.registry.url": https://kafka-prod-endpoint.com
and finally then:
docker-compose.yml
environment:
- SPRING_PROFILES_ACTIVE=development
If you wanna check further, you have more information about that here: https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-set-active-spring-profiles
I am developing a web application by using the Spring Boot, Spring Cloud Config and Docker. There are 3 projects. One is springboot web, Spring Cloud Config Server and MySql Database. I have use the Spring Boot Web with Themeleaf. It's basically perform a CRUD operation. In the development phase it's working fine. But when I deploy it in the Docker
Spring-Boot webapp container(springboot-thymeleaf-web-crud) is giving me the following error--
APPLICATION FAILED TO START
Description:
Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured.
Reason: Failed to determine a suitable driver class
Action:
Consider the following:
If you want an embedded database (H2, HSQL or Derby), please put it on the classpath.
If you have database settings to be loaded from a particular profile you may need to activate it (no profiles are currently active).
I have created 3 container on Docker Compose file.
version: '3'
services:
springboot-thymeleaf-web-crud:
image: numery/springboot-thymeleaf-web-crud:latest
networks:
- employee-network
ports:
- 8080:8080
depends_on:
- employee-database
- spring-cloud-config-server-employee
spring-cloud-config-server-employee:
image: numery/spring-cloud-config-server-employee:latest
networks:
- employee-network
ports:
- 8888:8888
employee-database:
image: mysql:5
networks:
- employee-network
environment:
- MYSQL_ROOT_PASSWORD=rootpassword
- MYSQL_DATABASE=employee_directory
volumes:
- /home/numery/Docker-Database/employee_directory:/var/lib/mysql
networks:
employee-network:
SpringBoot Web Application.properties is given below
spring.application.name=employee-service-mysql
server.port=8080
spring.cloud.config.uri=http://spring-cloud-config-server-
employee:8888
spring.profiles.active=dev
SpringBoot Config Server is providing the following properties--
spring.datasource.url: jdbc:mysql://employee-
database:3306/employee_directory?
useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
spring.datasource.username: root
spring.datasource.password: rootpassword
spring.datasource.validationQuery: SELECT 1
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.tomcat.max-wait: 20000
spring.tomcat.max-active: 50
spring.tomcat.max-idle: 20
spring.tomcat.min-idle: 15
spring.jpa.properties.hibernate.dialect:
org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.show-sql: true
spring.jpa.format-sql: true
spring.jpa.database: MYSQL
spring.jpa.hibernate.ddl-auto: create
Out of these 3 containers Mysql and Spring Cloud Config container is working fine. But for the Spring Boot Webapp(springboot-thymeleaf-web-crud) is exiting by giving the error above.
Note: I use the same datasouce(Spring Data JPA) configuration on some other SpringBoot Rest API. Which are working fine. But this is the first time I am using SpringBoot Web. Do I need to explicitly define any configuration on the data-source.
Please help!!
The datasource URL, that is spring.datasource.url: jdbc:mysql://employee- may be wrong.
OR Should add your datasource url to your Application.properties file
The url should be in the format jdbc:oracle:thin:#hostname:portNumber/schemaName
For further details and clarification, Not using the hostname and port in jdbc url
I perform the following task to solve the problem--
1) We should have up and running the mysql
container.
2) Inspect mysql container which ip running--
docker inspect mysqlcontainer | grep IP
3) Add the IP of that mysql container to my
springboot web app--
"spring.datasource.url:
jdbc:mysql://172.20.0.2:3306/employee_directory?
useSSL=false&serverTimezone
=UTC&useLegacyDatetimeCode=false"
4) run mvn clean, mvn install and build the
SpringBoot Web image.
5) If we down the container we need to perform the
same task again. Because each time Mysql
container obtain the different ip.