Spring application properties (map) from docker environment - spring

I would like to configure my dockerized spring boot application using Docker Environments. The property is a map, which I configure it in application.yml like below.
spring:
kafka:
producer:
properties:
"schema.registry.url": http://schema-registry.com:8081
I tried the following, but it didn't worked,
environment:
- SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL=http://schema-registry.com:8081
How can I configure this schema.registry.url parameter from docker environment ?

Well, first of all, I would require a little bit more of information about how do you use that containerized application: Do you deploy it with docker-compose? Does it forms part of a Docker Swarm?
Depending of this, the possible solutions can vary.
Docker Swarm
For example, if you are using Docker Swarm, you can define your application.yml as a template:
application.yml.template
spring:
kafka:
producer:
properties:
"schema.registry.url": {{ env "schema_registry" }}
Then, you will have to parse that template. For that, I will suppose you have located your Spring Boot executable JAR under /usr/app in the container and that your image is named springboot-app.
docker-compose.yml
version: "3.8"
services:
springboot-app:
image: springboot-app:latest
environment:
SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL: 'http://schema-registry.com:8081'
configs:
- source: springboot-app.application.yml
target: /usr/app/config/application.yaml
mode: 0440
configs:
springboot-app.application.yml:
template_driver: golang
file: ./application.yml.template
So you can deploy now your Swarm with docker stack deploy -c docker-compose.yml springboot-app.
Or even better, if you are working in a production environment, you can separate the environment variables from the common configuration:
docker-compose.yml
version: "3.8"
services:
springboot-app:
image: springboot-app:latest
configs:
- source: springboot-app.application.yml
target: /usr/app/config/application.yaml
mode: 0440
configs:
springboot-app.application.yml:
template_driver: golang
file: ./application.yml.template
docker-compose.dev.yml
version: "3.8"
services:
springboot-app:
environment:
SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL: 'http://schema-registry.com:8081'
And deploy it as docker stack deploy -c docker-compose.yml -c docker-compose.dev.yml springboot-app.
Docker Compose
Since you mentioned in a lately comment that you are using docker-compose, the way of working with isn't the same.
First of all, not all the properties in Spring can be overridden in the Docker Compose file, only the ones that you can pass to Maven at the time of building or starting the application.
Also, it seems you have wrongly defined the environment Property, since normally all those that you provide you should change the dots '.' by underscores '_', but anyway, since normally the configuration of a Kafka Producer goes further than just defining an URL, I would use the profiles feature of Spring.
You can create several profiles with the configuration combinations that you want, and inform Spring via Compose which one you want to use. Let's see an example.
application.yml
spring:
config:
activate:
on-profile: "development"
kafka:
producer:
properties:
"schema.registry.url": https://kafka-dev-endpoint.com
---
spring:
config:
activate:
on-profile: "production"
kafka:
producer:
properties:
"schema.registry.url": https://kafka-prod-endpoint.com
and finally then:
docker-compose.yml
environment:
- SPRING_PROFILES_ACTIVE=development
If you wanna check further, you have more information about that here: https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-set-active-spring-profiles

Related

Passing list of YAML properties from Docker compose to Spring Boot application

My spring Boot application contains the following list of properties in application.yml:
hs:
targets:
- name: system_a
url: https://system-a
username: username-a
password: password-a
- name: system_b
url: https://system-b
username: username-b
password: password-b
Instead of defining them there, I'd like to pass them from Docker Compose via docker-compose.yml.
my-app:
image: my-image
ports:
- 9002:9002
environment:
SPRING_PROFILES_ACTIVE: dev
HS_TARGETS_NAME: system-a
HS_TARGETS_URL: https://system-a
HS_TARGETS_USERNAME: username_a
HS_TARGETS_PASSWORD: password_a
HS_TARGETS_NAME: system-b
HS_TARGETS_URL: https://system-b
HS_TARGETS_USERNAME: username_b
HS_TARGETS_PASSWORD: password_b
But I doubt that this will work because the variables all have the same name. Is there any way to do that or do I have to restructore my Spring Boot properties?
Pass the properties as
HS_TARGETS_0_NAME: system-a
HS_TARGETS_0_URL: https://system-a
HS_TARGETS_0_USERNAME: username_a
HS_TARGETS_0_PASSWORD: password_a
HS_TARGETS_1_NAME: system-b
HS_TARGETS_1_URL: https://system-b
HS_TARGETS_1_USERNAME: username_b
HS_TARGETS_1_PASSWORD: password_b
Tested with Spring Boot 2.1.17

How do I keep the Eureka server url dynamic in the Eureka Client while using docker?

I am using the fabric8 docker-maven-plugin to build image for my Spring boot microservices.
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
The problem is that while running the application in docker containers I have to specify the Eureka Server Container name to Eureka Client. But if I run it directly as a "Spring Boot APP" I have to use "Localhost:8761/Eureka". Is there a way to make it work both with/without docker something like given below ?
eureka:
client:
service-url:
defaultZone: ${EUREKA_SERVER:http://localhost:8761/eureka}
I am not able to pass the value of "EUREKA_SERVER" from the fabrib8 plugin. I have tried the below code to pass the value but it does not work.
<docker.env.JAVA_OPTS>-DEUREKA_SERVER=http://discovery:8761/eureka</docker.env.JAVA_OPTS>
Spring can pickup Environment Variables. So if you add Environment Variables to the Docker Container that Spring Boot is running in, they will work. This avoids the need to provide a static URL up front.
If you use Docker Compose, it could look like this:
services:
eureka:
image: springcloud/eureka
container_name: eureka
ports:
- "8761:8761"
networks:
- "discovery"
environment:
- EUREKA_INSTANCE_PREFERIPADDRESS=true
spring:
build:
context: .
dockerfile: ./src/main/docker/Dockerfile
depends_on:
- eureka
container_name: spring
ports:
- "8080:8080"
networks:
- "discovery"
environment:
- EUREKA_SERVICE_URL=http://eureka:8761 // This overrides your Spring Property
- EUREKA_INSTANCE_PREFER_IP_ADDRESS=true
- LOGGING_FILE=/tmp/admin.log
Note: Since Environment Variables are not YAML, you need to change the format a bit. https://docs.spring.io/spring-boot/docs/1.5.5.RELEASE/reference/html/boot-features-external-config.html#boot-features-external-config-relaxed-binding

Read docker env variable inside Spring application.yaml

I'm setting an environment variable inside my docker-compose.yaml file and want to use that variable's value inside my Spring Boot's application.yaml. I was told that doing something like
app:
auth:
tokenSecret: tokensecretvaluehere
tokenExpirationMsec: 864000000
oauth2:
sso:
url: ${SSO_URL}
(where SSO_URL is defined in my docker-compose.yaml) in my Spring application.yaml. However, this causes an error when I run docker-compose up --build because it can't find that variable (error is like: Could not resolve placeholder SSO_URL in value "${SSO_URL}"). This is an example of what my docker-compose.yaml:
api:
restart: always
ports:
- "8080:8080"
links:
- redis
- db
environment:
- SERVER_SERVLET_SESSION_COOKIE_DOMAIN=localhost
- SSO_URL=myvaluehere
I was asked to not uses the System.getenv functions in Java and instead set the variable like above. From there I would just use the #Value annotation to get it in my Java code as like below:
#Value("${app.oauth2.sso.url}")
private String ssoUrl;
This is more of the application.yaml:
heb:
togglr:
jwt:
secret:
id: 101
session:
seconds: 600
tokenheader: X-TOGGLR-TOKEN
logging:
level:
com:
heb: debug
default: debug
path: logs
server:
error:
whitelabel:
enabled: false
port: 8080
servlet:
context-path: /togglr-api
use-forward-headers: true
spring:
application:
name: togglr_api
freemarker:
enabled: false
groovy:
template:
enabled: false
jmx:
enabled: false
main:
banner-mode: 'off'
thymeleaf:
cache: false
security:
oauth2:
client:
registration:
github:
clientId:
clientSecret:
redirectUri:
scope:
- user:email
- read:user
app:
auth:
tokenSecret:
tokenExpirationMsec: 864000000
oauth2:
sso:
url: ${SSO_URL}
In general spring boot applications are able to read the Environment variables accessible in the docker container. Its stated in the documentation (see item 10 in the list at the very beginning of the document).
So the problem might be elsewhere:
It might be a typo in the question, but if you're using application.yaml as opposed to application properties, then you should have something like:
sso:
url: ${SSO_URL}
Make sure, that the env variable SSO_URL is indeed accessible in the container even before you start the spring boot application. In java (for debugging purposes only) you can do something like:
#SpringBootApplication
public class MyApp {
public static void main(String [] args) {
System.out.println(System.getenv("SSO_URL"));
SpringApplication.run(MyApp.class);
}
}
Solution was to not use an underscore character in the variable name.
I feel what you are missing is the build context within the docker-compose file. Or, you have multiple profile based application.yml and not the correct profile is being set.
Below is the working code / config. So we are creating a springboot application from docker-compose, where docker-compose builds the image of springboot application and passes the required environment variables.
Snippet from Spring application.yml where we are using the env variable named API_BASE and TEST_API_PATH
third-party-api:
base-url: ${API_BASE}
test-api-path: ${TEST_API_PATH}
Below is the snippet from docker-compose.yml
my-app:
image: my-app-image-name
build:
dockerfile: Dockerfile
context: .
ports:
- '9080:8080'
environment:
API_BASE: http://mock-api:8080
TEST_API_PATH: /someTestApiPath
SPRING_PROFILES_ACTIVE: dev
Docker file of my application is pretty simple.
FROM openjdk:17-alpine
ARG JAR_FILE=build/libs/*.jar
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
Dockerfile and docker-compose.yml are on the same hierarcy, if there is any difference in your structure then it should reflect in below config of docker-compose.yml
build:
dockerfile: Dockerfile

Docker Stack "configs" with Spring Boot 2

This is a fairly simple question I think, however I haven't seen too many examples or really anything that explains the connection between using docker configs (v 3.3+) and loading that config into Spring Boot for reference.
sample docker-stack.yml
version: '3.3'
services:
test-service:
image: myrepo/test-service:1.0.0
configs:
- service-config
networks:
- test-network
configs:
service-config:
external:true
networks:
test-network:
sample swarm "service-config".
I entered this as a new "configs" entry in Portainer.
services:
test-service:
key1: sample value
key2: sample two
What I'm trying to do it load this config into Spring such that I can reference the values from this config within a component.
Either via #ConfigurationProperties
#ConfigurationProperties("services.test-service")
public MyBeanName myBean() { return new MyBeanName(); }
or via the #Value:
#Value("${services.test-service.key1}")
private String key1;
How can I get this docker "configs" configuration loaded into Spring. This has got to be simple enough.. lol. Thanks!
Sorry for the delay in responding to this question, or at least posting the solution...
It took a bit more digging into how configs work with docker, however as it turns out you need to specify a target for the "config" held within your "configs" entries in your swarm cluster, and then map it into your container, and load as an external config to your spring application. In my case, I didn't want to override the the application.yml in my spring boot app, just wanted to pick up additional configs. So I went with the setting:
--spring.config.additional-location=file:/configs/sample-config.yml
Lets say I create a docker configs entry with the name "sample-config" and having the following data:
Configs Entry => "sample-config"
services:
test-service:
key1: sample value
key2: sample two
Then on my compose/stack file I need to reference the "configs" entry, and provide a target file that corresponds to the file I specified in my "spring.config.additional-location" setting.
version: '3.3'
services:
test-service:
image: myrepo/test-service:1.0.0
configs:
- source: sample-config
target: /configs/sample-config.yml
networks:
- test-network
configs:
sample-config:
external:true
networks:
test-network:
In my Dockerfile then, I'd specify the following to essentially load the "sample-config" configs entry when starting the jar/app:
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar", "--spring.config.additional-location=file:/configs/sample-config.yml"]
This allowed me to access the configuration entries, loaded externally to my spring application, with the #Value and #ConfigurationProperties annotations.

Configuring ssl in rabbitmq.config using rabbitmq docker image

My goal is to set rabbitmq with ssl support, which was achieved previously using below rabbitmq.config file, which resides in host's /etc/rabbitmq path.
Now I want to be able to configure other rabbitmq user and password than defaults guest guest.
I'm using rabbitmq docker image with following docker-compose configuration:
version: '2'
services:
rabbitmq:
build: ./rabbitmq
ports:
- "8181:8181"
expose:
- "15672"
- "8181"
volumes:
- /etc/rabbitmq:/etc/rabbitmq
environment:
RABBITMQ_DEFAULT_USER: user123
RABBITMQ_DEFAULT_PASS: 1234
Rabbitmq config:
[{rabbit,
[
{loopback_users, []},
{heartbeat,0},
{ssl_listeners, [8181]},
{ssl_options, [{cacertfile, "/etc/rabbitmq/ca/cacert.pem"},
{certfile, "/etc/rabbitmq/server/cert.pem"},
{keyfile, "/etc/rabbitmq/server/key.pem"},
{verify,verify_none},
{fail_if_no_peer_cert,false}]}
]}
].
Rabbitmq dockerfile:
from rabbitmq:management
#and some certificate generating logic
I noticed that once upon adding environment section, current rabbitmq.config file is overriden with auto generated configuration possibly by docker-entrypoint.sh file.
For building configuration using the certs I found environment variables that can do this (look here).
However didn't found any reference for defining ssl_listeners section with its port, as seen in below rabbitmq.config
My question is: how can I create the exact configuration as mentioned below using env variables OR how can I remain with mine rabbitmq.config defining rabbitmq with new user and password in some dynamic way (maybe templating the config file)?
Try this
version: '2'
services:
rabbitmq:
build: ./rabbitmq
ports:
- "8181:8181"
expose:
- "15672"
- "8181"
volumes:
- /etc/rabbitmq:/etc/rabbitmq
command: rabbitmq-server
entrypoint: ""
environment:
RABBITMQ_DEFAULT_USER: user123
RABBITMQ_DEFAULT_PASS: 1234
This will override the docker-entrpoint and just run the rabbitmq server. Now the ./docker-entrypoint.sh sets certain environment variables also. Which may be needed in your case. So to make sure you have everything needed

Resources