Spring "Failed to bind properties under..." on Kubernetes cluster - spring

I've got a problem with my spring configuration on kubernetes.
I have given yaml configuration for my spring boot application.
spring:
application:
name: "Application name"
banner:
location: classpath:banner.txt
server:
port: 8081
some:
data:
active: true
# other data
So I created a ConfigMap for my app in that way
kubectl create --save-config configmap spring-application-config --from-file=application.yml --namespace dev -o yaml --dry-run | kubectl apply -f -
But unfortunately I receive that error from spring:
Failed to bind properties under 'some' to my.app.ApplicationConfig:
I've checked ApplicationConfig class and every property is mapped properly and also tested it locally and this problem only appears on kubernetes.

Related

How to build a Docker image for client service of Spring Cloud Config Server

I have built a Docker image for cloud config service. I have another service employee for which I want to build a Docker image. This client service will read data source configuration properties from the config server
I am able to connect the employee service successfully with config service on localhost without Docker. The problem is only that I am not able to build a Docker image for the employee service.
On running mvn package dockerfile:build command for the employee service I get the following error
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured.
Reason: Failed to determine a suitable driver class
Below is the bootstrap.yml file located at src/main/resources directory
spring:
application:
name: employee
profiles:
active: dev
cloud:
config:
uri: http://configserver:8071
Config service's bootstrap.yml file
spring:
application:
name: config-server
profiles:
active: native,dev
cloud:
config:
server:
#Local configuration: This locations can either of classpath or locations in the filesystem.
native:
#Reads from a specific filesystem folder
search-locations: classpath:/config
server:
port: 8071
employee.properties file in config directory
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.jpa.database-platform=org.hibernate.dialect.MySQL5InnoDBDialect
employee-dev.properties file
spring.datasource.url=jdbc:mysql://db:3306/employee?allowPublicKeyRetrieval=true&useSSL=false
spring.datasource.username=root
spring.datasource.password=root
This is probably because the container of config server with hostname configserver is not up. I am not sure what hostname I should provide. Even if I want to build the image of employee service by starting the container of config server, it won't be on configserver host but on localhost. This is probably why the build is failing?
I want to build the Docker image of employee service so that its container can be fired up using the docker-compose up command
Edit: This question is similar but I already have the spring-cloud-starter-config dependency added in my client service and it still gives me the same error on build
Edit 2
You don't have to separately build a Docker image for the client service at all!
When I ran docker-compose up command I noticed the following line in output
c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://configserver:8071
But I was still not sure if it was actually reading the data from the config server. So I purposely added incorrect database credentials in config server properties file for my service and the service startup failed! Now I am sure that my employee service is reading the data from the config server
I am still not sure how can I achieve creating a new Docker image for my employee service.
Did you try changing the URL param db to actual host of vhostname ?
db =change it to IP addess or accessable load balancer URL
spring.datasource.url=jdbc:mysql://db:3306/employee?allowPublicKeyRetrieval=true&useSSL=false

Invalid config server configuration

I am on learning stage of Spring Cloud & using spring version 2.4.3 and spring-cloud-version
2020.0.1 and I created two property file
application.yml
spring:
application:
name: cloud-server
server:
port: 8888
bootstrap.yml
spring:
cloud:
config:
server:
git:
uri: https://github.com/************/insurance-config-server
default-label: main
but still I got following error
***************************
APPLICATION FAILED TO START
***************************
Description:
Invalid config server configuration.
Action:
If you are using the git profile, you need to set a Git URI in your configuration. If you are using a native profile and have spring.cloud.config.server.bootstrap=true, you need to use a composite configuration.
How to solve this kind of error?
Did you have a bootstrap.yml file?
I had the same problem. My project did not have a bootstrap.yml, but I specified a dev activity file in the IDEA startup configuration, which caused the same error as you did. The error could not be found bootstrap.yml on startup. After deleting dev in IDEA startup configuration, it can start normally.
If you are following the example in the book you referenced then I will suggest you move this
spring:
cloud:
config:
server:
git:
uri: https://github.com/************/insurance-config-server
default-label: main
to application.yml
That is what I did and it worked for me

Spring application properties (map) from docker environment

I would like to configure my dockerized spring boot application using Docker Environments. The property is a map, which I configure it in application.yml like below.
spring:
kafka:
producer:
properties:
"schema.registry.url": http://schema-registry.com:8081
I tried the following, but it didn't worked,
environment:
- SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL=http://schema-registry.com:8081
How can I configure this schema.registry.url parameter from docker environment ?
Well, first of all, I would require a little bit more of information about how do you use that containerized application: Do you deploy it with docker-compose? Does it forms part of a Docker Swarm?
Depending of this, the possible solutions can vary.
Docker Swarm
For example, if you are using Docker Swarm, you can define your application.yml as a template:
application.yml.template
spring:
kafka:
producer:
properties:
"schema.registry.url": {{ env "schema_registry" }}
Then, you will have to parse that template. For that, I will suppose you have located your Spring Boot executable JAR under /usr/app in the container and that your image is named springboot-app.
docker-compose.yml
version: "3.8"
services:
springboot-app:
image: springboot-app:latest
environment:
SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL: 'http://schema-registry.com:8081'
configs:
- source: springboot-app.application.yml
target: /usr/app/config/application.yaml
mode: 0440
configs:
springboot-app.application.yml:
template_driver: golang
file: ./application.yml.template
So you can deploy now your Swarm with docker stack deploy -c docker-compose.yml springboot-app.
Or even better, if you are working in a production environment, you can separate the environment variables from the common configuration:
docker-compose.yml
version: "3.8"
services:
springboot-app:
image: springboot-app:latest
configs:
- source: springboot-app.application.yml
target: /usr/app/config/application.yaml
mode: 0440
configs:
springboot-app.application.yml:
template_driver: golang
file: ./application.yml.template
docker-compose.dev.yml
version: "3.8"
services:
springboot-app:
environment:
SPRING_KAFKA_PRODUCER_PROPERTIES_SCHEMA.REGISTRY.URL: 'http://schema-registry.com:8081'
And deploy it as docker stack deploy -c docker-compose.yml -c docker-compose.dev.yml springboot-app.
Docker Compose
Since you mentioned in a lately comment that you are using docker-compose, the way of working with isn't the same.
First of all, not all the properties in Spring can be overridden in the Docker Compose file, only the ones that you can pass to Maven at the time of building or starting the application.
Also, it seems you have wrongly defined the environment Property, since normally all those that you provide you should change the dots '.' by underscores '_', but anyway, since normally the configuration of a Kafka Producer goes further than just defining an URL, I would use the profiles feature of Spring.
You can create several profiles with the configuration combinations that you want, and inform Spring via Compose which one you want to use. Let's see an example.
application.yml
spring:
config:
activate:
on-profile: "development"
kafka:
producer:
properties:
"schema.registry.url": https://kafka-dev-endpoint.com
---
spring:
config:
activate:
on-profile: "production"
kafka:
producer:
properties:
"schema.registry.url": https://kafka-prod-endpoint.com
and finally then:
docker-compose.yml
environment:
- SPRING_PROFILES_ACTIVE=development
If you wanna check further, you have more information about that here: https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-set-active-spring-profiles

Kubernetes Endpoint created for Kafka but not reflecting in POD

In Kubernetes cluster I have created Endpoint pointing to Kafka cluster. Endpoint created successfully.
Name - kafka
Endpoint - X.X.X.X:9092
In my Spring Boot application's deployment yaml I have kept environment variable BROKER_IP. For this environment variable I have pointed:
env:
- name: BROKER_IP
value: kafka
The POD is in Error state. In my bootstrap-server I am getting kafka and not the actual Endpoint that was created. Any thoughts?
UPDATE - Just tried kafka:9092 and it worked. So wondering does the ENDPOINT maps to IP only and not the Port? Is my understanding correct??
Is it possible that you forgot to create the Service object matching the Endpoints? Because you are providing the ip-port pairs yourself the Service would need to be selectorless.
This works for me:
kind: Endpoints
apiVersion: v1
metadata:
name: kafka
subsets:
- addresses: [{ip: "1.2.3.4"}]
ports: [{port: 9092}]
---
kind: Service
apiVersion: v1
metadata:
name: kafka
spec:
ports: [{port: 9092}]
Testing it:
$ kubectl run kafka-dns-test --image=busybox --attach --rm --restart=Never -- nslookup kafka
If you don't see a command prompt, try pressing enter.
Server: 10.96.0.10
Address: 10.96.0.10:53
Name: kafka.default.svc.cluster.local
Address: 10.96.220.40
Successful lookup, ignore extra *** Can't find xxx: No answer messages
Also, because there is a Service object you get some environment variables in your Pods (without having to declare them):
KAFKA_PORT='tcp://10.96.220.40:9092'
KAFKA_PORT_9092_TCP='tcp://10.96.220.40:9092'
KAFKA_PORT_9092_TCP_ADDR='10.96.220.40'
KAFKA_PORT_9092_TCP_PORT='9092'
KAFKA_PORT_9092_TCP_PROTO='tcp'
KAFKA_SERVICE_HOST='10.96.220.40'
KAFKA_SERVICE_PORT='9092'
But the most flexible way to use a Service is still to use the dns name (kafka in this case).

Read values from consul while bootstrap spring boot

I have question is there any way to retrieve certain values and inject them to bootstrap.yml while application is coming up.
I have configuration file like this:
spring:
application:
name: myApp
cloud:
consul:
enabled: true
host: localhost
port: 8500
config:
enabled: true
datasource:
url: jdbc:oracle:thin:#localhost:1111:XXXX
username: ${nameOfVariable1}
password: ${nameOfVariable1}
driver-class-name: oracle.jdbc.OracleDriver
For example, I need to configure embedded tomcat port, or DB credentials, I don't want to put it hardcoded in .yml properties file, instead I want to put some variable name in .yml so Spring will go and bring value from Consul. Is it possible?
You can use Spring Cloud Consul Config project that helps to load configuration into the Spring Environment during the special "bootstrap" phase.
3 steps:
add pom dependency: spring-cloud-starter-consul-config
enable consul config: spring.cloud.consul.config.enabled=true
add some config in consul kv in specific folder, such as key: config/testConsulApp/server.port, value:8081
and then start the sample web app, it will listen 8081.
more detail at spring cloud consul doc.
and demo code here

Resources