Overriding/Externalizing application.properties into docker-compose.yml - spring

There are plenty of answers already but none of them solves my problem, please help!
version: '3.8'
services:
mysql:
image: mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=****
- MYSQL_DATABASE=portfolio
- MYSQL_USER=manager
- MYSQL_PASSWORD=******
volumes:
- mysql-data:/var/lib/mysql
ports:
- "8010:3306"
api:
build: ./api
ports:
- "8005:8080"
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://mysql:3306/portfolio
- SPRING_DATASOURCE_USERNAME=manager
- SPRING_DATASOURCE_PASSWORD=******
- SECRET_REGISTRATION_KEY=&&&&&&
depends_on:
- mysql
volumes:
mysql-data:
Inside my application.properties:
spring.datasource.url=jdbc:mysql://localhost:3306/portfolio_dev
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=manager
spring.datasource.password=$#(&%(#%(&%
secret.registration.key=[RANDOM_KEY]
Expected behavior: environment variable like SPRING_DATASOURCE_URL would override spring.datasource.url in application.properties in Spring Boot project.
Actual behavior: api container with Spring Boot inside does not pick up the environment variable in docker-compose.yml, defaulting to the ones inside application.properties.
Reproduction: I ran docker-compose up --build to make sure the JAR files are rebuilt and the mysql server runs fine.

It is really a weird thing when a developer found his solution 10 minutes after posting his desperate StackOverflow question.
Answer: https://stackoverflow.com/a/64940105/15119843
The Spring Boot container starts up right after the mysql starts working, but not ready for connection, so the SB container could not connect to that one.
EDIT: Add restart: on-failure to the Spring Boot container. It restarts the api whenever a failure (that stops the process) as such occur.

Related

How can I dockerize my springboot application which has multiple datasources?

I have a springboot application which uses a postgresql database and a mongoDB database , I have been able to correctly configure them but now when I want to dockerize my application to later deploy it on a Kubernetes cluster, I am completely clueless. Most of the youtube tutorials and articles are on how to dockerize simple springboot applications or springboot applications that use only one database, thus any input on how I can proceed to dockerize my application would be really appreciated!
Edit:
I am following this tutorial -
https://www.section.io/engineering-education/running-a-multi-container-springboot-postgresql-application-with-docker-compose/
Here in the docker-compose.yml file-
version: '3.1'
services:
API:
image: 'blog-api-docker.jar'
ports:
- "8080:8080"
depends_on:
PostgreSQL:
condition: service_healthy
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://PostgreSQL:5432/postgres
- SPRING_DATASOURCE_USERNAME=postgres
- SPRING_DATASOURCE_PASSWORD=password
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
PostgreSQL:
image: postgres
ports:
- "5432:5432"
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_USER=postgres
- POSTGRES_DB=postgres
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
Only one postgreSQL datasource is defined ,in my project with a similar postgreSQL datasource as given in the tutorial I am also using a mongoDB database which is running on atlas.
I am also including my application.properties file for your reference-
spring.ds-psql.datasource.jdbcUrl=jdbc:postgresql://localhost:5432/devicestatspsql
spring.ds-psql.datasource.username=
spring.ds-psql.datasource.password=
spring.data.mongodb.users-mongo-atlas.uri=*mongodb database url here*
spring.jpa.generate-ddl=true
spring.jpa.show-sql=true
So I just need to know what changes are required in my docker-compose.yml file to accommodate this mongodb database in the docker image
You can use the Kubernetes Configmap and Secret as of now to store the configuration of your application.
Configmap and Secret are mostly for storing configurations like database connection strings, usernames, and passwords.
You can create different configuration maps as per requirement for Dev,Stag and Prod then inject the specific to your deployment so the application will get those values from either .env file or from OS environment.
Here's the reference article.

Cannot connect my springboot docker with my cassandra docker

I have some issues to dockerize my Spring boot Cassandra application.
When I use Idea to run my project it's work fine correctly but when I turn my application into a docker container I have this issues :
"RetryingCassandraClusterFactoryBean : All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [localhost/127.0.0.1:9042] Cannot connect))"
my application.properties:
spring.data.cassandra.keyspace-name=keyspace
spring.data.cassandra.contact-points=127.0.0.1
spring.data.cassandra.port=9042
spring.data.cassandra.schema-action=create_if_not_exists
server.port=8081
my cassandra :
cassandradb:
image: "cassandra:latest"
container_name: "cassandradb"
ports:
- "9042:9042"
- "7191:7191"
- "7001:7001"
- "9160:9160"
- "7000:7000"
environment:
CASSANDRA_CLUSTER: "myCluster"
CASSANDRA_ENDPOINT_SNITCH: "GossipingPropertyFileSnitch"
CASSANDRA_DC: "data"
CASSANDRA_LISTEN_ADDRESS: "auto"
photos:
image: "photos:latest"
container_name: "photo"
ports:
- "8081:8081"
links:
- "cassandradb"
Error happens, becouse your spring cant reach cassandradb. You should connect or via host ip, or usecassandradb` host, anc configure docker-compose.yml links part
I develop a simple example about the CRUD spring boot application by Cassandra.
You can check it in github. here are a few differences in the comparison of mine.

How do I keep the Eureka server url dynamic in the Eureka Client while using docker?

I am using the fabric8 docker-maven-plugin to build image for my Spring boot microservices.
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
The problem is that while running the application in docker containers I have to specify the Eureka Server Container name to Eureka Client. But if I run it directly as a "Spring Boot APP" I have to use "Localhost:8761/Eureka". Is there a way to make it work both with/without docker something like given below ?
eureka:
client:
service-url:
defaultZone: ${EUREKA_SERVER:http://localhost:8761/eureka}
I am not able to pass the value of "EUREKA_SERVER" from the fabrib8 plugin. I have tried the below code to pass the value but it does not work.
<docker.env.JAVA_OPTS>-DEUREKA_SERVER=http://discovery:8761/eureka</docker.env.JAVA_OPTS>
Spring can pickup Environment Variables. So if you add Environment Variables to the Docker Container that Spring Boot is running in, they will work. This avoids the need to provide a static URL up front.
If you use Docker Compose, it could look like this:
services:
eureka:
image: springcloud/eureka
container_name: eureka
ports:
- "8761:8761"
networks:
- "discovery"
environment:
- EUREKA_INSTANCE_PREFERIPADDRESS=true
spring:
build:
context: .
dockerfile: ./src/main/docker/Dockerfile
depends_on:
- eureka
container_name: spring
ports:
- "8080:8080"
networks:
- "discovery"
environment:
- EUREKA_SERVICE_URL=http://eureka:8761 // This overrides your Spring Property
- EUREKA_INSTANCE_PREFER_IP_ADDRESS=true
- LOGGING_FILE=/tmp/admin.log
Note: Since Environment Variables are not YAML, you need to change the format a bit. https://docs.spring.io/spring-boot/docs/1.5.5.RELEASE/reference/html/boot-features-external-config.html#boot-features-external-config-relaxed-binding

Running Sonarqube with docker-compose using bind mount volumes

I’m trying to run Sonarqube in a Docker container on a Centos 7 server using docker-compose. Everything works as expected using named volumes as configured in this docker-compose.yml file:
version: "3"
services:
sonarqube:
image: sonarqube
ports:
- "9000:9000"
networks:
- sonarnet
environment:
- sonar.jdbc.url=jdbc:postgresql://db:5432/sonar
volumes:
- sonarqube_conf:/opt/sonarqube/conf
- sonarqube_data:/opt/sonarqube/data
- sonarqube_extensions:/opt/sonarqube/extensions
- sonarqube_bundled_plugins:/opt/sonarqube/lib/bundled-plugins
db:
image: postgres
networks:
- sonarnet
environment:
- POSTGRES_USER=sonar
- POSTGRES_PASSWORD=sonar
volumes:
- postgresql:/var/lib/postgresql
- postgresql_data:/var/lib/postgresql/data
networks:
sonarnet:
driver: bridge
volumes:
sonarqube_conf:
sonarqube_data:
sonarqube_extensions:
sonarqube_bundled_plugins:
postgresql:
postgresql_data:
However, my /var/lib/docker/volumes directory is not large enough to house the named volumes. So, I changed the docker-compose.yml file to use bind mount volumes as shown below.
version: "3"
services:
sonarqube:
image: sonarqube
ports:
- "9000:9000"
networks:
- sonarnet
environment:
- sonar.jdbc.url=jdbc:postgresql://db:5432/sonar
volumes:
- /data/sonarqube/conf:/opt/sonarqube/conf
- /data/sonarqube/data:/opt/sonarqube/data
- /data/sonarqube/extensions:/opt/sonarqube/extensions
- /data/sonarqube/bundled_plugins:/opt/sonarqube/lib/bundled-plugins
db:
image: postgres
networks:
- sonarnet
environment:
- POSTGRES_USER=sonar
- POSTGRES_PASSWORD=sonar
volumes:
- /data/postgresql:/var/lib/postgresql
- /data/postgresql_data:/var/lib/postgresql/data
networks:
sonarnet:
driver: bridge
However, after running docker-compose up -d, the app starts up but none of the bind mount volumes are written to. As a result, the Sonarqube plugins are not loaded and the sonar postgreSQL database is not initialized. I thought it may be a selinux issue, but I temporarily disabled it with no success. I’m unsure what to look at next.
I think my answer from "How to persist configuration & analytics across container invocations in Sonarqube docker image" would help you as well.
For good measure I have also pasted it in here:
.....
Notice this line SONARQUBE_HOME in the Dockerfile for the docker-sonarqube image. We can control this environment variable.
When using docker run. Simply do:
txt
docker run -d \
...
...
-e SONARQUBE_HOME=/sonarqube-data
-v /PERSISTENT_DISK/sonarqubeVolume:/sonarqube-data
This will make Sonarqube create the conf, data and so forth folders and store data therein. As needed.
Or with Kubernetes. In your deployment YAML file. Do:
txt
...
...
env:
- name: SONARQUBE_HOME
value: /sonarqube-data
...
...
volumeMounts:
- name: app-volume
mountPath: /sonarqube-data
And the name in the volumeMounts property points to a volume in the volumes section of the Kubernetes deployment YAML file.
This again will make Sonarqube use the /sonarqube-data mountPath for creating extenions, conf and so forth folders, then save data therein.
And voila your Sonarqube data is thereby persisted.
I hope this will help others.
N.B. Notice that the YAML and Docker run examples are not exhaustive. They focus on the issue of persisting Sonarqube data.
Try it out BobC and let me know.
Have a great day.
The below code will help you in a single command I hope so.
Create a new docker-compose file named as docker-compose.yaml,
version: "3"
services:
sonarqube:
image: sonarqube:8.2-community
depends_on:
- db
ports:
- "9000:9000"
networks:
- sonarqubenet
environment:
SONAR_JDBC_URL: jdbc:postgresql://db:5432/sonarqube
SONAR_JDBC_USERNAME: sonar
SONAR_JDBC_PASSWORD: sonar
volumes:
- sonarqube_data:/opt/sonarqube/data
- sonarqube_extensions:/opt/sonarqube/extensions
- sonarqube_logs:/opt/sonarqube/logs
- sonarqube_temp:/opt/sonarqube/temp
restart: on-failure
container_name: sonarqube
db:
image: postgres
networks:
- sonarqubenet
environment:
POSTGRES_USER: sonar
POSTGRES_PASSWORD: sonar
volumes:
- postgresql:/var/lib/postgresql
- postgresql_data:/var/lib/postgresql/data
restart: on-failure
container_name: postgresql
networks:
sonarqubenet:
driver: bridge
volumes:
sonarqube_data:
sonarqube_extensions:
sonarqube_logs:
sonarqube_temp:
postgresql:
postgresql_data:
Then, execute the command,
$ docker-compose up -d
$ docker container ps
Sounds like the container is running and, as you mentioned, Sonarqube starts-up. When it starts, is it showing that it's using the H2 in memory db? After running docker-compose up -d, use docker logs -f <container_name> to see what's happening on Sonarqube startup.
To simplify viewing your logs with a known name, I suggest you also add a container name to your Sonarqube service. For example, container_name: sonarqube.
Also, while I know the plan is to deprecate the use of environment variables for the username, password and jdbc connection, I've had better luck in docker-compose using environment variables rather than the corresponding property value. For the connection string, try: SONARQUBE_JDBC_URL: jdbc:postgresql://db/sonar without specifying the default port for postgres.

Docker Compose + Spring Boot + Postgres connection

I have a Java Spring Boot app which works with a Postgres database. I want to use Docker for both of them. I initially put just the Postgres in Docker, and I had a docker-compose.yml file defined like this:
version: '2'
services:
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
Then, when I issued the commands sudo dockerd and sudo docker-compose -f docker-compose.yml up, it was starting the database. I could connect using pgAdmin for example, by using localhost as server and port 5432. Then, in my Spring Boot app, inside the application.properties file I defined the following properties.
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
spring.datasource.username=sample
spring.datasource.password=sample
spring.jpa.generate-ddl=true
At this point I could run my Spring Boot app locally through Spring Suite, and it all was working fine. Then, I wanted to also add my Spring Boot app as Docker image. I first of all created a Dockerfile in my project directory, which looks like this:
FROM java:8
EXPOSE 8080
ADD /target/manager.jar manager.jar
ENTRYPOINT ["java","-jar","manager.jar"]
Then, I entered to the directory of the project issued mvn clean followed by mvn install. Next, issued docker build -f Dockerfile -t manager . followed by docker tag 9c6b1e3f1d5e myuser/manager:latest (the id is correct). Finally, I edited my existing docker-compose.yml file to look like this:
version: '2'
services:
web:
image: myuser/manager:latest
ports:
- 8080:8080
depends_on:
- db
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
But, now if I issue sudo docker-compose -f docker-compose.yml up command, the database again starts correctly, but I get errors and exit code 1 for the web app part. The problem is the connection string. I believe I have to change it to something else, but I don't know what it should be. I get the following error messages:
web_1 | 2017-06-27 22:11:54.418 ERROR 1 --- [ main] o.a.tomcat.jdbc.pool.ConnectionPool : Unable to create initial connections of pool.
web_1 |
web_1 | org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections
Any ideas?
Each container has its own network interface with its own localhost. So change how Java points to Postgres:
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
To:
spring.datasource.url=jdbc:postgresql://db:5432/sample
db will resolve to the proper Postgres IP.
Bonus. With docker-compose you don't need to build your image by hand. So change:
web:
image: myuser/manager:latest
To:
web:
build: .
I had the same problem and I lost some time to understand and solve this problem:
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
I show all the properties so that everyone understands.
application.properties:
spring.datasource.url=jdbc:postgresql://localhost:5432/testdb
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.username=postgres
spring.datasource.password=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL82Dialect
spring.jpa.hibernate.ddl-auto=update
docker-compose.yml:
version: "3"
services:
springapp:
build: .
container_name: springapp
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
ports:
- 8000:8080
restart: always
depends_on:
- db
db:
image: postgres
container_name: db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=testdb
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5000:5432
volumes:
- pgdata:/var/lib/postgresql/data
restart: always
volumes:
pgdata:
For start spring application with local database we use url localhost.
For connect to container with database we need change 'localhost' on your database service, in my case 'localhost' to 'db'.
Solution: add SPRING_DATASOURCE_URL environment in docker-compose.yml wich rewrite spring.datasource.url value for connect:
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
I hope this helps someone save his time.
You can use this.
version: "2"
services:
sample_db-postgresql:
image: postgres:9.5
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
volumes:
- sample_db:/var/lib/postgresql/data
volumes:
sample_db:
You can use ENV variable to change the db address in your docker-compose.
Dockerfile:
FROM java:8
EXPOSE 8080
ENV POSTGRES localhost
ADD /target/manager.jar manager.jar
ENTRYPOINT exec java $JAVA_OPTS -jar manager.jar --spring.datasource.url=jdbc:postgresql://$POSTGRES:5432/sample
docker-compose:
`
container_name: springapp
environment:
- POSTGRES=db`

Resources