Access to local database denied through docker container - spring-boot

I am having a problem connecting my compiled Spring-Boot app to the database that I have running on another container on my server.
I have tried different configurations, changing from localhost to the IP address of my server for the connection. I also double checked that the credentials matched by logging in via Adminer. Finally, I did a rebuild of the compose and image files several times to ensure that I have all the latest versions.
Compose file:
version: '3.1'
services:
db:
image: mariadb
restart: always
environment:
MYSQL_ROOT_PASSWORD: mypassword
MYSQL_DATABASE: marketingappdb
ports:
- "3306:3306"
expose:
- 3306
volumes:
- ./mariadbvolume:/var/lib/mariadb
networks:
- marketingapp
adminer:
image: adminer
restart: always
ports:
- "8086:8080"
expose:
- 8086
depends_on:
- db
networks:
- marketingapp
springserver:
image: marketingapp
restart: always
ports:
- "8091:8091"
expose:
- 8091
depends_on:
- db
networks:
- marketingapp
networks:
marketingapp:
Spring Server Image:
FROM openjdk:latest
COPY /marketing-app-final.jar .
EXPOSE 8091
ENTRYPOINT ["java", "-jar", "marketing-app-final.jar"]
Application properties for Spring:
server.port = 8091
spring.datasource.url=jdbc:mariadb://0.0.0.0:3306/marketingappdb
spring.datasource.username=root
spring.datasource.password=mypassword
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.jpa.hibernate.ddl-auto=update
I can connect from my PC to the database using the app from the remote same configuration (obviously replacing localhost with the IP) and don't see why I shouldn't be able to do the same from the actual server. Thanks in advance for any help!

Use the docker dns to connect your spring App to the mariabd:
jdbc:mariadb://db:3306/marketingappdb
Just a few other hints: you don't need to expose port 3306, you already bind it to 3306 on Host (if you just want to use it from within the docker Services you don't need to bind/expose it at all). And the mariabd persistent storage is var/lib/mysql and not var/lib/mariadb

Related

No resolvable bootstrap urls given in bootstrap.servers on maven install

I am building up a jar file for my spring boot microservice, but upon maven install i am getting the error as stated below.
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:89) ~[kafka-clients-3.1.1.jar:na]
at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:48) ~[kafka-clients-3.1.1.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:730) ~[kafka-clients-3.1.1.jar:na]
my docker-compose file for the application is
version: '2'
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
restart: always
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- 2181:2181
networks:
- app-network
kafka:
image: confluentinc/cp-kafka:latest
container_name: kafka
restart: always
depends_on:
- zookeeper
ports:
- 9092:9092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
networks:
- app-network
mongodb:
image: mongo:latest
restart: always
container_name: mongodb
networks:
- app-network
ports:
- 27017:27017
matching-engine-helper:
image: matching-engine-helper:latest
restart: always
container_name: "matching-engine-helper"
networks:
- app-network
ports:
- 9192:9192
depends_on:
- mongodb
- zookeeper
- kafka
matching-engine-core:
image: matching-engine-core:latest
restart: always
container_name: "matching-engine-core"
networks:
- app-network
ports:
- 9191:9191
depends_on:
- matching-engine-helper
- zookeeper
- kafka
- mongodb
networks:
app-network:
driver: bridge
and the application.yml is
spring:
data:
mongodb:
database: matching-engine-mongodb
host: mongodb
port: 27017
kafka:
bootstrap-servers: kafka:9092
server:
port: 9192
let me know if i need some environment variable's configuration here. as its running fine on local environment on local host but not on docker containers as i am unable to make a jar out of it.
Assuming mvn install is running on your host, then your host doesn't know how to resolve kafka as a DNS name running in a container. It that's what you wanted, then you should rename your file to application-docker.yml and use Spring profiles to properly override any defaults when you actually do run your code in a container. In your case, you would need to have localhost:29092 for Kafka, and similarly use localhost:27017 for Mongo in your default application.yml. You can also use environment variables for those two properties rather than hard-code them.
Or, if mvn install is running in a Docker layer itself, then that is completely isolated from other Docker networks. You can pass --network flag to docker build, though.
Assuming you don't want to mvn install -DskipTests, ideally, your tests do not rely on external services to be running. If you want to run a Kafka unit test, which is failing to connect, then you should either mock that, or using Spring-Kafka's EmbeddedKafka for integration-tests.
Worth mentioning that Spring boot has a Maven plugin for building Docker images, and it doesn't need a JAR to do so.

Problem in communication of docker mySQL container and my spring boot application container

When i start application in IDE, it works correctly, but when i use docker-compose for starting my project, something goes wrong and my application container restarts, i have shared my project on github https://github.com/MatveyAndrosyukk/sweater , maybe somebody can help.
dockerfile:
FROM adoptopenjdk/openjdk15
VOLUME /main-app
ADD target/sweater-0.0.1-SNAPSHOT.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar","/app.jar"]
docker-compose.yml:
version: "3.7"
services:
api_service:
build: .
restart: always
ports:
- 8080:8080
depends_on:
- mysql_db
links:
- mysql_db:sweater_db
mysql_db:
image: "mysql:8.0"
restart: always
ports:
- 3307:3306
environment:
MYSQL_DATABASE: sweater_db
MYSQL_USER: user228
MYSQL_PASSWORD: user228
MYSQL_ROOT_PASSWORD: 13574637480326
application.properties:
#DB properties------------
spring.datasource.url=jdbc:mysql://localhost:3306/sweater_db?useSSL=false
spring.datasource.username=user228
spring.datasource.password=user228
spring.jpa.show-sql=true
#hibernate will automatically create, update, validate database tables
spring.jpa.hibernate.ddl-auto=validate
spring.jpa.generate-ddl=true
#generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQLDialect
#-------------------------
#Logs properties----------
logging.level.org.hibernate.SQL=DEBUG
#-------------------------
#Mail properties----------
spring.mail.host=smtp.yandex.ru
spring.mail.username=matveyandrosyuk2002#yandex.ru
spring.mail.password=rjpyptztrbbivjec
spring.mail.port=465
spring.mail.protocol=smtps
mail.debug=true
#-------------------------
#Captcha properties-------
recaptcha.secret=6LcXGhEgAAAAAFUMjaB1kpj4NHzBHyAnCXcMPVhW
#-------------------------
#Image path properties----
upload.path=D:/Projects/Intellij IDEA/Web Applications/Spring/sweater/uploads
#-------------------------
#Spring Session properties
spring.session.jdbc.initialize-schema=always
spring.session.jdbc.table-name=SPRING_SESSION
#-------------------------
hostname=localhost:8080
Throws exception:
SQL State  : 08S01
Error Code : 0
Message    : Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1804) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:233) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveNamedBean(DefaultListableBeanFactory.java:1282) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveNamedBean(DefaultListableBeanFactory.java:1243) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveBean(DefaultListableBeanFactory.java:494) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:349) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:342) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.orm.jpa.EntityManagerFactoryUtils.findEntityManagerFactory(EntityManagerFactoryUtils.java:121) ~[spring-orm-5.3.18.jar!/:5.3.18]
at org.springframework.orm.jpa.JpaTransactionManager.setBeanFactory(JpaTransactionManager.java:333) ~[spring-orm-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeAwareMethods(AbstractAutowireCapableBeanFactory.java:1826) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1791) ~[spring-beans-5.3.18.jar!/:5.3.18]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620) ~[spring-beans-5.3.18.jar!/:5.3.18]
... 94 common frames omitted
Caused by: org.flywaydb.core.internal.exception.FlywaySqlException: Unable to obtain connection from database: Communications link failure
If you look at https://docs.docker.com/compose/networking/#links:
Links allow you to define extra aliases by which a service is reachable from another service. They are not required to enable services to communicate
However lets delete links and add hostname:
version: "3.7"
services:
api_service:
build: .
hostname: api_service
restart: always
ports:
- 8080:8080
depends_on:
- mysql_db
mysql_db:
image: "mysql:8.0"
hostname: mysql_db
restart: always
ports:
- 3307:3306
environment:
MYSQL_DATABASE: sweater_db
MYSQL_USER: user228
MYSQL_PASSWORD: user228
MYSQL_ROOT_PASSWORD: 13574637480326
The big problem is here: The "localhost" occurences in your application.properties refers to the api_service container localhost and not to your host machine localhost. So you can try:
spring.datasource.url=jdbc:mysql://mysqldb:3306/sweater_db?useSSL=false
you can also replace:
hostname=localhost:8080 by hostname=api_service:8080
Since both services created in the docker compose are created in the same network, you could ping each one with its hostname.
try it and come back please

How to access an external server folder from a spring boot docker container?

I'm using docker-compose to run spring boot application inside a docker container. A spring boot uses embedded tomcat to run.
There is an external folder on a server (/opt/cp/uploads) with images which I would like to access from a spring boot docker container.
Within the docker-compose.yaml file there are following containers defined:
nginx
mysql
springboot-app
nginx and CloudFlare redirect the domain to the spring app on the port 8080.
I'd like to access images like this:
https://domainname.com/uploads/imageName.png
Using tomcat this was done in server.xml:
<Context docBase="/opt/uploads" path="/uploads"/>
but this option is obviously not available via application.properties with embedded tomcat.
How does one come about this?
My docker-compose.yaml:
version: '3.8'
services:
nginx:
container_name: some-nginx
hostname: nginx
image: nginx:1.19.2-alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/conf.d:/etc/nginx/conf.d
restart: unless-stopped
network_mode: host
mysqldb:
image: mysql:8.0.20
hostname: mysqldb
container_name: cp-mysqldb
environment:
- MYSQL_ROOT_PASSWORD=pass1234
- MYSQL_DATABASE=db_name
- MYSQL_USER=root
- MYSQL_PASSWORD=pass1234
ports:
- "3306:3306"
volumes:
- cp-mysqldb-data:/opt/mysql
restart: unless-stopped
springboot-app:
image: openjdk:8
hostname: cp
container_name: cp-springboot
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://mysqldb:3306/db_name?autoReconnect=true&useSSL=false&useUnicode=yes&characterEncoding=UTF-8&characterSetResults=UTF-8&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&allowPublicKeyRetrieval=true&serverTimezone=UTC
ports:
- "8080:8080"
depends_on:
- mysqldb
volumes:
- cp-springboot-data:/opt/cp
- ./target/cp-springboot-0.0.1-SNAPSHOT.war:/ROOT.war
command: ["java", "-jar",
"-Dspring.profiles.active=prod",
"ROOT.war"]
restart: always
volumes:
cp-springboot-data: {
}
cp-mysqldb-data: {
}
Maybe better use Tomcat for docker image and not OpenJDK.
That way you can setup your Tomcat configuration for the external path
Take a look here: https://medium.com/#iamvickyav/deploying-spring-boot-war-in-tomcat-based-docker-2b689b206496
Actually, this option IS available via application.properties with embedded tomcat:
spring.mvc.static-path-pattern=/uploads/**
spring.resources.static-locations=file:/opt/uploads

Docker Container Connection Refused MacOS

I have this docker-compose file:
networks:
default:
ipam:
config:
- subnet: 10.48.0.0/16
gateway: 10.48.0.1
services:
haproxy:
build: haproxy
container_name: haproxy
volumes:
- ./haproxy/conf/:/usr/local/etc/haproxy/
- ./haproxy/ssl/:/etc/ssl/xip.io/
ports:
- "80:80"
- "443:443"
networks:
default:
ipv4_address: 10.48.0.2
server:
build: server
container_name: server
restart: always
environment:
- ENV=env=production db=true
ports:
- "8081:8081"
volumes:
- ./server/config:/usr/src/app/config
depends_on:
- haproxy
networks:
default:
ipv4_address: 10.48.0.4
frontend:
build: frontend
container_name: frontend
restart: always
ports:
- "8080:8080"
volumes:
- ./frontend/config:/usr/src/app/config
depends_on:
- server
networks:
default:
ipv4_address: 10.48.0.5
version: '2'
In order to deploy a backend server and a frontend interface inside a subnet defined in the range 10.48.0.0/16.
So I tried to assign fixed ip to each container. On Linux everything is ok, so I can reach 10.48.0.4_8081/api, but on MacOS when I try to do the same thing, I have ERR_CONNECTION_REFUSED.
If I try to connect without using IP, but with localhost:8081/api, this is ok. But with multiple containers, I have to access directly with the IP.
Inside each container, if I try to ping the other ip address (example from container frontend with IP 10.48.0.5 I try to ping 10.48.0.4) everything is OK.
So my question is, How can I do in order make an http call to an api that is on another service? thanks for your help.
I've read everywhere that is a well know situation under windows and mac, but not on linux, where is possible from the client side making request directly on the ip address of the container. This is not possible on mac and is still open an issue on github.
In this case, I've used haproxy in order to proxy requests to each container.

Connecting Spring Cloud Applications in Docker Container

I am attempting to host a Spring Cloud application in Docker containers.The underlying exception is as follows:
search_1 | Caused by: java.lang.IllegalStateException: Invalid URL: config:8888
I understand the reason is because of the URL specified in my config server.
spring.application.name=inventory-client
#spring.cloud.config.uri=http://localhost:8888
spring.cloud.config.uri=config:8888
On my development machine, I am able to use localhost. However, based on a past question (relating to connecting to my database), I learned that localhost is not appropriate in containers. For my database, I was able to use the following:
spring.jpa.hibernate.ddl-auto=none
spring.jpa.show-sql=false
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.datasource.url=jdbc:postgresql://db:5432/leisurely_diversion
#spring.datasource.url=jdbc:postgresql://localhost:5000/leisurely_diversion
spring.datasource.driver-class-name=org.postgresql.Driver
but this obviously did not work as expected for the configuration server.
My docker-compose file:
# Use postgres/example user/password credentials
version: '3.2'
services:
db:
image: postgres
ports:
- 5000:5432
environment:
POSTGRES_PASSWORD: example
volumes:
- type: volume
source: psql_data
target: /var/lib/postgresql/data
networks:
- app
restart: always
config:
image: kellymarchewa/config_server
networks:
- app
volumes:
- /root/.ssh:/root/.ssh
restart: always
search:
image: kellymarchewa/search_api
networks:
- app
restart: always
ports:
- 8082:8082
depends_on:
- db
- config
- inventory
inventory:
image: kellymarchewa/inventory_api
depends_on:
- db
- config
ports:
- 8081:8081
networks:
- app
restart: always
volumes:
psql_data:
networks:
app:
Both services are running under the same user defined network; how I allow the services to find the configuration service?
Thanks.

Resources