Connect external MQTT docker container with springboot docker-compose file - spring

I have a Springboot application with mysql database and now I want to connect to an external MQTT broker.
Following are my configs:
docker-compose.yml
version: '3.5'
networks:
foo:
name: foo
driver: bridge
services:
app-server:
build: .
container_name: Player1
ports:
- "8080:8080"
restart: always
networks:
- foo
depends_on:
- mydb
external_links:
- mosquitto:eclipse-mosquitto
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://mydb:3306/takeaway_db?serverTimezone=UTC&useLegacyDatetimeCode=false&createDatabaseIfNotExist=true
mydb:
image: mysql
ports:
- "3306:3306"
restart: always
networks:
- foo
environment:
- MYSQL_ROOT_USER=root
- MYSQL_ROOT_PASSWORD=P#ssword1234
volumes:
- /var/lib/mysql
and Dockerfile
FROM adoptopenjdk/openjdk11:alpine-jre
ARG JAR_FILE=target/spring-boot-web.jar
WORKDIR /opt/app
COPY target/*.jar app.jar
ENTRYPOINT ["java","-jar","app.jar"]
application.properties
mqtt.borker.url=tcp://172.18.0.2:1883 #container ip address from container inspect
mqtt.borker.username=username
mqtt.borker.password=password
command for starting MQTT broker
docker run -it -p 1883:1883 --net foo --name mosquitto eclipse-mosquitto
When I run docker-compose up, I get the followig error:
org.eclipse.paho.client.mqttv3.MqttException: Unable to connect to server
................
at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:na]

Basically the idea behind this solution should apply to connect any docker-compose containers with any other external containers.
Create or add existing containers to a network.
docker network create foo
docker run -it -d -p 1883:1883 --net foo --name mosquitto eclipse-mosquitto
Update docker compose file.
a. Add network foo to the containers.
b. Connect/expose network foo in compose file to the external network foo.
c. Link app container to the external container.
version: '3.5'
services:
app-server:
build: .
container_name: ...
ports:
- "8080:8080"
restart: always
networks:
- foo // =========> (a)
depends_on:
- mydb
external_links:
- mosquitto:eclipse-mosquitto //============> (c)
mydb:
image: mysql
container_name: ...
ports:
- "3306:3306"
restart: always
networks:
- foo // =========> (a)
networks:
foo:
external:
name: foo // ========> (b)
The complete docker file can be found here.
Update links in the application.properties file.
spring.datasource.url=jdbc:mysql://mydb:3306/takeaway_db?createDatabaseIfNotExist=true
mqtt.borker.url=tcp://mosquitto:1883

Related

How to get hostname from one container to another using docker compose?

I have two docker containers. One backend, and the other db (postgres). They both are linked. How do I utilise the backend environment HOST variable in the golang docker container?
From my understanding, both containers have their own IP addresses. I cannot use 'localhost' in the golang container because postgres isn't on localhost, but in an isolated container.
version: "3.7"
services:
backend:
image: golang:1.16
build: ./
working_dir: /app
volumes:
- ./backend/:/app
environment:
HOST: db
command: go run main.go
ports:
- 8080:8080
depends_on:
- db
db:
image: postgres
restart: always
environment:
POSTGRES_USER: gorm
POSTGRES_PASSWORD: gorm
POSTGRES_DB: gorm
ports:
- 9920:9920
I've tried researching how to access this variable as well as check Docker tutorials/documentation, but haven't found a solution.
Docker compose does DNS resolution. You should be able to access your database by name.
Remove:
environment:
HOST: db
Correct the postgres port to 5432:
db:
...
ports:
- 5432:5432
You must be able to connect like so:
db := pg.Connect(&pg.Options{
Addr: "db:5432",
User: "gorm",
Database: "gorm",
Password: "gorm",
})
As for environment variables, you can declare and access them like this:
backend:
environment:
POSTGRES_USER: gorm
...
os.Getenv("POSTGRES_USER")
Docker compose will create a network for your containers where they can communicate and reach each other.
You can make a simple change to your docker compose file by adding a name to your services which will make sure they get the same name each time.
version: "3.7"
services:
backend:
image: golang:1.16
build: ./
container_name: backend
working_dir: /app
volumes:
- ./backend/:/app
command: go run main.go
ports:
- "8080:8080"
depends_on:
- db
db:
image: postgres
restart: always
container_name: db
environment:
POSTGRES_USER: gorm
POSTGRES_PASSWORD: gorm
POSTGRES_DB: gorm
ports:
- "9920:5432"
The default port for Postgres is 5432, so I mapped it to your 9920. Then, you can access the db container from the backend container by specifying:
db:9920

class path resource cannot be resolved to URL when running docker compose

I am trying to create a docker-compose.yml since I have a lot of microservices to bootRun. It works properly when I run them one by one via gradle bootRun but encounter a lot of errors when doing docker-compose up
I want to use my local postgres database.
Here's my docker-compose.yml:
version: '3'
services:
redis:
image: redis:5.0.3-alpine
ports:
- '6379:6379'
rabbitmq:
image: rabbitmq:3.5.3-management
ports:
- "5672:5672"
- "15672:15672"
config-server:
network_mode: host
container_name: config-server
image: config-server:1.0.0-SNAPSHOT
customer-app:
network_mode: host
container_name: customer
image: customer:1.0.0-SNAPSHOT
environment:
- 'SPRING_DATASOURCE_URL=jdbc:postgresql://host.docker.internal:5432/demoapp?customer'
depends_on:
- config-server
restart: on-failure
authentication-app:
network_mode: host
container_name: authentication
image: authentication:1.0.0-SNAPSHOT
depends_on:
- config-server
restart: on-failure
admin-app:
network_mode: host
container_name: admin
image: admin:1.0.0-SNAPSHOT
environment:
- 'SPRING_DATASOURCE_URL=jdbc:postgresql://host.docker.internal:5432/demoapp?admin'
depends_on:
- config-server
restart: on-failure
Here's the error (1):
2021-03-23 13:40:36.772 DEBUG 1 --- [ main] o.a.catalina.core.AprLifecycleListener : The Apache Tomcat Native library could not be found using names [tcnative-1, libtcnative-1] on the java.library.path [/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib]. The errors reported were [Can't load library: /bin/libtcnative-1.so, Can't load library: /bin/liblibtcnative-1.so, no tcnative-1 in java.library.path: [/usr/java/packages/lib, /usr/lib/x86_64-linux-gnu/jni, /lib/x86_64-linux-gnu, /usr/lib/x86_64-linux-gnu, /usr/lib/jni, /lib, /usr/lib], no libtcnative-1 in java.library.path: [/usr/java/packages/lib, /usr/lib/x86_64-linux-gnu/jni, /lib/x86_64-linux-gnu, /usr/lib/x86_64-linux-gnu, /usr/lib/jni, /lib, /usr/lib]]
Here's the error (2):
Caused by: java.io.FileNotFoundException: class path resource [db/changelog/0.0.1-SNAPSHOT//data/sql/1597842399712-changelog.sql] cannot be resolved to URL because it does not exist
This is the structure of the project:
admin
admin-app
src
main
Dockerfile
java
resources
db.changelog
0.0.1-SNAPSHOT
data.sql
1597842399712-changelog.sql
target
libs
admin-app-1.0.0-SNAPSHOT.jar
Dockerfile:
FROM openjdk:11.0.2-jdk-slim
VOLUME /tmp
COPY target/libs/*.jar app.jar
EXPOSE 8081
ENTRYPOINT ["java","-jar","/app.jar"]
Assuming all other things work fine. In your Dockerfile
FROM openjdk:11.0.2-jdk-slim
VOLUME /tmp
RUN mkdir -p /data/sopie/your-project
COPY target/libs/*.jar /data/sopie/your-project/app.jar
EXPOSE 8081
ENTRYPOINT ["java","-jar","app.jar"]

Connect service container to db container

I'm new to docker and started to play with it on my small project.
I have dockerized the service itself with the following Docker file:
ROM adoptopenjdk:11-jdk-hotspot AS DEPENDENCIES_BUILD_IMAGE
ENV APP_HOME=/usr/app/
WORKDIR $APP_HOME
COPY build.gradle settings.gradle gradlew $APP_HOME
COPY gradle $APP_HOME/gradle
RUN ./gradlew build || return 0
COPY . .
RUN ./gradlew build
FROM adoptopenjdk/openjdk11:jdk-11.0.7_10-alpine AS FINAL
ENV JAR_TEMPLATE=myapp-0.0.1-SNAPSHOT.jar
ENV ARTIFACT_NAME=myapp.jar
ENV APP_HOME=/usr/app
WORKDIR $APP_HOME
COPY --from=DEPENDENCIES_BUILD_IMAGE $APP_HOME/build/libs/$JAR_TEMPLATE .
RUN mv $JAR_TEMPLATE $ARTIFACT_NAME
EXPOSE 8080
CMD ["java", "-jar", "budget-calculator.jar"]
Side note - I know that there's a problem that I'm always copying 0.0.1-SNAPSHOT - but I'm not sure how to solve it at the moment.
After that I wanted to connect my service to a Postgres DB with docker-compose using this confirmation:
version: '3'
services:
backend:
build: .
container_name: myapp
ports:
- "8080:8080"
links:
- "db"
depends_on:
- db
networks:
- backend
db:
restart: unless-stopped
image: postgres:10
container_name: myapp-db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=myapp
ports:
- 5436:5436
networks:
- backend
networks:
backend:
After that I've updated my application.properties file to indicate that the DB link is on the other container as follow:
spring.flyway.url=jdbc:postgresql://db:5436/myapp
spring.flyway.user=postgres
spring.flyway.password=secret
spring.flyway.baseline-on-migrate=true
spring.datasource.url=jdbc:postgresql://db:5436/myapp
spring.datasource.username=postgres
spring.datasource.password=secret
spring.datasource.driverClassName=org.postgresql.Driver
Now I had 2 problems:
While I assumed that build: . will rebuild my image every time
that I'm running docker-compose up if something changed in
practice I saw that it's not the case.
When the backend service starts flyway (a migration DB library) try
to connect to the database and cannot resolve the connection.
I've seen online that the usage of - links is deprecated and I should use the networks but both do not seem to work - what am I missing?
There are 2 problems with my configurations, the first one - the internal port of Postgres was configured as 5436 while the default port of the image is 5432 (I've updated both of them to 5432)
the second one, in order to pass the IP of the DB to the service I've added the following environment variables to the service image:
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/budget
SPRING_DATASOURCE_USERNAME: postgres
SPRING_DATASOURCE_PASSWORD: secret
SPRING_FLYWAY_URL: jdbc:postgresql://db:5432/budget
SPRING_FLYWAY_USER: postgres
SPRING_FLYWAY_PASSWORD: secret
So my current working configuration is this:
version: '3.8'
services:
backend:
build: .
container_name: app-service
ports:
- "8080:8080"
depends_on:
- db
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/myapp
SPRING_DATASOURCE_USERNAME: postgres
SPRING_DATASOURCE_PASSWORD: secret
SPRING_FLYWAY_URL: jdbc:postgresql://db:5432/myapp
SPRING_FLYWAY_USER: postgres
SPRING_FLYWAY_PASSWORD: secret
db:
restart: unless-stopped
image: postgres:10
environment:
- POSTGRES_DB=myapp
- POSTGRES_PASSWORD=postgres
- POSTGRES_USER=secret
volumes:
- myapp_data:/var/lib/postgresql/data
ports:
- 5432:5432
volumes:
myapp_data:

Docker postgres container loses data that should be stored in volume

I am running a postgres database generated by the below docker-compose file on Windows. Before running docker-compose up --build, I created a docker volume with docker volume --name postgresdata --driver local. The latter is done to avoid mounting a Windows folder into Postgres.
However, when I run docker-compose down followed by docker-compose up --build, the database is empty which I would not have expected. Any ideas or suggestions?
This is the docker-compose.yml file I am using:
version: '3.0'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
env_file:
- env_file
volumes:
- postgresdata
networks:
- db1
market_data:
build: .
environment:
PYTHONUNBUFFERED: 'true'
stdin_open: true
tty: true
links:
- db:db
container_name: market_data_container
volumes:
- '.:/market_data'
depends_on:
- db
networks:
- db1
adminer:
image: adminer
restart: always
ports:
- 8080:8080
networks:
- db1
depends_on:
- db
volumes:
market_data:
postgresdata:
external: true
networks:
db1:
driver: bridge
Postgres uses already a volume to persist data, but docker-compose down deletes this volume. You are using named volumes in your compose file, but don't mount it correctly.
version: '3.0'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
env_file:
- env_file
volumes:
- postgresdata:/var/lib/postgresql/data
networks:
- db1
Add the default path for postgres data to your volume postgresdata:/var/lib/postgresql/data. This should fix it.

Docker compose not using external named volume on Mac

Though I mount data on external volume, upon docker-compose down I am losing my data. I am using docker-compose version 1.14.0, build c7bdf9e and docker version 17.06.1-ce, build 874a737
# docker-compose.yml
version: '3'
services:
db:
image: postgres
volumes:
- data:/var/lib/postgres/data
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
volumes:
data:
external: true

Resources