I am trying to create a docker-compose.yml since I have a lot of microservices to bootRun. It works properly when I run them one by one via gradle bootRun but encounter a lot of errors when doing docker-compose up
I want to use my local postgres database.
Here's my docker-compose.yml:
version: '3'
services:
redis:
image: redis:5.0.3-alpine
ports:
- '6379:6379'
rabbitmq:
image: rabbitmq:3.5.3-management
ports:
- "5672:5672"
- "15672:15672"
config-server:
network_mode: host
container_name: config-server
image: config-server:1.0.0-SNAPSHOT
customer-app:
network_mode: host
container_name: customer
image: customer:1.0.0-SNAPSHOT
environment:
- 'SPRING_DATASOURCE_URL=jdbc:postgresql://host.docker.internal:5432/demoapp?customer'
depends_on:
- config-server
restart: on-failure
authentication-app:
network_mode: host
container_name: authentication
image: authentication:1.0.0-SNAPSHOT
depends_on:
- config-server
restart: on-failure
admin-app:
network_mode: host
container_name: admin
image: admin:1.0.0-SNAPSHOT
environment:
- 'SPRING_DATASOURCE_URL=jdbc:postgresql://host.docker.internal:5432/demoapp?admin'
depends_on:
- config-server
restart: on-failure
Here's the error (1):
2021-03-23 13:40:36.772 DEBUG 1 --- [ main] o.a.catalina.core.AprLifecycleListener : The Apache Tomcat Native library could not be found using names [tcnative-1, libtcnative-1] on the java.library.path [/usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib]. The errors reported were [Can't load library: /bin/libtcnative-1.so, Can't load library: /bin/liblibtcnative-1.so, no tcnative-1 in java.library.path: [/usr/java/packages/lib, /usr/lib/x86_64-linux-gnu/jni, /lib/x86_64-linux-gnu, /usr/lib/x86_64-linux-gnu, /usr/lib/jni, /lib, /usr/lib], no libtcnative-1 in java.library.path: [/usr/java/packages/lib, /usr/lib/x86_64-linux-gnu/jni, /lib/x86_64-linux-gnu, /usr/lib/x86_64-linux-gnu, /usr/lib/jni, /lib, /usr/lib]]
Here's the error (2):
Caused by: java.io.FileNotFoundException: class path resource [db/changelog/0.0.1-SNAPSHOT//data/sql/1597842399712-changelog.sql] cannot be resolved to URL because it does not exist
This is the structure of the project:
admin
admin-app
src
main
Dockerfile
java
resources
db.changelog
0.0.1-SNAPSHOT
data.sql
1597842399712-changelog.sql
target
libs
admin-app-1.0.0-SNAPSHOT.jar
Dockerfile:
FROM openjdk:11.0.2-jdk-slim
VOLUME /tmp
COPY target/libs/*.jar app.jar
EXPOSE 8081
ENTRYPOINT ["java","-jar","/app.jar"]
Assuming all other things work fine. In your Dockerfile
FROM openjdk:11.0.2-jdk-slim
VOLUME /tmp
RUN mkdir -p /data/sopie/your-project
COPY target/libs/*.jar /data/sopie/your-project/app.jar
EXPOSE 8081
ENTRYPOINT ["java","-jar","app.jar"]
Related
I am building up a jar file for my spring boot microservice, but upon maven install i am getting the error as stated below.
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:89) ~[kafka-clients-3.1.1.jar:na]
at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:48) ~[kafka-clients-3.1.1.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:730) ~[kafka-clients-3.1.1.jar:na]
my docker-compose file for the application is
version: '2'
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
restart: always
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- 2181:2181
networks:
- app-network
kafka:
image: confluentinc/cp-kafka:latest
container_name: kafka
restart: always
depends_on:
- zookeeper
ports:
- 9092:9092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
networks:
- app-network
mongodb:
image: mongo:latest
restart: always
container_name: mongodb
networks:
- app-network
ports:
- 27017:27017
matching-engine-helper:
image: matching-engine-helper:latest
restart: always
container_name: "matching-engine-helper"
networks:
- app-network
ports:
- 9192:9192
depends_on:
- mongodb
- zookeeper
- kafka
matching-engine-core:
image: matching-engine-core:latest
restart: always
container_name: "matching-engine-core"
networks:
- app-network
ports:
- 9191:9191
depends_on:
- matching-engine-helper
- zookeeper
- kafka
- mongodb
networks:
app-network:
driver: bridge
and the application.yml is
spring:
data:
mongodb:
database: matching-engine-mongodb
host: mongodb
port: 27017
kafka:
bootstrap-servers: kafka:9092
server:
port: 9192
let me know if i need some environment variable's configuration here. as its running fine on local environment on local host but not on docker containers as i am unable to make a jar out of it.
Assuming mvn install is running on your host, then your host doesn't know how to resolve kafka as a DNS name running in a container. It that's what you wanted, then you should rename your file to application-docker.yml and use Spring profiles to properly override any defaults when you actually do run your code in a container. In your case, you would need to have localhost:29092 for Kafka, and similarly use localhost:27017 for Mongo in your default application.yml. You can also use environment variables for those two properties rather than hard-code them.
Or, if mvn install is running in a Docker layer itself, then that is completely isolated from other Docker networks. You can pass --network flag to docker build, though.
Assuming you don't want to mvn install -DskipTests, ideally, your tests do not rely on external services to be running. If you want to run a Kafka unit test, which is failing to connect, then you should either mock that, or using Spring-Kafka's EmbeddedKafka for integration-tests.
Worth mentioning that Spring boot has a Maven plugin for building Docker images, and it doesn't need a JAR to do so.
I have discovery service: https://github.com/Naresh-Chaurasia/API-MicroServices-Kafka/tree/master/Microservices-CQRS-SAGA-Kafka/DiscoveryService
I have product service: https://github.com/Naresh-Chaurasia/API-MicroServices-Kafka/tree/master/Microservices-CQRS-SAGA-Kafka/ProductsService
Following is my docker-compose.yml file: https://github.com/Naresh-Chaurasia/API-MicroServices-Kafka/tree/master/Microservices-CQRS-SAGA-Kafka/docker-compose.yml
version: "3.8"
services:
axon-server:
image: axoniq/axonserver
container_name: axon-server
ports:
- 8124:8124
- 8024:8024
networks:
- axon-demo
discovery-service:
build:
context: ./DiscoveryService
container_name: discovery-service
ports:
- 8010:8010
networks:
- axon-demo
networks:
axon-demo:
driver: bridge
When i run the following command docker-compose up, I get axon-server and discovery-service running.
I now run ProductService using the following file src/main/java/com/appsdeveloperblog/estore/ProductsService/ProductsServiceApplication.java which is under https://github.com/Naresh-Chaurasia/API-MicroServices-Kafka/tree/master/Microservices-CQRS-SAGA-Kafka/ProductsService.
It works fine.
The problem start when I try to run ProductService as a microservice, and it fails to connect to axon server. To do that I modify the docker-compose.yml as follows:
version: "3.8"
services:
axon-server:
image: axoniq/axonserver
container_name: axon-server
ports:
- 8124:8124
- 8024:8024
networks:
- axon-demo
discovery-service:
build:
context: ./DiscoveryService
container_name: discovery-service
ports:
- 8010:8010
networks:
- axon-demo
product-service:
build:
context: ./ProductsService
ports:
- 8090:8090
depends_on:
- axon-server
- discovery-service
networks:
- axon-demo
networks:
axon-demo:
driver: bridge
Now if I try to run docker-compose up, I get the following error:
product-service_1 | 2021-08-20 03:06:27.973 INFO 1 --- [#29db04f6c0a4-0] i.a.a.c.impl.AxonServerManagedChannel : Requesting connection details from localhost:8124
product-service_1 | 2021-08-20 03:06:30.004 WARN 1 --- [#29db04f6c0a4-0] i.a.a.c.impl.AxonServerManagedChannel : Connecting to AxonServer node [localhost:8124] failed: UNAVAILABLE: io exception
I have gone through the following link, Spring Boot Microservices are unable to connect to Axon Server, which looks like similar problem but still not able to fix my problem.
Please guide.
Thanks.
I made the following changes and it works fine now.
In the file ProductsService\src\main\resources\application.properties
Old Entry: eureka.client.serviceUrl.defaultZone = http://localhost:8010/eureka
New Entry: eureka.client.serviceUrl.defaultZone = http://discovery-service:8010/eureka
Added the following line in the above file: axon.axonserver.servers=axon-server:8124
In the File: Microservices-CQRS-SAGA-Kafka\docker-compose.yml
Old Entry: version: "3.8"
New Entry: version: "2"
Added following code in yml file
product-service:
build:
context: ./ProductsService
ports:
- 8090:8090
networks:
- axon-demo
I have also checked in the code in git. The micro services can talk to each other, and connect to axon server and the problem reported earlier is resolved.
Thanks.
Just a thought.
Try to add to each service in application.properties
axon.axonserver.servers=localhost:8124
and to each service which connects to axon-server in docker-compose.yml
in environment variables section
axon-server:
image: axoniq/axonserver
container_name: axon-server
ports:
- 8124:8124
- 8024:8024
product-service:
...
environment:
AXON_AXONSERVER_SERVERS: axon:8124
I had previously built a Docker image of each Spring micro-service, and in the docker-compose.yml file this is what I did to start them and have some other services being registered by Eureka's service, by overriding defaultZone passing the container's name:
version: '3'
services:
eureka-discovery:
image: eureka-discovery-service:0.0.1
ports:
- 8761:8761
zuul-gateway:
image: zuul-gateway-service:0.0.1
environment:
- eureka.client.serviceUrl.defaultZone=http://eureka-discovery:8761/eureka
depends_on:
- eureka-discovery
ports:
- 8765:8765
It worked. Now, I did things a little different, by bind mounting my project into a container for local development:
version: '3.1'
services:
eureka-discovery-service:
container_name: eureka-discovery-service
image: openjdk:11.0.9.1-jdk
ports:
- 8761:8761
volumes:
- ./eureka-discovery-service:/usr/src/app
working_dir: /usr/src/app
command: ./mvnw spring-boot:run
zuul-gateway-service:
container_name: zuul-gateway-service
image: openjdk:11.0.9.1-jdk
environment:
- eureka.client.serviceUrl.defaultZone=http://eureka-discovery-service:8761/eureka
ports:
- 8765:8765
volumes:
- ./zuul-gateway-service:/usr/src/app
working_dir: /usr/src/app
command: ./mvnw spring-boot:run
depends_on:
- eureka-discovery-service
links:
- eureka-discovery-service
But with things like this, zuul-gateway-service doesn't see eureka-discovery-service like it did before.
2020-12-17 23:26:29.997 ERROR 78 --- [freshExecutor-0] com.netflix.discovery.DiscoveryClient : DiscoveryClient_ZUUL-GATEWAY-SERVICE/7c580b412063:zuul-gateway-service:8765 - was unable to refresh its cache! status = Cannot execute request on any known server
I notice that it's using the Eureka container's ID instead of its name.
I managed to make it work by altering the command and overriding the property there instead of on environments:
command: ./mvnw spring-boot:run -Dspring-boot.run.arguments=--eureka.client.serviceUrl.defaultZone=http://eureka-discovery-service:8761/eureka
If there'd be a better way, I'd like to know.
I'm using docker-compose to run spring boot application inside a docker container. A spring boot uses embedded tomcat to run.
There is an external folder on a server (/opt/cp/uploads) with images which I would like to access from a spring boot docker container.
Within the docker-compose.yaml file there are following containers defined:
nginx
mysql
springboot-app
nginx and CloudFlare redirect the domain to the spring app on the port 8080.
I'd like to access images like this:
https://domainname.com/uploads/imageName.png
Using tomcat this was done in server.xml:
<Context docBase="/opt/uploads" path="/uploads"/>
but this option is obviously not available via application.properties with embedded tomcat.
How does one come about this?
My docker-compose.yaml:
version: '3.8'
services:
nginx:
container_name: some-nginx
hostname: nginx
image: nginx:1.19.2-alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/conf.d:/etc/nginx/conf.d
restart: unless-stopped
network_mode: host
mysqldb:
image: mysql:8.0.20
hostname: mysqldb
container_name: cp-mysqldb
environment:
- MYSQL_ROOT_PASSWORD=pass1234
- MYSQL_DATABASE=db_name
- MYSQL_USER=root
- MYSQL_PASSWORD=pass1234
ports:
- "3306:3306"
volumes:
- cp-mysqldb-data:/opt/mysql
restart: unless-stopped
springboot-app:
image: openjdk:8
hostname: cp
container_name: cp-springboot
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://mysqldb:3306/db_name?autoReconnect=true&useSSL=false&useUnicode=yes&characterEncoding=UTF-8&characterSetResults=UTF-8&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&allowPublicKeyRetrieval=true&serverTimezone=UTC
ports:
- "8080:8080"
depends_on:
- mysqldb
volumes:
- cp-springboot-data:/opt/cp
- ./target/cp-springboot-0.0.1-SNAPSHOT.war:/ROOT.war
command: ["java", "-jar",
"-Dspring.profiles.active=prod",
"ROOT.war"]
restart: always
volumes:
cp-springboot-data: {
}
cp-mysqldb-data: {
}
Maybe better use Tomcat for docker image and not OpenJDK.
That way you can setup your Tomcat configuration for the external path
Take a look here: https://medium.com/#iamvickyav/deploying-spring-boot-war-in-tomcat-based-docker-2b689b206496
Actually, this option IS available via application.properties with embedded tomcat:
spring.mvc.static-path-pattern=/uploads/**
spring.resources.static-locations=file:/opt/uploads
I'm new to docker and started to play with it on my small project.
I have dockerized the service itself with the following Docker file:
ROM adoptopenjdk:11-jdk-hotspot AS DEPENDENCIES_BUILD_IMAGE
ENV APP_HOME=/usr/app/
WORKDIR $APP_HOME
COPY build.gradle settings.gradle gradlew $APP_HOME
COPY gradle $APP_HOME/gradle
RUN ./gradlew build || return 0
COPY . .
RUN ./gradlew build
FROM adoptopenjdk/openjdk11:jdk-11.0.7_10-alpine AS FINAL
ENV JAR_TEMPLATE=myapp-0.0.1-SNAPSHOT.jar
ENV ARTIFACT_NAME=myapp.jar
ENV APP_HOME=/usr/app
WORKDIR $APP_HOME
COPY --from=DEPENDENCIES_BUILD_IMAGE $APP_HOME/build/libs/$JAR_TEMPLATE .
RUN mv $JAR_TEMPLATE $ARTIFACT_NAME
EXPOSE 8080
CMD ["java", "-jar", "budget-calculator.jar"]
Side note - I know that there's a problem that I'm always copying 0.0.1-SNAPSHOT - but I'm not sure how to solve it at the moment.
After that I wanted to connect my service to a Postgres DB with docker-compose using this confirmation:
version: '3'
services:
backend:
build: .
container_name: myapp
ports:
- "8080:8080"
links:
- "db"
depends_on:
- db
networks:
- backend
db:
restart: unless-stopped
image: postgres:10
container_name: myapp-db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=myapp
ports:
- 5436:5436
networks:
- backend
networks:
backend:
After that I've updated my application.properties file to indicate that the DB link is on the other container as follow:
spring.flyway.url=jdbc:postgresql://db:5436/myapp
spring.flyway.user=postgres
spring.flyway.password=secret
spring.flyway.baseline-on-migrate=true
spring.datasource.url=jdbc:postgresql://db:5436/myapp
spring.datasource.username=postgres
spring.datasource.password=secret
spring.datasource.driverClassName=org.postgresql.Driver
Now I had 2 problems:
While I assumed that build: . will rebuild my image every time
that I'm running docker-compose up if something changed in
practice I saw that it's not the case.
When the backend service starts flyway (a migration DB library) try
to connect to the database and cannot resolve the connection.
I've seen online that the usage of - links is deprecated and I should use the networks but both do not seem to work - what am I missing?
There are 2 problems with my configurations, the first one - the internal port of Postgres was configured as 5436 while the default port of the image is 5432 (I've updated both of them to 5432)
the second one, in order to pass the IP of the DB to the service I've added the following environment variables to the service image:
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/budget
SPRING_DATASOURCE_USERNAME: postgres
SPRING_DATASOURCE_PASSWORD: secret
SPRING_FLYWAY_URL: jdbc:postgresql://db:5432/budget
SPRING_FLYWAY_USER: postgres
SPRING_FLYWAY_PASSWORD: secret
So my current working configuration is this:
version: '3.8'
services:
backend:
build: .
container_name: app-service
ports:
- "8080:8080"
depends_on:
- db
environment: # Pass environment variables to the service
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/myapp
SPRING_DATASOURCE_USERNAME: postgres
SPRING_DATASOURCE_PASSWORD: secret
SPRING_FLYWAY_URL: jdbc:postgresql://db:5432/myapp
SPRING_FLYWAY_USER: postgres
SPRING_FLYWAY_PASSWORD: secret
db:
restart: unless-stopped
image: postgres:10
environment:
- POSTGRES_DB=myapp
- POSTGRES_PASSWORD=postgres
- POSTGRES_USER=secret
volumes:
- myapp_data:/var/lib/postgresql/data
ports:
- 5432:5432
volumes:
myapp_data: