Docker-compose migrating from links to networking MongoDB database issue - spring

I am trying to migrate by Dockerfile from using links to networking but I am experiencing some problems. I will first sketch my original setup:
Original - Links
Dockerfile:
FROM java:8
VOLUME /tmp
ADD docker_micro_maven-0.0.1-SNAPSHOT.jar app.jar
RUN bash -c 'touch /app.jar'
EXPOSE 8080
ENTRYPOINT ["java","-Dspring.data.mongodb.uri=mongodb://mongodb/micros", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
docker-compose.yml:
mongodb:
image: mongo
employee:
image: jdruwe/docker_micro_maven
links:
- mongodb
ports:
- "8080"
New - Networking
Dockerfile:
FROM java:8
VOLUME /tmp
ADD docker_micro_maven-0.0.1-SNAPSHOT.jar app.jar
RUN bash -c 'touch /app.jar'
EXPOSE 8080
ENTRYPOINT ["java","-Dspring.data.mongodb.uri=mongodb://${MONGO_URI_DOCKER}/micros", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
docker-compose.yml:
mongodb:
container_name: "db"
image: mongo
employee:
image: jdruwe/docker_micro_maven
environment:
- MONGO_URI_DOCKER=db:27017
ports:
- "8080"
When I called my rest endpoint in the spring boot app (original situation) I always got a response (some json data). When I call it now I get the following error response:
{
"timestamp": 1453405937191,
"status": 500,
"error": "Internal Server Error",
"exception": "org.springframework.dao.DataAccessResourceFailureException",
"message": "Timed out after 10000 ms while waiting to connect. Client view of cluster state is {type=Unknown, servers=[{address=db:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.UnknownHostException: db: unknown error}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 10000 ms while waiting to connect. Client view of cluster state is {type=Unknown, servers=[{address=db:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.UnknownHostException: db: unknown error}}]",
"path": "/employee"
}
The video from Devoxx I used as reference: https://youtu.be/aSATsLG59Zs?t=53m33s
It seems that I keep doing something wrong but I am not really sure what it is, any help is welcome. Thanks in advance!

In docker-compose 1.5.x, the new networking is still an experimental feature that needs to be enabled using the --x-networking option.
The new networking features will be moved out of experimental in docker-compose 1.6 and are automatically used if the new (2.0) file format is used for your docker-compose.yml.
Read the release notes of docker-compose 1.6-rc1 for more information;
https://github.com/docker/compose/releases/tag/1.6.0-rc1

Related

Connecting to a Mongo container from Spring container

I have a problem here that I really cannot understand. I already saw few topics here with the same problem and those topics was successfully solved. I basically did the same thing and cannot understand what I'm doing wrong.
I have a Spring application container that tries to connect to a Mongo container through the following Docker Composer:
version: '3'
services:
app:
build: .
ports:
- "8080:8080"
links:
- db
db:
image: mongo
volumes:
- ./database:/data
ports:
- "27017:27017"
In my application.properties:
spring.data.mongodb.uri=mongodb://db:27017/app
Finally, my Dockerfile:
FROM eclipse-temurin:11-jre-alpine
WORKDIR /home/java
RUN mkdir /home/java/bar
COPY ./build/libs/foo.jar /home/java/bar/foo.jar
CMD ["java","-jar", "/home/java/bar/foo.jar"]
When I run docker compose up --build I got:
2022-11-17 12:08:53.452 INFO 1 --- [null'}-db:27017] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server db:27017
Caused by: java.net.UnknownHostException: db
Running the docker compose ps I can see the mongo container running well, and I am able to connect to it through Mongo Compass and with this same Spring Application but outside of container. The difference running outside of container is the host from spring.data.mongodb.uri=mongodb://db:27017/app to spring.data.mongodb.uri=mongodb://localhost:27017/app.
Also, I already tried to change the host for localhost inside of the spring container and didnt work.
You need to specify MongoDB host, port and database as different parameters as mentioned here.
spring.data.mongodb.host=db
spring.data.mongodb.port=27017
spring.data.mongodb.authentication-database=admin
As per the official docker-compose documentation the above docker-compose file should worked since both db and app are in the same network (You can check if they are in different networks just in case)
If the networking is not working, as a workaround, instead of using localhost inside the spring container, use the server's IP, i.e, mongodb://<server_ip>:27017/app (And make sure there is no firewall blocking it)

Error connecting ScyllaDB in Docker from Spring Boot app

I hope someone can help me with this issues, as I'm no expert with docker.
I have a Java Spring Boot application (let's call it my-app) that uses ScyllaDB. So far, I have been running the application with Spring Boot embedded Apache Tomcat build, and the database is running in Docker with no issues.
Here is the docker-compose file for the 3 Scylla nodes:
version: "3"
services:
scylla-node1:
container_name: scylla-node1
image: scylladb/scylla:4.5.0
restart: always
command: --seeds=scylla-node1,scylla-node2 --smp 1 --memory 750M --overprovisioned 1 --api-address 0.0.0.0
ports:
- 9042:9042
volumes:
- "./scylla/scylla.yaml:/etc/scylla/scylla.yaml"
- "./scylla/cassandra-rackdc.properties.dc1:/etc/scylla/cassandra-rackdc.properties"
networks:
- scylla-network
scylla-node2:
container_name: scylla-node2
image: scylladb/scylla:4.5.0
restart: always
command: --seeds=scylla-node1,scylla-node2 --smp 1 --memory 750M --overprovisioned 1 --api-address 0.0.0.0
ports:
- 9043:9042
volumes:
- "./scylla/scylla.yaml:/etc/scylla/scylla.yaml"
- "./scylla/cassandra-rackdc.properties.dc1:/etc/scylla/cassandra-rackdc.properties"
networks:
- scylla-network
scylla-node3:
container_name: scylla-node3
image: scylladb/scylla:4.5.0
restart: always
command: --seeds=scylla-node1,scylla-node2 --smp 1 --memory 750M --overprovisioned 1 --api-address 0.0.0.0
ports:
- 9044:9042
volumes:
- "./scylla/scylla.yaml:/etc/scylla/scylla.yaml"
- "./scylla/cassandra-rackdc.properties.dc1:/etc/scylla/cassandra-rackdc.properties"
networks:
- scylla-network
Using the node tool, I can see the DB is fine:
Datacenter: DC1
-- Address Load Tokens Owns Host ID Rack
UN 172.27.0.3 202.92 KB 256 ? 4e2690ec-393b-426d-8956-fb775ab5b3f9 Rack1
UN 172.27.0.2 99.5 KB 256 ? ae6a0b9f-d0e7-4740-8ebe-0ce1d2e9ea7e Rack1
UN 172.27.0.4 202.68 KB 256 ? 7a4b39bf-f38a-41ab-be33-c11a4e4e352c Rack1
In the application, the Java driver I'm using is the DataStax Java driver 3.11.2.0 for Apache Cassandra. The way I connect with the DB is the following:
#Bean
public Cluster cluster() {
Cluster cluster = Cluster.builder().addContactPointsWithPorts(
new InetSocketAddress("127.0.0.1", 9042),
new InetSocketAddress("127.0.0.1", 9043),
new InetSocketAddress("127.0.0.1", 9044))
.build();
return cluster;
}
#Bean
public Session session(Cluster cluster, #Value("${scylla.keyspace}") String keyspace) throws IOException {
final Session session = cluster.connect();
setupKeyspace(session, keyspace);
return session;
}
When running the application with the tomcat server, I receive a lot of connection errors at the start:
2022-07-19 22:42:38.424 WARN 28228 --- [r1-nio-worker-3] com.datastax.driver.core.Connection : Error creating netty channel to /172.27.0.4:9042
However, after a small spam of log errors, the app eventually connects and its totally usable. I do have to wait for the node tool to execute and confirm that all nodes are up, though.
2022-07-19 23:25:12.324 INFO 25652 --- [ restartedMain] c.d.d.c.p.DCAwareRoundRobinPolicy : Using data-center name 'DC1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2022-07-19 23:25:12.324 INFO 25652 --- [ restartedMain] com.datastax.driver.core.Cluster : New Cassandra host /172.27.0.3:9042 added
2022-07-19 23:25:12.324 INFO 25652 --- [ restartedMain] com.datastax.driver.core.Cluster : New Cassandra host /172.27.0.2:9042 added
2022-07-19 23:25:12.324 INFO 25652 --- [ restartedMain] com.datastax.driver.core.Cluster : New Cassandra host /127.0.0.1:9044 added
Then, I recently added "my-app" to my docker-compose file, but the app can't start and instantly shuts down even if I wait for the node status tool to confirm that all nodes are up.
Caused by: java.net.ConnectException: Connection refused
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042
Is there something wrong with the way I'm connecting with the DB? I wonder why the embedded tomcat build works and the docker one instantly shuts down. I was hoping someone here could help me find a way for the docker-compose build to wait for all the scylla nodes to be up before starting my-app (I assume I can do it with a script in the dockerfile? Maybe?), but I can't even seem to start the app in docker the same way I did with the tomcat. Maybe I'm missing smething regarding the port and host when using docker.
Any ideas in what I could try to solve this? Thanks in advance!
Docker compose file edited with the app:
my-app:
container_name: my-app
build:
context: .
dockerfile: Dockerfile
image: my-app
ports:
- 8082:8082
depends_on:
- scylla-node1
- scylla-node2
- scylla-node3
networks:
- scylla-network
You need to use IP addresses in the contact points which are accessible from outside the containers, not localhost.
Typically, it will be the IP address you've configured for CASSANDRA_RPC_ADDRESS (environment variable) or rpc_address (in your yaml).
If you didn't set the RPC addresses for the containers, you need to tell Cassandra what IP address to advertise to other nodes and clients by specifying a broadcast address with CASSANDRA_BROADCAST_ADDRESS or broadcast_rpc_address.
The important thing is that you need to use IP addresses which are reachable from your Spring Boot app. Cheers!

Register Dockerized microservice to Eureka server - Springboot

In my production environment I have an Eureka Server running inside a docker container.
I can register to it other basic microservices with this kind of Application.yml
Application.yml:
server:
port: '8095'
spring:
application:
name: sap-listener
eureka:
instance:
preferIpAddress: true
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://172.17.0.2:8761/eureka
I create a DockerImage with this Dockerfile:
Dockerfile
FROM openjdk:17-jdk
ARG JAR_FILE=target/*.jar
COPY target/sap-listener-*.jar /sap-listener.jar
ENTRYPOINT ["java", "-jar", "/sap-listener.jar" ]
EXPOSE 8095
and then I run it in production with this command:
docker run -d -p 8095:8095 --name sap myrepo/sap-listener1.0:latest
The service is successfully registered to the Eureka server.
I came across to some problems when I try to run a bigger microservice which have a docker-compose file.
I send directly this docker-compose file in production:
Docker-compose
version: "3.3"
services:
docker-mysql:
image: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'password'
MYSQL_DATABASE: 'db'
ports:
- "3007:3306"
phpmyadmin:
image: phpmyadmin
restart: always
container_name: php-my-admin-users
ports:
- "8081:80"
ldap-app:
image: myRepo/service1:latest
ports:
- "8090:8090"
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://docker-mysql:3306/db
depends_on:
- docker-mysql
And I run it with docker-compose -f docker-compose.yml up -d
service1 application.yml have the same type of connection with Eureka server of the previous microservice.
service1 is correctly deployed but It can't register himself to the Eureka server, if I log out the container output I have this error:
2022-06-29 15:45:20.551 INFO 1 --- [ main] c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://172.17.0.2:8761/eureka/}, exception=I/O error on GET request for "http://172.17.0.2:8761/eureka/apps/": Connect to 172.17.0.2:8761 [/172.17.0.2] failed: Connection timed out; nested exception is org.apache.http.conn.ConnectTimeoutException: Connect to 172.17.0.2:8761 [/172.17.0.2] failed: Connection timed out stacktrace=org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://172.17.0.2:8761/eureka/apps/": Connect to 172.17.0.2:8761 [/172.17.0.2] failed: Connection timed out; nested exception is org.apache.http.conn.ConnectTimeoutException: Connect to 172.17.0.2:8761 [/172.17.0.2] failed: Connection timed out
I read that someone directly insert the Eureka Server data as a service inside the Docker-Compose.yml file, but my Eureka Server is already deployed and is already listening to a specific port.
That is probably happening because docker-compose automatically assign a network to the containers.
Try adding network_mode: host to your services in the compose file, like so:
version: "3.3"
services:
docker-mysql:
network_mode: host
image: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'password'
MYSQL_DATABASE: 'db'
ports:
- "3007:3306"
phpmyadmin:
network_mode: host
image: phpmyadmin
restart: always
container_name: php-my-admin-users
ports:
- "8081:80"
ldap-app:
network_mode: host
image: myRepo/service1:latest
ports:
- "8090:8090"
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://docker-mysql:3306/db
depends_on:
- docker-mysql
First I would suggest you get familiar with how networking in Docker works and then have a look at networking in Docker-Compose.
When you run docker network ls when your containers are deployed you will see that they are running on different networks, which isolates them. Inspect the networks using docker network inspect <id> and you'll see they have different subnets. So for the services to be able to communicate they need to be on the same network.
You can manually create a network and use it in both compose and the docker cli.

Windows10 docker: Grafana not able to access influxdb-502 Bad Gateway

I have installed two containers(influxdb and grafana) inside docker.
Two containers up and running. Able to access grafana(http://localhost:3000/datasources/edit/1/) and influxdb(http://localhost:8086/query?q=show%20databases) urls from browser.
But when we configure influxdb in grafana, grafana api throwing 502: Bad Gateway.
Grafana log message
t=2020-12-23T23:53:36+0000 lvl=eror msg="Data proxy error" logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/1/query remote_addr=xxx.xx.x.x referer=http://localhost:3000/datasources/edit/1/ error="http: proxy error: dial tcp localhost:8086: connect: connection refused"
t=2020-12-23T23:53:36+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=GET path=/api/datasources/proxy/1/query status=502 remote_addr=1xxx.xx.x.x time_ms=26 size=0 referer=http://localhost:3000/datasources/edit/1/
Docker Compose file:
version: “2”
services:
grafana:
image: grafana/grafana
container_name: grafana
restart: always
ports:
3000:3000
volumes:
grafana_storage:/var/lib/grafana_storage
networks:
ranga_docker_network
influxdb:
image: influxdb
container_name: influxdb
restart: always
ports:
8086:8086
volumes:
influxdb_storage:/var/lib/influxdb_storage
networks:
ranga_docker_network
environment:
INFLUXDB_DB=telegraf
INFLUXDB_USER=telegraf
INFLUXDB_ADMIN_ENABLED=true
INFLUXDB_ADMIN_USER=admin
INFLUXDB_ADMIN_PASSWORD=telegraf1234
networks:
ranga_docker_network:
volumes:
grafana_storage:
influxdb_storage:
Docker network also up and running.
C:\WINDOWS\system32>docker network ls
NETWORK ID NAME DRIVER SCOPE
f42bea59b834 bridge bridge local
5ae575311d19 host host local
d7f57802a04c monitoring_default bridge local
2419751561dd monitoring_monitoring bridge local
4829f3a73843 monitoring_ranga_docker_network bridge local
4962bd901e12 none null local
ad245f671c22 ranga_docker_network bridge local
Thanks,
Ranga Rao
Issue Root cause: Connect VPN but didn't restart Docker daemon.
Resolution: Restart Docker daemon after connecting VPN.

Spring Boot, PostgreSQL, and Docker - Connection Refused whil Running in Container

I am attempting to build a "service" consisting of a Spring Boot application and PostgreSQL database. I have been able to access the database (running in a container) from the Spring Boot app while the Spring Boot application was running on my local machine. Now, when I attempt to move the Spring Boot application to a container, I am received the following error:
inventory_1 | 2018-01-20 18:43:06.108 ERROR 1 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection] with root cause
inventory_1 |
inventory_1 | java.net.ConnectException: Connection refused (Connection refused)
However, I am able to connect to DB from my local machine:
psql -h localhost -p 5000 -U kelly_psql -d leisurely_diversion
My application.properties file:
spring.jpa.hibernate.ddl-auto=none
spring.jpa.show-sql=false
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.datasource.url=jdbc:postgresql://localhost:5432/leisurely_diversion
spring.datasource.username=kelly_psql
spring.datasource.password=pass
spring.datasource.driver-class-name=org.postgresql.Driver
My docker-compose file:
# Use postgres/example user/password credentials
version: '3.2'
services:
db:
image: postgres
ports:
- 5000:5432
environment:
POSTGRES_PASSWORD: example
volumes:
- type: volume
source: psql_data
target: /var/lib/postgresql/data
networks:
- app
restart: always
inventory:
image: kellymarchewa/inventory_api
depends_on:
- db
ports:
- 8080:8080
networks:
- app
restart: always
volumes:
psql_data:
networks:
app:
My Dockerfile (from the Spring website)
FROM openjdk:8-jdk-alpine
VOLUME /tmp
ARG JAR_FILE
ADD ${JAR_FILE} app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
I suspect the issue lies in a misunderstanding (on my part) of Docker or containers, but I am not sure. Any advice would be appreciated.
You are pointing your application towards localhost, but this is not shared between containers.
To access another container you have to refer to its hostname.
In your case, I understand that you want the inventory service to access the db service. So you should use the following datasource url:
spring.datasource.url=jdbc:postgresql://db:5432/leisurely_diversion
See this simple tutorial about connecting to a container from another container with docker compose: https://docs.docker.com/compose/gettingstarted/
Like in my case if you are using Docker Toolbox for windows 8.1 then you cannot use "localhost",
Instead you have to use docker machine ip;
host> docker-machine ip default
192.168.99.100
After that your url will look like;
spring.datasource.url=jdbc:postgresql://192.168.99.100:5432/bankdb
This will successfully connect to docker Postgres DB.
Cheers!!
Elaborating a little upon the answer given by ESala:
I agree, it is a networking issue and the given solution works fine, but you can also use localhost (e.g. if you really really want to), by switching to network host mode when running your containers (cf here). Like done here for nginx.
I'd say you won't want this most of the time, since it messes with the sandbox you gain. But the option exists.

Resources