RabbitMQ not working when services are dockerized - spring-boot

So I have 2 services. Kweet (which is a tweet) and User. When I run the 2 services manually + rest of the services in docker, it works. Rest of services include MongoDB, RabbitMQ, Spring Cloud Gateway, Eureka Discovery. But the moment I run the 2 (micro)services in Docker (I just use a docker compose), the rabbitmq functionality stops working. The normal API calls work, it's specifically the RabbitMQ calls that fail.
RabbitMQ functionality:
EditUsername, edits username in user-service.
Than sends data via RabbitMQ (this is where it goes wrong I think) to kweet-service where it edits the username of a kweet.
Docker-compose file:
version: '3.8'
services:
eureka-service:
build: ./eureka-discovery-service
restart: always
container_name: eureka-service
ports:
- 8087:8087
api-gateway:
build: ./api-gateway
restart: always
container_name: api-gateway
depends_on:
- eureka-service
ports:
- 8080:8080
user:
build: ./user
restart: unless-stopped
container_name: user-ms
ports:
- 8081:8081
depends_on:
- eureka-service
kweet:
build: ./kweet
restart: unless-stopped
container_name: kweet-ms
depends_on:
- eureka-service
ports:
- 8082:8082
mongodb:
image: mongo
restart: always
container_name: mongodb
ports:
- 27017:27017
rabbitmq:
image: rabbitmq:management
restart: always
container_name: rabbitmq
hostname: rabbitmq
ports:
- 5672:5672
- 15672:15672
When I try to make a call the console shows:
user-ms | 2022-04-27 08:52:04.823 INFO 1 --- [nio-8081-exec-4] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:5672]
The postman status I get back is 503 Service Unavailable which isn't from any try-catch's I made. Anybody have any clue where the problem might be?
EDIT[ConnectionFactory]:
I tried to use the documentation and added a the CachingConnectionFactory but it had the same result. Am I doing it wrong?
I added this to the RabbitMQ/Message-config (HOST, USERNAME, PASSWORD come from application.properties:
#Bean
public AmqpTemplate template() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(HOST);
connectionFactory.setUsername(USERNAME);
connectionFactory.setPassword(PASSWORD);
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(converter());
return rabbitTemplate;
}
EDIT [docker-compose]:
Found this source (https://www.linkedin.com/pulse/binding-your-docker-app-container-rabbitmq-phani-bushan/) that got rid of my 503 Service Unavailable error. The problem I found now is that whenever I start up the containers, it generates new queues and exchanges that aren't the ones I set up in my application.properties.
Now whenever I make a call, it shows this log:
user-ms | 2022-04-28 07:36:28.825 INFO 1 --- [nio-8081-exec-1] o.s.a.r.c.CachingConnectionFactory : Created new connection: rabbitConnectionFactory#2ca65ce4:0/SimpleCo
nnection#7e7052f [delegate=amqp://guest#172.23.0.4:5672/, localPort= 43208]
Things tried:
change host to [rabbitmq-container-name] in code via CachingConnectionFactory
change host to [rabbitmq-container-name] in docker compose with environment: - spring_rabbitmq_host=[rabbitmq-container-name]
build: ./user
restart: unless-stopped
container_name: user-ms
depends_on:
- eureka-service
- rabbitmq
ports:
- 8081:8081
environment:
- spring_rabbitmq_host=[rabbitmq-container-name]
Instead of [rabbitmq-container-name] I've tried host.docker.internal and localhost

When you dockerize your services are no longer listening in localhost. If you need to network connect services you need to use container_name instead of localhost.
localhost points to the container itself, where only one service is listening. Do not mistake for when you develop on your laptop without containers, where everything is in localhost
More about this here
By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name.
You must configure, somewhere in your user-ms application (we do not know what kind of applicatin is), that RabbitMQ service is listening at rabbitmq (container_name) not localhost.

Try to use the container with environment variables. For me it is enough for working.
rabbitmq:
image: rabbitmq:management
restart: always
container_name: rabbitmq
environment:
RABBITMQ_DEFAULT_USER: guest
RABBITMQ_DEFAULT_PASS: guest
RABBITMQ_DEFAULT_VHOST: /
ports:
- 5672:5672
- 15672:15672
I can open this container in a browser: http://localhost:15672/
I use PHP FPM and Symfony and I pass env value to this container to connect to rabbitmq.
services:
php-fpm:
environment:
ENQUEUE_DSN: amqp://guest:guest#rabbitmq:5672
For Java you need to find and define or redefine application properties. The config may use the default value for localhost, like this spring.rabbitmq.host=localhost, but you need to use Docker's host, it is rabbitmq.
spring.rabbitmq.host=rabbitmq
spring.rabbitmq.port=5672
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest

I have made a grave mistake. 2 things that I did before it started working.
Change Dockerfile ADD command to COPY (Don't think that was the problem though)
Deleted all my images and containers and re-made them a.k.a instead of docker-compose up I should've been typing docker-compose up --build. This was most likely the issue
brb gonna cry in a corner

Related

Register Dockerized microservice to Eureka server - Springboot

In my production environment I have an Eureka Server running inside a docker container.
I can register to it other basic microservices with this kind of Application.yml
Application.yml:
server:
port: '8095'
spring:
application:
name: sap-listener
eureka:
instance:
preferIpAddress: true
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://172.17.0.2:8761/eureka
I create a DockerImage with this Dockerfile:
Dockerfile
FROM openjdk:17-jdk
ARG JAR_FILE=target/*.jar
COPY target/sap-listener-*.jar /sap-listener.jar
ENTRYPOINT ["java", "-jar", "/sap-listener.jar" ]
EXPOSE 8095
and then I run it in production with this command:
docker run -d -p 8095:8095 --name sap myrepo/sap-listener1.0:latest
The service is successfully registered to the Eureka server.
I came across to some problems when I try to run a bigger microservice which have a docker-compose file.
I send directly this docker-compose file in production:
Docker-compose
version: "3.3"
services:
docker-mysql:
image: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'password'
MYSQL_DATABASE: 'db'
ports:
- "3007:3306"
phpmyadmin:
image: phpmyadmin
restart: always
container_name: php-my-admin-users
ports:
- "8081:80"
ldap-app:
image: myRepo/service1:latest
ports:
- "8090:8090"
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://docker-mysql:3306/db
depends_on:
- docker-mysql
And I run it with docker-compose -f docker-compose.yml up -d
service1 application.yml have the same type of connection with Eureka server of the previous microservice.
service1 is correctly deployed but It can't register himself to the Eureka server, if I log out the container output I have this error:
2022-06-29 15:45:20.551 INFO 1 --- [ main] c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://172.17.0.2:8761/eureka/}, exception=I/O error on GET request for "http://172.17.0.2:8761/eureka/apps/": Connect to 172.17.0.2:8761 [/172.17.0.2] failed: Connection timed out; nested exception is org.apache.http.conn.ConnectTimeoutException: Connect to 172.17.0.2:8761 [/172.17.0.2] failed: Connection timed out stacktrace=org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://172.17.0.2:8761/eureka/apps/": Connect to 172.17.0.2:8761 [/172.17.0.2] failed: Connection timed out; nested exception is org.apache.http.conn.ConnectTimeoutException: Connect to 172.17.0.2:8761 [/172.17.0.2] failed: Connection timed out
I read that someone directly insert the Eureka Server data as a service inside the Docker-Compose.yml file, but my Eureka Server is already deployed and is already listening to a specific port.
That is probably happening because docker-compose automatically assign a network to the containers.
Try adding network_mode: host to your services in the compose file, like so:
version: "3.3"
services:
docker-mysql:
network_mode: host
image: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'password'
MYSQL_DATABASE: 'db'
ports:
- "3007:3306"
phpmyadmin:
network_mode: host
image: phpmyadmin
restart: always
container_name: php-my-admin-users
ports:
- "8081:80"
ldap-app:
network_mode: host
image: myRepo/service1:latest
ports:
- "8090:8090"
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://docker-mysql:3306/db
depends_on:
- docker-mysql
First I would suggest you get familiar with how networking in Docker works and then have a look at networking in Docker-Compose.
When you run docker network ls when your containers are deployed you will see that they are running on different networks, which isolates them. Inspect the networks using docker network inspect <id> and you'll see they have different subnets. So for the services to be able to communicate they need to be on the same network.
You can manually create a network and use it in both compose and the docker cli.

feign.RetryableException Connection refused executing ... Spring Boot + Feign + Docker Compose

I'm facing this issue when trying to use a microservice in docker (this doesn't happen in local).
An exception occurred in RetryableException, in the line -2:
Connection refused executing GET http://localhost:8082/api/car/findAll
this is my docker-compose file
version: '3.9'
services:
people:
build:
context: peoplems
ports:
- '8081:8081'
networks:
- host
vehicles:
build:
context: vehiclesms
ports:
- '8082:8082'
networks:
- host
api:
build:
context: apigateway
dockerfile: Dockefile
ports:
- '8080:8080'
networks:
- host
networks:
host:
driver: bridge
The services are running
The error was that I was trying to call "localhost" using feignclient
url = http://localhost:8081/api/car
So, when the app was 'deployed' in docker, docker wouldn't find the localhost address. I had to change "localhost" by the name of the container where the service was deployed: parking-people-1. My new url in feignclient is now
url = http://parking-people-1:8081/api/car
and it works.
NOTE: with this solution, you don't have to create a new network anymore. (at least in this case.)
I hope this could be useful for someone.

Not able to connect ActiveMQ docker container from another docker container

version: '3.9'
services:
activemq:
image: rmohr/activemq:5.15.9-alpine
restart: always
ports:
- 61616:61616
- 8161:8161
- 5672:5672
container_name: activemq
app-service:
image: app-service:v1
restart: always
ports:
- 8080:8080
container_name: app-service
links:
- activemq
depends_on:
- activemq
In my app service I've configured the ActiveMQ broker URL using Spring Boot spring.activemq.broker-url=tcp://activemq:61616 and also username and password.
When I am trying to run docker-compose up the app service showing below error
DefaultMessageListenerContainer : Could not refresh JMS Connection for
destination 'queueName' - retrying using FixedBackOff{interval=5000,
currentAttempts=5, maxAttempts=unlimited}. Cause: Java.lang.NullPointerException.
I can access the ActiveMQ web console on browser (e.g. using http://localhost:8161).
Without docker container the same code is working fine in localhost.
I also had this exact problem and what helped me is adding spring.activemq.broker-url=tcp://activemq:61616 to docker-compose for app environment tag. For me it's like that:
app:
build:
context: .
container_name: app
ports:
- 8080:8080
environment:
- spring.activemq.broker-url=tcp://activemq:61616
depends_on:
- activemq
I think containerized spring app doesn't see broker-url from app properties for whatever reason
Yes, The big reason is your app run before activemq service.
You can try docker-compose up and see the console log in terminal.
Fixed:
It is not a good idea yet,but you can go to docker app and click on restart with you app container's name and then everything will work.

Docker Container Connection Refused MacOS

I have this docker-compose file:
networks:
default:
ipam:
config:
- subnet: 10.48.0.0/16
gateway: 10.48.0.1
services:
haproxy:
build: haproxy
container_name: haproxy
volumes:
- ./haproxy/conf/:/usr/local/etc/haproxy/
- ./haproxy/ssl/:/etc/ssl/xip.io/
ports:
- "80:80"
- "443:443"
networks:
default:
ipv4_address: 10.48.0.2
server:
build: server
container_name: server
restart: always
environment:
- ENV=env=production db=true
ports:
- "8081:8081"
volumes:
- ./server/config:/usr/src/app/config
depends_on:
- haproxy
networks:
default:
ipv4_address: 10.48.0.4
frontend:
build: frontend
container_name: frontend
restart: always
ports:
- "8080:8080"
volumes:
- ./frontend/config:/usr/src/app/config
depends_on:
- server
networks:
default:
ipv4_address: 10.48.0.5
version: '2'
In order to deploy a backend server and a frontend interface inside a subnet defined in the range 10.48.0.0/16.
So I tried to assign fixed ip to each container. On Linux everything is ok, so I can reach 10.48.0.4_8081/api, but on MacOS when I try to do the same thing, I have ERR_CONNECTION_REFUSED.
If I try to connect without using IP, but with localhost:8081/api, this is ok. But with multiple containers, I have to access directly with the IP.
Inside each container, if I try to ping the other ip address (example from container frontend with IP 10.48.0.5 I try to ping 10.48.0.4) everything is OK.
So my question is, How can I do in order make an http call to an api that is on another service? thanks for your help.
I've read everywhere that is a well know situation under windows and mac, but not on linux, where is possible from the client side making request directly on the ip address of the container. This is not possible on mac and is still open an issue on github.
In this case, I've used haproxy in order to proxy requests to each container.

Docker Compose + Spring Boot + Postgres connection

I have a Java Spring Boot app which works with a Postgres database. I want to use Docker for both of them. I initially put just the Postgres in Docker, and I had a docker-compose.yml file defined like this:
version: '2'
services:
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
Then, when I issued the commands sudo dockerd and sudo docker-compose -f docker-compose.yml up, it was starting the database. I could connect using pgAdmin for example, by using localhost as server and port 5432. Then, in my Spring Boot app, inside the application.properties file I defined the following properties.
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
spring.datasource.username=sample
spring.datasource.password=sample
spring.jpa.generate-ddl=true
At this point I could run my Spring Boot app locally through Spring Suite, and it all was working fine. Then, I wanted to also add my Spring Boot app as Docker image. I first of all created a Dockerfile in my project directory, which looks like this:
FROM java:8
EXPOSE 8080
ADD /target/manager.jar manager.jar
ENTRYPOINT ["java","-jar","manager.jar"]
Then, I entered to the directory of the project issued mvn clean followed by mvn install. Next, issued docker build -f Dockerfile -t manager . followed by docker tag 9c6b1e3f1d5e myuser/manager:latest (the id is correct). Finally, I edited my existing docker-compose.yml file to look like this:
version: '2'
services:
web:
image: myuser/manager:latest
ports:
- 8080:8080
depends_on:
- db
db:
container_name: sample_db
image: postgres:9.5
volumes:
- sample_db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
volumes:
sample_db: {}
But, now if I issue sudo docker-compose -f docker-compose.yml up command, the database again starts correctly, but I get errors and exit code 1 for the web app part. The problem is the connection string. I believe I have to change it to something else, but I don't know what it should be. I get the following error messages:
web_1 | 2017-06-27 22:11:54.418 ERROR 1 --- [ main] o.a.tomcat.jdbc.pool.ConnectionPool : Unable to create initial connections of pool.
web_1 |
web_1 | org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections
Any ideas?
Each container has its own network interface with its own localhost. So change how Java points to Postgres:
spring.datasource.url=jdbc:postgresql://localhost:5432/sample
To:
spring.datasource.url=jdbc:postgresql://db:5432/sample
db will resolve to the proper Postgres IP.
Bonus. With docker-compose you don't need to build your image by hand. So change:
web:
image: myuser/manager:latest
To:
web:
build: .
I had the same problem and I lost some time to understand and solve this problem:
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
I show all the properties so that everyone understands.
application.properties:
spring.datasource.url=jdbc:postgresql://localhost:5432/testdb
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.username=postgres
spring.datasource.password=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL82Dialect
spring.jpa.hibernate.ddl-auto=update
docker-compose.yml:
version: "3"
services:
springapp:
build: .
container_name: springapp
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
ports:
- 8000:8080
restart: always
depends_on:
- db
db:
image: postgres
container_name: db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=testdb
- PGDATA=/var/lib/postgresql/data/pgdata
ports:
- 5000:5432
volumes:
- pgdata:/var/lib/postgresql/data
restart: always
volumes:
pgdata:
For start spring application with local database we use url localhost.
For connect to container with database we need change 'localhost' on your database service, in my case 'localhost' to 'db'.
Solution: add SPRING_DATASOURCE_URL environment in docker-compose.yml wich rewrite spring.datasource.url value for connect:
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/testdb
I hope this helps someone save his time.
You can use this.
version: "2"
services:
sample_db-postgresql:
image: postgres:9.5
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=sample
- POSTGRES_USER=sample
- POSTGRES_DB=sample
volumes:
- sample_db:/var/lib/postgresql/data
volumes:
sample_db:
You can use ENV variable to change the db address in your docker-compose.
Dockerfile:
FROM java:8
EXPOSE 8080
ENV POSTGRES localhost
ADD /target/manager.jar manager.jar
ENTRYPOINT exec java $JAVA_OPTS -jar manager.jar --spring.datasource.url=jdbc:postgresql://$POSTGRES:5432/sample
docker-compose:
`
container_name: springapp
environment:
- POSTGRES=db`

Resources