We are facing one problem, that sometimes our RabbitMQ server crashes due to some reasons. To connect RabbitMQ again with micro-services we need to restart the spring boot micro-services, Now is there a way that we can skip the restarts, so whenever RabbitMQ comes up, services' connection to RabbitMQ should automatically be created and should start working as expected.
Related
We use Consul by HashiCorp for configuration management in our spring boot application. Sometimes consul agent is occasionally unavailable when our app starts. It is inconvenient that the app fails if consul isn’t available for configuration.
I read about Consul Retry https://cloud.spring.io/spring-cloud-static/spring-cloud-consul/1.2.3.RELEASE/single/spring-cloud-consul.html#spring-cloud-consul-retry
Although it is a good solution for retrying, is there any solution in which the application retries indefinitely until it connects with consul?
We deploy this app on tomcat. Is there a better solution than consul-retry for reconnection? Should we try reloading app on tomcat whenever it fails ?
Can anyone tell me the best practice for reconnection in this scenario?
I am developing a Spring Boot Client cache application (spring-geode-starter version 1.7.5), which connects a gfsh locator and server ( say locator1 , server1).
During startup of the client application, spring boot automatically connects the locator and server running.
Now if I stop server from gfsh terminal , I observe in log, the client application gets notified with some log console messages.. also when i restart server from gfsh, i see client log message of auto reconnection done towards server.
My objective is : how to handle in some event listener in my client application code to get notified these disconnect and reconnect events, so that I need to do some extra logic work during these events.
can any one suggest and help me..
I am new to PCF and need some help understanding the root cause of an issue I am facing.
We have one PCF application that is connecting to Rabbit MQ (no binding done on the Rabbit MQ service yet). the messages get passed through the application to queues created on this Rabbit MQ instance. so far so good.
But when we bind again the application to the same Rabbit MQ instance and restage the application, it starts getting connected to a different messaging instance and the message is being relayed to that messaging service and no message transfer happened on expected Rabbit MQ instance although I can't see the actual movement of messages because I might not be having access to the mystery messaging service (I say that because two spring boot services are communicating as expected and the process gets completed).
I am clueless as to what is changing in the application by binding it to Rabbit MQ. is it a default spring implementation of spring AMQP which is interfering here? what can I do to stop this behavior? I haven't provided any parameters while binding the app to Rabbit MQ.
I tried to put the loggers to find out the name of the exchange it's getting connected to but it's still the same exchange, but I can't see the messages flowing and can't explain the phenomenon to the client.
Try to look at the service key of the your rabbit instance and that will give you hostname etc. i.e. rabbitmq details where you are connecting to after binding.
cf service-key SERVICE_INSTANCE SERVICE_KEY
Not sure if there is any default setting in the app that's being used for setting up the connection to a different rabbit instance in absence of binding.
I have a Spring Boot 2 standalone application( not REST service) which connect to rabbit MQ and process message. The application is deployed in kubernetes. While it work great, but when Rabbit MQ remain down for little longer and in logs I see hearbeat exception 60sec and eventually connection get drop even if the rabbit mq comes up after certain time:
Automatic retry connection to broker by spring-rabbitmq
https://www.rabbitmq.com/heartbeats.html
While I try to manage above issue by increasing number of retry :https://stackoverflow.com/questions/45385119/how-configure-timeouts-retries-or-max-attempts-in-differents-queues-with-spring
but after expiry of retry still above issue comes.
How can I reboot/delete-recreate pod if I see above issue in logs from kubernetes.
The easiest way is to use actuator, which has a /actuator/health endpoint. (Note that the recent version also add /actuator/health/liveness and /actuator/health/readiness).
You can assign the endpoint to livenessProbe property of k8s. Then it will automatically restart when it is necessary. You can parameterize, when your app is down if necessary.
See the docs:
Kubernetes liveness probe
Spring actuator health
I use spring cloud config bus (rabbitmq) in my micro-service. Only purpose for me to use rabbitmq in my microservice is spring cloud bus... I have 2 questions below.
When I was experimenting, I found that spring expects rabbitmq to be UP and running during application start. Which is contrary to what Spring cloud evangelises... (Circuit breakers...) To be fair, even service discovery is not expected to be up and running before starting an application. Is there any sensible reason behind this...?
Say, I start my application when rabbitmq is up and running. For some reason, rabbitmq goes down... What I should be losing is just my ability to work with rabbitmq... instead, /health endpoint responds back as DOWN for my micro-service. Any eureka instance listening to heart beats from my micro-service is also marking the instance as down. Any reasons for doing this...?
To my knowledge, this is against the circuit breaker pattern that spring cloud has evangelised.
I personally feel that spring cloud config bus is not an important feature to mark an application as down...
Is there any alternatives to tell my spring boot micro-service that connection to rabbitmq is not a critical service?
Thanks in advance!