We use Consul by HashiCorp for configuration management in our spring boot application. Sometimes consul agent is occasionally unavailable when our app starts. It is inconvenient that the app fails if consul isn’t available for configuration.
I read about Consul Retry https://cloud.spring.io/spring-cloud-static/spring-cloud-consul/1.2.3.RELEASE/single/spring-cloud-consul.html#spring-cloud-consul-retry
Although it is a good solution for retrying, is there any solution in which the application retries indefinitely until it connects with consul?
We deploy this app on tomcat. Is there a better solution than consul-retry for reconnection? Should we try reloading app on tomcat whenever it fails ?
Can anyone tell me the best practice for reconnection in this scenario?
Related
We are facing one problem, that sometimes our RabbitMQ server crashes due to some reasons. To connect RabbitMQ again with micro-services we need to restart the spring boot micro-services, Now is there a way that we can skip the restarts, so whenever RabbitMQ comes up, services' connection to RabbitMQ should automatically be created and should start working as expected.
I have a springboot microservice application, deployed as docker image in a Kubernetes cluster.
This microservice is (also) an Ignite Client.
If for some reason Ignite Bean fails the startup (for example because there is no Ingnite Server service), the overall application is not able to startup.
I would like to make not mandatory the complete startup of Ignite Bean, so my service can be up & running, even if it cannot use Ignite.
Thanks in advance
I have Spring boot mongo db app, when i start running this application if mongo host is alive, am able to see the my app is up and running successfully,
if my mongo host is down when i start my application my app failed to start.
is there any way even thought if mongo host is down my application should be up and running.
could someone please help me on this?
am using spring boot mongo properties in my application
spring.data.mongodb.repositories.enabled=true
spring.data.mongodb.port=27017
spring.data.mongodb.database=db
spring.data.mongodb.uri=mongodb://mongo-node-1.ballu/db
have same problem with spring boot kafka also.
Sorry for the previous comment. It was for excluding auto config beans, anyway
Is there any way even though if mongo host is down my application
should be up and running.
Yes,
spring.datasource.continue-on-error=true #Whether to stop if an error occurs while initializing the database.
as per spring doc
By default, Spring Boot enables the fail-fast feature of the Spring
JDBC initializer. This means that, if the scripts cause exceptions,
the application fails to start. You can tune that behavior by setting
spring.datasource.continue-on-error.
and as of spring kafka try this(i'm not sure if this meets your requirement)
spring.kafka.admin.fail-fast=true # Whether to fail fast if the broker is not available on startup.
I have created a spring boot integration app and deployed it to Pivotal Cloud Foundry (PCF) environment. It works for couple of days and then it starts to crash randomly afterwards. I checked the PCF logs and found this information about the crash.
OUTApp instance exited with guid 3c348d47-48c4-403f-950a-29af1efa551d
payload: {"instance"=>"e2122543-214f-4806-62c7-00e1", "index"=>2,
"reason"=>"CRASHED", "exit_description"=>"Instance became unhealthy: Failed
to make HTTP request to '/health' on port 8080: timed out after 1.00
seconds", "crash_count"=>1, "crash_timestamp"=>1511959503256098495,
"version"=>"10cea919-d490-460d-83d6-5132c96ef781"}
My CPU utilization is not much. My memory is also not leaking.
Information about the application deployed in PCF:
Spring boot integration app connects to IBM MQ queues and polls for messages and then calls couple of web services.
There is also another application Service Bus, which makes the health check call on PCF application to check if the PCF app is available or not. If Service Bus finds that PCF app is available then the requests are routed to PCF else they are processed at Service Bus end itself.
Please let me know, how to find the root cause of the CRASH and fix it.
Thanks in advance. Please let me know, if you need further details.
I have changed the health check type to port type from http in manifest.yml file.
configuration change in manifest file is as follows:
health-check-type: port
Now the app is not crashing. It is working fine. Hope this helps.
We've three Spring Boot applications:
Eureka Service
Config Server
Simple Web Service making use of Eureka and Config Server
I've set up the services so that we use a Eureka First Discovery, i.e. the simple web application finds out about the config server from the eureka service.
When started separately (either locally or by starting them as individual docker images) everything is ok, i.e. start config server after discovery service is running, and the Simple web service is started once the config server is running.
When docker-compose is used to start the services, they obviously start at the same time and essentially race to get up and running. This isn't an issue as we've added failFast: true and retry values to the simple web service and also have the docker container restarting so that the simple web service will eventually restart at a time when the discovery service and config server are both running but this doesn't feel optimal.
The unexpected behaviour we noticed was the following:
The simple web service reattempts a number of times to connect to the discovery service. This is sensible and expected
At the same time the simple web service attempts to contact the config server. Because it cannot contact the discovery service, it retries to connect to a config server on localhost, e.g. logs show retries going to http://localhost:8888. This wasn't expected.
The simple web service will eventually successfully connect to the discovery service but the logs show it stills tries to establish communication to the config server by going to http://localhost:8888. Again, this wasn't ideal.
Three questions/observations:
Is it a sensible strategy for the config client to fall back to trying localhost:8888 when it has been configured to use discovery to find the config server?
When the eureka connections is established, should the retry mechanism not now switch to trying the config server endpoint as indicated by Eureka? Essentially putting in higher/longer retry intervals and periods for the config server connection is pointless in this case as it's never going to connect to it if it's looking at localhost so we're better just failing fast.
Are there any properties that can override this behaviour?
I've created a sample github repo that demonstrates this behaviour:
https://github.com/KramKroc/eurekafirstdiscovery/tree/master