I'm learning Consul.Got confused by relation between Spring Cloud Consul and Consul Client.
I found that, in Spring boot applications, we can use #EnableDiscoveryClient to contact to a Consul Agent. But if I want to election a leader of my own service, can Spring Cloud Consul provides these interfaces？Or I need to relay on consul-client?
The election of the leader in the consul cluster is make of internal algorithm of consul based in RAFT that is an algoritm of consensum, for more information you can see:
https://www.consul.io/docs/internals/consensus.html
The clients of consul is a wrapper of API REST expoused by Consul, normaly you must have you consul servers behind of the load balancer and this url is that you configure in your clients for example in spring boot.
spring.cloud.consul.host = url_load_balancer
spring.cloud.consul.port = port_load_balancer
In conclusion the client can't manage the leader election of a consul cluster, but the closest approach that i find is:
https://www.consul.io/docs/guides/leader-election.html
i hope that it help you
Related
I'm working with spring cloud consul as a config server and service discovery tool in test environment. according to the consul documentation, consul server need to be at least 3 to 5 recommended servers, and consul will automatically choose a leader server with internal election. The problem is in consul cluster, which server I need to choose to connect in my spring boot service? if leader server changed, what will happen to my service? what is the best practice of using consul server in a real production environment?
I need to mention that, I already studied consul website documentation, but what I need is a real example for architecture and best practice for production environment. note that my services have high TPS due to USSD transactions in a telecom company. Thanks in advance.
I have tried to connecting to consul server and I'm using it in a test environment as config and service discovery tool. I have nearly 100 service to use consul as their config and discovery server within an internal network with 100 VMs and containers.
I expect a architecture for production environment to use consul in real production environment with pros and cons.
I am trying to register a spring boot app to a consul cluster.
I have 3 node consul cluster 1 master 2 agents.
I have a load-balancer in front of 2 consul agents, so that it is HA.
In my application.yml. I ask the services to join via load-balancer
spring
cloud:
consul:
enabled: true
port: loadbalancer_port
host: http://loadbalancer
discovery:
instance-id: ${info.app.environment}:${spring.application.name}
tags:
- ${spring.profiles.active}
Now, when my service restarts it is creating a duplicate entry in consul.
I figured that, because it is being registered in 2 different agents.
Does this mean, I cant have HA consul with loadbalancer ? or should I ask the services to register to particular agents with out load-balancer?
Please help!!
Consul is designed to have a Consul client agent deployed on each server in your data center (see Consul Reference Architecture). Instead of registering services centrally, services running on a machine are registered with the local/co-located Consul agent. The agents then submit the list of services registered against them to the Consul servers, which then aggregates this info from each agent to form the service catalog. The catalog maintains the high-level view of the cluster, including which services are available, which nodes run those services, health information, etc.
TLDR; Remove the load balancer and register the services directly with the agents in order to avoid this issue where service registrations are duplicated across hosts.
I have a Spring Boot (2.3.3) service using spring-kafka to currently access a dedicated Kafka/Zookeeper configuration. I have been using the application.properties setting spring.kafka.bootstrap-servers=localhost:9092 to access my dev/test Apache Kafka service.
However, in production, we have a Cluster of Kafka Brokers (on many servers) configured in Zookeeper, and I have been asked to modify my service to query Zookeeper to get the list of brokers and use that list instead of the bootstrap servers configuration. Reason, our DevOps folks have been known to reconfigure servers/nodes and Kafka brokers.
Basically, I have been asked to make my service agnostic to where the Apache Kafka brokers are running. All my service needs to know is how to get the list of brokers (bootstrap server info including host and port) from Zookeeper.
Is there a way in spring-boot and spring-kafka to retrieve from Zookeeper the broker list and use that broker (aka bootstrap server) list in my service?
Spring delegates to the kafka-clients for all connections; for a long time now, the kafka-clients no longer connect to Zookeeper, only to the brokers themselves.
There is no built-in support in Spring for querying the Zookeeper to determine the broker list.
Furthermore, in a future Kafka version, Zookeeper is going away altogether; see KIP-500.
I use Spring Cloud Consul to discover the services I need for my application. The discovery client returns all registered instances for the requested service although I also returns failing instances.
Consul itself marks the failing instances corretly as failed (critical).
So, why does the discovery client not remove critical/failed instances? Or is where is the documentation which describes this?
After reading the source of Spring Cloud Consul I found out that I have to set the property spring.cloud.consul.discovery.queryPassing as follows
java -Dspring.cloud.consul.discovery.queryPassing=true ...`
The relevant source files are:
ConsulDiscoveryClient.java
ConsulDiscoveryProperties.java
After setting this property the discovery client for Consul will return only those instance which pass all Consul health checks.
I use spring cloud config bus (rabbitmq) in my micro-service. Only purpose for me to use rabbitmq in my microservice is spring cloud bus... I have 2 questions below.
When I was experimenting, I found that spring expects rabbitmq to be UP and running during application start. Which is contrary to what Spring cloud evangelises... (Circuit breakers...) To be fair, even service discovery is not expected to be up and running before starting an application. Is there any sensible reason behind this...?
Say, I start my application when rabbitmq is up and running. For some reason, rabbitmq goes down... What I should be losing is just my ability to work with rabbitmq... instead, /health endpoint responds back as DOWN for my micro-service. Any eureka instance listening to heart beats from my micro-service is also marking the instance as down. Any reasons for doing this...?
To my knowledge, this is against the circuit breaker pattern that spring cloud has evangelised.
I personally feel that spring cloud config bus is not an important feature to mark an application as down...
Is there any alternatives to tell my spring boot micro-service that connection to rabbitmq is not a critical service?
Thanks in advance!