I am in a situation where I have a microservice environment but for one service I want that it should not be load balanced rather work in a active standby(one at a time serves the request). When one service instance goes down then only the requests should be routed to the other instance even if the first one comes up the 2nd instance should be the one servicing the requests.
I am looking for the options like - overriding ribbon's IRule or doing this in a #PreFilter which is there on each of these services.
Let me know if anyone has any implementation for the above case.
Related
I am working on the micro-service developed using Spring Boot . I have implemented following layers:
Controller layer: Invoked when user sends API request
Service layer: Processes the request. Either sends request to third-part service or sends request to database
Repository layer: Used to interact with the
database
.
Methods in all of above layers returns the CompletableFuture. I have following questions related to this setup:
Is it good practice to return Completable future from all methods across all layers?
Is it always recommended to use #Async annotation when using CompletableFuture? what happens when I use default fork-join pool to process the requests?
How can I configure the threads for above methods? Will it be a good idea to configure the thread pool per layer? what are other configurations I can consider here?
Which metrics I should focus while optimizing performance for this micro-service?
If the work your application is doing can be done on the request thread without too much latency, I would recommend it. You can always move to an async model if you find that your web server is running out of worker threads.
The #Async annotation is basically helping with scheduling. If you can, use it - it can keep the code free of the references to the thread pool on which the work will be scheduled. As for what thread actually does your async work, that's really up to you. If you can, use your own pool. That will make sure you can add instrumentation and expose configuration options that you may need once your service is running.
Technically you will have two pools in play. One that Spring will use to consume the result of your future, and another that you will use to do the async work. If I recall correctly, Spring Boot will configure its pool if you don't already have one, and will log a warning if you didn't explicitly configure one. As for your worker threads, start simple. Consider using Spring's ThreadPoolTaskExecutor.
Regarding which metrics to monitor, start first by choosing how you will monitor. Using something like Spring Sleuth coupled with Spring Actuator will give you a lot of information out of the box. There are a lot of services that can collect all the metrics actuator generates into time-based databases that you can then use to analyze performance and get some ideas on what to tweak.
One final recommendation is that Spring's Web Flux is designed from the start to be async. It has a learning curve for sure since reactive code is very different from the usual MVC stuff. However, that framework is also thinking about all the questions you are asking so it might be better suited for your application, specially if you want to make everything async by default.
If multiple request are hit to a single RestController at the same time in a application, how it is handle for different scenarios (Multiple request to a single endpoints (only GET), or Multiple requests for multiple endpoints(GET, POST, PUT...))
Is multi-threading concept utilized? If yes is it possible to handle the requests in FIFO pattern?
What is the maximum request a RestController can take ?
Does RestController scope affect handling of requests (behavior of request scope with default scope-singleton) ?
Also how it is handle by Application context (example with flow will be helpful)
Considering building Micro-services with Spring Boot 2.
From the point of view of Spring (Application Context) rest controller is a singleton if not specified otherwise.
So the code of controller must be ready to be invoked by multiple threads simultaneously.
When a new request reaches the server, in a tradition thread-per-request model the web server (like tomcat) is responsible to allocate a thread from the predefined pool of threads to the request. Then the request gets processed by controller in the context of this thread.
The actual thread pool implementation can in general vary from server to server, but in general, its something that can be configured (number of threads per loop, queue size to store requests for future processing if the pool is full, etc.)
Now regarding the Scope of RestController. If the controller is stateless (and it should be for many cases, just keep it singleton). If you need the new Instance of controller to be created per request, than change the scope. Obviously each thread will have to use the same (in case of singleton scope) instance of rest controller or spring mvc will create a new instance of controller if you specify another scope.
All the answer above applies to a "traditional" thread-per-request model.
Note that since spring 5 / spring boot 2 spring also supports "Reactive" model with a webflux. It works on top of netty and doesn't utilize a thread-per-request model. Please specify in the question if you're interested in this model rather than a tradition model that I've tried to briefly describe.
I'm using Spring Boot for microservices, and I came accross and issue with load balancing.
Spring Actuator adds special health and metrics endpoint to the apps; with this, some basic information can be acquired from the running instances.
What I would like to do, is to a create a (reverse)proxy (e.g. with Zuul and/or Ribbon?), which creates a centralized load balancer, that selects instances by their health status.
For example, I have the following microservices
client
proxy (<- I would like to implement this)
server 1
server 2
When the client sends an http request to the proxy, the proxy should be able to decide, which of the to server instances has the least load, and forward request to that one.
Is there an easy way to do this?
Thanks,
krisy
If you want to make a choice on various load-data, you could implement custom HealthIndicators that accumulate some kind of 'load over time' data, use this in your load balancer to decide where to send traffic.
All custom health indicators will be picked up by spring-boot, and invoked on the actuator /health endpoint.
#Component
class LoadIndicator implements HealthIndicator {
#Override
Health health() {
def loadData = ... do stuff to gather whatever load
return Health.up()
.withDetail("load", loadData)
.build();
}
}
Perhaps you could already use some of spring-boots metrics already, there's multiple endpoints in the actuator. /beans, /trace, /metrics. Should be possible to find that data in your application too.
I want the following features in spring-cloud-Eureka backed microservices application.
1) Load balancing - if I have 3 nodes for one service, load balancing should happen between them
2)Retry logic - if one of the nodes did not respond, retry should happen for certain number ( eg 3. should be configurable) before falling back to another node.
3)circuit breaker - if for some reasons, all the 3 nodes of service is having some issue accessing db and throwing exceptions or not responding, the circuit should get open, fall back method called and circuit automatically closes after the services recovers.
Looking at many examples of Spring-cloud, I figured out
1) RestTemplate will help with option 1. but when RestTemplate access one instance of service and if the node fails, will it try with other two nodes?
2) Hystix will help with circuit breaker option (3 above). but if just one node is not responding, will it try other nodes, before opening up circuit and call fallback method. and will it automatically close circuit once the service recovers?
3) how to get retryLogic with spring-cloud? I do know about #Retryable annotation. But will it help in the following situation?
Retry with one node for 3 times and after it fails, try the next node 3 times and the last node 3 times before circuit breaker kicks in.
I see that all these configurations are available in spring cloud. but having a hard-time understanding how to configure for all these for efficient solution.
Here is one proposed:
#HystrixCommand
#Retryable
public Object doSomething() {
// use your RestTemplate here
}
But I don't totally know if it is going to help me with all the subtleties I mentioned above.
I do see there is a #FeignClient. But from this blog, I understand that it provides a high level feature for HTTP client requests. Does it help with retry and circuit breaker and load balancing all-in-one?
Thanks
I do see there is a #FeignClient. Does it help with retry and circuit breaker and load balancing all-in-one?
If you are using the full spring-cloud stack, it actually solves everything you mentioned.
The netflix components in this scenario are the following in spring-cloud:
Eureka - Service Registry
Let's you dyanmically register your services so you only need to fix one host in your app (eureka).
Ribbon - Load balancer
Out of the box it's providing you with round robin loadbalancing, but you can implement your own #RibbonClient (even for a specific service) and design your custom loadbalancing for example based on eureka metadata. The loadbalancing happens on the client side.
Feign - Http client
With #FeignClient you can rapidly develop clients for you other services (or services outside of your infrastructure). It is integrated with ribbon and eureka so you can refer to your services #FeignClient(yourServiceNameInEureka) and what you end up with is a client which loadbalances between the registered instances with your preferred logic. If you are using spring you can use the familiar #RequestMapping annotation to describe the endpoint you are using.
Hystrix - Circuit breaker
By default your feign clients will use hystrix, every request will be wrapped in a hystrix command. You can of course create hytrix commands by hand and configure them for your needs.
You have to configure a little to get thees working (actually just a few #Enable annotation on your configuration).
I highly recommend reading the provided spring documentation because it wraps up almost all of your aspects in a fairly quick read.
http://cloud.spring.io/spring-cloud-netflix/spring-cloud-netflix.html
I'm using Curator service discovery with Spring MVC rest controllers where each controller registers itself with ZooKeeper in #PostConstruct and de-registers itself in #PreDestroy.
The problem I'm having is that by the time the #PreDestroy method is called, the controller is already no longer servicing requests. I need to de-register the controller before the controller stops servicing requests to avoid throwing exceptions for the small number of requests that occur between the controller stopping and de-registration.
I've tried the spring ApplicationListener interface, SmartLifecycle, and ServletContextListener and in all shutdown/close related hooks, the controller has already stopped servicing requests.
I need a shutdown hook where I can deregister before the controller stops servicing requests, and I'm not sure one is available.
I don't believe such hook exists in Spring.
However, you could have a special controller, mapped to a specific reserved url, i.e. ending in /activity/suspend-traffic. You might want to have some sort of basic security for that, maybe an application specific token, i.e. /activity/suspend-traffic/{token}.
When this special controller receives the correct token, it deregisters all application controllers from ZK.
This way you wouldn't loose any request. Then, when the application is no longer receiving traffic, you could safely shutdown the server. After sending /activity/suspend-traffic/{token}, you could wait a fixed amount of time before shuting down, so that the server finishes processing the requests it has received after the suspend command.
Or you might implement a more sophisticated mechanism, i.e. via a filter or mvc interceptor that counts how many requests are "inside" the application. You could have another mapping in that special controller, i.e. /activity/request-count that returns the actual number of requests being processed. When this request returns 0, it would be safe to shutdown the server.