Rate limiting on top of WebFlux retry
I want to limit the number of retires from WebFlux. The use case is that if the service to be invoked goes down then i end up retrying for all read timeouts which in-turn creates double the load.
I figured out a way to write custom methods for checking the feasibility of retry but that looks to be more of a hack. Is there any cleaner approach to follow for this use case?
Based on the question tags, you already figured out what you need: circuit breaker.
Resilience4j circuit breaker has support for Project Reactor: https://resilience4j.readme.io/docs/examples-1#decorate-flowable-with-a-circuitbreaker
Related
I am building a Spring batch job and in the Item processor step I am consuming an external end-point and saving the values to DB. The external point at times is very slow and takes more than 60 sec to respond. So, as a work around I implemented restTemplate timeout(15s) but, how to implement circuit breaker techniques here. As a result of this my transaction is timing out (even after implementing timeout). Are there any solutions to overcome this out of box in spring-batch.
how to implement circuit breaker techniques here
You can annotate the ItemProcessor#process with #CircuitBreaker (see attributes like maxAttempts, resetTimeout, etc) from the spring-retry library and add a recovery method that you annotate with #Recover.
Michael Minella gives a complete sample of this very scenario in his talk: Cloud Native Batch Processing. And you can find the code example here.
Can we use both together in Spring Boot during the development of microservice?
These are fundamentally different patterns.
A circuit breaker pattern is implemented on the caller, to avoid overwhelming a service which may be struggling to handle calls. A sample implementation in Spring can be found here.
A bulkhead pattern is implemented on the service, to prevent a failure during the handling of a single incoming call impacting the handling of other incoming calls. A sample implementation in Spring can be found here.
The only thing these patters have in common is that they are both designed to increase the resilience of a distributed system.
While you can certainly use them together in the same service, you must understand that they are not related to each other, as one is concerned with making calls and the other is concerned with handling calls.
Yes, they can be used together, but it's not always necessary.
As #tom redfern said, circuit breaker is implemented on the caller side. So, if you are sending request to another service, you should wrap those requests into a circuit breaker specific to that service. Keep in mind that every other third party system or service should have it's own circuit breaker. Otherwise, the unavailability of one system will impact the requests that you are sending to the other by opening the circuit breaker.
More informations about circuit breaker can be found here: https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker
Also, #tom redfern is right again in the case of bulkheading, this is a pattern which is implemented in the service that is called. So, if you are reacting to external requests by spanning other multiple requests or worloads, you should avoid doing all those worloads into a single unit (thread). Instead, separate the worloads into pieces (thread pools) for each request that you have spanned.
More information about bulkheading can be found here: https://learn.microsoft.com/en-us/azure/architecture/patterns/bulkhead
Your question was if it's possible to use both these patterns in the same microservice. The answer is: yes, you can and very often the situation implies this.
I am trying to add exception filtering to MassTransit circuit breaker. I'd like to use CircuitBreaker only for TimeoutExceptions. Is this possible?
Exception filtering is available for retry/redelivery:
endpoint.UseRetry(config =>
{
config.Handle<TimeoutException>();
config.Interval(10, TimeSpan.FromSeconds(5));
});
Yes, in the above example, the message would retry up to ten times, waiting five seconds between attempts, but only if the exception thrown is a TimeoutException.
Also, I'd suggest using UseMessageRetry instead of UseRetry going forward.
You can configure multiple retry filters, each with different policies and filters.
The details of filters is in the documentation.
Also, make sure the retry configuration is prior to any consumer configuration. Pipelines are built in-order, so order matters.
I am starting a project using spring webflux reactive stack which by default uses Reactor Netty as the server. Pls correct me if i'm wrong, but i read that Netty can only have maximum number of event loops as the amount of processors on the instance.
This means that if a request gets blocked for a second (which should not be the use case i know, just for example), we would only be able to get max 1 Transaction Per Second if there is only 1 processor on instance.
I am wondering how scalable Netty is compared to servlet container like Tomcat? What are the pros and cons of using Netty vs Tomcat?
I also want to know the ways to optimize Netty configurations to make sure it is production ready.
This means that if a request gets blocked for a second (which should not be the use case i know, just for example)
the whole purpose of this stack is to scale hugely on a limited amount of resources (here, threads). This is all built on the critical requirement that every step is asynchronous and non-blocking.
So your "just for example" doesn't make any sense. Yes, if you block for one second that CPU will only process that single request during that second. That is also completely wrong of you to do so, and everything in the stack is made to help you avoid blocking.
Is it possible to stop the flow execution in SI based on a header/message value ?
Thanks.
You can use a Control Bus to start and stop an inbound-adapter.
If you want to stop an existing flow mid-execution, I'm not aware of any standard ESB component that will enable you to do that. You could perhaps use a Channel Interceptor and lock the thread execution manually, but this approach would only be as granular as your message endpoints.
Also, if you find a way to interrupt the execution, be careful of any timeout values you set in your flow configuration. Otherwise you may find the flow will fail when you eventually resume it!