resilience4j Spring Boot 2 - spring-boot

Trying out a simple Spring Boot 2 + Resilience4j project.
But facing an issue that the circuit breaker is always CLOSED though the host application is down.
Service class
#Autowired
private RestTemplate restTemplate;
#Bean
public RestTemplate restTemplate() {
return new RestTemplate();
}
#CircuitBreaker(name = "mainService", fallbackMethod="testFallBack")
public ResponseEntity<String> invokeService(int i) {
return restTemplate.exchange(
"http://localhost:9092/", // This service is always down
HttpMethod.GET,
null,
String.class
);
}
private ResponseEntity<String> testFallBack(int i, Exception e) {
return new ResponseEntity<String>("In fallback method", HttpStatus.INTERNAL_SERVER_ERROR);
}
Resilience4J Config
management.endpoint.health.show-details: always
management.health.circuitbreakers.enabled: true
resilience4j:
circuitbreaker:
configs:
default:
registerHealthIndicator: true
minimumNumberOfCalls: 5
permittedNumberOfCallsInHalfOpenState: 3
automaticTransitionFromOpenToHalfOpenEnabled: true
waitDurationInOpenState: 5s
failureRateThreshold: 50
eventConsumerBufferSize: 10
instances:
mainService:
baseConfig: default
The Service is being called multiple times from the Controller and I expect it to fallback after minimum 5 calls but the circuit breaker is always CLOSED and for each call from controller the host service is being called and up with Connection refused.
Dependencies: spring-boot-starter-web, resilience4j-spring-boot2, spring-aop, spring-boot-starter-actuator
Earlier I tried out programmatic approach using CircuitBreakerRegistry and Decorator Function which works as expected.

Actually, you misunderstand the circuit breaker parameters. See the documentation:
minimumNumberOfCalls, default: 100
Configures the minimum number of calls which are required (per sliding window period) before the CircuitBreaker can calculate the error rate or slow call rate.
For example, if minimumNumberOfCalls is 10, then at least 10 calls must be recorded, before the failure rate can be calculated.
If only 9 calls have been recorded the CircuitBreaker will not transition to open even if all 9 calls have failed.
slidingWindowSize, default: 100
Configures the size of the sliding window which is used to record the outcome of calls when the CircuitBreaker is closed.
In your configuration, there is
minimumNumberOfCalls: 5
slidingWindowSize: 100 ## implicitly, because you have not set this parameters
And you
expect it to fallback after minimum 5 calls
However, your circuit breaker opens after 100 failures, not after 5.

Related

Resilience4j How to route to fallback method then return back to original method after specific amount of time

I am working with resilience4j and spring boot,
I need to accomplish the below scenario,
When I have a failure in the originalMethod
After 5 attempts route to the fallback method
After a specific time like 5 minutes return back to the originalMethod
I tried with retry as below but does not fit the problem ,
#Retry(name = "retryService", fallbackMethod = "fallback")
public String originalMethod(String data) throws InterruptedException {
//..... call external service
}
public String fallback(String data, Throwable t) {
logger.error("Inside retryfallback, cause – {}", t.toString());
return "Inside retryfallback method. Some error occurred ";
}
Added properties
resilience4j.retry:
instances:
retryService:
maxRetryAttempts: 5
waitDuration: 50000
I think you can use a circuit breaker for sometime when a failure limit reached to achieve the behavior you want.
By adding #CircuitBreaker(...) annotation and specifying the failureRateThreshold, waitDurationInOpenState and the other needed config properties for that instance.

Resiliency4j circuit breaker with retry configuration not working

I am using both #CircuitBreaker and #Retry annotations on a service method. When I apply both the configurations, the retry configurations are not taking affect.
Below is the configuration:
resilience4j:
circuitbreaker:
instances:
inventorymicroservice:
registerHealthIndicator: true
ringBufferSizeInClosedState: 5
ringBufferSizeInHalfOpenState: 3
waitDurationInOpenState: 30000
failureRateThreshold: 50
slowCallRateThreshold: 50
recordExceptions:
- java.io.IOException
- java.util.concurrent.TimeoutException
- java.net.ConnectException
- org.springframework.web.reactive.function.client.WebClientRequestException
retry:
instances:
retryConfig:
maxAttempts: 3
waitDuration: 10s
enableExponentialBackoff: true
exponentialBackoffMultiplier: 2
retryExceptions:
- org.springframework.web.client.HttpServerErrorException
- java.io.IOException
- java.io.IOException
- java.util.concurrent.TimeoutException
- java.net.ConnectException
- org.springframework.web.reactive.function.client.WebClientRequestException
Service method:
#CircuitBreaker(name = "inventorymicroservice", fallbackMethod = "fallBack")
#Retry(name = "retryConfig", fallbackMethod = "fallBack")
public Order saveOrder(Order order){
Order savedOrder = this.orderRepository.save(order);
log.info("Calling the inventory service to update the quantity :: ");
//ResponseEntity<Integer> integerResponseEntity = this.restTemplate.postForEntity("http://localhost:9222/api/inventory", null, Integer.class);
Integer response = this.webClient
.post()
.uri("/api/inventory")
.retrieve()
.bodyToMono(Integer.class)
.block();
log.info("Response from the inventory microservice :: {}", response);
return savedOrder;
}
private Order fallBack(Throwable exception){
log.error("Exception while invoking the REST endpoint :: ", exception.getMessage());
return Order.builder().build();
}
Where am I going wrong? Also, how to convert this configuration to programmatic configuration using functional programming.
The default Resilience4j aspect order is
Retry( CircuitBreaker( RateLimiter( TimeLimiter( Bulkhead( function)))))
Your CircuitBreaker has a fallback, so it never throws an exception, so Retry never sees a failed invocation to retry.
Remove the Retry fallback, and change the aspect order so CircuitBreaker does its work after Retry.
resilience4j:
circuitbreaker:
circuitBreakerAspectOrder: 1
retry:
retryAspectOrder: 2

Change timeout for Spring Cloud Circuit Breaker at runtime?

I'm using Spring Cloud Circuit Breaker 2.0.0 (resilience4j implementation) for circuit breaking and timeouts in my application. I've created the following configuration:
#Bean
public Customizer<Resilience4JCircuitBreakerFactory> defaultCustomizer() {
return factory ->
factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
.timeLimiterConfig(TimeLimiterConfig.custom().timeoutDuration(Duration.ofSeconds(10)).build())
.circuitBreakerConfig(..)
.build());
}
Now I want to write an integration test to verify that my behavior is correct when a timeout occurs. For this to work, I'd like to temporarily change the timeout duration specified in the configuration above to something like 1 millisecond instead of 10 seconds.
So my question is: How can I change the value of the timeout of the TimeLimiterConfig(temporarily) when I'm writing a Spring Boot integration test?
You can use the #Value Spring annotation that retrieves the value at a configuration file from your resource folder src/main/resources/common.properties.
#Bean
public Customizer<Resilience4JCircuitBreakerFactory> defaultCustomizer(
#Value("${duration.milli:600}") int durationMilli) {
return factory ->
factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
.timeLimiterConfig(TimeLimiterConfig.custom().timeoutDuration(Duration.ofMilli(durationMilli)).build())
.circuitBreakerConfig(..)
.build());
}
Then you set the value at src/main/resources/common.properties
duration.milli=600
When you are doing your test you can configure another resource file at the test folder src/test/resources/common.properties with a different value.
duration.milli=1

Get Instance of circuit breaker from configuration file

This is my configuration file.
resilience4j.circuitbreaker:
instances:
backendB:
registerHealthIndicator: true
slidingWindowSize: 10
minimumNumberOfCalls: 10
permittedNumberOfCallsInHalfOpenState: 3
waitDurationInOpenState: 5s
failureRateThreshold: 50
eventConsumerBufferSize: 10
I am trying to create bean of my feign client using Resilience4jFeign but the circuit breaker object is initialized with default configuration as the name suggest CircuitBreaker.ofDefaults. I can't find any ways to get my instance of circuit breaker from configuration to an object.
#Bean
public CommunicationServiceProxy communicationServiceProxy(){
CircuitBreaker circuitBreaker = CircuitBreaker.ofDefaults("backendB");
FeignDecorators decorators = FeignDecorators.builder()
.withCircuitBreaker(circuitBreaker)
.withFallbackFactory(CommunicationFallBack::new)
.build();
return Resilience4jFeign
.builder(decorators)
.encoder(new JacksonEncoder())
.decoder(new JacksonDecoder())
.target(CommunicationServiceProxy.class, "http://localhost:8081");
}
I found the solution and it was using the CircuitBreakerRegistry.
Instead of creating registry using
CircuitBreakerRegistry.ofDefaults()
one need to autowired them.
#Autowired
private CircuitBreakerRegistry circuitBreakerRegistry;

resilience4j + Spring instance of CircuitBreaker

I would like to use Resilience4j to deal with fault tolerance, I am using CircuitBreaker and TimerLimit.
I would like to separate business logic of fault tolerance behavior, to not "dirty" my business code.
So, I am thinking to use Command pattern to execute my methods that will be treat, like Hystrix do with HystrixCommand.
Example:
public class MyCommand {
private static final CircuitBreaker circuitBreaker;
private Long param1, param2;
private MyService myService;
private static final TimeLimiter timeLimiter;
static {
long ttl = 50000;
TimeLimiterConfig configTimerLimit
= TimeLimiterConfig.custom().timeoutDuration(Duration.ofMillis(ttl)).build();
timeLimiter = TimeLimiter.of(configTimerLimit);
// I got the configuration from a class that I created.
circuitBreaker = CircuitBreaker.of("my", CircuitBreakerConfigOptions.defaultForExternalService());
}
public MyCommand(Long param1, Long param2, MyService myService) {
this.param1 = param1;
this.param2 = param2;
this.myService = myService;
}
public String run() {
Callable<String> stringCallable = TimeLimiter.decorateFutureSupplier(timeLimiter,
() -> CompletableFuture.supplyAsync(() -> myService.hello(param1, param2)));
Callable<String> callable = CircuitBreaker.decorateCallable(circuitBreaker, stringCallable);
return Try.of(callable::call).recover(t -> fallback(t)).get();
}
protected String fallback(Throwable throwable) {
Callable<String> stringCallable = TimeLimiter.decorateFutureSupplier(timeLimiter,
() -> CompletableFuture.supplyAsync(() -> myService.otherHello(param1, param2)));
return Try.of(stringCallable::call).getOrElse("Fallback");
}
}
Calling in my Controller:
#ApiOperation(value = "Only to test")
#GetMapping(value = "/execute", produces = MediaType.APPLICATION_JSON)
public String execute() {
return new MyCommand(1L, 2L, new MyService()).run();
}
My doubts:
1 - In this case, circuitBreaker really needs to be static, because I understood that the same object needs to be shared between the same method that you want to threat, I am right?
2 - How I have many instances of this application, circuitBreaker works individually for each instance? I am right?
What I understand from your question is - You need a Resilience4j's circuit breaker which should be sperate i.e. not messed with your business logic.
So I would suggest put circuit breaker protection around run() method. below code will elaborate -
Your Controller -
#ApiOperation(value = "Only to test")
#GetMapping(value = "/execute", produces = MediaType.APPLICATION_JSON)
public String execute() {
return new MyCommand().run();
}
Now Write your run() method of MyCommand class with #CircuitBreaker
public class MyCommand {
#CircuitBreaker(name = "RUN_METHOD_PROTECTION") // <---- here is our circuit breaker annotation code top of below your business code... and that’s it.
Your_Response run(Your_Request){
// Your business logic written here...
}
Further add circuit breaker configuration in your YAML property file as below (Count based I have used instead of time-based) –
resilience4j.circuitbreaker:
backends:
RUN_METHOD_PROTECTION:
registerHealthIndicator: true
slidingWindowSize: 100 # start rate calc after 100 calls
minimumNumberOfCalls: 100 # minimum calls before the CircuitBreaker can calculate the error rate.
permittedNumberOfCallsInHalfOpenState: 10 # number of permitted calls when the CircuitBreaker is half open
waitDurationInOpenState: 10s # time that the CircuitBreaker should wait before transitioning from open to half-open
failureRateThreshold: 50 # failure rate threshold in percentage
slowCallRateThreshold: 100 # consider all transactions under interceptor for slow call rate
slowCallDurationThreshold: 2s # if a call is taking more than 2s then increase the error rate
recordExceptions: # increment error rate if following exception occurs
- org.springframework.web.client.HttpServerErrorException
- java.io.IOException
- org.springframework.web.client.ResourceAccessException
Now if you are unable to use annotation #CircuitBreaker in your project then you can also do things in functional way i.e.
Let say we you have defined a bean in your configuration,
#Bean
public CircuitBreaker MyCircuitBreaker(){
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
.slidingWindow(100,100, CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
.failureRateThreshold(50)
.build();
CircuitBreakerRegistry registry = CircuitBreakerRegistry.of(config);
CircuitBreaker circuitBreaker = registry.circuitBreaker("your_run_method_CircuitBreker"); // here you are registering your circuit breaker with a unique tag. And in future you refer this tag you get a same circuit breaker.
return circuitBreaker;
}
Now Your controller code would be below -
private CircuitBreaker circuitBreaker; // Assume you have injected the value from CircuitBreaker bean
#ApiOperation(value = "Only to test")
#GetMapping(value = "/execute", produces = MediaType.APPLICATION_JSON)
public String execute() {
Function<Your_Request, Your_Response> decorated = CircuitBreaker
.decorateFunction(circuitBreaker, new MyCommand().run());
return decorated.apply();
}
This way too you have never interfered with your business logic.
Since it seems that you are using Spring Boot, you could use the resilience4j-spring-boot-2 starter module which also has support for annotations.
https://resilience4j.readme.io/docs/getting-started-3

Resources