Resiliency4j circuit breaker with retry configuration not working - spring-boot

I am using both #CircuitBreaker and #Retry annotations on a service method. When I apply both the configurations, the retry configurations are not taking affect.
Below is the configuration:
resilience4j:
circuitbreaker:
instances:
inventorymicroservice:
registerHealthIndicator: true
ringBufferSizeInClosedState: 5
ringBufferSizeInHalfOpenState: 3
waitDurationInOpenState: 30000
failureRateThreshold: 50
slowCallRateThreshold: 50
recordExceptions:
- java.io.IOException
- java.util.concurrent.TimeoutException
- java.net.ConnectException
- org.springframework.web.reactive.function.client.WebClientRequestException
retry:
instances:
retryConfig:
maxAttempts: 3
waitDuration: 10s
enableExponentialBackoff: true
exponentialBackoffMultiplier: 2
retryExceptions:
- org.springframework.web.client.HttpServerErrorException
- java.io.IOException
- java.io.IOException
- java.util.concurrent.TimeoutException
- java.net.ConnectException
- org.springframework.web.reactive.function.client.WebClientRequestException
Service method:
#CircuitBreaker(name = "inventorymicroservice", fallbackMethod = "fallBack")
#Retry(name = "retryConfig", fallbackMethod = "fallBack")
public Order saveOrder(Order order){
Order savedOrder = this.orderRepository.save(order);
log.info("Calling the inventory service to update the quantity :: ");
//ResponseEntity<Integer> integerResponseEntity = this.restTemplate.postForEntity("http://localhost:9222/api/inventory", null, Integer.class);
Integer response = this.webClient
.post()
.uri("/api/inventory")
.retrieve()
.bodyToMono(Integer.class)
.block();
log.info("Response from the inventory microservice :: {}", response);
return savedOrder;
}
private Order fallBack(Throwable exception){
log.error("Exception while invoking the REST endpoint :: ", exception.getMessage());
return Order.builder().build();
}
Where am I going wrong? Also, how to convert this configuration to programmatic configuration using functional programming.

The default Resilience4j aspect order is
Retry( CircuitBreaker( RateLimiter( TimeLimiter( Bulkhead( function)))))
Your CircuitBreaker has a fallback, so it never throws an exception, so Retry never sees a failed invocation to retry.
Remove the Retry fallback, and change the aspect order so CircuitBreaker does its work after Retry.
resilience4j:
circuitbreaker:
circuitBreakerAspectOrder: 1
retry:
retryAspectOrder: 2

Related

Resilience4j How to route to fallback method then return back to original method after specific amount of time

I am working with resilience4j and spring boot,
I need to accomplish the below scenario,
When I have a failure in the originalMethod
After 5 attempts route to the fallback method
After a specific time like 5 minutes return back to the originalMethod
I tried with retry as below but does not fit the problem ,
#Retry(name = "retryService", fallbackMethod = "fallback")
public String originalMethod(String data) throws InterruptedException {
//..... call external service
}
public String fallback(String data, Throwable t) {
logger.error("Inside retryfallback, cause – {}", t.toString());
return "Inside retryfallback method. Some error occurred ";
}
Added properties
resilience4j.retry:
instances:
retryService:
maxRetryAttempts: 5
waitDuration: 50000
I think you can use a circuit breaker for sometime when a failure limit reached to achieve the behavior you want.
By adding #CircuitBreaker(...) annotation and specifying the failureRateThreshold, waitDurationInOpenState and the other needed config properties for that instance.

Spring Data Neo4j: Unable to acquire connection from the pool within configured maximum time

I am doing stress testing of my Neo4j client which is built using spring-boot-starter-data-neo4j.
After sometime passed, application throwing error Unable to acquire connection for any request.
This is my bean configuration for Neo4j connection. can someone check and tell what can be done
id("org.springframework.boot") version "2.4.0"
#Configuration
#EnableTransactionManagement
class Neo4jConfig(
#Value("\${spring.neo4j.uri}")
val neo4jDBUri: String,
#Value("\${spring.neo4j.authentication.username}")
val neo4jDbUserName: String,
#Value("\${spring.neo4j.authentication.password}")
val neo4jDbUserPassword: String
) : AbstractNeo4jConfig() {
// https://medium.com/neo4j/try-and-then-retry-there-can-be-failure-30bf336383da
#Bean
override fun driver(): Driver {
val config: Config = Config.builder()
.withMaxTransactionRetryTime(60, TimeUnit.SECONDS)
.withLeakedSessionsLogging()
.withRoutingTablePurgeDelay(1, TimeUnit.SECONDS)
.withConnectionAcquisitionTimeout(2 * 60, TimeUnit.SECONDS)
.withConnectionLivenessCheckTimeout(2 * 60, TimeUnit.SECONDS)
.withDriverMetrics()
.withMaxConnectionPoolSize(100)
.build()
return GraphDatabase
.driver(neo4jDBUri, AuthTokens.basic(neo4jDbUserName, neo4jDbUserPassword), config)
}
}
// Refer: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.4.0-M2-Release-Notes#neo4j-1
#Bean(ReactiveNeo4jRepositoryConfigurationExtension.DEFAULT_TRANSACTION_MANAGER_BEAN_NAME)
fun reactiveTransactionManager(
driver: Driver,
databaseNameProvider: ReactiveDatabaseSelectionProvider
): ReactiveTransactionManager = ReactiveNeo4jTransactionManager(driver, databaseNameProvider)
// Refer: https://hantsy.medium.com/data-auditing-with-spring-data-neo4j-11d6461146ff
#Configuration(proxyBeanMethods = false)
#EnableReactiveNeo4jAuditing
internal class DataConfig {
#Bean
fun reactiveAuditorAware(): ReactiveAuditorAware<String> = ReactiveAuditorAware { Mono.just("Audit enabled") }
}
Error message
webFilters.ControllerConfig in Neo4jDriverIO-2-2 - Returning HTTP 400 Bad Requestorg.springframework.dao.InvalidDataAccessResourceUsageException: Unable to acquire connection from the pool within configured maximum time of 120000ms; Error code 'N/A' at org.springframework.data.neo4j.repository.support.Neo4jPersistenceExceptionTranslator.translateImpl(Neo4jPersistenceExceptionTranslator.java:105) Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: Error has been observed at the following site(s): |_ checkpoint ⇢ Handler
controller.UserApiController#createUser(UserReqModel, Continuation) [DispatcherHandler]Stack trace: at org.springframework.data.neo4j.repository.support.Neo4jPersistenceExceptionTranslator.translateImpl(Neo4jPersistenceExceptionTranslator.java:105) at org.springframework.data.neo4j.repository.support.Neo4jPersistenceExceptionTranslator.translateExceptionIfPossible(Neo4jPersistenceExceptionTranslator.java:91) at org.springframework.data.neo4j.core.DefaultReactiveNeo4jClient.potentiallyConvertRuntimeException(DefaultReactiveNeo4jClient.java:271) at org.springframework.data.neo4j.core.DefaultReactiveNeo4jClient.access$200(DefaultReactiveNeo4jClient.java:54) at org.springframework.data.neo4j.core.DefaultReactiveNeo4jClient$DefaultRecordFetchSpec.la

Get Instance of circuit breaker from configuration file

This is my configuration file.
resilience4j.circuitbreaker:
instances:
backendB:
registerHealthIndicator: true
slidingWindowSize: 10
minimumNumberOfCalls: 10
permittedNumberOfCallsInHalfOpenState: 3
waitDurationInOpenState: 5s
failureRateThreshold: 50
eventConsumerBufferSize: 10
I am trying to create bean of my feign client using Resilience4jFeign but the circuit breaker object is initialized with default configuration as the name suggest CircuitBreaker.ofDefaults. I can't find any ways to get my instance of circuit breaker from configuration to an object.
#Bean
public CommunicationServiceProxy communicationServiceProxy(){
CircuitBreaker circuitBreaker = CircuitBreaker.ofDefaults("backendB");
FeignDecorators decorators = FeignDecorators.builder()
.withCircuitBreaker(circuitBreaker)
.withFallbackFactory(CommunicationFallBack::new)
.build();
return Resilience4jFeign
.builder(decorators)
.encoder(new JacksonEncoder())
.decoder(new JacksonDecoder())
.target(CommunicationServiceProxy.class, "http://localhost:8081");
}
I found the solution and it was using the CircuitBreakerRegistry.
Instead of creating registry using
CircuitBreakerRegistry.ofDefaults()
one need to autowired them.
#Autowired
private CircuitBreakerRegistry circuitBreakerRegistry;

resilience4j Spring Boot 2

Trying out a simple Spring Boot 2 + Resilience4j project.
But facing an issue that the circuit breaker is always CLOSED though the host application is down.
Service class
#Autowired
private RestTemplate restTemplate;
#Bean
public RestTemplate restTemplate() {
return new RestTemplate();
}
#CircuitBreaker(name = "mainService", fallbackMethod="testFallBack")
public ResponseEntity<String> invokeService(int i) {
return restTemplate.exchange(
"http://localhost:9092/", // This service is always down
HttpMethod.GET,
null,
String.class
);
}
private ResponseEntity<String> testFallBack(int i, Exception e) {
return new ResponseEntity<String>("In fallback method", HttpStatus.INTERNAL_SERVER_ERROR);
}
Resilience4J Config
management.endpoint.health.show-details: always
management.health.circuitbreakers.enabled: true
resilience4j:
circuitbreaker:
configs:
default:
registerHealthIndicator: true
minimumNumberOfCalls: 5
permittedNumberOfCallsInHalfOpenState: 3
automaticTransitionFromOpenToHalfOpenEnabled: true
waitDurationInOpenState: 5s
failureRateThreshold: 50
eventConsumerBufferSize: 10
instances:
mainService:
baseConfig: default
The Service is being called multiple times from the Controller and I expect it to fallback after minimum 5 calls but the circuit breaker is always CLOSED and for each call from controller the host service is being called and up with Connection refused.
Dependencies: spring-boot-starter-web, resilience4j-spring-boot2, spring-aop, spring-boot-starter-actuator
Earlier I tried out programmatic approach using CircuitBreakerRegistry and Decorator Function which works as expected.
Actually, you misunderstand the circuit breaker parameters. See the documentation:
minimumNumberOfCalls, default: 100
Configures the minimum number of calls which are required (per sliding window period) before the CircuitBreaker can calculate the error rate or slow call rate.
For example, if minimumNumberOfCalls is 10, then at least 10 calls must be recorded, before the failure rate can be calculated.
If only 9 calls have been recorded the CircuitBreaker will not transition to open even if all 9 calls have failed.
slidingWindowSize, default: 100
Configures the size of the sliding window which is used to record the outcome of calls when the CircuitBreaker is closed.
In your configuration, there is
minimumNumberOfCalls: 5
slidingWindowSize: 100 ## implicitly, because you have not set this parameters
And you
expect it to fallback after minimum 5 calls
However, your circuit breaker opens after 100 failures, not after 5.

How to add resilience4j retry to a spring boot 2 webclient call?

I'm trying to add retry mechanism to a webclient rest call using resilience4j retry which is not working. The method is only getting called once in case of exception. I'm using spring boot 2 with kotlin.
This is the caller
GlobalScope.launch {
println(service.callRest())
}
This is the config
resilience4j.retry:
configs:
default:
maxRetryAttempts: 3
waitDuration: 100
retryExceptions:
- java.lang.IllegalArgumentException
- java.util.concurrent.TimeoutException
- org.springframework.web.client.HttpServerErrorException
- java.io.IOException
- java.net.UnknownHostException
- org.springframework.web.reactive.function.client.WebClientResponseException
- org.springframework.web.reactive.function.client.WebClientResponseException$NotFound
- org.springframework.web.client.HttpClientErrorException$NotFound
instances:
backendA:
baseConfig: default
this is my method:
#Retry(name = BACKEND_A)
suspend fun callRest(): String {
println("tried calling")
return webClient.get()
.uri("/api/v1/dummy1")
.accept(APPLICATION_JSON)
.retrieve()
.awaitBody()
}
If I throw a hardcoded exception from the method, the retry works correctly
#Retry(name = BACKEND_A)
#Throws(WebClientResponseException::class)
suspend fun callRest(): String {
println("tried calling")
throw WebClientResponseException("abc", 404, "abc", null, null, null)
Also, it works with restTemplate
#Retry(name = BACKEND_A)
fun callRestTemplate(): String {
println("tried calling")
return restTemplate
.getForObject("/api/v1/dummy1")
}
Try to return a Future for your asynchronous function.
#Retry(name = BACKEND_A)
fun callRestTemplate(): Future<String> {
Also I cannot see you service declaration but I had the same issue. To add the retry annotation on the class resolved it.
#Retry(name = BACKEND_A)
#Service
class BackendServiceA() {

Resources