I'm trying to add retry mechanism to a webclient rest call using resilience4j retry which is not working. The method is only getting called once in case of exception. I'm using spring boot 2 with kotlin.
This is the caller
GlobalScope.launch {
println(service.callRest())
}
This is the config
resilience4j.retry:
configs:
default:
maxRetryAttempts: 3
waitDuration: 100
retryExceptions:
- java.lang.IllegalArgumentException
- java.util.concurrent.TimeoutException
- org.springframework.web.client.HttpServerErrorException
- java.io.IOException
- java.net.UnknownHostException
- org.springframework.web.reactive.function.client.WebClientResponseException
- org.springframework.web.reactive.function.client.WebClientResponseException$NotFound
- org.springframework.web.client.HttpClientErrorException$NotFound
instances:
backendA:
baseConfig: default
this is my method:
#Retry(name = BACKEND_A)
suspend fun callRest(): String {
println("tried calling")
return webClient.get()
.uri("/api/v1/dummy1")
.accept(APPLICATION_JSON)
.retrieve()
.awaitBody()
}
If I throw a hardcoded exception from the method, the retry works correctly
#Retry(name = BACKEND_A)
#Throws(WebClientResponseException::class)
suspend fun callRest(): String {
println("tried calling")
throw WebClientResponseException("abc", 404, "abc", null, null, null)
Also, it works with restTemplate
#Retry(name = BACKEND_A)
fun callRestTemplate(): String {
println("tried calling")
return restTemplate
.getForObject("/api/v1/dummy1")
}
Try to return a Future for your asynchronous function.
#Retry(name = BACKEND_A)
fun callRestTemplate(): Future<String> {
Also I cannot see you service declaration but I had the same issue. To add the retry annotation on the class resolved it.
#Retry(name = BACKEND_A)
#Service
class BackendServiceA() {
Related
In a spring boot project, I would like to implement CicrcuitBreaker when connecting to 3rd party services using spring frameworks new declarative client
What I have done till now
Added below dependency in POM.xml as declarative client uses webclient underneath
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>
Added below properties in application.properties
resilience4j.circuitbreaker.configs.default.registerHealthIndicator=true
resilience4j.circuitbreaker.configs.default.slidingWindowSize=10
resilience4j.circuitbreaker.configs.default.minimumNumberOfCalls=5
resilience4j.circuitbreaker.configs.default.permittedNumberOfCallsInHalfOpenState=3
resilience4j.circuitbreaker.configs.default.automaticTransitionFromOpenToHalfOpenEnabled=true
resilience4j.circuitbreaker.configs.default.waitDurationInOpenState=5s
resilience4j.circuitbreaker.configs.default.failureRateThreshold=50
resilience4j.circuitbreaker.configs.default.eventConsumerBufferSize=10
resilience4j.circuitbreaker.configs.default.recordExceptions[0]=org.springframework.web.reactive.function.client.WebClientRequestException
resilience4j.circuitbreaker.configs.default.recordExceptions[1]=java.net.ConnectException
resilience4j.circuitbreaker.configs.default.recordExceptions[2]=java.io.IOException
resilience4j.timelimiter.configs.default.timeoutDuration=5s
resilience4j.timelimiter.configs.default.cancelRunningFuture=true
management.health.circuitbreakers.enabled=true
Created a Bean of type ReactiveResilience4JCircuitBreakerFactory as below.
#Bean
Customizer<ReactiveResilience4JCircuitBreakerFactory> defaultCustomizer() {
return factory ->
factory.configureDefault(
id ->
new Resilience4JConfigBuilder(id)
.circuitBreakerConfig(CircuitBreakerConfig.ofDefaults())
.timeLimiterConfig(TimeLimiterConfig.ofDefaults())
.build());
}
Added CircuitBreaker to the HttpServiceProxyFactory Bean as below
#Bean
public HttpServiceProxyFactory httpServiceProxyFactory(
WebClient.Builder builder,
ReactiveResilience4JCircuitBreakerFactory resilience4JCircuitBreakerFactory) {
CircuitBreaker circuitBreaker =
resilience4JCircuitBreakerFactory
.getCircuitBreakerRegistry()
.circuitBreaker("default");
WebClient webClient =
builder.baseUrl(applicationProperties.getInventoryServiceUrl())
.filter(
(request, next) ->
next.exchange(request)
.transform(
CircuitBreakerOperator.of(circuitBreaker)))
.defaultHeaders(
httpHeaders -> {
httpHeaders.setContentType(MediaType.APPLICATION_JSON);
httpHeaders.setAccept(List.of(MediaType.APPLICATION_JSON));
})
.build();
return HttpServiceProxyFactory.builder(WebClientAdapter.forClient(webClient)).build();
}
When 3rd party service is down it is not giving any fallback response, how to fix this.
Another approach tried
Annotating the method with #CircuitBreaker and creating fallback method, still it is giving me exception.
#CircuitBreaker(
name = "getInventoryByProductCode",
fallbackMethod = "getInventoryByProductCodeFallBack")
private InventoryDto getInventoryByProductCode(String code) {
return inventoryServiceProxy.getInventoryByProductCode(code);
}
private InventoryDto getInventoryByProductCodeFallBack(String code, Exception e) {
log.error("Exception occurred while fetching product details", e);
return new InventoryDto(code, 0);
}
How to fix this?
I have service A that calls downstream service B.
Service A code
#RestController
#RequestMapping(value = "", produces = MediaType.APPLICATION_JSON_VALUE)
public class GreetingController {
private final GreetingService greetingService;
public GreetingController(GreetingService greetingService){
this.greetingService = greetingService;
}
#GetMapping(value = "/greetings")
public Mono<String> getGreetings() {
return greetingService.callServiceB();
}
}
#Component
#RequiredArgsConstructor
public class GreetingService {
CircuitBreaker circuitBreaker = CircuitBreaker.ofDefaults("greetingService");
Callable<Mono<String>> callable = CircuitBreaker.decorateCallable(circuitBreaker, this::clientCall);
Future<Mono<String>> future = Executors.newSingleThreadExecutor().submit(callable);
public Mono<String> callServiceB() {
try {
return future.get();
} catch (CircuitBreakerOpenException | InterruptedException | ExecutionException ex){
return Mono.just("Service is down!");
}
}
private final String url = "/v1/holidaysgreetings";
private Mono<String> clientCall(){
WebClient client = WebClient.builder().baseUrl("http://localhost:8080").build();
return client
.get()
.uri(url)
.retrieve()
.bodyToMono(String.class);
}
when i shut down downstream service B(running on localhost:8080) and hit /greetings endpoint in GreetingsController class to see if my circuit breaker is working properly or not, i get very this nasty error
2021-06-28 21:27:31.431 ERROR 10285 --- [nio-8081-exec-7] o.a.c.c.C.[.[.[.[dispatcherServlet]: Servlet.service() for servlet [dispatcherServlet] in context with path [/v1/holidaysgreetings]
threw exception [Request processing failed; nested exception is org.springframework.web.reactive.function.client.WebClientRequestException: Connection refused: localhost/127.0.0.1:8080;
nested exception is io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8080] with root cause
java.net.ConnectException: Connection refused
Anyone knows why i am getting this? What i am missing here? Am i implementing circuit breaker correctly?
You are mixing reactive libraries with regular non-reactive libraries. If you aim to use spring-webflux it is better to use the reactor-resilience4j together with the regular reactor-adapter library.
Use these imports:
implementation 'org.springframework.boot:spring-boot-starter-webflux'
implementation 'org.springframework.cloud:spring-cloud-starter-circuitbreaker-reactor-resilience4j'
implementation "io.projectreactor.addons:reactor-adapter:${reactorVersion}"
You are also not creating the circuit-breaker service that you can rely on. After creating it you can call the " Mono run(Mono toRun, Function<Throwable, Mono> fallback)" (to the one that return a Flux if you want) to execute your service and provide a fallback.
Here is one example from a demo code.
#RestController
public class CompletableFutureDemoController {
Logger LOG = LoggerFactory.getLogger(CompletableFutureDemoController.class);
private CompletableFutureHttpBinService httpBin;
private ReactiveCircuitBreakerFactory reactiveCircuitBreakerFactory;
public CompletableFutureDemoController(CompletableFutureHttpBinService httpBin, ReactiveCircuitBreakerFactory reactiveCircuitBreakerFactory) {
this.httpBin = httpBin;
this.reactiveCircuitBreakerFactory = reactiveCircuitBreakerFactory;
}
#GetMapping("/completablefuture/delay/{seconds}")
public Mono<Map> delay(#PathVariable int seconds) {
return reactiveCircuitBreakerFactory.create("completablefuturedelay")
.run(Mono.fromFuture(httpBin.delay(seconds)), t -> {
LOG.warn("delay call failed error", t);
Map<String, String> fallback = new HashMap();
fallback.put("hello", "world");
return Mono.just(fallback);
}
);
}
}
I had the following function (the function is not really important):
fun myRandomFunc(something: String?): List<Int> {
return listOf(5)
}
And you can imagine it was doing some API calls, returning list of some objects, etc. I could easily mock this function in test like this:
doReturn(
listOf(
5
)
)
.whenever(...).myRandomFunc("something")
But after I introduced (retry/recover) in the mix, that mock is now throwing
org.mockito.exceptions.misusing.NotAMockException at .... Any idea why?
This is the code with spring retry:
#Retryable(
value = [ApiException::class], maxAttempts = MAX_RETRIES,
backoff = Backoff(delay = RETRY_DELAY, multiplier = RETRY_MULTIPLIER, random = true)
)
fun myRandomFunc(something: String?): List<Int> {
return listOf(5)
}
#Recover
fun testMyRandomFunc(exception: Exception): List<Int> {
log.error("Exception occurred ...", exception)
throw RemoteServiceNotAvailableException("Call failed after $MAX_RETRIES retries")
}
The code works, it's functional, just the mocking of tests is now broken. Would appreciate some help
Spring retry creates a proxy around the object.
If there is an interface, the proxy is a JDK proxy; if not, CGLIB is used.
Mockito can't mock CGLIB (final) methods.
I implemented feign client and hystrix to my spring boot microservice application.
I first tried to test to communicate users service to albums service with feign client,
so I threw an exception at albums service to check if users service Error Decoder can catch the exception and then make the fallback method triggered.
It worked, but the cause is always null only at the first time, and after that I can see the error message that I wanted to see.
Can anyone tell me if something is wrong or not.
This is my code.
Users Service Feign Client
#FeignClient(name = "albums-ws", fallbackFactory = AlbumsFallbackFactory.class)
public interface AlbumServiceClient {
#GetMapping(path = "users/{userId}/albums")
List<AlbumDetailResponse> getAlbums(#PathVariable("userId") String userId);
}
Fallback Factory
#Component
public class AlbumsFallbackFactory implements FallbackFactory<AlbumServiceClient> {
#Override
public AlbumServiceClient create(Throwable cause) {
return new AlbumServiceClientFallback(cause);
}
}
public class AlbumServiceClientFallback implements AlbumServiceClient {
private final Throwable cause;
private final Logger logger = LoggerFactory.getLogger(this.getClass());
public AlbumServiceClientFallback(Throwable cause) {
this.cause = cause;
}
#Override
public List<AlbumDetailResponse> getAlbums(String userId) {
logger.error("An exception took place: " + cause.getMessage());
return new ArrayList<>();
}
}
Feign Error Decoder
#Component
public class FeignErrorDecoder implements ErrorDecoder {
#Override
public Exception decode(String methodKey, Response response) {
switch(response.status()) {
case 400:
break;
case 404:
if(methodKey.contains("getAlbums")) {
return new ResponseStatusException(HttpStatus.valueOf(response.status()), response.reason());
}
break;
default:
return new Exception(response.reason());
}
return null;
}
}
First fallback triggered
2020-08-02 12:42:27.836 ERROR 24772 --- [ HystrixTimer-1] c.a.p.a.u.P.f.AlbumServiceClientFallback : An exception took place: null
After
2020-08-02 12:43:07.672 DEBUG 24772 --- [rix-albums-ws-2] c.a.p.a.u.P.feign.AlbumServiceClient : [AlbumServiceClient#getAlbums] User not found with id: f5b313e2-411f-4fc3-95e7-9aa5c43c286c
Hystrix has class org.springframework.cloud.netflix.feign.HystrixTargeter. There is a comment in targetWithFallbackFactory method:
We take a sample fallback from the fallback factory to check if it
returns a fallback that is compatible with the annotated feign
interface.
and code after:
Object exampleFallback = fallbackFactory.create(new RuntimeException());
It is why you don't have cause in exception.
I am using Spring Boot 2.1.14.RELEASE, Java8, Spring Boot.
I have a client from which I have to access another rest service.
I need to retry an Http404 and HTTP500 2 times whereas not retry any other exceptions.
I am using RestTemplate to invoke the rest service like this:
restTemplate.postForEntity(restUrl, requestEntity, String.class);
I looked into using Retryable as well as RetryTemplate and implemented the retry functionality using RetryTemplate.
I have implemented this in 2 ways:
OPTION1:
The RetryTemplate bean is:
#Bean
public RetryTemplate retryTemplate() {
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(retryProperties.getDelayForCall());
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
retryTemplate.setRetryPolicy(exceptionClassifierRetryPolicy);
return retryTemplate;
}
ClassifierRetryPolicy is:
#Component
public class ExceptionClassifierRetryPolicy1 extends ExceptionClassifierRetryPolicy {
#Inject
private RetryProperties retryProperties;
public ExceptionClassifierRetryPolicy1(){
final SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(2);
this.setExceptionClassifier(new Classifier<Throwable, RetryPolicy>() {
#Override
public RetryPolicy classify(Throwable classifiable) {
if (classifiable instanceof HttpServerErrorException) {
// For specifically 500
if (((HttpServerErrorException) classifiable).getStatusCode() == HttpStatus.INTERNAL_SERVER_ERROR) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
else if (classifiable instanceof HttpClientErrorException) {
// For specifically 404
if (((HttpClientErrorException) classifiable).getStatusCode() == HttpStatus.NOT_FOUND) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
return new NeverRetryPolicy();
}
});
}
}
In my client class, I am using retryTemplate like this:
public void postToRestService(...,...){
...
retryTemplate.execute(context -> {
logger.info("Processing request...");
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
return null;
}, context -> recoveryCallback(context));
...
}
The rest service being invoked is throwing HTTP404 on every request.
My expectation is: The client should submit one request, receive HTTP404, and perform 2 retries. So a total of 3 requests submitted to rest service before invoking recovery callback method.
My observation is: The client is submitting 2 requests to rest service.
Above observation makes sense from what I have read about RetryTemplate.
So the questions are:
Is the above implementation of retryTemplate correct? If not, how to implement and invoke it? Another option that I tried implementing (but didn't get any far) was using a RetryListenerSupport on the client method and invoking the retryTemplate inside the onError method.
Are we supposed to bump up the retry count by 1 to achieve what is desired? I have tried this and it gets me what I need but the RetryTemplate isn't created with this purpose in mind.
OPTION2: Code implementing option mentioned in #1 above:
Client method:
#Retryable(listeners = "RestClientListener")
public void postToRestService(...,...){
...
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
...
}
Listener:
public class RestClientListener extends RetryListenerSupport {
private static final Logger logger = LoggerFactory.getLogger(RestClientListener.class);
#Inject
RestTemplate restTemplate;
#Inject
RetryTemplate retryTemplate;
public <T, E extends Throwable> void onError(RetryContext context, RetryCallback<T, E> callback, Throwable throwable) {
logger.info("Retrying count for RestClientListener "+context.getRetryCount());
...
final ResponseEntity<String>[] responseEntity = new ResponseEntity[]{null};
if( context.getLastThrowable().getCause() != null &&
(context.getLastThrowable().getCause() instanceof RestClientResponseException &&
((RestClientResponseException) context.getLastThrowable().getCause()).getRawStatusCode() == HttpStatus.NOT_FOUND.value()))
{
logger.info("Retrying now: ", context.getLastThrowable().toString());
retryTemplate.execute(context2 -> {
logger.info("Processing request...: {}", context2);
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
return responseEntity;
}, context2 -> recoveryCallback(context2));
}
else {
// Only retry for the above if condition
context.setExhaustedOnly();
}
}
}
The problem with this approach is that I cannot find a way to share objects between my client and clientListener classes. These objects are required in order to create requestEntity and header objects. How can this be achieved?
simpleRetryPolicy.setMaxAttempts(2);
Means 2 attempts total, not 2 retries.