I am evaluating spring-retry for a use case where we need to automatically retry certain API Calls Based on Status Code. I am able to customize the retry policy like this
#Component("httpStatusCodeRetryPolicy")
public class HttpStatusRetryPolicy extends ExceptionClassifierRetryPolicy
{
#PostConstruct
public void init()
{
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
this.setExceptionClassifier( new Classifier<Throwable, RetryPolicy>()
{
#Override
public RetryPolicy classify(Throwable classifiable )
{
if ( classifiable instanceof HttpStatusCodeException)
{
var exception = (HttpStatusCodeException)classifiable;
if(exception.getStatusCode() == HttpStatus.REQUEST_TIMEOUT){
retryPolicy.setMaxAttempts(3);
}
else if (exception.getStatusCode() == HttpStatus.valueOf(429) ||
exception.getStatusCode() == HttpStatus.valueOf(502) ||
exception.getStatusCode() == HttpStatus.valueOf(503) ||
exception.getStatusCode() == HttpStatus.valueOf(504)) {
retryPolicy.setMaxAttempts(4);
}
return retryPolicy;
}
return new NeverRetryPolicy();
}
});
}
}
However, I want to also customize the backoff policy based on these status codes. I want to use a FixedBackOff policy for some response status code and an ExponentialBackOffPolicy for the rest. I looked around and have not found any pointers.
Use a custom BackOffPolicy that delegates to the desired BackOffPolicy, depending on the lastThrowable in the retryContext.
Related
I am using Spring Boot 2.1.14.RELEASE, Java8, Spring Boot.
I have a client from which I have to access another rest service.
I need to retry an Http404 and HTTP500 2 times whereas not retry any other exceptions.
I am using RestTemplate to invoke the rest service like this:
restTemplate.postForEntity(restUrl, requestEntity, String.class);
I looked into using Retryable as well as RetryTemplate and implemented the retry functionality using RetryTemplate.
I have implemented this in 2 ways:
OPTION1:
The RetryTemplate bean is:
#Bean
public RetryTemplate retryTemplate() {
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(retryProperties.getDelayForCall());
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
retryTemplate.setRetryPolicy(exceptionClassifierRetryPolicy);
return retryTemplate;
}
ClassifierRetryPolicy is:
#Component
public class ExceptionClassifierRetryPolicy1 extends ExceptionClassifierRetryPolicy {
#Inject
private RetryProperties retryProperties;
public ExceptionClassifierRetryPolicy1(){
final SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(2);
this.setExceptionClassifier(new Classifier<Throwable, RetryPolicy>() {
#Override
public RetryPolicy classify(Throwable classifiable) {
if (classifiable instanceof HttpServerErrorException) {
// For specifically 500
if (((HttpServerErrorException) classifiable).getStatusCode() == HttpStatus.INTERNAL_SERVER_ERROR) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
else if (classifiable instanceof HttpClientErrorException) {
// For specifically 404
if (((HttpClientErrorException) classifiable).getStatusCode() == HttpStatus.NOT_FOUND) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
return new NeverRetryPolicy();
}
});
}
}
In my client class, I am using retryTemplate like this:
public void postToRestService(...,...){
...
retryTemplate.execute(context -> {
logger.info("Processing request...");
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
return null;
}, context -> recoveryCallback(context));
...
}
The rest service being invoked is throwing HTTP404 on every request.
My expectation is: The client should submit one request, receive HTTP404, and perform 2 retries. So a total of 3 requests submitted to rest service before invoking recovery callback method.
My observation is: The client is submitting 2 requests to rest service.
Above observation makes sense from what I have read about RetryTemplate.
So the questions are:
Is the above implementation of retryTemplate correct? If not, how to implement and invoke it? Another option that I tried implementing (but didn't get any far) was using a RetryListenerSupport on the client method and invoking the retryTemplate inside the onError method.
Are we supposed to bump up the retry count by 1 to achieve what is desired? I have tried this and it gets me what I need but the RetryTemplate isn't created with this purpose in mind.
OPTION2: Code implementing option mentioned in #1 above:
Client method:
#Retryable(listeners = "RestClientListener")
public void postToRestService(...,...){
...
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
...
}
Listener:
public class RestClientListener extends RetryListenerSupport {
private static final Logger logger = LoggerFactory.getLogger(RestClientListener.class);
#Inject
RestTemplate restTemplate;
#Inject
RetryTemplate retryTemplate;
public <T, E extends Throwable> void onError(RetryContext context, RetryCallback<T, E> callback, Throwable throwable) {
logger.info("Retrying count for RestClientListener "+context.getRetryCount());
...
final ResponseEntity<String>[] responseEntity = new ResponseEntity[]{null};
if( context.getLastThrowable().getCause() != null &&
(context.getLastThrowable().getCause() instanceof RestClientResponseException &&
((RestClientResponseException) context.getLastThrowable().getCause()).getRawStatusCode() == HttpStatus.NOT_FOUND.value()))
{
logger.info("Retrying now: ", context.getLastThrowable().toString());
retryTemplate.execute(context2 -> {
logger.info("Processing request...: {}", context2);
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
return responseEntity;
}, context2 -> recoveryCallback(context2));
}
else {
// Only retry for the above if condition
context.setExhaustedOnly();
}
}
}
The problem with this approach is that I cannot find a way to share objects between my client and clientListener classes. These objects are required in order to create requestEntity and header objects. How can this be achieved?
simpleRetryPolicy.setMaxAttempts(2);
Means 2 attempts total, not 2 retries.
Thanks for reading ahead of time. In my main method I have a PublishSubscribeChannel
#Bean(name = "feeSchedule")
public SubscribableChannel getMessageChannel() {
return new PublishSubscribeChannel();
}
In a service that does a long running process it creates a fee schedule that I inject the channel into
#Service
public class FeeScheduleCompareServiceImpl implements FeeScheduleCompareService {
#Autowired
MessageChannel outChannel;
public List<FeeScheduleUpdate> compareFeeSchedules(String oldStudyId) {
List<FeeScheduleUpdate> sortedResultList = longMethod(oldStudyId);
outChannel.send(MessageBuilder.withPayload(sortedResultList).build());
return sortedResultList;
}
}
Now this is the part I'm struggling with. I want to use completable future and get the payload of the event in the future A in another spring bean. I need future A to return the payload from the message. I think want to create a ServiceActivator to be the message end point but like I said, I need it to return the payload for future A.
#org.springframework.stereotype.Service
public class SFCCCompareServiceImpl implements SFCCCompareService {
#Autowired
private SubscribableChannel outChannel;
#Override
public List<SFCCCompareDTO> compareSFCC(String state, int service){
ArrayList<SFCCCompareDTO> returnList = new ArrayList<SFCCCompareDTO>();
CompletableFuture<List<FeeScheduleUpdate>> fa = CompletableFuture.supplyAsync( () ->
{ //block A WHAT GOES HERE?!?!
outChannel.subscribe()
}
);
CompletableFuture<List<StateFeeCodeClassification>> fb = CompletableFuture.supplyAsync( () ->
{
return this.stateFeeCodeClassificationRepository.findAll();
}
);
CompletableFuture<List<SFCCCompareDTO>> fc = fa.thenCombine(fb,(a,b) ->{
//block C
//get in this block when both A & B are complete
Object theList = b.stream().forEach(new Consumer<StateFeeCodeClassification>() {
#Override
public void accept(StateFeeCodeClassification stateFeeCodeClassification) {
a.stream().forEach(new Consumer<FeeScheduleUpdate>() {
#Override
public void accept(FeeScheduleUpdate feeScheduleUpdate) {
returnList new SFCCCompareDTO();
}
});
}
}).collect(Collectors.toList());
return theList;
});
fc.join();
return returnList;
}
}
Was thinking there would be a service activator like:
#MessageEndpoint
public class UpdatesHandler implements MessageHandler{
#ServiceActivator(requiresReply = "true")
public List<FeeScheduleUpdate> getUpdates(Message m){
return (List<FeeScheduleUpdate>) m.getPayload();
}
}
Your question isn't clear, but I'll try to help you with some info.
Spring Integration doesn't provide CompletableFuture support, but it does provide an async handling and replies.
See Asynchronous Gateway for more information. And also see Asynchronous Service Activator.
outChannel.subscribe() should come with the MessageHandler callback, by the way.
I want to add a custom error Zuul Filter and want to make sure SendErrorFilter does not execute. I have looked at few github links including Spring-cloud/spring-cloud-netflix and various stack-overflow questions:-
Customizing Zuul Exception
Overriding Zuul Filter SendErrorFilter
My code is as follows-
public class CustomErrorFilter extends ZuulFilter {
private static final Logger LOG = LoggerFactory.getLogger(CustomErrorFilter.class);
#Override
public String filterType() {
return "post";
}
#Override
public int filterOrder() {
return -1;
}
#Override
public boolean shouldFilter() {
RequestContext ctx=RequestContext.getCurrentContext();
if(ctx.getThrowable()!=null)
return true;
else
return false;
}
#Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
ctx.setThrowable(null); // response is not returned unless
throwable is set to null.
ctx.remove("error.status_code");
ctx.setResponseBody(“Error”);
ctx.getResponse().setContentType("text/plain");
ctx.setResponseStatusCode(400);
}
return null;
}
I am having the following issues-
Setting filter order to -1 does not prohibit sendErrorFilter from running.
To stop sendErrorFilter from running, I need to set
zuul.SendErrorFilter.error.disable=true in bootstrap.yml
To get a response body which is set in the custom error filter, i need to set throwable to null as mentioned in the github.
Setting a filter as type "error" does nothing, and the custom filer does not run.
I would like someone to explain, what I am doing wrong and what is the most correct way of handling custom error filters, because there is lot of conflicting information available on the web.
Dependencies-
spring cloud - Edgware.RELEASE
spring cloud netflix starter zuul- 1.4.3.RELEASE
filtertype() should return "error" if you want to handle error scenario
filterOrder() should be -1 to execute before SendErrorFilter
Add the following lines to your filter(CustomErrorFilter )
protected static final String SEND_ERROR_FILTER_RAN = "sendErrorFilter.ran";
and
#Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
ctx.set(SEND_ERROR_FILTER_RAN);
// rest of your code
return null;
}
ctx.set(SEND_ERROR_FILTER_RAN); will block the SendErrorFilter from running.
Update:
Check shouldFilter() method in SendErrorFilter
#Override
public boolean shouldFilter() {
RequestContext ctx = RequestContext.getCurrentContext();
// only forward to errorPath if it hasn't been forwarded to already
return ctx.getThrowable() != null
&& !ctx.getBoolean(SEND_ERROR_FILTER_RAN, false);
}
For every request a RequestContext gets created, but spring is not setting SEND_ERROR_FILTER_RAN in the context. May be for older version you have to configure from yaml file (zuul.SendErrorFilter.error.disable=true) for newer version(1.4.3.RELEASE) its from code like ctx.set(SEND_ERROR_FILTER_RAN).
By default !ctx.getBoolean(SEND_ERROR_FILTER_RAN, false) this will evaluates to true and run() method of SendErrorFilter will execute.
if you put ctx.set(SEND_ERROR_FILTER_RAN) in your CustomErrorFilter with filterOrder() as -1, your CustomErrorFilter will execute first and we are setting the RequestContext with SEND_ERROR_FILTER_RAN as true. Now when it goes to shouldFilter() method of SendErrorFilter evaluate to false and it won't execute run() method of SendErrorFilter.
I have the following configuration
spring.rabbitmq.listener.prefetch=1
spring.rabbitmq.listener.concurrency=1
spring.rabbitmq.listener.retry.enabled=true
spring.rabbitmq.listener.retry.max-attempts=3
spring.rabbitmq.listener.retry.max-interval=1000
spring.rabbitmq.listener.default-requeue-rejected=false //I have also changed it to true but the same behavior still happens
and in my listener I throw the exception AmqpRejectAndDontRequeueException to reject the message and enforce rabbit not to try to redeliver it...But rabbit redilvers it for 3 times then finally route it to dead letter queue.
Is that the standard behavior according to my provided configuration or do I miss something?
You have to configure the retry policy to not retry for that exception.
You can't do that with properties, you have to configure the retry advice yourself.
I'll post an example later if you need help with that.
requeue-rejected is at the container level (below retry on the stack).
EDIT
#SpringBootApplication
public class So39853762Application {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = SpringApplication.run(So39853762Application.class, args);
Thread.sleep(60000);
context.close();
}
#RabbitListener(queues = "foo")
public void foo(String foo) {
System.out.println(foo);
if ("foo".equals(foo)) {
throw new AmqpRejectAndDontRequeueException("foo"); // won't be retried.
}
else {
throw new IllegalStateException("bar"); // will be retried
}
}
#Bean
public ListenerRetryAdviceCustomizer retryCustomizer(SimpleRabbitListenerContainerFactory containerFactory,
RabbitProperties rabbitPropeties) {
return new ListenerRetryAdviceCustomizer(containerFactory, rabbitPropeties);
}
public static class ListenerRetryAdviceCustomizer implements InitializingBean {
private final SimpleRabbitListenerContainerFactory containerFactory;
private final RabbitProperties rabbitPropeties;
public ListenerRetryAdviceCustomizer(SimpleRabbitListenerContainerFactory containerFactory,
RabbitProperties rabbitPropeties) {
this.containerFactory = containerFactory;
this.rabbitPropeties = rabbitPropeties;
}
#Override
public void afterPropertiesSet() throws Exception {
ListenerRetry retryConfig = this.rabbitPropeties.getListener().getRetry();
if (retryConfig.isEnabled()) {
RetryInterceptorBuilder<?> builder = (retryConfig.isStateless()
? RetryInterceptorBuilder.stateless()
: RetryInterceptorBuilder.stateful());
Map<Class<? extends Throwable>, Boolean> retryableExceptions = new HashMap<>();
retryableExceptions.put(AmqpRejectAndDontRequeueException.class, false);
retryableExceptions.put(IllegalStateException.class, true);
SimpleRetryPolicy policy =
new SimpleRetryPolicy(retryConfig.getMaxAttempts(), retryableExceptions, true);
ExponentialBackOffPolicy backOff = new ExponentialBackOffPolicy();
backOff.setInitialInterval(retryConfig.getInitialInterval());
backOff.setMultiplier(retryConfig.getMultiplier());
backOff.setMaxInterval(retryConfig.getMaxInterval());
builder.retryPolicy(policy)
.backOffPolicy(backOff)
.recoverer(new RejectAndDontRequeueRecoverer());
this.containerFactory.setAdviceChain(builder.build());
}
}
}
}
NOTE: You cannot currently configure the policy to retry all exceptions, "except" this one - you have to classify all exceptions you want retried (and they can't be a superclass of AmqpRejectAndDontRequeueException). I have opened an issue to support this.
The other answers posted here didn't work me when using Spring Boot 2.3.5 and Spring AMQP Starter 2.2.12, but for these versions I was able to customize the retry policy to not retry AmqpRejectAndDontRequeueException exceptions:
#Configuration
public class RabbitConfiguration {
#Bean
public RabbitRetryTemplateCustomizer customizeRetryPolicy(
#Value("${spring.rabbitmq.listener.simple.retry.max-attempts}") int maxAttempts) {
SimpleRetryPolicy policy = new SimpleRetryPolicy(maxAttempts, Map.of(AmqpRejectAndDontRequeueException.class, false), true, true);
return (target, retryTemplate) -> retryTemplate.setRetryPolicy(policy);
}
}
This lets the retry policy skip retries for AmqpRejectAndDontRequeueExceptions but retries all other exceptions as usual.
Configured this way, it traverses the causes of an exception, and skips retries if it finds an AmqpRejectAndDontRequeueException.
Traversing the causes is needed as org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter#invokeHandler wraps all exceptions as a ListenerExecutionFailedException
Is it possible to set RetryPolicy in spring retry (https://github.com/spring-projects/spring-retry) based on error status code? e.g. I want to retry on HttpServerErrorException with HttpStatus.INTERNAL_SERVER_ERROR status code, which is 503. Therefore it should ignore all other error codes -- [500 - 502] and [504 - 511].
The RestTemplate has setErrorHandler option and DefaultResponseErrorHandler is the default one.
Its code looks like:
public void handleError(ClientHttpResponse response) throws IOException {
HttpStatus statusCode = getHttpStatusCode(response);
switch (statusCode.series()) {
case CLIENT_ERROR:
throw new HttpClientErrorException(statusCode, response.getStatusText(),
response.getHeaders(), getResponseBody(response), getCharset(response));
case SERVER_ERROR:
throw new HttpServerErrorException(statusCode, response.getStatusText(),
response.getHeaders(), getResponseBody(response), getCharset(response));
default:
throw new RestClientException("Unknown status code [" + statusCode + "]");
}
}
So, you can provide your own implementation for that method to simplify your RetryPolicy around desired status codes.
For others who are facing same problem, I'm posting this answer.
Implement custom retry policy as follows:
class InternalServerExceptionClassifierRetryPolicy extends ExceptionClassifierRetryPolicy {
public InternalServerExceptionClassifierRetryPolicy() {
final SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(3);
this.setExceptionClassifier(new Classifier<Throwable, RetryPolicy>() {
#Override
public RetryPolicy classify(Throwable classifiable) {
if (classifiable instanceof HttpServerErrorException) {
// For specifically 500 and 504
if (((HttpServerErrorException) classifiable).getStatusCode() == HttpStatus.INTERNAL_SERVER_ERROR
|| ((HttpServerErrorException) classifiable)
.getStatusCode() == HttpStatus.GATEWAY_TIMEOUT) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
return new NeverRetryPolicy();
}
});
}}
Ans the simply call it as below:
RetryTemplate template = new RetryTemplate();
template.setRetryPolicy(new InternalServerExceptionClassifierRetryPolicy())
You can also add the specific error code in the retryableExceptions list of the SinmpleRetryPolicy.
Map<Class<? extends Throwable>, Boolean> retryableExceptions = new HashMap<>();
retryableExceptions.put(HttpClientErrorException.Unauthorized.class, true);
retryTemplate.setRetryPolicy(new SimpleRetryPolicy(5, retryableExceptions));