I am using RetryTemplate of spring boot i am trying hit third party URL if not able to connect then i want to retry.it is working in local machine as expected but if i deployed my application on UAT and tried for retry it working but it retries more then mentioned maxattempts and at the end it giving 504 bad_gateway exception without any response on swagger?
any one help me in this
retryTemplate.execute(arg0 -> {
log.error("Sending Company Request to SIRA.");
final String xmlResponse = submitPayload(request, ResponseTimer, Client);
final ScreeningResponse screeningResponseFromXml = resultMapper
.getScreeningResponseFromXml(soapRequestXml, xmlResponse, screeningProvider, sourceDataId);
if (!screeningResponseFromXml.getErrors().isEmpty()
&& Arrays.asList(SiraRetryableFaultCode.values()).toString()
.contains(screeningResponseFromXml.getErrors().stream().findFirst().get().getCode())) {
throw new FraudFaultCodeException(screeningResponseFromXml.getErrors().stream().findFirst().get().getCode(),
sourceDataId, screeningResponseFromXml.getErrors().stream().findFirst().get().getMessage());
}
return screeningResponseFromXml;
});
========my retry config file====
#Configuration
public class RetryAppConfig {
#Bean
public RetryTemplate retryTemplate() {
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(2000L);
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
ExceptionClassifierRetryPolicy exRetryPolicy = new ExceptionClassifierRetryPolicy();
exRetryPolicy.setPolicyMap(new HashMap<Class<? extends Throwable>, RetryPolicy>() {
{
put(RetryableException.class, new SimpleRetryPolicy(5));
put(FraudFaultCodeException.class, new SimpleRetryPolicy(5));
}
});
retryTemplate.setRetryPolicy(exRetryPolicy);
return retryTemplate;
}
}
Related
I am using spring inbound channel adapter to poll files from sftp server. Application needs to poll from multiple directories from single sftp server. Since Inbound channel adapter does not allow to poll multiple directories I tried creating multiple beans of same type with different values. Since number of directories can increase in future, I want to control it from application properties and want to register beans dynamically.
My code -
#Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
beanFactory.registerSingleton("sftpSessionFactory", sftpSessionFactory(host, port, user, password));
beanFactory.registerSingleton("sftpInboundFileSynchronizer",
sftpInboundFileSynchronizer((SessionFactory) beanFactory.getBean("sftpSessionFactory")));
}
public SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory(String host, String port, String user, String password) {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(host);
factory.setPort(Integer.parseInt(port));
factory.setUser(user);
factory.setPassword(password);
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<>(factory);
}
private SftpInboundFileSynchronizer sftpInboundFileSynchronizer(SessionFactory sessionFactory) {
SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sessionFactory);
fileSynchronizer.setDeleteRemoteFiles(true);
fileSynchronizer.setPreserveTimestamp(true);
fileSynchronizer.setRemoteDirectory("/mydir/subdir);
fileSynchronizer.setFilter(new SftpSimplePatternFileListFilter("*.pdf"));
return fileSynchronizer;
}
#Bean
#InboundChannelAdapter(channel = "sftpChannel", poller = #Poller(fixedDelay = "2000"))
public MessageSource<File> sftpMessageSource(String s) {
SftpInboundFileSynchronizingMessageSource source = new SftpInboundFileSynchronizingMessageSource(
(AbstractInboundFileSynchronizer<ChannelSftp.LsEntry>) applicationContext.getBean("sftpInboundFileSynchronizer"));
source.setLocalDirectory(new File("/dir/subdir"));
source.setAutoCreateLocalDirectory(true);
source.setLocalFilter(new AcceptOnceFileListFilter<>());
source.setMaxFetchSize(Integer.parseInt(maxFetchSize));
source.setAutoCreateLocalDirectory(true);
return source;
}
#Bean
#ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler handler() {
return message -> {
LOGGER.info("Payload - {}", message.getPayload());
};
}
This code works fine. But If I create sftpMessageSource dynamically, then #InboundChannelAdapter annotation won't work. Please suggest a way to dynamically create sftpMessageSource and handler beans also and add respective annotations.
Update:
Following Code Worked :
#PostConstruct
void init() {
int index = 0;
for (String directory : directories) {
index++;
int finalI = index;
IntegrationFlow flow = IntegrationFlows
.from(Sftp.inboundAdapter(sftpSessionFactory())
.preserveTimestamp(true)
.remoteDirectory(directory)
.autoCreateLocalDirectory(true)
.localDirectory(new File("/" + directory))
.localFilter(new AcceptOnceFileListFilter<>())
.maxFetchSize(10)
.filter(new SftpSimplePatternFileListFilter("*.pdf"))
.deleteRemoteFiles(true),
e -> e.id("sftpInboundAdapter" + finalI)
.autoStartup(true)
.poller(Pollers.fixedDelay(2000)))
.handle(handler())
.get();
this.flowContext.registration(flow).register();
}
}
#Bean
public SessionFactory<ChannelSftp.LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(host);
factory.setPort(Integer.parseInt(port));
factory.setUser(user);
factory.setPassword(password);
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<>(factory);
}
Annotations in Java are static. You can't add them at runtime for created objects. Plus the framework reads those annotation on application context startup. So, what you are looking for is just not possible with Java as language per se.
You need consider to switch to Java DSL in Spring Integration to be able to use its "dynamic flows": https://docs.spring.io/spring-integration/docs/5.3.1.RELEASE/reference/html/dsl.html#java-dsl-runtime-flows.
But, please, first of all study more what Java can do and what cannot.
I am using Spring Boot 2.1.14.RELEASE, Java8, Spring Boot.
I have a client from which I have to access another rest service.
I need to retry an Http404 and HTTP500 2 times whereas not retry any other exceptions.
I am using RestTemplate to invoke the rest service like this:
restTemplate.postForEntity(restUrl, requestEntity, String.class);
I looked into using Retryable as well as RetryTemplate and implemented the retry functionality using RetryTemplate.
I have implemented this in 2 ways:
OPTION1:
The RetryTemplate bean is:
#Bean
public RetryTemplate retryTemplate() {
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(retryProperties.getDelayForCall());
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
retryTemplate.setRetryPolicy(exceptionClassifierRetryPolicy);
return retryTemplate;
}
ClassifierRetryPolicy is:
#Component
public class ExceptionClassifierRetryPolicy1 extends ExceptionClassifierRetryPolicy {
#Inject
private RetryProperties retryProperties;
public ExceptionClassifierRetryPolicy1(){
final SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(2);
this.setExceptionClassifier(new Classifier<Throwable, RetryPolicy>() {
#Override
public RetryPolicy classify(Throwable classifiable) {
if (classifiable instanceof HttpServerErrorException) {
// For specifically 500
if (((HttpServerErrorException) classifiable).getStatusCode() == HttpStatus.INTERNAL_SERVER_ERROR) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
else if (classifiable instanceof HttpClientErrorException) {
// For specifically 404
if (((HttpClientErrorException) classifiable).getStatusCode() == HttpStatus.NOT_FOUND) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
return new NeverRetryPolicy();
}
});
}
}
In my client class, I am using retryTemplate like this:
public void postToRestService(...,...){
...
retryTemplate.execute(context -> {
logger.info("Processing request...");
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
return null;
}, context -> recoveryCallback(context));
...
}
The rest service being invoked is throwing HTTP404 on every request.
My expectation is: The client should submit one request, receive HTTP404, and perform 2 retries. So a total of 3 requests submitted to rest service before invoking recovery callback method.
My observation is: The client is submitting 2 requests to rest service.
Above observation makes sense from what I have read about RetryTemplate.
So the questions are:
Is the above implementation of retryTemplate correct? If not, how to implement and invoke it? Another option that I tried implementing (but didn't get any far) was using a RetryListenerSupport on the client method and invoking the retryTemplate inside the onError method.
Are we supposed to bump up the retry count by 1 to achieve what is desired? I have tried this and it gets me what I need but the RetryTemplate isn't created with this purpose in mind.
OPTION2: Code implementing option mentioned in #1 above:
Client method:
#Retryable(listeners = "RestClientListener")
public void postToRestService(...,...){
...
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
...
}
Listener:
public class RestClientListener extends RetryListenerSupport {
private static final Logger logger = LoggerFactory.getLogger(RestClientListener.class);
#Inject
RestTemplate restTemplate;
#Inject
RetryTemplate retryTemplate;
public <T, E extends Throwable> void onError(RetryContext context, RetryCallback<T, E> callback, Throwable throwable) {
logger.info("Retrying count for RestClientListener "+context.getRetryCount());
...
final ResponseEntity<String>[] responseEntity = new ResponseEntity[]{null};
if( context.getLastThrowable().getCause() != null &&
(context.getLastThrowable().getCause() instanceof RestClientResponseException &&
((RestClientResponseException) context.getLastThrowable().getCause()).getRawStatusCode() == HttpStatus.NOT_FOUND.value()))
{
logger.info("Retrying now: ", context.getLastThrowable().toString());
retryTemplate.execute(context2 -> {
logger.info("Processing request...: {}", context2);
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
return responseEntity;
}, context2 -> recoveryCallback(context2));
}
else {
// Only retry for the above if condition
context.setExhaustedOnly();
}
}
}
The problem with this approach is that I cannot find a way to share objects between my client and clientListener classes. These objects are required in order to create requestEntity and header objects. How can this be achieved?
simpleRetryPolicy.setMaxAttempts(2);
Means 2 attempts total, not 2 retries.
I'm pretty new to spring-integration anyway I'm using it in order to receive mails and elaborate them.
I used this spring configuration class:
#Configuration
#EnableIntegration
#PropertySource(value = { "classpath:configuration.properties" }, encoding = "UTF-8", ignoreResourceNotFound = false)
public class MailReceiverConfiguration {
private static final Log logger = LogFactory.getLog(MailReceiverConfiguration.class);
#Autowired
private EmailTransformerService emailTransformerService;
// Configurazione AE
#Bean
public MessageChannel inboundChannelAE() {
return new DirectChannel();
}
#Bean(name= {"aeProps"})
public Properties aeProps() {
Properties javaMailPropertiesAE = new Properties();
javaMailPropertiesAE.put("mail.store.protocol", "imap");
javaMailPropertiesAE.put("mail.debug", Boolean.TRUE);
javaMailPropertiesAE.put("mail.auth.debug", Boolean.TRUE);
javaMailPropertiesAE.put("mail.smtp.socketFactory.fallback", "false");
javaMailPropertiesAE.put("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
return javaMailPropertiesAE;
}
#Bean(name="mailReceiverAE")
public MailReceiver mailReceiverAE(#Autowired MailConfigurationBean mcb, #Autowired #Qualifier("aeProps") Properties javaMailPropertiesAE) throws Exception {
return ConfigurationUtil.getMailReceiver("imap://USERNAME:PASSWORD#MAILSERVER:PORT/INBOX", new BigDecimal(2), javaMailPropertiesAE);
}
#Bean
#InboundChannelAdapter( autoStartup = "true",
channel = "inboundChannelAE",
poller = {#Poller(fixedRate = "${fixed.rate.ae}",
maxMessagesPerPoll = "${max.messages.per.poll.ae}") })
public MailReceivingMessageSource pollForEmailAE(#Autowired MailReceiver mailReceiverAE) {
MailReceivingMessageSource mrms = new MailReceivingMessageSource(mailReceiverAE);
return mrms;
}
#Transformer(inputChannel = "inboundChannelAE", outputChannel = "transformerChannelAE")
public MessageBean transformitAE( MimeMessage mailMessage ) throws Exception {
// amministratore email inbox
MessageBean messageBean = emailTransformerService.transformit(mailMessage);
return messageBean;
}
#Splitter(inputChannel = "transformerChannelAE", outputChannel = "nullChannel")
public List<Message<?>> splitIntoMessagesAE(final MessageBean mb) {
final List<Message<?>> messages = new ArrayList<Message<?>>();
for (EmailFragment emailFragment : mb.getEmailFragments()) {
Message<?> message = MessageBuilder.withPayload(emailFragment.getData())
.setHeader(FileHeaders.FILENAME, emailFragment.getFilename())
.setHeader("directory", emailFragment.getDirectory()).build();
messages.add(message);
}
return messages;
}
}
So far so good.... I start my micro-service and there is this component listening on the specified mail server and mails are downloaded.
Now I have this requirement: mail server configuration (I mean the string "imap://USERNAME:PASSWORD#MAILSERVER:PORT/INBOX") must be taken from a database and it can be configurable. In any time a system administrator can change it and the mail receiver must use the new configuration.
As far as I understood I should create a new instance of MailReceiver when a new configuration is present and use it in the InboundChannelAdapter
Is there any best practice in order to do it? I found this solution: ImapMailReceiver NO STORE attempt on READ-ONLY folder (Failure) [THROTTLED];
In this solution I can inject the ThreadPoolTaskScheduler if I define it in my Configuration class; I can also inject the DirectChannel but every-time I should create a new MailReceiver and a new ImapIdleChannelAdapter without considering this WARN message I get when the
ImapIdleChannelAdapter starts:
java.lang.RuntimeException: No beanfactory at org.springframework.integration.expression.ExpressionUtils.createStandardEvaluationContext(ExpressionUtils.java:79) at org.springframework.integration.mail.AbstractMailReceiver.onInit(AbstractMailReceiver.java:403)
Is there a better way to satisfy my scenario?
Thank you
Angelo
The best way to do this is to use the Java DSL and dynamic flow registration.
Documentation here.
That way, you can unregister the old flow and register a new one, each time the configuration changes.
It will automatically handle injecting dependencies such as the bean factory.
Using Spring FTP Integration and Annotation configuration, I downloaded files from the FTP server. After downloaded still our application is trigger to connect the server and find the any newly added files, if any files added it will download from the server. But I don't need to maintain the FTP server session alive and disconnect the server after first connection or first time downloaded.
Code :
public class FtpServices {
#Bean(name="ftpSessionFactory")
public DefaultFtpSessionFactory ftpSessionFactory() {
System.out.println("session");
DefaultFtpSessionFactory sf = new DefaultFtpSessionFactory();
sf.setHost("localhost");
sf.setPort(21);
sf.setUsername("user");
sf.setPassword("password");
return sf;
}
#Bean
public FtpInboundFileSynchronizer ftpInboundFileSynchronizer() {
System.out.println("2");
FtpInboundFileSynchronizer fileSynchronizer = new FtpInboundFileSynchronizer(ftpSessionFactory());
fileSynchronizer.setDeleteRemoteFiles(false);
fileSynchronizer.afterPropertiesSet();
fileSynchronizer.setRemoteDirectory("/test/");
// fileSynchronizer.setFilter(new FtpSimplePatternFileListFilter("*.docx"));
fileSynchronizer.setFilter(filter);
return fileSynchronizer;
}
#Bean()
#InboundChannelAdapter(value="ftpChannel", poller = #Poller(fixedDelay = "50", maxMessagesPerPoll = "1"))
public FtpInboundFileSynchronizingMessageSource ftpMessageSource() {
System.out.println(3);
FtpInboundFileSynchronizingMessageSource source =
new FtpInboundFileSynchronizingMessageSource(ftpInboundFileSynchronizer());
source.setLocalDirectory(new File("D:/Test-downloaded/"));
//source.stop();
return source;
}
#Bean
#ServiceActivator(inputChannel = "ftpChannel", requiresReply="false")
public MessageHandler handler() {
System.out.println(4);
MessageHandler handler = new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
System.out.println(message.getPayload()+" #ServiceActivator");
System.out.println(" Message Header :"+message.getHeaders());
}
};
return handler;
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata defaultPoller() {
PollerMetadata pollerMetadata = new PollerMetadata();
pollerMetadata.setTrigger(triggerOnlyOnce());
return pollerMetadata;
}
}
and also I override the AbtractFTPSessionFactory.java to test FTP server connection and disconnection process.
protected void postProcessClientAfterConnect(T t) throws IOException {
System.out.println("After connect");
}
protected void postProcessClientBeforeConnect(T client) throws IOException {
System.out.println("Before connect");
}
Console :
INFO : org.springframework.context.support.DefaultLifecycleProcessor - Starting beans in phase -2147483648
INFO : org.springframework.context.support.DefaultLifecycleProcessor - Starting beans in phase 0
Before connect
After connect
D:\Test-downloaded\demo 1.txt #ServiceActivator
Message Header :{id=e4a1fd7f-0bbf-9692-f70f-b0ac68b4dec4, timestamp=1477317086272}
D:\Test-downloaded\demo.txt #ServiceActivator
Message Header :{id=9115ee92-12b4-bf1f-d592-9c13bf7a27fa, timestamp=1477317086324}
Before connect
After connect
Before connect
After connect
Before connect
After connect
Before connect
After connect
Before connect
After connect
Before connect
After connect
Thanks.
That is really a purpose of any #InboundChannelAdapter: poll the target system for new data periodically.
To do that once we sometimes suggest OnlyOnceTrigger:
public class OnlyOnceTrigger implements Trigger {
private final AtomicBoolean done = new AtomicBoolean();
#Override
public Date nextExecutionTime(TriggerContext triggerContext) {
return !this.done.getAndSet(true) ? new Date() : null;
}
}
But this might not work for your case, because there might not be desired files in the source FTP directory yet.
Therefore we have to poll until you will receive required files and .stop() an adapter when that condition is met.
For this purpose you can use any downstream logic to determine the state or consider to implement AbstractMessageSourceAdvice to be injected to the PollerMetadata of the #Poller: http://docs.spring.io/spring-integration/reference/html/messaging-channels-section.html#conditional-pollers
Is it possible to set RetryPolicy in spring retry (https://github.com/spring-projects/spring-retry) based on error status code? e.g. I want to retry on HttpServerErrorException with HttpStatus.INTERNAL_SERVER_ERROR status code, which is 503. Therefore it should ignore all other error codes -- [500 - 502] and [504 - 511].
The RestTemplate has setErrorHandler option and DefaultResponseErrorHandler is the default one.
Its code looks like:
public void handleError(ClientHttpResponse response) throws IOException {
HttpStatus statusCode = getHttpStatusCode(response);
switch (statusCode.series()) {
case CLIENT_ERROR:
throw new HttpClientErrorException(statusCode, response.getStatusText(),
response.getHeaders(), getResponseBody(response), getCharset(response));
case SERVER_ERROR:
throw new HttpServerErrorException(statusCode, response.getStatusText(),
response.getHeaders(), getResponseBody(response), getCharset(response));
default:
throw new RestClientException("Unknown status code [" + statusCode + "]");
}
}
So, you can provide your own implementation for that method to simplify your RetryPolicy around desired status codes.
For others who are facing same problem, I'm posting this answer.
Implement custom retry policy as follows:
class InternalServerExceptionClassifierRetryPolicy extends ExceptionClassifierRetryPolicy {
public InternalServerExceptionClassifierRetryPolicy() {
final SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(3);
this.setExceptionClassifier(new Classifier<Throwable, RetryPolicy>() {
#Override
public RetryPolicy classify(Throwable classifiable) {
if (classifiable instanceof HttpServerErrorException) {
// For specifically 500 and 504
if (((HttpServerErrorException) classifiable).getStatusCode() == HttpStatus.INTERNAL_SERVER_ERROR
|| ((HttpServerErrorException) classifiable)
.getStatusCode() == HttpStatus.GATEWAY_TIMEOUT) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
return new NeverRetryPolicy();
}
});
}}
Ans the simply call it as below:
RetryTemplate template = new RetryTemplate();
template.setRetryPolicy(new InternalServerExceptionClassifierRetryPolicy())
You can also add the specific error code in the retryableExceptions list of the SinmpleRetryPolicy.
Map<Class<? extends Throwable>, Boolean> retryableExceptions = new HashMap<>();
retryableExceptions.put(HttpClientErrorException.Unauthorized.class, true);
retryTemplate.setRetryPolicy(new SimpleRetryPolicy(5, retryableExceptions));