How to retry a failed call to a Webflux.outboundgateway in Spring integration - spring-boot

I have a spring integration flow defined in the flow DSL syntax. One of my handlers is a Webflux.outboundGateway. When the remote URI is not accessible, an exception is thrown and sent to the "errorChannel". I'm trying to have the flow to retry, but so far with no success (the call is never retried). Here is what my configuration looks like:
#Bean
public IntegrationFlow retriableFlow() {
return IntegrationFlows
.from(...)
.handle(
WebFlux.outboundGateway(m ->
UriComponentsBuilder.fromUriString(remoteGateway + "/foo/bar")
.build()
.toUri(), webClient)
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class)
.replyPayloadToFlux(true), e -> e.advice(retryAdvice())
)
// [ ... ]
.get();
}
#Bean
public Advice retryAdvice() {
RequestHandlerRetryAdvice advice = new RequestHandlerRetryAdvice();
RetryTemplate retryTemplate = new RetryTemplate();
ExponentialBackOffPolicy retryPolicy = new ExponentialBackOffPolicy();
retryPolicy.setInitialInterval(1000);
retryPolicy.setMaxInterval(20000);
retryTemplate.setBackOffPolicy(retryPolicy);
advice.setRetryTemplate(retryTemplate);
return advice;
}
Should I be using something different than the RequestHandlerRetryAdvice? If so, what should it be?

Webflux is, by definition, async, which means the Mono (reply) is satisfied asynchronously when the request completes/fails, not on the calling thread. Hence the advice won't help because the "send" part of the request is always successful.
You would have to perform retries via a flow on the error channel (assigned somewhere near the start of the flow). With, perhaps, some header indicating how many times you have retried.
The ErrorMessage has properties failedMessage and cause; you can resend the failedMessage.
You could turn off async so the calling thread blocks, but that really defeats the whole purpose of using WebFlux.

Related

suppress stacktrace from spring webclient/netty layer

In one of the apps my team manages, there's a GraphQL orchestration layer that calls a downstream service.
We use Spring's webclient for it.
WebClient Config.
WebClient.Builder webClientBuilder(MetricsWebClientCustomizer metricsCustomizer) {
final HttpClient httpClient = HttpClient.create(ConnectionProvider.fixed("webClientPool", maxConnections))
.tcpConfiguration(client ->
client.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, connectionTimeoutMillis)
.option(EpollChannelOption.TCP_KEEPIDLE, tcpKeepIdleInSec)
.option(EpollChannelOption.TCP_KEEPINTVL, tcpKeepIntvlInSec)
.option(EpollChannelOption.TCP_KEEPCNT, tcpKeepCount)
.doOnConnected(conn -> conn
.addHandlerLast(new ReadTimeoutHandler(readTimeoutInSec))
));
final WebClient.Builder webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(httpClient));
metricsCustomizer.customize(webClient);
return webClient;
}
return client.post()
.uri(uriBuilder -> buildUri())
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromObject(request))
.retrieve()
.bodyToMono(Result.class)
.compose(CircuitBreakerOperator.of(cb))
.block(Duration.ofSeconds(blockDuration));
This setup works well. However, I see a lot of
io.netty.handler.timeout.ReadTimeoutException: null
Suppressed: java.lang.Exception: #block terminated with an error
at reactor.core.publisher. .blockingGet(BlockingSingleSubscriber.java:133)
at reactor.core.publisher.Mono.block(Mono.java:1518)
at com.xxxx.c.g.xx.client.Client.get(Client.java:293)
at com.xxxx.c.g.xx.resolver.impl.xxQueryImpl.lambda$xx$171(xxQueryImpl.java:2187)
at io.micrometer.core.instrument.AbstractTimer.record(AbstractTimer.java:149)
Every timeout results in this chunky stacktrace. It's the same message that gets repeated. Does give any useful info. Is there a way to get WebClient/Netty print this once and ignore the rest?
BlockingSingleSubscriber.blockingGet seems to be doing this. Keeps track of previous exceptions.
Throwable e = this.error;
if (e != null) {
RuntimeException re = Exceptions.propagate(e);
re.addSuppressed(new Exception("#block terminated with an error"));
throw re;
} else {
return this.value;
}
I tried adding error handlers to the calling function bodyToMono(Result.class).doOnError("log").block();
This results in the line "log" from the doOnError consumer getting printed along with the chunky stacktraces.
Any ideas?
Full stacktrace:
https://pastebin.com/kdpspCEY
Not sure if you've solved the issue yet, but I had the same problem. For anyone with a similar issue, I solved it by setting the following in the application.properties file:
# Logging Information
logging.level.reactor.netty=off
This disabled Reactor Netty internal logging, which was a bit verbose for what I needed.

Async RabbitMQ communcation using Spring Integration

I have two spring boot services that communicate using RabbitMQ.
Service1 sends request for session creation to Service2.
Service2 handles request and should return response.
Service1 should handle the response.
Service1 method for requesting session:
public void startSession()
{
ListenableFuture<SessionCreationResponseDTO> sessionCreationResponse = sessionGateway.requestNewSession();
sessionCreationResponse.addCallback(response -> {
//handle success
}, ex -> {
// handle exception
});
}
On Service1 I have defined AsyncOutboundGateway, like:
#Bean
public IntegrationFlow requestSessionFlow(MessageChannel requestNewSessionChannel,
AsyncRabbitTemplate amqpTemplate,
SessionProperties sessionProperties)
{
return flow -> flow.channel(requestNewSessionChannel)
.handle(Amqp.asyncOutboundGateway(amqpTemplate)
.exchangeName(sessionProperties.getRequestSession().getExchangeName())
.routingKey(sessionProperties.getRequestSession().getRoutingKey()));
}
On Service2, I have flow for receiving these messages:
#Bean
public IntegrationFlow requestNewSessionFlow(ConnectionFactory connectionFactory,
SessionProperties sessionProperties,
MessageConverter messageConverter,
RequestNewSessionHandler requestNewSessionHandler)
{
return IntegrationFlows.from(Amqp.inboundGateway(connectionFactory,
sessionProperties.requestSessionProperties().queueName())
.handle(requestNewSessionHandler)
.get();
Service2 handles there requests:
#ServiceActivator(async = "true")
public ListenableFuture<SessionCreationResponseDTO> handleRequestNewSession()
{
SettableListenableFuture<SessionCreationResponseDTO> settableListenableFuture = new SettableListenableFuture<>();
// Goes through asynchronous process of creating session and sets value in listenable future
return settableListenableFuture;
}
Problem is that Service2 immediately returns ListenableFuture to Service1 as message payload, instead of waiting for result of future and sending back result.
If I understood documentation correctly Docs by setting async parameter in #ServiceActivator to true, successful result should be returned and in case of exception, error channel would be used.
Probably I misunderstood documentation, so that I need to unpack ListenableFuture in flow of Service2 before returning it as response, but I am not sure how to achieve that.
I tried something with publishSubscribeChannel but without much luck.
Your problem is here:
.handle(requestNewSessionHandler)
Such a configuration doesn't see your #ServiceActivator(async = "true") and uses it as a regular blocking service-activator.
Let's see if this helps you:
.handle(requestNewSessionHandler, "handleRequestNewSession", e -> e.async(true))
It is better to think about it like: or only annotation configuration. or only programmatic, via Java DSL.

Spring WebClient toFuture() vs. block() - What's the main difference?

My Spring Boot application uses WebClient to make calls to a remote API. I do have some difficulty understanding the difference between the following modes on how to use the WebClient.
Option 1 - using block()
// WebClient
public Boolean updateUser(long id) {
return webClient.post()
.uri(uriBuilder -> uriBuilder.path(USER_PATH).build(id))
.body(Mono.just(payload), User.class)
.exchangeToMono(clientResponse -> Mono.just(clientResponse.statusCode().is2xxSuccessful()))
.block();
}
// Caller
Boolean result = updateUser(5);
Option 2 - using toFuture():
// WebClient
public CompletableFuture<Boolean> updateUser(long id) {
return webClient.post()
.uri(uriBuilder -> uriBuilder.path(USER_PATH).build(id))
.body(Mono.just(payload), User.class)
.exchangeToMono(clientResponse -> Mono.just(clientResponse.statusCode().is2xxSuccessful()))
.toFuture();
}
// Caller
CompletableFuture<Boolean> future = updateUser(5);
Boolean result = future.get();
As far as I understand, using .block() blocks the thread when the WebClient makes its request and waits for a response.
When using toFuture() instead, then the WebClient runs on a different thread, thus it does not block. But is the thread not blocked anyways using the .get() method on the CompletableFuture?
When would I choose one over the other?
In the second option, you allow the caller to decide when to wait, this looks more flexible than the first option.
TL;DR
Mono.toFuture() is not blocking but Mono.toFuture().get() is blocking. block() is technically the same as toFuture().get() and both are blocking.
Mono.toFuture() just transforms Mono into a CompletableFuture by subscribing to it and resolving immediately. But it doesn't mean that you can access result of the corresponding Mono after this. CompletableFuture is still async and you can use methods like thenApply(), thenCompose(), thenCombine(), ... to continue async processing. toFuture().get() is a blocking operation.
CompletableFuture<Double> result = getUserDetail(userId)
.toFuture()
.thenCompose(user -> getCreditRating(user));
where getUserDetail is defined as
Mono<User> getUserDetail(String userId);
Mono.toFuture is useful when you need to combine different async APIs. For example, AWS Java v2 API is async but based on CompletableFuture but we can combine APIs using Mono.toFuture or Mono.fromFuture.

Is there a way to give dynamic timeouts to Rest Template?

I am using Spring Rest template along with apache's PoolingHttpClientConnectionManager for making API calls. The project in which I am working on requires setting custom timeout for each of the HTTP request I make via rest template. In order to achieve this, I am using CompletableFuture with a separate ExecutorService and calling get(Timeout) method.
try{
CompletableFuture<BidResponse> future = CompletableFuture.supplyAsync(() -> bidderService.getBid(), executorService);
bidResponse = future.get(bidderTimeout, TimeUnit.MILLISECONDS);
} catch (InterruptedException | TimeoutException | ExecutionException e) {
bidResponse = getTimeoutBidResponse();
}
Unfortunately, the problem with this approach is that in cases of timeout, the underlying thread keeps on working until the rest template finishes its call. So I am kind of losing out a thread from the thread pool, as well as a connection from the HTTP connection pool.
Is there a way to close the HTTP connection as soon as we receive a Timeout exception, and return the HTTP connection back to the pool ?
p.s. I also tried using Spring Webclient with Mono.timeout. Turns out it actually closes the HTTP connection immediately, but does not return it back to the HTTP pool.
#Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder)
{
return restTemplateBuilder
.setConnectTimeout(...)
.setReadTimeout(...)
.build();
}

Testing error handling with direct channels and source with poller

I have a following flow implemented in Spring Integration DSL:
Take feed from HTTP
Enrich errorChannel header (point of handling all exception downstream here).
Transform message to be a Collection
Split Collection into sepearate messages
Send each message to next processing channels
#Bean
public IntegrationFlow inboundHttpFlow(
Puller puller,
HeaderEnricher errorHandlingChannelHeaderEnricher,
FeedTransformer feedTransformer,
MessageChannel outputFeedIntegrationChannel
) {
final Consumer<SourcePollingChannelAdapterSpec> pollingSpec = spec ->
spec
.poller(Pollers.cron(SCHEDULE_EVERY_HALF_MINUTE)
.errorChannel(INBOUND_ERROR_CHANNEL));
return IntegrationFlows
.from(puller, pollingSpec)
.enrichHeaders(errorHandlingChannelHeaderEnricher)
.transform(feedTransformer)
.split()
.channel(outputFeedIntegrationChannel)
.get();
}
Where my errorHandlingChannelHeaderEnricher is:
#Component
public class ErrorHandlingChannelHeaderEnricher implements HeaderEnricher {
#Override
public void accept(HeaderEnricherSpec spec) {
spec.header(
MessageHeaders.ERROR_CHANNEL,
INBOUND_ERROR_CHANNEL,
true
);
}
}
When feedTransformer throws an exception in working app then it goes to set errorChannel as expected. But I don't know how to write a test to test if thrown exception goes to errorChannel defined in header?
When I'm trying to simulate it in test given way, it doesn't work because exception is thrown back into caller instead of errorChannel:
// given
Throwable transformerException = new IllegalStateException();
when(feedTransformerMock.apply(any())).thenThrow(transformerException);
// when
var testFeedMessage = MessageBuilder
.withPayload(pullerResult)
.build();
inboundHttpFlow.getInputChannel().send(testFeedMessage); // excetpion returns to caller here
// then
verify(errorHandlerSpy).accept(transformerException);
And exception is typical:
org.springframework.integration.transformer.MessageTransformationException: Failed to transform Message; nested exception is org.springframework.messaging.MessageHandlingException: nested exception is java.lang.IllegalStateException, failedMessage=GenericMessage [payload=test-payload, headers={errorChannel=inboundErrorChannel, id=f77a6a01-9bca-5af3-8352-7edb4c5e94b0, timestamp=1556019833867}]
, failedMessage=GenericMessage [payload=test-payload, headers={errorChannel=inboundErrorChannel, id=f77a6a01-9bca-5af3-8352-7edb4c5e94b0, timestamp=1556019833867}]
I assume that because of DirectChannel and lack of poller in this test example in compare to real flow (with poller).
Is there any way to simulate that throwing exception and checking if it really goes to errorChannel defined in header?
It's not clear what you are trying to test.
Why do you need to test the framework?
You don't need to enrich the headers since you already have an error channel on the poller.
You are correct; sending to a DirectChannel will result in the exception being thrown to the caller.
If you really want to test the framework, mock the Puller instead of sending to the input channel.

Resources