I'm using a WebClient object to send Http Post request to a server.
It's sending a huge amount of requests quite rapidly (there is about 4000 messages in a QueueChannel). The problem is... it seems the server can't respond fast enough... so I'm getting a lot of server error 500 and connexion closed prematurely.
Is there a way to limit the number of request per seconds ? Or limit the number of threads it's using ?
EDIT :
The Message endpoint processe message in a QueueChannel :
#MessageEndpoint
public class CustomServiceActivator {
private static final Logger logger = LogManager.getLogger();
#Autowired
IHttpService httpService;
#ServiceActivator(
inputChannel = "outputFilterChannel",
outputChannel = "outputHttpServiceChannel",
poller = #Poller( fixedDelay = "1000" )
)
public void processMessage(Data data) {
httpService.push(data);
try {
Thread.sleep(20);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
The WebClient service class :
#Service
public class HttpService implements IHttpService {
private static final String URL = "http://www.blabla.com/log";
private static final Logger logger = LogManager.getLogger();
#Autowired
WebClient webClient;
#Override
public void push(Data data) {
String body = constructString(data);
Mono<ResponseEntity<Response>> res = webClient.post()
.uri(URL + getLogType(data))
.contentLength(body.length())
.contentType(MediaType.APPLICATION_JSON)
.syncBody(body)
.exchange()
.flatMap(response -> response.toEntity(Response.class));
res.subscribe(new Consumer<ResponseEntity<Response>>() { ... });
}
}
Resilience4j has excellent support for non-blocking rate limiting with Project Reactor.
Required dependencies (beside Spring WebFlux):
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-reactor</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-ratelimiter</artifactId>
<version>1.6.1</version>
</dependency>
Example:
import io.github.resilience4j.ratelimiter.RateLimiter;
import io.github.resilience4j.ratelimiter.RateLimiterConfig;
import io.github.resilience4j.reactor.ratelimiter.operator.RateLimiterOperator;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.time.Duration;
import java.time.LocalDateTime;
import java.util.concurrent.atomic.AtomicInteger;
public class WebClientRateLimit
{
private static final AtomicInteger COUNTER = new AtomicInteger(0);
private final WebClient webClient;
private final RateLimiter rateLimiter;
public WebClientRateLimit()
{
this.webClient = WebClient.create();
// enables 3 requests every 5 seconds
this.rateLimiter = RateLimiter.of("my-rate-limiter",
RateLimiterConfig.custom()
.limitRefreshPeriod(Duration.ofSeconds(5))
.limitForPeriod(3)
.timeoutDuration(Duration.ofMinutes(1)) // max wait time for a request, if reached then error
.build());
}
public Mono<?> call()
{
return webClient.get()
.uri("https://jsonplaceholder.typicode.com/todos/1")
.retrieve()
.bodyToMono(String.class)
.doOnSubscribe(s -> System.out.println(COUNTER.incrementAndGet() + " - " + LocalDateTime.now()
+ " - call triggered"))
.transformDeferred(RateLimiterOperator.of(rateLimiter));
}
public static void main(String[] args)
{
WebClientRateLimit webClientRateLimit = new WebClientRateLimit();
long start = System.currentTimeMillis();
Flux.range(1, 16)
.flatMap(x -> webClientRateLimit.call())
.blockLast();
System.out.println("Elapsed time in seconds: " + (System.currentTimeMillis() - start) / 1000d);
}
}
Example output:
1 - 2020-11-30T15:44:01.575003200 - call triggered
2 - 2020-11-30T15:44:01.821134 - call triggered
3 - 2020-11-30T15:44:01.823133100 - call triggered
4 - 2020-11-30T15:44:04.462353900 - call triggered
5 - 2020-11-30T15:44:04.462353900 - call triggered
6 - 2020-11-30T15:44:04.470399200 - call triggered
7 - 2020-11-30T15:44:09.461199100 - call triggered
8 - 2020-11-30T15:44:09.463157 - call triggered
9 - 2020-11-30T15:44:09.463157 - call triggered
11 - 2020-11-30T15:44:14.461447700 - call triggered
10 - 2020-11-30T15:44:14.461447700 - call triggered
12 - 2020-11-30T15:44:14.461447700 - call triggered
13 - 2020-11-30T15:44:19.462098200 - call triggered
14 - 2020-11-30T15:44:19.462098200 - call triggered
15 - 2020-11-30T15:44:19.468059700 - call triggered
16 - 2020-11-30T15:44:24.462615 - call triggered
Elapsed time in seconds: 25.096
Docs: https://resilience4j.readme.io/docs/examples-1#decorate-mono-or-flux-with-a-ratelimiter
Question Limiting rate of requests with Reactor provides two answrers (one in comment)
zipWith another flux that acts as rate limiter
.zipWith(Flux.interval(Duration.of(1, ChronoUnit.SECONDS)))
just delay each web request
use delayElements function
edit: answer below is valid for blocking RestTemplate but do not really fit well into reactive pattern.
WebClient does not have ability to limit request, but you could easily add this feature using composition.
You may throttle your client externally using RateLimiter from Guava/
(https://google.github.io/guava/releases/19.0/api/docs/index.html?com/google/common/util/concurrent/RateLimiter.html)
In this tutorial http://www.baeldung.com/guava-rate-limiter you will find how to use Rate limiter in blocking way, or with timeouts.
I would decorate all calls that need to be throttled in separate class that
limits number of calls per second
performs actual web call using WebClient
I hope I'm not late for the party. Anyway, limiting the rate of the request is just one of the problem I faced a week ago as I was creating a crawler. Here are the issues:
I have to do a recursive, paginated sequential request. Pagination parameters are included in the API that I'm calling for.
Once a response is received, pause for 1 second before doing the next request.
For certain errors encountered, do a retry
On retry, pause for certain seconds
Here's the solution:
private Flux<HostListResponse> sequentialCrawl() {
AtomicLong pageNo = new AtomicLong(2);
// Solution for #1 - Flux.expand
return getHosts(1)
.doOnRequest(value -> LOGGER.info("Start crawling."))
.expand(hostListResponse -> {
final long totalPages = hostListResponse.getData().getTotalPages();
long currPageNo = pageNo.getAndIncrement();
if (currPageNo <= totalPages) {
LOGGER.info("Crawling page " + currPageNo + " of " + totalPages);
// Solution for #2
return Mono.just(1).delayElement(Duration.ofSeconds(1)).then(
getHosts(currPageNo)
);
}
return Flux.empty();
})
.doOnComplete(() -> LOGGER.info("End of crawling."));
}
private Mono<HostListResponse> getHosts(long pageNo) {
final String uri = hostListUrl + pageNo;
LOGGER.info("Crawling " + uri);
return webClient.get()
.uri(uri)
.exchange()
// Solution for #3
.retryWhen(companion -> companion
.zipWith(Flux.range(1, RETRY + 1), (error, index) -> {
String message = "Failed to crawl uri: " + error.getMessage();
if (index <= RETRY && (error instanceof RequestIntervalTooShortException
|| error instanceof ConnectTimeoutException
|| "Connection reset by peer".equals(error.getMessage())
)) {
LOGGER.info(message + ". Retries count: " + index);
return Tuples.of(error, index);
} else {
LOGGER.warn(message);
throw Exceptions.propagate(error); //terminate the source with the 4th `onError`
}
})
.map(tuple -> {
// Solution for #4
Throwable e = tuple.getT1();
int delaySeconds = tuple.getT2();
// TODO: Adjust these values according to your needs
if (e instanceof ConnectTimeoutException) {
delaySeconds = delaySeconds * 5;
} else if ("Connection reset by peer".equals(e.getMessage())) {
// The API that this app is calling will sometimes think that the requests are SPAM. So let's rest longer before retrying the request.
delaySeconds = delaySeconds * 10;
}
LOGGER.info("Will retry crawling after " + delaySeconds + " seconds to " + uri + ".");
return Mono.delay(Duration.ofSeconds(delaySeconds));
})
.doOnNext(s -> LOGGER.warn("Request is too short - " + uri + ". Retried at " + LocalDateTime.now()))
)
.flatMap(clientResponse -> clientResponse.toEntity(String.class))
.map(responseEntity -> {
HttpStatus statusCode = responseEntity.getStatusCode();
if (statusCode != HttpStatus.OK) {
Throwable exception;
// Convert json string to Java POJO
HostListResponse response = toHostListResponse(uri, statusCode, responseEntity.getBody());
// The API that I'm calling will return error code of 06 if request interval is too short
if (statusCode == HttpStatus.BAD_REQUEST && "06".equals(response.getError().getCode())) {
exception = new RequestIntervalTooShortException(uri);
} else {
exception = new IllegalStateException("Request to " + uri + " failed. Reason: " + responseEntity.getBody());
}
throw Exceptions.propagate(exception);
} else {
return toHostListResponse(uri, statusCode, responseEntity.getBody());
}
});
}
I use this to limit the number of active requests:
public DemoClass(WebClient.Builder webClientBuilder) {
AtomicInteger activeRequest = new AtomicInteger();
this.webClient = webClientBuilder
.baseUrl("http://httpbin.org/ip")
.filter(
(request, next) -> Mono.just(next)
.flatMap(a -> {
if (activeRequest.intValue() < 3) {
activeRequest.incrementAndGet();
return next.exchange(request)
.doOnNext(b -> activeRequest.decrementAndGet());
}
return Mono.error(new RuntimeException("Too many requests"));
})
.retryWhen(Retry.anyOf(RuntimeException.class)
.randomBackoff(Duration.ofMillis(300), Duration.ofMillis(1000))
.retryMax(50)
)
)
.build();
}
public Mono<String> call() {
return webClient.get()
.retrieve()
.bodyToMono(String.class);
}
We can customize ConnectionBuilder to rate limit the active connections on WebClient.
Need to add pendingAquiredMaxCount for number of waiting requests on queue as the default queue size is always 2 * maxConnections.
This rate limits the webclient to serve the requests at a time.
ConnectionProvider provider = ConnectionProvider.builder('builder').maxConnections(maxConnections).pendingAcquireMaxCount(maxPendingRequests).build()
TcpClient tcpClient = TcpClient
.create(provider)
WebClient client = WebClient.builder()
.baseUrl('url')
.clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient)))
Related
I've set up a spring-reactive server and an use a while loop inside of a Scheduled job to make fire and forget requests from a httpClient (backend just returns the same simple string via tomcat) it is configured as follows:
public ConnectionProvider getConnectionProvider(){
int maxConnections = 20;
ConnectionProvider connProvider = ConnectionProvider
.builder("webclient-conn-pool")
.maxConnections(maxConnections)
.maxIdleTime(Duration.of(20, ChronoUnit.SECONDS))
.maxLifeTime(Duration.of(1000, ChronoUnit.SECONDS))
.pendingAcquireMaxCount(2000)
.pendingAcquireTimeout(Duration.ofMillis(20000))
.build();
return connProvider;
}
public HttpClient getHttpClient(){
return HttpClient
.create(getConnectionProvider());
//.secure(sslContextSpec -> sslContextSpec.sslContext(webClientSslHelper.getSslContext()))
/*.tcpConfiguration(tcpClient -> {
LoopResources loop = LoopResources.create("webclient-event-loop",
selectorThreadCount, workerThreadCount, Boolean.TRUE);
return tcpClient
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 10000)
.option(ChannelOption.TCP_NODELAY, true);*/
}
I've also created a job that uses #Autowired kafkaTemplate and makes a while loop that sends a short text string
For http / tomcat calls I am getting around 65 requests a second
For kafka I am maxing out around 50000 requests with a half second lag
application.props
spring.kafka.bootstrap-servers=PLAINTEXT://localhost:9092,PLAINTEXT://localhost:9093
host.name=localhost
Job to make calls
#Async
#Scheduled(fixedDelay = 15000)
public void scheduleTaskUsingCronExpression() {
generateCalls();
}
private void generateCalls() {
try{
int i = 0;
System.out.println("start");
long startTime = System.currentTimeMillis();
while(i <= 5000){
//Thread.sleep(5);
String message = "Test Message sadg sad-";
kafkaTemplate.send(TOPIC, message + i);
i++;
}
long endTime = System.currentTimeMillis();
System.out.println((endTime - startTime));
System.out.println("done");
}
catch(Exception e){
e.printStackTrace();
}
System.out.println("RUNNING");
}
Kakfa partition config
#Bean
public KafkaAdmin kafkaAdmin() {
//String bootstrapAddress = "localhost:29092";
String bootstrapAddress = "localhost:9092";
Map<String, Object> configs = new HashMap<>();
configs.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);
return new KafkaAdmin(configs);
}
#Bean
public NewTopic testTopic() {
return new NewTopic("test-topic", 6, (short) 1);
}
Kafka consumer consuming messge
#KafkaListener(topics = "test-topic", groupId = "one", concurrency = "6" )
public void listenGroupFoo(String message) {
if(message.indexOf("-0") != -1){
startTime = new Date().getTime();
System.out.println("Starting Message in group foo: " + message);
}
else if(message.indexOf("-100000") != -1){
endTime = new Date().getTime();
System.out.println("Received Message in group foo: " + message);
System.out.println(endTime - startTime);
}
}
For hardware I have a 10900k with 64gb ram
5ghz clock speed
970 Evo single nvme disk
10 core 20 thread
All requests are from the same local server to the same local server
I can add more local servers for testing if needed
Is there a better way to organize / optimize the code to make a massive number of requests?
Theories:
Multiple Threads?
Changing configurations of servers such as tomcat configs (receiving or sending side)?
Not use the kafkaTemplate that is autowired or creating multiple?
Modify Hardware to have multiple disks?
Improve server receiving http to allow more connections?
Not use a job to make the requests?
Anything else anyone can think of to help?
I'm using Spring's 'WebClient` and project reactor to make non-blocking calls to a list of URLs. My requirements are:
Asynchronously call GET on a list of URLs
Log the URL when each URL is called
Log the URL of a call that results in a exception
Log the URL of a call that is successful
Log the URL of a call that results in a non 2xx HTTP status
Send an email containing a list of URLs where the call resulted in an exception or non 2xx HTTP status
Here's my attempt to do this:
List<Mono<ClientResponse>> restCalls = new ArrayList<>();
List<String> failedUrls = new ArrayList<>();
for (String serviceUrl : serviceUrls.getServiceUrls()) {
restCalls.add(
webClientBuilder
.build()
.get()
.uri(serviceUrl)
.exchange()
.doOnSubscribe(c -> log.info("calling service URL {}", serviceUrl))
.doOnSuccess(response -> log.info("{} success status {}", serviceUrl, response.statusCode().toString()))
.doOnError(response -> {log.info("{} error status {}", serviceUrl, response); failedUrls.add(serviceUrl);}));
}
Flux.fromIterable(restCalls)
.map((data) -> data.subscribe())
.onErrorContinue((throwable, e) -> {
log.info("Exception for URL {}", ((WebClientResponseException) throwable).getRequest().getURI());
failedUrls.add(serviceUrl);
})
.collectList()
.subscribe((data) -> {
log.info("all called");
email.send("Failed URLs are {}", failedUrls);
});
The problem is the email is sent before the calls respond. How can I wait until all URLs calls have been completed prior to calling email.send?
As stated in comment, the main error in your example is the use of 'subscribe', that launch queries, but in a context independant from the main flux, so you cannot get back errors or results.
subscribe is sort of a trigger operation on the pipeline, it's not used for chaining.
Here is a full example (except email, replaced by logging):
package fr.amanin.stackoverflow;
import org.springframework.http.HttpStatus;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.util.Arrays;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.stream.Collectors;
public class WebfluxURLProcessing {
private static final Logger LOGGER = Logger.getLogger("example");
public static void main(String[] args) {
final List<String> urls = Arrays.asList("https://www.google.com", "https://kotlinlang.org/kotlin/is/wonderful/", "https://stackoverflow.com", "http://doNotexists.blabla");
final Flux<ExchangeDetails> events = Flux.fromIterable(urls)
// unwrap request async operations
.flatMap(url -> request(url))
// Add a side-effect to log results
.doOnNext(details -> log(details))
// Keep only results that show an error
.filter(details -> details.status < 0 || !HttpStatus.valueOf(details.status).is2xxSuccessful());
sendEmail(events);
}
/**
* Mock emails by collecting all events in a text and logging it.
* #param report asynchronous flow of responses
*/
private static void sendEmail(Flux<ExchangeDetails> report) {
final String formattedReport = report
.map(details -> String.format("Error on %s. status: %d. Reason: %s", details.url, details.status, details.error.getMessage()))
// collecting (or reducing, folding, etc.) allows to gather all upstream results to use them as a single value downstream.
.collect(Collectors.joining(System.lineSeparator(), "REPORT:"+System.lineSeparator(), ""))
// In a real-world scenario, replace this with a subscribe or chaining to another reactive operation.
.block();
LOGGER.info(formattedReport);
}
private static void log(ExchangeDetails details) {
if (details.status >= 0 && HttpStatus.valueOf(details.status).is2xxSuccessful()) {
LOGGER.info("Success on: "+details.url);
} else {
LOGGER.log(Level.WARNING,
"Status {0} on {1}. Reason: {2}",
new Object[]{
details.status,
details.url,
details.error == null ? "None" : details.error.getMessage()
});
}
}
private static Mono<ExchangeDetails> request(String url) {
return WebClient.create(url).get()
.retrieve()
// workaround to counter fail-fast behavior: create a special error that will be converted back to a result
.onStatus(status -> !status.is2xxSuccessful(), cr -> cr.createException().map(err -> new RequestException(cr.statusCode(), err)))
.toBodilessEntity()
.map(response -> new ExchangeDetails(url, response.getStatusCode().value(), null))
// Convert back custom error to result
.onErrorResume(RequestException.class, err -> Mono.just(new ExchangeDetails(url, err.status.value(), err.cause)))
// Convert errors that shut connection before server response (cannot connect, etc.) to a result
.onErrorResume(Exception.class, err -> Mono.just(new ExchangeDetails(url, -1, err)));
}
public static class ExchangeDetails {
final String url;
final int status;
final Exception error;
public ExchangeDetails(String url, int status, Exception error) {
this.url = url;
this.status = status;
this.error = error;
}
}
private static class RequestException extends RuntimeException {
final HttpStatus status;
final Exception cause;
public RequestException(HttpStatus status, Exception cause) {
this.status = status;
this.cause = cause;
}
}
}
I haven't tested this, but this should work
public void check() {
List<Flux<String>> restCalls = new ArrayList<>();
for (String serviceUrl : serviceUrls.getServiceUrls()) {
restCalls.add(rest.getForEntity(serviceUrl, String.class));
}
Flux.fromIterable(restCalls)
.map((data) -> data.blockFirst())
.onErrorContinue((throwable, e) -> {
((WebClientResponseException) throwable).getRequest().getURI(); // get the failing URI
// do whatever you need with the failed service
})
.collectList() // Collects all the results into a list
.subscribe((data) -> {
// from here do whatever is needed from the results
});
}
So if you haven't done so, your service call must be non blocking, so you should turn the type into Flux.
So inside your restService your method should be something like this
public Flux<String> getForEntity(String name) {
return this.webClient.get().uri("url", name)
.retrieve().bodyToFlux(String.class);
}
I hope it helps out
restCalls.add(
webClientBuilder
.build()
.get()
.uri(serviceUrl)
.exchange()
.doOnSubscribe(c -> log.info("calling service URL {}", serviceUrl))
.doOnSuccess(response -> log.info("{} success status {}", serviceUrl, response.statusCode().toString()))
.doOnError(response -> {log.info("{} error status {}", serviceUrl, response); failedUrls.add(serviceUrl);}));
Flux.fromIterable(restCalls)
.map((data) -> data.subscribe())
.onErrorContinue((throwable, e) -> {
log.info("Exception for URL {}", ((WebClientResponseException) throwable).getRequest().getURI());
failedUrls.add(serviceUrl);
})
.collectList()
.subscribeOn(Schedulers.elastic())
.subscribe((data) -> {
log.info("all called");
email.send("Failed URLs are {}", failedUrls);
});
Move from https://github.com/square/okhttp/issues/6051
This is a problem with okhttp keepalived connections, I wonder how to deal with this issue.
OkHttpClient httpClient = new OkHttpClient.Builder()
.sslSocketFactory(factory, (X509TrustManager) TRUST_MANAGERS[0])
.hostnameVerifier((s, sslSession) -> true)
.protocols(Arrays.asList(Protocol.HTTP_2, Protocol.HTTP_1_1))
.retryOnConnectionFailure(true)
.connectionPool(new ConnectionPool(5, 60, TimeUnit.SECONDS))
.build();
Request request = new Request.Builder()
.url("https://example.com/")
.build();
int count = 2;
long startTime = System.currentTimeMillis();
final CountDownLatch countDownLatch = new CountDownLatch(count);
for (int i = 1; i <= count; ++i) {
httpClient.newCall(request).enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
System.out.println(Thread.currentThread().getName() + "-failed-" + e.getMessage());
finish();
}
#Override
public void onResponse(Call call, Response response) throws IOException {
finish();
}
private void finish() {
countDownLatch.countDown();
}
});
}
countDownLatch.await();
System.out.println("Time: " + (System.currentTimeMillis() - startTime) );
System.exit(0);
}
Use LoggingEventListener if you want to see what is going on.
Generally when we first connect we can't know ahead of time that a single connection will be http/2 and able to serve both requests. So when the connection pool is empty, OkHttp will establish 2 connections, discover they are both http/2 and then disconnect one.
So likely both requests use a single connection, but you see two established.
You have a few options
1) Don't worry, let it create two connections temporarily, and OkHttp will close one as soon as it realises it has two HTTP/2 connections.
2) Use a warmup request to create the first request, then only create secondary requests to the same host once you have received a connectionAcquired event on that first request.
But I'd encourage you not to worry and just let OkHttp do it's thing.
We are trying to follow this blog and understand deferredresult . https://www.linkedin.com/pulse/building-async-non-blocking-microservices-using-spring-patnaik/
In the blog, the person has given a normal blocking and non blocking code. I have copied it here.
The person ran jmeter to concurrently test 1000 threads for 5 minutes. He mentioned that latency and tps were different for non blocking with the code. When I try to see it using jmeter with same details, i see the same latency for both non blocking and blocking.
I have already tried decreasing and increasing the times etc.
public SpringBootAppController() {
timer = new Timer();
ses = new ScheduledThreadPoolExecutor(10);
}
#RequestMapping("/blockingprocess")
public String blockingProcessing(#RequestParam(value="processingtime") long processingtime) throws InterruptedException
{
long startTime = System.currentTimeMillis();
Thread.sleep(processingtime);
//add more processing later
long endTime = System.currentTimeMillis();
long timeTaken = endTime-startTime;
return "SUCCESS. Blocking process completed in " + timeTaken + " Ms";
}
#RequestMapping("/nonblockingprocess")
public DeferredResult<String> nonBlockingProcessing(#RequestParam(value="processingtime") long processingtime) throws InterruptedException
{
DeferredResult<String> deferredResult = new DeferredResult<String>();
NewProcess j = new NewProcess(deferredResult, processingtime);
ses.schedule(j,processingtime, TimeUnit.MILLISECONDS);
System.out.println("hello");
return deferredResult;
}
Another class.
public NewProcess(DeferredResult<String> deferredresult, long processingtime)
{
this.deferredresult = deferredresult;
this.processingtime = processingtime;
}
#Override
public void run()
{
String result = "SUCCESS non blocking process completed in " + processingtime + " Ms";
deferredresult.setResult(result);
}
Expected is difference and better performance for non blocking compared to blocking.
My goal here is to log time of a process without using xml files for configurations. By reading other posts I came up with enriching headers in the integration flow. This kinda works, but not for the right purpose. For every new started process it gives me a startTime when the application is launched (i.e. a constant). See below:
#Bean
public IntegrationFlow processFileFlow() {
return IntegrationFlows
.from(FILE_CHANNEL_PROCESSING)
.transform(fileToStringTransformer())
.enrichHeaders(h -> h.header("startTime", String.valueOf(System.currentTimeMillis())))
.handle(FILE_PROCESSOR, "processFile").get();
}
My goal is to properly log the process without using xml files like I said above but I don't manage to do this. I found an example and tried a solution with ChannelInterceptorAdapter like this:
#Component(value = "integrationLoggingInterceptor")
public class IntegrationLoggingInterceptor extends ChannelInterceptorAdapter {
private static final Logger log = LoggerFactory.getLogger(IntegrationLoggingInterceptor.class);
#Override
public void postSend(Message<?> message, MessageChannel channel, boolean sent) {
log.debug("Post Send - Channel " + channel.getClass());
log.debug("Post Send - Headers: " + message.getHeaders() + " Payload: " + message.getPayload() + " Message sent?: " + sent);
}
#Override
public Message<?> postReceive(Message<?> message, MessageChannel channel) {
try {
log.debug("Post Receive - Channel " + channel.getClass());
log.debug("Post Receive - Headers: " + message.getHeaders() + " Payload: " + message.getPayload());
} catch (Exception ex) {
log.error("Error in post receive : ", ex);
}
return message;
}
}
But I receive no logs at all. Any ideas?
The .enrichHeaders(h -> h.header("startTime", String.valueOf(System.currentTimeMillis()))) falls to this:
public <V> HeaderEnricherSpec header(String name, V value, Boolean overwrite) {
AbstractHeaderValueMessageProcessor<V> headerValueMessageProcessor =
new StaticHeaderValueMessageProcessor<>(value);
headerValueMessageProcessor.setOverwrite(overwrite);
return header(name, headerValueMessageProcessor);
}
Pay attention to the StaticHeaderValueMessageProcessor. So, what you show is really a constant.
If you need a value calculated for each message to process, you should consider to use Function-based variant:
.enrichHeaders(h ->
h.headerFunction("startTime",
m -> String.valueOf(System.currentTimeMillis())))