This question already has answers here:
Restrict Spring WebClient call at application level
(2 answers)
Closed 4 years ago.
We are using Spring WebClient for calling web services using the same.
However, i don't know how to create/manage connection pool in Spring WebClient.
I got to know that we have use 'ReactorClientHttpConnector' but just don't get any sample code.
Basically, i want to have WebClient pool with maxTotal, maxWaitMillis etc.
Spring WebClient is a No-Blocking IO http client while ReactorClientHttpConnector is a Reactor-Netty based implementation. Said that I can suggest to do not warry about connection pool but focus on a complete no blocking service call. The key of succes using this kind of technology is all on focus on a complete no blocking service call chain, the model do not involve a thread per request, it is like a browser or node js development if you has something that block your code you block anything. I know that it is so not usual but the base implementation on a event-loop model force you to think about a completely different model.
I can comfort you by telling you that typically in a Netty based implementation you have a number of event loop that is the same of the number of your core, it is configurable of course but is think that it is enough, remember the power of reactive and no blocking IO programming is to embrace the no blockin io in all pieces of your code and add more event loop per processor will bring you to add some of concurrency while having one event loop per processor will enabling you to a fully parallel usage of your processor.
I hoe that this reflection can help you
TIP. for timeout on your http service call you can add a timeout on your like in the test below:
#Test
#WithMockUser(username = "user")
fun `read basic personal details data`() {
personalDetailsRepository.save("RESUME_ID", TestCase.personalDetails()).toMono().block();
val expectedJson = TestCase.readFileAsString("personal-details.json")
webClient.get()
.uri("/resume/RESUME_ID/personal-details")
.accept(MediaType.APPLICATION_JSON)
.exchange().toMono().timeout(Duration.ofMinutes(1))
}
Update
Considering the request of restrict on application level the webclient of course it is possible use the Backpressure feature in order to deal with a data stream that may be too large at times to be reliably processed or in case if a stream response like a Flux with the Flux limitRate() operator can be useful taking the official documentation:
/**
* Ensure that backpressure signals from downstream subscribers are split into batches
* capped at the provided {#code prefetchRate} when propagated upstream, effectively
* rate limiting the upstream {#link Publisher}.
* <p>
* Note that this is an upper bound, and that this operator uses a prefetch-and-replenish
* strategy, requesting a replenishing amount when 75% of the prefetch amount has been
* emitted.
* <p>
* Typically used for scenarios where consumer(s) request a large amount of data
* (eg. {#code Long.MAX_VALUE}) but the data source behaves better or can be optimized
* with smaller requests (eg. database paging, etc...). All data is still processed,
* unlike with {#link #limitRequest(long)} which will cap the grand total request
* amount.
* <p>
* Equivalent to {#code flux.publishOn(Schedulers.immediate(), prefetchRate).subscribe() }.
* Note that the {#code prefetchRate} is an upper bound, and that this operator uses a
* prefetch-and-replenish strategy, requesting a replenishing amount when 75% of the
* prefetch amount has been emitted.
*
* #param prefetchRate the limit to apply to downstream's backpressure
*
* #return a {#link Flux} limiting downstream's backpressure
* #see #publishOn(Scheduler, int)
* #see #limitRequest(long)
*/
public final Flux<T> limitRate(int prefetchRate) {
return onAssembly(this.publishOn(Schedulers.immediate(), prefetchRate));
}
said that I suggest to use this features and do not attempt to limit the consuming of data in a forced way like a connection limit. One of the point of strength of reactive programming and No blocking IO is on the incredible efficiency to use the resource and limit the resource usage appear like a against sense of the spirit of paradigm
Related
I have a legacy Spring Boot REST app that interacts with downstream services that block. I'm new to reactive programming, and am unsure how to handle these blocking requests. Most Webflux examples I've seen are pretty trivial. Here's the flow-of-control of my app:
User queries MyApp at http://myapp.com
MyApp then queries partner REST API, which is BLOCKING.
Depending on account type, data from the blocking app needs to be queried to make another call to another blocking REST application.
All data is enriched and rendered by MyApp to the browser.
Where to start? I'm using WebClient currently, so that part's done. I know I should perform the blocking steps on a different scheduler (parallel or boundedElastic?) Should I use a Flux or Mono, since the partner APIs return the data all at once?
Both apps return thousands of rows of data, and the user just waits... Steps 1-2 take about 4 secs; add in step 3, and we're looking at over 30 seconds due to the inefficiency of the API. Can Flux help my users' wait time at all?
EDIT Below is a (long) example of what my application is doing. Notice that I block my first call to the API to get a count of what's being returned, then I fetch the rest in batches of TASK_QUERY_LIMIT.
#Bean
public WebClient authWebClient(WebClient.Builder builder) {
MultiValueMap<String, String> map = new LinkedMultiValueMap<>();
map.set(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
final int size = 48 * 1024 * 1024;
final ExchangeStrategies strategies = ExchangeStrategies.builder()
.codecs(codecs -> codecs.defaultCodecs().maxInMemorySize(size))
.build();
return builder.baseUrl(configProperties.getUrl())
.exchangeStrategies(strategies)
.defaultHeaders(httpHeaders -> httpHeaders.addAll(map))
.filters(exchangeFilterFunctions -> {
exchangeFilterFunctions.add(logResponseStatus());
exchangeFilterFunctions.add(logRequest());
})
.build();
}
public Mono<Task> getTasksMono() {
return getAuthWebClient()
.baseUrl("http://MyApp.com")
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(HttpStatus::isError, this::onHttpStatusError)
.bodyToMono(new ParameterizedTypeReference<Response<Task>>() {}));
}
// Service method
public List<Task> getTasksMono() {
Mono<Response<Task>> monoTasks = getTasksMono();
Task tasks = monoTasks.block();
int taskCount = tasks.getCount();
List<Task> returnTasks = new ArrayList<>(tasks.getData());
List<Mono<<Task>> tasksMonoList = new ArrayList<>();
// query API-ONE for all remaining tasks
if (taskCount > TASK_QUERY_LIMIT) {
retrieveAdditionalTasks(key, taskCount, tasksMonoList);
}
// Send out all of the calls at once, and subscribe to their results.
Flux.mergeSequential(tasksMonoList)
.map(Response::getData)
.doOnNext(returnTasks::addAll)
.blockLast();
return returnTasks.stream()
.map(this::transform) // This method performs business logic on the data before returning to user
.collect(Collectors.toList());
}
private void retrieveAdditionalTasks(String key, int taskCount,
List<Mono<Response<Task>>> tasksMonoList) {
int offset = TASK_QUERY_LIMIT;
int numRequests = (taskCount - offset) / TASK_QUERY_LIMIT + 1;
for (int i = 0; i < numRequests; i++) {
tasksMonoList.add(getTasksMono(processDefinitionKey, encryptedIacToken,
TASK_QUERY_LIMIT, offset));
offset += TASK_QUERY_LIMIT;
}
}
There are multiple questions here. Will try to highlight main points
1. Does it make sense refactoring to Reactive API?
From the first look your application is IO bound and typically reactive applications are much more efficient because all IO operations are async and non-blocking. Reactive application will not be faster but you will need less resources to The only caveat is that in order to get all benefits from the reactive API, your app should be reactive end-to-end (reactive drivers for DB, reactive WebClient, …). All reactive logic is executed on Schedulers.parallel() and you need small number of threads (by default, number of CPU cores) to execute non-blocking logic. It’s still possible use blocking API by “offloading” them to Schedulers.boundedElastic() but it should be an exception (not the rule) to make your app efficient. For more details, check Flight of the Flux 3 - Hopping Threads and Schedulers.
2. Blocking vs non-blocking.
It looks like there is some misunderstanding of the blocking API. It’s not about response time but about underlining API. By default, Spring WebFlux uses Reactor Netty as underlying Http Client library which itself is a reactive implementation of Netty client that uses Event Loop instead of Thread Per Request model. Even if request takes 30-60 sec to get response, thread will not be blocked because all IO operations are async. For such API reactive applications will behave much better because for non-reactive (thread per request) you would need large number of threads and as result much more memory to handle the same workload.
To quantify efficiency we could apply Little's Law to calculate required number of threads in a ”traditional” thread per request model
workers >= throughput x latency, where workers - number of threads
For example, to handle 100 QPS with 30 sec latency we would need 100 x 30 = 3000 threads. In reactive app the same workload could be handled by several threads only and, as result, much less memory. For scalability it means that for IO bound reactive apps you would typically scale by CPU usage and for “traditional” most probably by memory.
Sometimes it's not obvious what code is blocking. One very useful tool while testing reactive code is BlockHound that you could integrate into unit tests.
3. How to refactor?
I would migrate layer by layer but block only once. Moving remote calls to WebClient could be a first step to refactor app to reactive API. I would create all request/response logic using reactive API and then block (if required) at the very top level (e.g. in controller). Do’s and Don’ts: Avoiding First-Time Reactive Programmer Mines is a great overview of the common pitfalls and possible migration strategy.
4. Flux vs Mono.
Flux will not help you to improve performance. It’s more about downstream logic. If you process record-by-record - use Flux<T> but if you process data in batches - use Mono<List<T>>.
Your current code is not really reactive and very hard to understand mixing reactive API, stream API and blocking multiple times. As a first step try to rewrite it as a single flow using reactive API and block only once.
Not really sure about your internal types but here is some skeleton that could give you an idea about the flow.
// Service method
public Flux<Task> getTasks() {
return getTasksMono()
.flatMapMany(response -> {
List<Mono<Response<Task>>> taskRequests = new ArrayList<>();
taskRequests.add(Mono.just(response));
if (response.getCount() > TASK_QUERY_LIMIT) {
retrieveAdditionalTasks(key, response.getCount(), taskRequests);
}
return Flux.mergeSequential(taskRequests);
})
.flatMapIterable(Response::getData)
.map(this::transform); // use flatMap in case transform is async
}
As I mentioned before, try to keep internal API reactive returning Mono or Flux and block only once in the upper layer.
So I have an empty map referenced like:
private var labelsForGroupId: Map<GroupId, Label> = emptyMap()
to lower the amount of calls through network api. After first call I cache the response to the map.
However, I would love to add TTL to that map, (for example, every hour it should be empty again). I am quite new to Kotlin, so wondering what would be the best approach here with some examples?
Instead of using a Map, you could use Guava Cache. It works like a Map (key-value) and have expiration policies.
Expiration by time example:
CacheBuilder.newBuilder()
.expireAfterAccess(200, TimeUnit.MILLISECONDS)
.build(loader);
If you are not interested in caches at all, then you could try to setup a Coroutine with a ScheduledExecutorService as Dispatcher. I never did this before but is a way out. Take a look at the Executors documentation - Coroutine context and dispatchers
If the given [ExecutorService] is an instance of
[ScheduledExecutorService], then all time-related * coroutine
operations such as [delay], [withTimeout] and time-based [Flow]
operators will be scheduled * on this executor using
[schedule][ScheduledExecutorService.schedule] method. If the
corresponding * coroutine is cancelled, [ScheduledFuture.cancel] will
be invoked on the corresponding future.
Our system receives messages to fetch data from remote service and then store it into the database. Currently, it opens multiple connections with the database to save the fetched data for each request. We want to convert it into a process with multiple producers(fetching data from remote service) and a single consumer to persist data in the database. Doing this it will hold only one connection at most to persist data in the database.
We are using spring-boot with a reactor. We want to have a publisher publishing all the data fetched from the remote service which we can subscribe to and push this data in a batch of say 200 records in the database.
For example, I am planning to us following code to consume messages from ActiveMQ queue:
public Publisher<Message<RestoreMessage>> restoreMessagesSource() {
return IntegrationFlows
.from(Jms.messageDrivenChannelAdapter(this.connectionFactory)
.destination(RestoreMessage.class.getSimpleName() + "Queue"))
.channel(MessageChannels.queue())
.log(LoggingHandler.Level.DEBUG)
.log()
.toReactivePublisher();
}
In this code message from the ActiveMQ qeueu are put into a ReactivePublisher. This publisher has been subsribed. This way we are conusming the messages from the queue.
In a similar fashion, we want the response of all the remote API to be pushed to a publisher which we can process in a subscriber at one place.
Sounds like you are going to have several Publisher<Message<?>> and you want to consume them all in a single subscriber. For this reason you can use:
/**
* Merge data from {#link Publisher} sequences contained in an array / vararg
* into an interleaved merged sequence. Unlike {#link #concat(Publisher) concat},
* sources are subscribed to eagerly.
* <p>
* <img class="marble" src="doc-files/marbles/mergeFixedSources.svg" alt="">
* <p>
* Note that merge is tailored to work with asynchronous sources or finite sources. When dealing with
* an infinite source that doesn't already publish on a dedicated Scheduler, you must isolate that source
* in its own Scheduler, as merge would otherwise attempt to drain it before subscribing to
* another source.
*
* #param sources the array of {#link Publisher} sources to merge
* #param <I> The source type of the data sequence
*
* #return a merged {#link Flux}
*/
#SafeVarargs
public static <I> Flux<I> merge(Publisher<? extends I>... sources) {
So, you are going to sink all your sources to one Flux and will subscribe to this one.
Pay attention to the Note. The .toReactivePublisher() indeed produces an infinite source, although, according the Jms.messageDrivenChannelAdapter() it is done in its specific thread from an executor in listener container. So, try it as is or wrap each source to the Flux with particular publishOn().
We can see SimpleMessageListenerContainer has property named concurrentConsumers, but SimpleJmsListenerContainerFactory did NOT support to config it. What's the root cause? Do not recommend to use concurrency for SimpleMessageListenerContainer ?
Set the concurrency on the #JmsListener annotation:
/**
* The concurrency limits for the listener, if any. Overrides the value defined
* by the container factory used to create the listener container.
* <p>The concurrency limits can be a "lower-upper" String — for example,
* "5-10" — or a simple upper limit String — for example, "10", in
* which case the lower limit will be 1.
* <p>Note that the underlying container may or may not support all features.
* For instance, it may not be able to scale, in which case only the upper limit
* is used.
*/
String concurrency() default "";
Looks like the Javadocs are wrong, though
Overrides the value defined
* by the container factory used to create the listener container.
Does ConsumerSeekAware interface, onPartitionsAssigned method call when rebalancing. Because I want to seek to specific offset which are in the offset when initializing and rebalancing. Could I use consumerSeekAware for both purposes or should I use ConsumerRebalanceListener for rebalancing purpose. Please give simple answers because I don't have depth knowledge about spring kafka yet. If you could please provide a sample code. Thank you
The ConsumerSeekAware has this method:
/**
* When using group management, called when partition assignments change.
* #param assignments the new assignments and their current offsets.
* #param callback the callback to perform an initial seek after assignment.
*/
void onPartitionsAssigned(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback);
It is called from the KafkaMessageListenerContainer.seekPartitions(Collection<TopicPartition> partitions, boolean idle), which, in turn, from the ConsumerRebalanceListener.onPartitionsAssigned() internal implementation. And thew last one has this JavaDocs:
* A callback method the user can implement to provide handling of customized offsets on completion of a successful
* partition re-assignment. This method will be called after an offset re-assignment completes and before the
* consumer starts fetching data.
So, yes, ConsumerSeekAware.onPartitionsAssigned() is always called during rebalancing. By the way there is no such a state for Apache Kafka as initializing. It is always rebalancing - the broker is in waiting state and starts rebalancing whenever a new consumer is joined.