I want to use Spring WebClient in a project to consume some external web service.
Can WebClient object be singleton or shared among all threads (requests)?
If my application is going to get millions of requests per second, then do I need to pool WebClient Objects? If yes, I am unable to find any documentation or examples.
Does mono.block() internally work similar to future.get() or latch.await()?
The WebClient is a non-blocking implementation of a REST client built on the Reactive Stack, so I guess the only issue you should focus on is to complete a non-blocking call.
Can WebClient object be a singleton or shared among all threads (requests)?
A standard way I have seen everywhere is to inject WebClient as a bean. I find no reason to do any different.
#Autowired
WebClient webClient;
If my application is going to get millions of requests per second, then do I need to pool WebClient Objects?
That's a lot! This should definitely need to be solved with service replication, load-balancers, bulkhead, etc. In terms of the client itself, see the following performance of reactive client using newer versions of Spring: WebFlux Reactive Programming Performance Test. Moreover, that is the expected maximal throughput?
Does mono.block() internally work similar to future.get() or latch.await()?
Yes, it does.
Related
I am working on the micro-service developed using Spring Boot . I have implemented following layers:
Controller layer: Invoked when user sends API request
Service layer: Processes the request. Either sends request to third-part service or sends request to database
Repository layer: Used to interact with the
database
.
Methods in all of above layers returns the CompletableFuture. I have following questions related to this setup:
Is it good practice to return Completable future from all methods across all layers?
Is it always recommended to use #Async annotation when using CompletableFuture? what happens when I use default fork-join pool to process the requests?
How can I configure the threads for above methods? Will it be a good idea to configure the thread pool per layer? what are other configurations I can consider here?
Which metrics I should focus while optimizing performance for this micro-service?
If the work your application is doing can be done on the request thread without too much latency, I would recommend it. You can always move to an async model if you find that your web server is running out of worker threads.
The #Async annotation is basically helping with scheduling. If you can, use it - it can keep the code free of the references to the thread pool on which the work will be scheduled. As for what thread actually does your async work, that's really up to you. If you can, use your own pool. That will make sure you can add instrumentation and expose configuration options that you may need once your service is running.
Technically you will have two pools in play. One that Spring will use to consume the result of your future, and another that you will use to do the async work. If I recall correctly, Spring Boot will configure its pool if you don't already have one, and will log a warning if you didn't explicitly configure one. As for your worker threads, start simple. Consider using Spring's ThreadPoolTaskExecutor.
Regarding which metrics to monitor, start first by choosing how you will monitor. Using something like Spring Sleuth coupled with Spring Actuator will give you a lot of information out of the box. There are a lot of services that can collect all the metrics actuator generates into time-based databases that you can then use to analyze performance and get some ideas on what to tweak.
One final recommendation is that Spring's Web Flux is designed from the start to be async. It has a learning curve for sure since reactive code is very different from the usual MVC stuff. However, that framework is also thinking about all the questions you are asking so it might be better suited for your application, specially if you want to make everything async by default.
Background : I am using spring boot with embedded jetty. My app calls bunch of rest apis. For calling these rest apis I use spring rest template.
Question: Is Spring rest template any good at high concurrency? Searching on the web search suggests moving to reactive but still there are apps which are written in blocking way and need to continue that way. Question is what alternate is there or what can be done to make rest template more responsive under heavy load. PoolingHttpClientConnectionManager improves things a bit but essentially still not at par with is required.
There are suggestions to move to rest easy and other http clients but no slid reasoning behind it. End of the day, they all make pool of connections and essentially works the same. Please note, reactive is not an option yet. This question is very specific to traditional blocking rest calls. Any suggestions in optimizing connection pooling or using rest template right will be of great help.
RestTemplate does not do an actual rest call by itself, its just a "wrapper" - a convenient API.
Now when it comes to connection pooling, by default it doesn't use any kind of pooling and just opens URL connections available in Java anyway. No third-parties are required, but performance is not so good.
You can configure rest template to use, say, OkHttp Client under the hood. See here for different ways to work with different clients. The interesting part is that its possible to configure connection pools there and achieve a better performance.
So you should really check what exactly the expected performance is and configure the connection pool accordingly.
Now one more thing about Reactive stuff - it won't give you a performance gain, however it will allow to serve better multiple concurrent requests by reusing resources more efficiently. However if you'll measure how long it takes to perform one single request - its not expected to be performed faster.
In other words you should consider the transition to reactive stack if the application has too many concurrent requests that it can't serve, but not if you want to process every single request faster.
Spring RestTemplate is used to write application level code. It obtains the HTTP connection from ClientHttpRequestFactory implementation which is what glues low-level HTTP client library to Spring e.g. HttpComponentsClientHttpRequestFactory for Apache HTTP Client.
Bottom line, in most cases you have to tune the underlying low-level HTTP client library and not RestTemplate when you are tuning outgoing requests to external APIs.
You are confusing a lot of concepts in your question. Try understanding what is Reactive programming, HTTP, HTTP pipelining, and TCP/IP before you start tuning anything. Otherwise you won't find where is your code's bottleneck and you will end up tuning wrong part of the software stack.
I have some microservices, which should work on top of WebFlux framework. Each server has own API with Mono or Flux. We are using MongoDB, which is supported by Spring (Spring Data MongoDb Reactive).
The problem is external blocking API, which I have to use in my system.
I have one solution. I can just wrap blocking API calls in dedicated thread pool and use it with CompletableFuture.
Is there anything else to solve my problem? I think, that brand new Rsocket cannot solve my problem.
1.If possible, you can change your blocking API call to the reactive way using the WebClient class.
References:
Reference guide
WebClient API
A simple, complete sample
2.If the blocking API can't be changed to reactive ones, we should have a dedicated, well-tuned thread pool and isolate the blocking code there.
There is also an example here.
I don't see why you cannot wrap a blocking API call in a Flux or a Mono. You can also integrate Akka with Spring if the actor model seems easier to you.
RSocket should be a perfect fit, good tutorials to get you started
https://www.baeldung.com/spring-boot-rsocket
https://spring.io/blog/2020/04/06/getting-started-with-rsocket-spring-boot-channels
As per spring 5:
WebClient is an interface representing the main entry point for performing web requests.
It has been created as a part of the Spring Web Reactive module and will be replacing the classic RestTemplate in these scenarios. The new client is a reactive, non-blocking solution that works over the HTTP/1.1 protocol
Does that mean, we need to recode for the old applications using RestTemplate if we want to upgrade to Spring 5?
Or there is some workaround to work with RestTemplate in Spring 5?
No, RestTemplate will continue to exist (at least for now). You don't have to replace it with WebClient.
One of the main differences is RestTemplate is synchronous and blocking i.e. when you do a rest call you need to wait till the response comes back to proceed further.
But WebClient is complete opposite of this. The caller need not wait till response comes back. Instead he will be notified when there is a response.
If you need such a functionality, then yes you need to replace your Resttemplate with WebClient.
You can in fact achieve Rest template like synchronous processing in webclient using .block(). But the other way is not possible.
EDIT:
RestTemplate will be deprecated in a future version(> 5.0) and will not have major new features added going forward
According to the Java Doc the RestTemplate will be in maintenance mode. Spring team advise to use the WebClient if possible:
NOTE: As of 5.0, the non-blocking, reactive
org.springframework.web.reactive.client.WebClient offers a modern
alternative to the RestTemplate with efficient support for both sync
and async, as well as streaming scenarios. The RestTemplate will be
deprecated in a future version and will not have major new features
added going forward.
RestTemplate is not really deprecated. But it will not be evolved in the future. So sticking to RestTemplate is perfectly valid if it does what you need.
Another way to put that is that if you need specific usage patterns like streaming, scatter/gatter, or custom timeouts, this won't be covered by RestTemplate and you need to use WebClient instead.
Now using WebClient in a blocking application is fine too. Using block() shouldn't hurt there and Spring MVC controller does partially support reactive return types.
WebClient is Non-Blocking Client, RestTemplate is Blocking Client.
For a long time, spring serves as a web customer. Under the hood, RestTemplate uses the Java API API, which is based on the subject model.This means that the matter will be blocked until the client receives a response. The problem with the blockage code is due to the existence of any string of memory and cpu cycles. Let us consider a lot of applications that are waiting for low services that are needed to produce the result.Sooner or later, requests for the results are collected. As a result, the program creates many issues, which result in depletion of a pool of thread or occupying all of the available memory. We can also experience performance performance due to the cpu switching.
Spring WebClient vs. RestTemplate
WebClient supports asynchronous as well as synchronous calls.
RestTemplate supports only synchronous calls.
No changes are required in old code even if the RestTemplate is depracated(as long as you don't need asynchronous behaviour)
I was experimenting with reactive web framework.I have certain question regarding how it will work.
In typical application,we have datastore(Relational or No SQL).
Application Layer(Controllers) to connect to store and get the Data.
Client Layer(Calls your API End Points) and get the data.
To best of my knowledge,there are no async or reactive drivers published by Vendors.Only Mongo and may be Cassandra has reactive drivers).
Controller layer will beam back the data using Mono or Flux or Single.
Client layer will be consuming this data.
Since HTTP is synchronous in nature,how will client layer or application benefit from reactive support in Spring.
Question:Let us says I have 10 records in JSON coming from my Flux response.Does it mean,my client will get data in stream or entire data set will be fetched first at client side and then process of consuming it will be reactive in nature.Currently ,we have InputStream as a response of service call,which is blocking in nature,due to design of HTTP protocol.
Question:Does it then make sense to have reactive architecture for typical web application,when very medium on which we are going to get response is Blocking in Nature.
Spring Web Reactive makes use of Servlet 3.1 non-blocking I/O and runs on Servlet 3.1 containers. It also runs on non-Servlet runtimes such as Netty and Undertow. Each runtime is adapted to a set of shared, reactive ServerHttpRequest and ServerHttpResponse abstractions that expose the request and response body as Flux with full backpressure support on the read and the write side.
Source:
https://docs.spring.io/spring-framework/docs/5.0.0.M1/spring-framework-reference/html/web-reactive.html
Datastore vendors and OSS communities are working on that. There's already support for Cassandra, Couchbase, MongoDB and Redis in Spring Data Kay.
I think you're conflating the HTTP protocol itself and blocking Java APIs. You're not getting the full HTTP request or response in one big block, so the HTTP procotol is not synchronous. The underlying networking library you choose also drives the choice between blocking or non-blocking I/O.
Now about your HTTP client question: if you're using WebClient, the returned Flux will emit elements as soon as they're available. The underlying libraries are reading and decoding messages as soon as possible, while still respecting backpressure.
I'm not sure I get your last question - but if you're wondering when and why you should use a reactive approach: this approach has benefits if you're already running into scalability/efficiency issues, or if your application is communicating with many external services and is then sensitive to latency. See more about that in the Spring Framework 5.0 FAQ.