Using Spring MVC, I have a controller that makes an API request to an external service that may take a long time. I am doing this synchronously because I want to return the external request result to the user. Ideally, this doesn't block an entire thread during this time, sort of like an async/await C# style idiom, but I'm not sure how to do that easily with Spring (or something else).
You can use #Async annotation on some service method for asynchronous execution.
Spring MVC is built on top of a Servlet, this means that (by default) anything you do inside a Controller method will block the request thread provided by Tomcat (Tomcat is the default servlet container for Spring Boot these days).
If you are calling another HTTP based service, or Query a database inside your controller, you are probably making a blocking IO call. Blocking calls will always block the calling thread, the async/await in C# is just syntactical sugar, like Springs #Async it uses a hidden ThreadPool to do the actual work.
Unless you are able to call the upstream service using a NIO selector, you will need a thread to wait for the response, and in that case it makes most sense to just block on the Tomcat request thread, I believe the default number of request threads is 150, but you can set it higher if needed. If the upstream service can be called in a NIO manner, you only need one thread for all calls to that service, and you can can return a DeferedResult from your controller, and write the response to the client once the response from the upstream service has been produced.
It's simple. Do not call the long running api from your main controller method. Just return the view (JSP). Then you can fire an ajax request from your document.ready method in your view (Javascript). You can have a seperate #ResourceRequest annotated method in your controller to handle the ajax call. Return the api result (as JSON or something) to the ajax caller. Once the view receives the ajax response, update the UI as you want.
Related
I have a simple Spring Boot 2.x RestController with an endpoint performing certain remote calls as well as controller is also calling an Async method that in turn makes several remote HTTP calls.
I'm having opentracing-spring-jaeger-web-starter in classpath with tracing enabled. If i invoke my REST endpoint, It creates a span for the endpoint call as well as remote calls that the controller is making synchronously.
However the remote calls made by Async method is getting reported in its own span. Is this by design or is there a way to propagate some context information to the Async method to better group/relate the spans ?
I solved it by using this library instead https://github.com/opentracing-contrib/java-spring-cloud
It seem to have an option to enable or disable different instrumentation feature. Read about opentracing.spring.cloud.async.enabled for more info.
I have an spring boot application that pulls message from an cloud message queue and put it back to a cloud db. I realize that my program is single thread(I am not using request mapping, just pull,process,put to db). I want Spring handle concurrency things. So can I make a dispatcher function, which calls controller in the application with #RequestMapping?
#RestController
#RequestMapping("/test")
public class GatewayController {
#RequestMapping("/service")
public void InvokeService(...) {...}
}
I need mutithread to call other service for response, which I don't want it to block others. If I recieve 10 messages, I want it to call /test/service... which have 10 threads processing them.
My question is:
Will Spring make the controller multithread?
How to call its own controller? Send request to the url? (I don't need response from controller, just let controller call a service to put response in a db on could)
RequestMapping is MVC thing - intended to issue http requests. And yes, it uses tomcat under the hood.
If you'll inject RestController into your class it won't issue any HTTP requests, you'll only call the controller as a regular bean. If you consume messages in one thread, it won't become multithreaded to answer your first question.
You can, of course, create HTTP request but frankly it's just wrong. So don't do it. This answers your second question to some extent :)
Now, there is nothing wrong conceptually if your microservice acts as a consumer and producer and deals with queues, not all microservices have to be accessible via HTTP.
In order to work in a multi threaded environment:
Check whether you can consume messages in a multi-threaded manner. Maybe the client of your "cloud message queue" offers multi-threaded configuration (thread pool or something).
If it's not possible, create a thread pool executor by yourself and upon each message submit the processing task to this thread pool. This will make the processing logic multithreaded with a parallelism level confined by the thread pool size and thread pool configurations.
If multiple request are hit to a single RestController at the same time in a application, how it is handle for different scenarios (Multiple request to a single endpoints (only GET), or Multiple requests for multiple endpoints(GET, POST, PUT...))
Is multi-threading concept utilized? If yes is it possible to handle the requests in FIFO pattern?
What is the maximum request a RestController can take ?
Does RestController scope affect handling of requests (behavior of request scope with default scope-singleton) ?
Also how it is handle by Application context (example with flow will be helpful)
Considering building Micro-services with Spring Boot 2.
From the point of view of Spring (Application Context) rest controller is a singleton if not specified otherwise.
So the code of controller must be ready to be invoked by multiple threads simultaneously.
When a new request reaches the server, in a tradition thread-per-request model the web server (like tomcat) is responsible to allocate a thread from the predefined pool of threads to the request. Then the request gets processed by controller in the context of this thread.
The actual thread pool implementation can in general vary from server to server, but in general, its something that can be configured (number of threads per loop, queue size to store requests for future processing if the pool is full, etc.)
Now regarding the Scope of RestController. If the controller is stateless (and it should be for many cases, just keep it singleton). If you need the new Instance of controller to be created per request, than change the scope. Obviously each thread will have to use the same (in case of singleton scope) instance of rest controller or spring mvc will create a new instance of controller if you specify another scope.
All the answer above applies to a "traditional" thread-per-request model.
Note that since spring 5 / spring boot 2 spring also supports "Reactive" model with a webflux. It works on top of netty and doesn't utilize a thread-per-request model. Please specify in the question if you're interested in this model rather than a tradition model that I've tried to briefly describe.
As of spring 5 seems like Flux/Mono can be also be a returned value of spring rest controller method.
I looked at spring reference guide 4.3 and read a pretty nice description of how Callable and DeferredResult are handled (https://docs.spring.io/spring/docs/4.3.x/spring-framework-reference/htmlsingle/#mvc-ann-async)
I also read on some blog posts that CompletableFuture can also be returned from controller methods. If it is true than can anybody shed some light how
CompletableFuture and Flux/Mono are handled in Spring MVC ?
Are they handled more less the same way as DeferredResult/Callable, i.e. "processing" done in an external thread and then Spring MVC dispatches the request back to DispatcherServlet (so the HTTP Response is written to and flushed on servlet container thread)
Indeed, as of Spring Framework 5.0 Spring MVC can handle Publisher-like return types in controller handlers. Depending on the cardinality of the returned type (one or multiple possible values?), Spring MVC will handle it with:
a DeferredResult if returning a single value
a DeferredResult<List<?>> if it's returning an asynchronous collection, collecting all elements in a list
a ResponseBodyEmitter or SseEmitter if you're streaming data to the client
Now the underlying processing model is using Servlet 3.0 async support, moving the blocking I/O operations to a non-container thread, dispatching back to the container when it's done.
On the other hand, Spring WebFlux is fully asynchronous and non-blocking (using Servlet 3.1 support for Servlet containers); in that case, fewer threads will be used.
More on that in the reference documentation.
This video nicely explains what's going on in spring mvc https://www.infoq.com/presentations/servlet-reactive-stack. Simply startAsync is used to resume(dispatch) to servlet thread to write (in a blocking way) to servlet response.
I'm using Curator service discovery with Spring MVC rest controllers where each controller registers itself with ZooKeeper in #PostConstruct and de-registers itself in #PreDestroy.
The problem I'm having is that by the time the #PreDestroy method is called, the controller is already no longer servicing requests. I need to de-register the controller before the controller stops servicing requests to avoid throwing exceptions for the small number of requests that occur between the controller stopping and de-registration.
I've tried the spring ApplicationListener interface, SmartLifecycle, and ServletContextListener and in all shutdown/close related hooks, the controller has already stopped servicing requests.
I need a shutdown hook where I can deregister before the controller stops servicing requests, and I'm not sure one is available.
I don't believe such hook exists in Spring.
However, you could have a special controller, mapped to a specific reserved url, i.e. ending in /activity/suspend-traffic. You might want to have some sort of basic security for that, maybe an application specific token, i.e. /activity/suspend-traffic/{token}.
When this special controller receives the correct token, it deregisters all application controllers from ZK.
This way you wouldn't loose any request. Then, when the application is no longer receiving traffic, you could safely shutdown the server. After sending /activity/suspend-traffic/{token}, you could wait a fixed amount of time before shuting down, so that the server finishes processing the requests it has received after the suspend command.
Or you might implement a more sophisticated mechanism, i.e. via a filter or mvc interceptor that counts how many requests are "inside" the application. You could have another mapping in that special controller, i.e. /activity/request-count that returns the actual number of requests being processed. When this request returns 0, it would be safe to shutdown the server.