If multiple request are hit to a single RestController at the same time in a application, how it is handle for different scenarios (Multiple request to a single endpoints (only GET), or Multiple requests for multiple endpoints(GET, POST, PUT...))
Is multi-threading concept utilized? If yes is it possible to handle the requests in FIFO pattern?
What is the maximum request a RestController can take ?
Does RestController scope affect handling of requests (behavior of request scope with default scope-singleton) ?
Also how it is handle by Application context (example with flow will be helpful)
Considering building Micro-services with Spring Boot 2.
From the point of view of Spring (Application Context) rest controller is a singleton if not specified otherwise.
So the code of controller must be ready to be invoked by multiple threads simultaneously.
When a new request reaches the server, in a tradition thread-per-request model the web server (like tomcat) is responsible to allocate a thread from the predefined pool of threads to the request. Then the request gets processed by controller in the context of this thread.
The actual thread pool implementation can in general vary from server to server, but in general, its something that can be configured (number of threads per loop, queue size to store requests for future processing if the pool is full, etc.)
Now regarding the Scope of RestController. If the controller is stateless (and it should be for many cases, just keep it singleton). If you need the new Instance of controller to be created per request, than change the scope. Obviously each thread will have to use the same (in case of singleton scope) instance of rest controller or spring mvc will create a new instance of controller if you specify another scope.
All the answer above applies to a "traditional" thread-per-request model.
Note that since spring 5 / spring boot 2 spring also supports "Reactive" model with a webflux. It works on top of netty and doesn't utilize a thread-per-request model. Please specify in the question if you're interested in this model rather than a tradition model that I've tried to briefly describe.
Related
I have an spring boot application that pulls message from an cloud message queue and put it back to a cloud db. I realize that my program is single thread(I am not using request mapping, just pull,process,put to db). I want Spring handle concurrency things. So can I make a dispatcher function, which calls controller in the application with #RequestMapping?
#RestController
#RequestMapping("/test")
public class GatewayController {
#RequestMapping("/service")
public void InvokeService(...) {...}
}
I need mutithread to call other service for response, which I don't want it to block others. If I recieve 10 messages, I want it to call /test/service... which have 10 threads processing them.
My question is:
Will Spring make the controller multithread?
How to call its own controller? Send request to the url? (I don't need response from controller, just let controller call a service to put response in a db on could)
RequestMapping is MVC thing - intended to issue http requests. And yes, it uses tomcat under the hood.
If you'll inject RestController into your class it won't issue any HTTP requests, you'll only call the controller as a regular bean. If you consume messages in one thread, it won't become multithreaded to answer your first question.
You can, of course, create HTTP request but frankly it's just wrong. So don't do it. This answers your second question to some extent :)
Now, there is nothing wrong conceptually if your microservice acts as a consumer and producer and deals with queues, not all microservices have to be accessible via HTTP.
In order to work in a multi threaded environment:
Check whether you can consume messages in a multi-threaded manner. Maybe the client of your "cloud message queue" offers multi-threaded configuration (thread pool or something).
If it's not possible, create a thread pool executor by yourself and upon each message submit the processing task to this thread pool. This will make the processing logic multithreaded with a parallelism level confined by the thread pool size and thread pool configurations.
A single controller in Spring usually has many Request Mappings. Many requests may hit APIs belonging to one controller(means one class) at the same time. Does n't it lead to thread safety issues?
Q1) Is Spring controller inherently thread-safe?
My answer: By default, Spring controller is a Singleton bean. Whether it is inherently thread-safe or not depends upon how Singleton pattern is implemented by Spring. It can be done thread-safe or non-threadsafe. Correct?
Q2) Does Spring provide any annotation or configuration to ensure that handling multiple requests does not run into thread safety issues?
Q3) Is ensuring thread safety the burden of the developer himself?
For your first question about how the Singleton pattern is implemented by Spring
see this link:
How does the singleton Bean serve the concurrent request?
Spring concurrency:
How does Spring bean Handle concurrency
How does Spring MVC handle multiple users
I'm using Curator service discovery with Spring MVC rest controllers where each controller registers itself with ZooKeeper in #PostConstruct and de-registers itself in #PreDestroy.
The problem I'm having is that by the time the #PreDestroy method is called, the controller is already no longer servicing requests. I need to de-register the controller before the controller stops servicing requests to avoid throwing exceptions for the small number of requests that occur between the controller stopping and de-registration.
I've tried the spring ApplicationListener interface, SmartLifecycle, and ServletContextListener and in all shutdown/close related hooks, the controller has already stopped servicing requests.
I need a shutdown hook where I can deregister before the controller stops servicing requests, and I'm not sure one is available.
I don't believe such hook exists in Spring.
However, you could have a special controller, mapped to a specific reserved url, i.e. ending in /activity/suspend-traffic. You might want to have some sort of basic security for that, maybe an application specific token, i.e. /activity/suspend-traffic/{token}.
When this special controller receives the correct token, it deregisters all application controllers from ZK.
This way you wouldn't loose any request. Then, when the application is no longer receiving traffic, you could safely shutdown the server. After sending /activity/suspend-traffic/{token}, you could wait a fixed amount of time before shuting down, so that the server finishes processing the requests it has received after the suspend command.
Or you might implement a more sophisticated mechanism, i.e. via a filter or mvc interceptor that counts how many requests are "inside" the application. You could have another mapping in that special controller, i.e. /activity/request-count that returns the actual number of requests being processed. When this request returns 0, it would be safe to shutdown the server.
If you define your service in singleton scope in your spring config, what would happen if more than one user try to access it (ie as dependency injected into your controller) at the same time? Should it cause any conflict? Or the IoC container will hold the later call until the first one finish? If so it should slow down the performance in large applications, which sounds not right to me. Could any one give me a correct answer?
BTW, as I can remember, if it is not a singleton one, IoC container will pool few instances based on the number of requests. Could some one confirm it?
what would happen if more than one user try to access it (ie as dependency injected into your controller) at the same time?
A singleton bean can be accessed many times concurrently. That's why it always has to be thread-safe
Should it cause any conflict?
Only if you fail to make it thread-safe
Or the IoC container will hold the later call until the first one finish?
No, that would be awful
BTW, as I can remember, if it is not a singleton one, IoC container will pool few instances based on the number of requests. Could some one confirm it?
Spring has the following scopes (see Bean Scopes reference):
singleton (only one instance is managed per application)
prototype (a new instance for every injection)
session (one instance per HTTP session, only in Spring MVC)
request (one instance per HTTP request, only in Spring MVC)
global session (one instance per global HTTP session, only in portlet-based Spring MVC)
Also:
As of Spring 3.0, a thread scope is available, but is not registered by default. For more information, see the documentation for SimpleThreadScope.
What you describe is an Object Pool. In Spring that would be implemented as a Prototype-scoped FactoryBean. And internally it would use a library like Apache Commons / Pool.
Singletons are just that - singletons. One instance is managed by the Spring context, and all requests go through that one instance concurrently. It's up to you to make that thread-safe.
If your bean isn't thread-safe, then consider using non-singleton-scoped beans. Spring lets you use request, session and prototype scopes.
So I'm using Spring MVC and in my controller I call several Utility classes. Do the Collections I use in those utility classes need to be synchronized? Similarly, are multiple threads spawned for each user when they access my webpage in the controller meaning I need to ensure thread-safety?
Each request will be handled by some arbitrary thread allocated by the servlet container (from a thread pool), so multiple requests will mean multiple concurrent executions of the controller. There is no direct correlation between users and threads, just requests and threads, but if you have multiple users, then you typically have concurrent requests, and so multiple threads.
Given that controllers should be thread-safe, you will then need to ensure your utility classes and collections used by the controller thread-safe, either by design (e.g. making them or the controller request or possibly session scope if you ensure the same session cannot be served concurrently) or by use of locking on shared resources.
Generally you should be good, but here is a very good article that talks about thread safety in spring web applications, Thread safe controllers in Spring MVC.