A single controller in Spring usually has many Request Mappings. Many requests may hit APIs belonging to one controller(means one class) at the same time. Does n't it lead to thread safety issues?
Q1) Is Spring controller inherently thread-safe?
My answer: By default, Spring controller is a Singleton bean. Whether it is inherently thread-safe or not depends upon how Singleton pattern is implemented by Spring. It can be done thread-safe or non-threadsafe. Correct?
Q2) Does Spring provide any annotation or configuration to ensure that handling multiple requests does not run into thread safety issues?
Q3) Is ensuring thread safety the burden of the developer himself?
For your first question about how the Singleton pattern is implemented by Spring
see this link:
How does the singleton Bean serve the concurrent request?
Spring concurrency:
How does Spring bean Handle concurrency
How does Spring MVC handle multiple users
Related
I am working on the micro-service developed using Spring Boot . I have implemented following layers:
Controller layer: Invoked when user sends API request
Service layer: Processes the request. Either sends request to third-part service or sends request to database
Repository layer: Used to interact with the
database
.
Methods in all of above layers returns the CompletableFuture. I have following questions related to this setup:
Is it good practice to return Completable future from all methods across all layers?
Is it always recommended to use #Async annotation when using CompletableFuture? what happens when I use default fork-join pool to process the requests?
How can I configure the threads for above methods? Will it be a good idea to configure the thread pool per layer? what are other configurations I can consider here?
Which metrics I should focus while optimizing performance for this micro-service?
If the work your application is doing can be done on the request thread without too much latency, I would recommend it. You can always move to an async model if you find that your web server is running out of worker threads.
The #Async annotation is basically helping with scheduling. If you can, use it - it can keep the code free of the references to the thread pool on which the work will be scheduled. As for what thread actually does your async work, that's really up to you. If you can, use your own pool. That will make sure you can add instrumentation and expose configuration options that you may need once your service is running.
Technically you will have two pools in play. One that Spring will use to consume the result of your future, and another that you will use to do the async work. If I recall correctly, Spring Boot will configure its pool if you don't already have one, and will log a warning if you didn't explicitly configure one. As for your worker threads, start simple. Consider using Spring's ThreadPoolTaskExecutor.
Regarding which metrics to monitor, start first by choosing how you will monitor. Using something like Spring Sleuth coupled with Spring Actuator will give you a lot of information out of the box. There are a lot of services that can collect all the metrics actuator generates into time-based databases that you can then use to analyze performance and get some ideas on what to tweak.
One final recommendation is that Spring's Web Flux is designed from the start to be async. It has a learning curve for sure since reactive code is very different from the usual MVC stuff. However, that framework is also thinking about all the questions you are asking so it might be better suited for your application, specially if you want to make everything async by default.
If multiple request are hit to a single RestController at the same time in a application, how it is handle for different scenarios (Multiple request to a single endpoints (only GET), or Multiple requests for multiple endpoints(GET, POST, PUT...))
Is multi-threading concept utilized? If yes is it possible to handle the requests in FIFO pattern?
What is the maximum request a RestController can take ?
Does RestController scope affect handling of requests (behavior of request scope with default scope-singleton) ?
Also how it is handle by Application context (example with flow will be helpful)
Considering building Micro-services with Spring Boot 2.
From the point of view of Spring (Application Context) rest controller is a singleton if not specified otherwise.
So the code of controller must be ready to be invoked by multiple threads simultaneously.
When a new request reaches the server, in a tradition thread-per-request model the web server (like tomcat) is responsible to allocate a thread from the predefined pool of threads to the request. Then the request gets processed by controller in the context of this thread.
The actual thread pool implementation can in general vary from server to server, but in general, its something that can be configured (number of threads per loop, queue size to store requests for future processing if the pool is full, etc.)
Now regarding the Scope of RestController. If the controller is stateless (and it should be for many cases, just keep it singleton). If you need the new Instance of controller to be created per request, than change the scope. Obviously each thread will have to use the same (in case of singleton scope) instance of rest controller or spring mvc will create a new instance of controller if you specify another scope.
All the answer above applies to a "traditional" thread-per-request model.
Note that since spring 5 / spring boot 2 spring also supports "Reactive" model with a webflux. It works on top of netty and doesn't utilize a thread-per-request model. Please specify in the question if you're interested in this model rather than a tradition model that I've tried to briefly describe.
I wanted to understand better how and when it makes sense to use the prototype scope in Spring. Seems that it is similar how the stateless session beans have been handled in the EJB world (although, the EJB container would release an instance from a pool of stateless session beans that would be created rather than creating a new instance on demand).
I have a few basic questions about that:
1) From an architecture standpoint, when does it make sense to use prototype-scoped beans in a typical J2EE web application (Spring MVC/Hibernate or JDBC template DB access)?
2) Is there a concept of creating a pool of such prototype instances similar to the pooling that the EJB server would do with the stateless session beans?
3) Should the service facade (that is similar to the Session Facade in EJB) be created with a prototype scope and would that help in dealing with the concurrent requests coming from a web tier? And how do I control the number of instances that are created (limit to certain manageable number and, preferably, pool them)?
you would want singletons for services, assuming your services are stateless. That way you only have one instance of each service, and since they are stateless they are threadsafe.
you would want prototypes for things like request actions (e.g. in struts), so a new object gets created to handle each request. Those prototypes can be wired up to singleton services.
from the documentation:
The non-singleton, prototype scope of bean deployment results in the
creation of a new bean instance every time a request for that specific
bean is made. That is, the bean is injected into another bean or you
request it through a getBean() method call on the container. As a
rule, use the prototype scope for all stateful beans and the singleton
scope for stateless beans.
If you define your service in singleton scope in your spring config, what would happen if more than one user try to access it (ie as dependency injected into your controller) at the same time? Should it cause any conflict? Or the IoC container will hold the later call until the first one finish? If so it should slow down the performance in large applications, which sounds not right to me. Could any one give me a correct answer?
BTW, as I can remember, if it is not a singleton one, IoC container will pool few instances based on the number of requests. Could some one confirm it?
what would happen if more than one user try to access it (ie as dependency injected into your controller) at the same time?
A singleton bean can be accessed many times concurrently. That's why it always has to be thread-safe
Should it cause any conflict?
Only if you fail to make it thread-safe
Or the IoC container will hold the later call until the first one finish?
No, that would be awful
BTW, as I can remember, if it is not a singleton one, IoC container will pool few instances based on the number of requests. Could some one confirm it?
Spring has the following scopes (see Bean Scopes reference):
singleton (only one instance is managed per application)
prototype (a new instance for every injection)
session (one instance per HTTP session, only in Spring MVC)
request (one instance per HTTP request, only in Spring MVC)
global session (one instance per global HTTP session, only in portlet-based Spring MVC)
Also:
As of Spring 3.0, a thread scope is available, but is not registered by default. For more information, see the documentation for SimpleThreadScope.
What you describe is an Object Pool. In Spring that would be implemented as a Prototype-scoped FactoryBean. And internally it would use a library like Apache Commons / Pool.
Singletons are just that - singletons. One instance is managed by the Spring context, and all requests go through that one instance concurrently. It's up to you to make that thread-safe.
If your bean isn't thread-safe, then consider using non-singleton-scoped beans. Spring lets you use request, session and prototype scopes.
So I'm using Spring MVC and in my controller I call several Utility classes. Do the Collections I use in those utility classes need to be synchronized? Similarly, are multiple threads spawned for each user when they access my webpage in the controller meaning I need to ensure thread-safety?
Each request will be handled by some arbitrary thread allocated by the servlet container (from a thread pool), so multiple requests will mean multiple concurrent executions of the controller. There is no direct correlation between users and threads, just requests and threads, but if you have multiple users, then you typically have concurrent requests, and so multiple threads.
Given that controllers should be thread-safe, you will then need to ensure your utility classes and collections used by the controller thread-safe, either by design (e.g. making them or the controller request or possibly session scope if you ensure the same session cannot be served concurrently) or by use of locking on shared resources.
Generally you should be good, but here is a very good article that talks about thread safety in spring web applications, Thread safe controllers in Spring MVC.