So I'm using Spring MVC and in my controller I call several Utility classes. Do the Collections I use in those utility classes need to be synchronized? Similarly, are multiple threads spawned for each user when they access my webpage in the controller meaning I need to ensure thread-safety?
Each request will be handled by some arbitrary thread allocated by the servlet container (from a thread pool), so multiple requests will mean multiple concurrent executions of the controller. There is no direct correlation between users and threads, just requests and threads, but if you have multiple users, then you typically have concurrent requests, and so multiple threads.
Given that controllers should be thread-safe, you will then need to ensure your utility classes and collections used by the controller thread-safe, either by design (e.g. making them or the controller request or possibly session scope if you ensure the same session cannot be served concurrently) or by use of locking on shared resources.
Generally you should be good, but here is a very good article that talks about thread safety in spring web applications, Thread safe controllers in Spring MVC.
Related
I am working on the micro-service developed using Spring Boot . I have implemented following layers:
Controller layer: Invoked when user sends API request
Service layer: Processes the request. Either sends request to third-part service or sends request to database
Repository layer: Used to interact with the
database
.
Methods in all of above layers returns the CompletableFuture. I have following questions related to this setup:
Is it good practice to return Completable future from all methods across all layers?
Is it always recommended to use #Async annotation when using CompletableFuture? what happens when I use default fork-join pool to process the requests?
How can I configure the threads for above methods? Will it be a good idea to configure the thread pool per layer? what are other configurations I can consider here?
Which metrics I should focus while optimizing performance for this micro-service?
If the work your application is doing can be done on the request thread without too much latency, I would recommend it. You can always move to an async model if you find that your web server is running out of worker threads.
The #Async annotation is basically helping with scheduling. If you can, use it - it can keep the code free of the references to the thread pool on which the work will be scheduled. As for what thread actually does your async work, that's really up to you. If you can, use your own pool. That will make sure you can add instrumentation and expose configuration options that you may need once your service is running.
Technically you will have two pools in play. One that Spring will use to consume the result of your future, and another that you will use to do the async work. If I recall correctly, Spring Boot will configure its pool if you don't already have one, and will log a warning if you didn't explicitly configure one. As for your worker threads, start simple. Consider using Spring's ThreadPoolTaskExecutor.
Regarding which metrics to monitor, start first by choosing how you will monitor. Using something like Spring Sleuth coupled with Spring Actuator will give you a lot of information out of the box. There are a lot of services that can collect all the metrics actuator generates into time-based databases that you can then use to analyze performance and get some ideas on what to tweak.
One final recommendation is that Spring's Web Flux is designed from the start to be async. It has a learning curve for sure since reactive code is very different from the usual MVC stuff. However, that framework is also thinking about all the questions you are asking so it might be better suited for your application, specially if you want to make everything async by default.
If multiple request are hit to a single RestController at the same time in a application, how it is handle for different scenarios (Multiple request to a single endpoints (only GET), or Multiple requests for multiple endpoints(GET, POST, PUT...))
Is multi-threading concept utilized? If yes is it possible to handle the requests in FIFO pattern?
What is the maximum request a RestController can take ?
Does RestController scope affect handling of requests (behavior of request scope with default scope-singleton) ?
Also how it is handle by Application context (example with flow will be helpful)
Considering building Micro-services with Spring Boot 2.
From the point of view of Spring (Application Context) rest controller is a singleton if not specified otherwise.
So the code of controller must be ready to be invoked by multiple threads simultaneously.
When a new request reaches the server, in a tradition thread-per-request model the web server (like tomcat) is responsible to allocate a thread from the predefined pool of threads to the request. Then the request gets processed by controller in the context of this thread.
The actual thread pool implementation can in general vary from server to server, but in general, its something that can be configured (number of threads per loop, queue size to store requests for future processing if the pool is full, etc.)
Now regarding the Scope of RestController. If the controller is stateless (and it should be for many cases, just keep it singleton). If you need the new Instance of controller to be created per request, than change the scope. Obviously each thread will have to use the same (in case of singleton scope) instance of rest controller or spring mvc will create a new instance of controller if you specify another scope.
All the answer above applies to a "traditional" thread-per-request model.
Note that since spring 5 / spring boot 2 spring also supports "Reactive" model with a webflux. It works on top of netty and doesn't utilize a thread-per-request model. Please specify in the question if you're interested in this model rather than a tradition model that I've tried to briefly describe.
A single controller in Spring usually has many Request Mappings. Many requests may hit APIs belonging to one controller(means one class) at the same time. Does n't it lead to thread safety issues?
Q1) Is Spring controller inherently thread-safe?
My answer: By default, Spring controller is a Singleton bean. Whether it is inherently thread-safe or not depends upon how Singleton pattern is implemented by Spring. It can be done thread-safe or non-threadsafe. Correct?
Q2) Does Spring provide any annotation or configuration to ensure that handling multiple requests does not run into thread safety issues?
Q3) Is ensuring thread safety the burden of the developer himself?
For your first question about how the Singleton pattern is implemented by Spring
see this link:
How does the singleton Bean serve the concurrent request?
Spring concurrency:
How does Spring bean Handle concurrency
How does Spring MVC handle multiple users
I have gone through various books on stateful and stateless session bean and how they work. I want to know the real usage of these ejbs and advantages over using plain java classes. Basically when do you go for stateless ejbs and when do you go for stateful ejbs. I want a real time application.
the usage of these type of ejbs are usually in service layer as service classes.
EJB3 stateless and stateful bean are actually POJO (with some annotations) and they don't have any big difference with normal classes.
but in term of usage, they have some abilities that you can't find in normal classes like:
they can be called remotely (e.g. RMI protocol).
they can use application server context resources like DB Connection and Transactions.
stateless or stateful:
- if a task or process can be done in a single step (by a single method call) stateless is the right option
like a authentication process
- if a task needs a series of method calls (more than one) and you need to keep previous results to use them in next call, then go for stateful.
like a shipping process (select items, add/remove and then do the transaction)
http session or stateful?
ejbs can be served in application server and they may have different type of clients like a normal swing application or ..., so you can't relay on http session in these cases.
if your appserver and webserver are different (distributed) its not good idea keep data in http session and pass/getback it to/from app server (network overhead).
Stateless session bean are lightweight: they do not store information about a specific user. They are usually used in a static way. For example a client ask for a product information will communicate with a stateless session bean. ("You want the price of product 'YXZ', here you go!")
Stateful session bean however remember's the client information. They contains data about the user actions. For example, let's say a user go through a shopping cart. The steps will be stored in a stateful session bean (for example, user it at the payment step).
You really need both type of session bean in any website. Unless you web site is so basic that anything can be done with stateless session bean (a read-only web site really).
Any web site that track a user through cookies, will need a stateful session bean. Be aware however that you can decide to put very little session information in a session bean and store that information in a database. But you still need some session management.
Developers prefer to maintain state in web layer in modern enterprise applications. I have never seen a real world web application using Stateful Session Bean. It is a scalability issue also.
An example is a shopping cart stateful session bean that tracks a client's product choices and can execute a sale when requested.
How is that instance pooling with EJBs can improve performance? Wouldn't you be able to accomplish the same performance just with threads like a java servlet?
Or perhaps instance pooling with EJBs happens for another reason?
Instance pooling not only helps because you can re-use objects (and avoid costly object creation), but also allows the app. server to manage the load correctly. It's app. server specific, but you can normally specify max-pool-size, min-pool-size, pool-resize and the timeout.
When the pool has reached its max-pool-size capacity, requests are served using the existing instances, and will time out if no instance is available within the expected time frame. That may degrade the quality of service of the application, but at least it doesn't blow the app. server itself. That's the same as with a web server.
A few notes about thread-safety:
Sect. 4.3.13 "Serializing Session Bean Methods"
The container serializes calls to each session bean instance. Most containers will support many
instances of a session bean executing concurrently; however, each instance sees only a serialized
sequence of method calls. Therefore, a session bean does not have to be coded as reentrant.
As per the EJB spec, all requests to a specific instance of a bean are synchronized by the app. server. This is, for instance, to allow a stateless session bean (SLSB) to store a database connection in one of its fields. The fields of an SLSB should be transient, though. (The bean instance can be destroyed/re-created any time.) With the synchronization, the app. server ensures the SLSB is thread-safe. Without the synchronization by the app. server, the developer should ensure that the SLSB is thread-safe, that is, it should have no fields.
Note: It's rare to have fields in an SLSB. Most SLSB are thread-safe by nature. I would not recommend storing the connection in field for instance. Better obtain one in the method, and release it in the method asap.
I think instance pooling is used when the beans are expensive to construct. By letting the next request reuse the same bean you don't have to construct another instance.
If the bean itself is cheap to construct and the cost is in the work that it's doing, then instance pooling isn't worth the hassle.
I think the advantages would be similar to those of connection pooling. Having instances ready in a pool avoids the overhead of creating a new instance every time the EJB is requested.
Another advantage, depending on how you look at it, is that by using a maximum pool size you can limit the damage a runaway application can do by forcing it to wait for an instance to become available. This can help prevent poorly written apps from monopolizing server resources.
AFAIK the fundamental reason is the different threading model compared to the servlet's one. In case of servlet, there is only one instance and many threads can operate on this intstance at the same time. It is the developers responsibility to ensure proper synchronisation.
In contrast to this, ejb container allows only one thread at the same time to operate on the bean instance (including necessary synchronisation at the background). Thanks to this, developer does not have to care about synchronisation, and what's more using synchronisation in the bean's code is forbidden by the spec (in fact you can do it, but you have to consider the performance consequences).
So to enable concurrent processing, you need to pool multiple bean instances so that multiple threads can access them simultaneously. The size of the pool can be tuned depending on what kind of work is done inside the bean. The basic rule is:if the task is I/O bound, you need big pool so that the cpu time is not wasted when waiting for i/o response. If the task is cpu bound, the pool should be as big as much processors/cores the machine does have. But the tuning should be based on measuring.
One more note on the servlets: you can enforce the same behaviour as ejb when using SingleThreadModel interface. But in fact this is not much used I guess.