I am using a proprietary IoC mechanism in my asp.net mvc3 application (on IIS7) that saves state in [ThreadStatic] fields and therefore relies on an assumption that HttpApplication.BeginRequest, HttpApplication.EndRequest and the whole synchronous execution of the (single) request they relate to are executed on the same thread.
Is that assumption correct?
Is that assumption correct?
No, this assumption is not correct and there's evidence for it. The only reliable per request storage mechanism in ASP.NET is HttpContext.Items.
Never use [ThreadStatic] fields to store per-request values in an ASP.NET application. For example if you have an asynchronous controller you could very well have the engine draw one request from the thread pool to begin serving the request, then initiate an asynchronous operation relying on an IOCP (I/O Completion Port) and finally draw another thread from the pool to finish the request. So you could have 2 different threads serving the same HTTP request.
Absolutely never rely on the fact that the HTTP request will be served by the same thread.
This could be true in some cases for synchronous requests but remember that this is just an implementation detail. This could change without any notice from one version of .NET to another. You should never rely on it and never use [ThreadStatic] in ASP.NET. This could bite you very badly.
Related
Given that beans in Spring, including Controllers, have a scope of singleton by default, will it not be a performance hit or delay in servicing requests if multiple requests are made by multiple users?
In a web application with tens of 1000s of active users, who may access the same or different end-points in the same controller, how will their request be handled if there is only one instance of this controller or even service?
Will it not require that each active request is serviced before the controller could handle the next one? Will the users not see too much delay if there are multiple active users making requests?
I know that the controllers and service beans must be stateless (and let's say they completely are stateless) but my question is more to do with how this single instance handles multiple requests without showing any type of delay.
Because the controller has no state, no synchronization is needed. And hence requests can be processed concurrently on the same controller; so no delays.
At a CPU level, the cache line with the controllerd data will be in the SHARED state (MESI), so no cache coherence traffic is needed because every CPU could have that cache line in that state.
Reading Spring in action 5th edition chapter 11, last paragraph in section 11.1.2
By accepting a Mono as input, the method is invoked immediately
without waiting for the Taco to be resolved from the request body. And
because the repository is also reactive, it’ll accept a Mono and
immediately return a Flux, from which you call next() and return
the resulting Mono … all before the request is even processed!
How the service will immediately return before the request is even processed? Isn't that counter-intuitive?
I mean should the request be processed first before returning a response?
The book has everything you need. It is a well-written book, just make sure to read carefully while actually (make sure to download the source code from Manning) running the code. It will help you understand better.
From the book (https://livebook.manning.com/book/spring-in-action-fifth-edition/chapter-11/v-7/6):
11.1 Working with Spring WebFlux
Typical Servlet-based web frameworks, such as Spring MVC, are blocking
and multithreaded in nature, using a single thread per connection. As
requests are handled, a worker thread is pulled from a thread pool to
process the request. Meanwhile, the request thread is blocked until it
is notified by the worker thread that it is finished.
Consequently, blocking web frameworks do not scale effectively under
heavy request volume. Latency in slow worker threads makes things even
worse, because it will take longer for the worker thread to be
returned to the pool to be ready to handle another request.
In some use cases, this arrangement is perfectly acceptable. In fact,
this is largely how most web applications have been developed for well
over a decade. But times are changing and the clients of these web
applications have grown from people occasionally viewing websites on
the web browser to people frequently consuming content and using
applications that consume APIs. And these days the so-called "Internet
of Things" where humans aren’t even involved while cars, jet engines,
and other non-traditional clients are constantly exchanging data with
our APIs. With an increasing number of clients consuming our web
applications, scalability is more important than ever.
Asynchronous web frameworks, in contrast, achieve higher scalability
with fewer threads—generally one per CPU core. By applying a technique
known as event looping (as illustrated in Figure 11.1), these
frameworks are able to handle many requests per thread, making the
per-connection cost much cheaper.
In an event loop, everything is handled as an event, including
requests and callbacks from intensive operations (such as database and
network operations). When a costly operation is needed, the event loop
registers a callback for that operation to be performed in parallel
while the event loop moves on to handle other events. When the
operation is complete, the completion is treated as an event by the
event loop the same as requests. As a result, asynchronous web
frameworks are able to scale better under heavy request volume with
fewer threads (and thus reduced overhead for thread management).
Read the rest of this section and it will clarify any other concern.
Also, check Reactor https://github.com/reactor/reactor-core
For a complete example if you are still having difficulties https://www.baeldung.com/spring-webflux
Does Spring Session management take care of asynchronous calls?
Say that we have multiple controllers and each one is reading/writing different session attributes. Will there be a concurrency issue as the session object is entirely written/read to/from external servers and not the attributes alone?
We are facing such an issue that the attributes set from a controller are not present in the next read... this is an intermittent issue depending on the execution of other controllers in parallel.
When we use the session object from the container we never faced this issue... assuming that it is a direct attribute set/get happening right on to the session object in the memory.
The general use case for the session is storing some user specific data. If I am understanding your context correctly, your issue describes the scenario in which a user, while for example being authenticated from two devices (for example a PC and a phone - hence withing the bounds of the same session) is hitting your backend with requests so fast you face concurrency issues around reading and writing the session data.
This is not a common (and IMHO reasonable) scenario for the session, so projects such as spring-data-redis or spring-data-gemfire won't support it out of the box.
The good news is that spring-session was built with flexibility in mind, so you could of course achieve what you want. You could implement your own version of SessionRepository and manually synchronize (for example via Redis distributed locks) the relevant methods. But, before doing that, check your design and make sure you are using session for the right data storage job.
This question is very similar in nature to your last question. And, you should read my answer to that question before reading my response/comments here.
The previous answer (and insight) posted by the anonymous user is fairly accurate.
Anytime you have a highly concurrent (Web) application/environment where many different, simultaneous HTTP requests are coming in, accessing the same HTTP session, there is always a possibility for lost updates caused by race conditions between competing concurrent HTTP requests. This is due to the very nature of a Servlet container (e.g. Apache Tomcat, or Eclipse Jetty) since each HTTP request is processed by, and in, a separate Thread.
Not only does the HTTP session object provided by the Servlet container need to be Thread-safe, but so too do all the application domain objects that your Web application puts into the HTTP session. So, be mindful of this.
In addition, most HTTP session implementations, such as Apache Tomcat's, or even Spring Session's session implementations backed by different session management providers (e.g. Spring Session Data Redis, or Spring Session Data GemFire) make extensive use of "deltas" to send only the changes (or differences) to the Session state, there by minimizing the chance of lost updates due to race conditions.
For instance, if the HTTP session currently has an attribute key/value of 1/A and HTTP request 1 (processed by Thread 1) reads the HTTP session (with only 1/A) and adds an attribute 2/B, while another concurrent HTTP request 2 (processed by Thread 2) reads the same HTTP session, by session ID (seeing the same initial session state with 1/A), and now wants to add 3/C, then as Web application developers, we expect the end result and HTTP session state to be, after request 1 & 2 in Threads 1 & 2 complete, to include attributes: [1/A, 2/B, 3/C].
However, if 2 (or even more) competing HTTP requests are both modifying say HTTP sessoin attribute 1/A and HTTP request/Thread 1 wants to set the attribute to 1/B and the competing HTTP request/Thread 2 wants to set the same attribute to 1/C then who wins?
Well, it turns out, last 1 wins, or rather, the last Thread to write the HTTP session state wins and the result could either be 1/B or 1/C, which is indeterminate and subject to the vagaries of scheduling, network latency, load, etc, etc. In fact, it is nearly impossible to reason which one will happen, much less always happen.
While our anonymous user provided some context with, say, a user using multiple devices (a Web browser and perhaps a mobile device... smart phone or tablet) concurrently, reproducing this sort of error with a single user, even multiple users would not be impossible, but very improbable.
But, if we think about this in a production context, where you might have, say, several hundred Web application instances, spread across multiple physical machines, or VMs, or container, etc, load balanced by some network load balancer/appliance, and then throw in the fact that many Web applications today are "single page apps", highly sophisticated non-dumb (no longer thin) but thick clients with JavaScript and AJAX calls, then we begin the understand that this scenario is much more likely, especially in a highly loaded Web application; think Amazon or Facebook. Not only many concurrent users, but many concurrent requests by a single user given all the dynamic, asynchronous calls that a Web application can make.
Still, as our anonymous user pointed out, this does not excuse the Web application developer from responsibly designing and coding our Web application.
In general, I would say the HTTP session should only be used to track very minimal (i.e. in quantity) and necessary information to maintain a good user experience and preserve the proper interaction between the user and the application as the user transitions through different parts or phases of the Web app, like tracking preferences or items (in a shopping cart). In general, the HTTP session should not be used to store "transactional" data. To due so is to get yourself into trouble. The HTTP session should be primarily a read heavy data structure (rather than write heavy), particularly because the HTTP session can be and most likely will be accessed from multiple Threads.
Of course, different backing data stores (like Redis, and even GemFire) provide locking mechanisms. GemFire even provides cache level transactions, which is very heavy and arguable not appropriate when processing Web interactions managed in and by an HTTP session object (not to be confused with transactions). Even locking is going to introduce serious contention and latency to the application.
Anyway, all of this is to say that you very much need to be conscious of the interactions and data access patterns, otherwise you will find yourself in hot water, so be careful, always!
Food for thought!
I'm experiencing an issue with an asp.net core application that virtually hangs when started under load. Once started, the application can handle the load no problem. It feels like initialization is happening for every concurrent request. Is anyone else experiencing similar behavior?
test scenario:
start a test application that hits the service with 50 simultaneous tasks
start the service
notice the requests getting started, but extremely long delays before any start to finish and most will time out
As a temporary work around, I created middleware that throttles subsequent requests until the very first finishes. This effectively lets asp.net mvc initialize before processing the bulk of the requests. This particular app is asp.net core 1.1 (web api) with EF core.
When using a real database located halfway across the country, I experience a good 900ms delay for the first request to my ASP.NET Core WebAPI. This is because it needs to get a connection pool established for use with connecting to said database, and I do not eagerly create a connection pool when running the service. Instead, it gets initialized lazily when I request a connection via a connection factory that is registered as a singleton in the services container.
Perhaps you're experiencing the same type of delay as you stated you're using Entity Framework Core, which is presumably backed by SQL Server. Is a connection pool with the database being initiated lazily as part of this initial request?
Try making a controller that does not have any dependencies that returns a vanilla 200 OK. Assuming you do not have global filters that have expensive services to hydrate, you should see the baseline initialization performance of your web service.
In a standalone windows application (Wpf/Winforms) I can see the benefits of using async implementation of long running methods to keep the UI responsive.
As I know, the IIS is will take care of the threading stuff e.g the web service will not block while its processing another request.
So my questions are:
When it comes to implementing a web service using async/await, does that makes sence ?
Will we gain any performance/scaling benefits for making the implementation async e.g using the async/await keywords.
To make a wcf service scale and perform well, can the only tuning be done in the config file or are there other things to think of ?
I'm interested in hearing your experiences and not only links to theoretical articles.
Thanks
Web services in general can gain a benefit from async/await if they use naturally-asynchronous operations. This is because async can return the IIS thread to the thread pool while it is waiting for the operation to complete, instead of blocking that thread.
I haven't heard many numbers on WCF services, but on the MVC/WebAPI side I've heard of scalability benefits in the 10x-100x range.
"Naturally-asynchronous" usually means "I/O-based". Obviously, there's no benefit if you just use Task.Run to offload a CPU-bound (or blocking) call onto a thread pool thread - you'd just be trading one thread for another. But if your implementation is I/O-bound, then you can use async to make maximum use of the thread pool.
Another thing to keep in mind is the scalability of the system as a whole. I.e., if your WCF calls all just turn around and call a single SQL Server backend, then you probably won't get any benefit from scaling WCF because the SQL Server will be your scalability bottleneck.
But in general, I recommend using async/await on your server-side code for any naturally-asynchronous work.