In a standalone windows application (Wpf/Winforms) I can see the benefits of using async implementation of long running methods to keep the UI responsive.
As I know, the IIS is will take care of the threading stuff e.g the web service will not block while its processing another request.
So my questions are:
When it comes to implementing a web service using async/await, does that makes sence ?
Will we gain any performance/scaling benefits for making the implementation async e.g using the async/await keywords.
To make a wcf service scale and perform well, can the only tuning be done in the config file or are there other things to think of ?
I'm interested in hearing your experiences and not only links to theoretical articles.
Thanks
Web services in general can gain a benefit from async/await if they use naturally-asynchronous operations. This is because async can return the IIS thread to the thread pool while it is waiting for the operation to complete, instead of blocking that thread.
I haven't heard many numbers on WCF services, but on the MVC/WebAPI side I've heard of scalability benefits in the 10x-100x range.
"Naturally-asynchronous" usually means "I/O-based". Obviously, there's no benefit if you just use Task.Run to offload a CPU-bound (or blocking) call onto a thread pool thread - you'd just be trading one thread for another. But if your implementation is I/O-bound, then you can use async to make maximum use of the thread pool.
Another thing to keep in mind is the scalability of the system as a whole. I.e., if your WCF calls all just turn around and call a single SQL Server backend, then you probably won't get any benefit from scaling WCF because the SQL Server will be your scalability bottleneck.
But in general, I recommend using async/await on your server-side code for any naturally-asynchronous work.
Related
Reading Spring in action 5th edition chapter 11, last paragraph in section 11.1.2
By accepting a Mono as input, the method is invoked immediately
without waiting for the Taco to be resolved from the request body. And
because the repository is also reactive, it’ll accept a Mono and
immediately return a Flux, from which you call next() and return
the resulting Mono … all before the request is even processed!
How the service will immediately return before the request is even processed? Isn't that counter-intuitive?
I mean should the request be processed first before returning a response?
The book has everything you need. It is a well-written book, just make sure to read carefully while actually (make sure to download the source code from Manning) running the code. It will help you understand better.
From the book (https://livebook.manning.com/book/spring-in-action-fifth-edition/chapter-11/v-7/6):
11.1 Working with Spring WebFlux
Typical Servlet-based web frameworks, such as Spring MVC, are blocking
and multithreaded in nature, using a single thread per connection. As
requests are handled, a worker thread is pulled from a thread pool to
process the request. Meanwhile, the request thread is blocked until it
is notified by the worker thread that it is finished.
Consequently, blocking web frameworks do not scale effectively under
heavy request volume. Latency in slow worker threads makes things even
worse, because it will take longer for the worker thread to be
returned to the pool to be ready to handle another request.
In some use cases, this arrangement is perfectly acceptable. In fact,
this is largely how most web applications have been developed for well
over a decade. But times are changing and the clients of these web
applications have grown from people occasionally viewing websites on
the web browser to people frequently consuming content and using
applications that consume APIs. And these days the so-called "Internet
of Things" where humans aren’t even involved while cars, jet engines,
and other non-traditional clients are constantly exchanging data with
our APIs. With an increasing number of clients consuming our web
applications, scalability is more important than ever.
Asynchronous web frameworks, in contrast, achieve higher scalability
with fewer threads—generally one per CPU core. By applying a technique
known as event looping (as illustrated in Figure 11.1), these
frameworks are able to handle many requests per thread, making the
per-connection cost much cheaper.
In an event loop, everything is handled as an event, including
requests and callbacks from intensive operations (such as database and
network operations). When a costly operation is needed, the event loop
registers a callback for that operation to be performed in parallel
while the event loop moves on to handle other events. When the
operation is complete, the completion is treated as an event by the
event loop the same as requests. As a result, asynchronous web
frameworks are able to scale better under heavy request volume with
fewer threads (and thus reduced overhead for thread management).
Read the rest of this section and it will clarify any other concern.
Also, check Reactor https://github.com/reactor/reactor-core
For a complete example if you are still having difficulties https://www.baeldung.com/spring-webflux
We are currently designing a web service based process, in which we will be using the web-service invoke and receive steps to communicate with a Microsoft biz-talk server.
Our main concern is that a task on the receive step can wait for some time (up to one week) until the biz-talk responds to us, which (we think) would incur a performance penalty on the workflow system as it will be polling for response.
My question is, is there any known performance considerations for the receive step, specially for leaving work items for extended periods?
No, I don't think there will be any undue "overhead". Yes, internally the process engine "polls". For just about anything. Including invoking components, or executing timers. But from a system perspective, you're just waiting for a request.
It sounds like a "receive" step is exactly the right solution here.
I'm trying to get to grips with service fabric and I'm struggling a little bit. Some questions:
are all service fabric service instances single-threaded? I created a stateless web api, one instance, with a method that did a Task.Delay, then returned a string. Two requests to this service were served one after the other, not concurrently. So am I right in thinking then that the number of concurrent requests that can be served is purely a function of the service instance count in the application manifest? Edit Thinking about this, it is probably to do with the set up of OWIN Wep Api. Could it be it is blocking by session? I assumed there is no session by default?
I have long-running operations that I need to perform in service fabric (that can take several hours). Is there a recommended pattern that I can use for this in service fabric? These are currently handled using a storage queue that triggers a webjob. Maybe something with Reliable Queues and a RunAsync loop?
It seems you handled the first part so I will comment on the second part: "long-running operations".
We can see long running operations / workflows being handled far before service fabric came about. For this reason, we can build on the shoulders of giants by looking on the design patterns that software experts have been using for decades. For example, the famous and all inclusive Process Manager. Mind you that this pattern is sometimes an overkill. If it is in your case, just check out the rest of the related patterns in the Enterprise Integration Patterns book (by Gregor Hohpe).
As for the use of reliable collections, those are implementation details when choosing a data structure supporting the chosen design pattern.
I hope that helps
With regards to your second point - It really depends on the nature of your long running task.
Is your long running task the kind of workload that runs on an isolated thread that depends on local OS/VM level resources and eventually comes back with a result (A)? or is it the kind of long running task that goes through stages and builds up a model of the result through a series of persisted state changes (B)?
From what I understand of Service Fabric, it isn't really designed for running long running workloads (A), but more for writing horizontally-scalable, highly-available systems.
If you were absolutely keen on using service fabric (and your kind of workload tends to be more like B than A) I would definitely find a way to break down those long running tasks that could be processed in parallel across the cluster. But even then, there is probably more appropriate technologies designed for this such as Azure Batch?
P.s. If you are going to put a long running process in the RunAsync method, you should design the workload so it is interruptable and its state can be persisted in a way that can be resumed from another node in the cluster
In a stateful service, only the primary replica has write access to
state and thus is generally when the service is performing actual
work. The RunAsync method in a stateful service is executed only when
the stateful service replica is primary. The RunAsync method is
cancelled when a primary replica's role changes away from primary, as
well as during the close and abort events.
P.s.s Long running operations are the devil when trying to write scalable systems. Try and tackle that now and save yourself the future pain if possibe.
To the first point - this is purely a client issue. Chrome saw my requests as indentical and so delayed the 2nd request until the 1st got a response. Varying the parameter of the requests allowed them to be served concurrently.
It seems this is a common question/problem but despite checking out a number of proposed solutions nothing worked for us so far.
The app
It's a simple chat app, that puts a new interface on an existing app's JSON library. We proxy all the calls to their app to avoid x-domain restrictions (IE8).
ASP.net MVC3 App;
It's hosted in IIS6, W2K3 SP2. DEV svr has 1gig ram, TEST svr has 4gig ram.
The problem
When we approach 20 concurrent users, requests start lagging - no issues in Event Viewer to be found. It looks like calls are just queued. There are NO 503's returned.
What we've tried
We're using AsyncController to long-poll a 3rd party webservice for results
Hosted in IIS6
We're using the TPL to call their service in our AsyncController method
We've modified processModel and set maxWorkerThreads=100.
We've looked at this how-to but the HTTP.SYS config looks to service an infinite number of threads so we haven't bothered adding the reg keys.
The 3rd party service can handle lots of concurrent requests (and is in a web farm, so we're fairly confident we're the weakest link)
what are we missing? - any help greatly appreciated
Well... almost four weeks later and I thought I'd update this in case anyone wants to find out what helped us overcome these limitations (we're cramming around 100 simul connections on our DEV server, 1gig Xeon).
AsyncControllers
If you've got a potentially long waiting request (i.e. long polling) then use them.
Feel free to use TaskFactory but be sure to mark it as a long running process, if there is risk you could exception in your thread, be sure to use ContinueWith so you can decrement the operations, and return the error to your caller.
ServicePointManager
If you're making downstream calls (i.e. WebService/3rd party API) then make sure you have increased DefaultConnectionLimit from the default of 2 simultaneous connections.
A rough guide is 8 * Num cores so you don't starve outgoing connection resources.
See this MSDN article on DefaultConnectionLimit for more info.
IOCP vs RestSharp
I love RestSharp's API, it's fantastic but it's probably meant more for client side programming, not for proxying requests. My mistake!! Use HttpWebRequest and the Begin/End methods to make use of IOCP
If you're looking to reverse proxy or url rewrite, check out URL Rewriter, a great library available freely on CodePlex
In the end, our issue wasn't with incoming requests, it was with requests being proxied to a third party, we weren't supplying enough connections and thus they all queued up lagging the whole system. Happy to say after lots of reading, investigation and coding we've resolved it.
I am using a proprietary IoC mechanism in my asp.net mvc3 application (on IIS7) that saves state in [ThreadStatic] fields and therefore relies on an assumption that HttpApplication.BeginRequest, HttpApplication.EndRequest and the whole synchronous execution of the (single) request they relate to are executed on the same thread.
Is that assumption correct?
Is that assumption correct?
No, this assumption is not correct and there's evidence for it. The only reliable per request storage mechanism in ASP.NET is HttpContext.Items.
Never use [ThreadStatic] fields to store per-request values in an ASP.NET application. For example if you have an asynchronous controller you could very well have the engine draw one request from the thread pool to begin serving the request, then initiate an asynchronous operation relying on an IOCP (I/O Completion Port) and finally draw another thread from the pool to finish the request. So you could have 2 different threads serving the same HTTP request.
Absolutely never rely on the fact that the HTTP request will be served by the same thread.
This could be true in some cases for synchronous requests but remember that this is just an implementation detail. This could change without any notice from one version of .NET to another. You should never rely on it and never use [ThreadStatic] in ASP.NET. This could bite you very badly.