What is the advantage of using threadpool in Hystrix?
Suppose we are calling a third party service ,when we call a service or DB that thread goes into waiting state than what is the use of keep creating thread for each call?
So,I mean how short circuited(Threadpooled) method is batter then normal(non short circuited) method?
Lets say when a remote service(any service) is started to respond slowly, but a typical application(service which is making call to remote service) will still continue to call that remote service. So short circuited(Threadpooled) method helps you build a Defensive system in this particular case.
As calling service does not know if the remote service is healthy or not and new threads are spawned every time a request comes in. This will cause threads on an already struggling server to be used.
We don’t want this to happen as we need these threads for other remote calls or processes running on our server and we also want to avoid CPU utilization spiking up. so this prevents resources from becoming blocked if latency occurs. Also Bounded thread pool also gives some breathing room for downstream services to recover.
For detail : ThreadPool in Hystrix
Related
We are conducting an experiment to gauge the benefits of service worker. As part of that we are recording pageLoadTime and service worker state. The idea is that the data which has controller state as activated will be treated as pages served via service worker and where service worker is not installed will have controller as null and so no service worker state. This way we can compare pageLoadTime with & without service worker.
Is this the right approach to know that the page is being served from SW? One concern that comes to mind is, if SW gets killed in between and gives us wrong info about the state. However we are recording service worker state way early in page load lifecycle as soon as our js code boots up. So at this point SW should be up & running intercepting traffic from the page. This looks to be a safe technique. Any thoughts are this ?
Another approach that comes to mind after reading PerformanceResourceTiming API on MDN is to use workerTime.
The workerStart read-only property of the PerformanceResourceTiming interface returns a DOMHighResTimeStamp
immediately before dispatching the FetchEvent if a Service Worker
thread is already running, or immediately before starting the Service
Worker thread if it is not already running. If the resource is not
intercepted by a Service Worker the property will always return 0.
As per the doc, it mentions workerTime will always be zero if SW is not there. Can there be a scenario when SW is installed and yet workerTime is zero?
As an trial, I crawled to Pinterest and queried following in Chrome dev console, workerTime is always zero though SW is there.
performance.getEntriesByType('navigation')[0]
For your own applications, where you're in control of the code that registers the service worker, the best way to determine whether a service worker was in control of the page at load time is to check navigator.serviceWorker.controller before you call navigator.serviceWorker.register(). If navigator.serviceWorker.controller is not null that means a service worker was in control when the page was loaded.
Note, it's important to check navigator.serviceWorker.controller before registering your service worker, because a service worker can start controlling the page after it's registered (if it calls clients.claim()) but in such cases it wasn't controlling the page at load time, which is what you care about.
On my website, here's the code I use on my blog to determine the initial service worker state:
export const initialSWState = !navigator.serviceWorker ? 'unsupported' :
navigator.serviceWorker.controller ? 'controlled' : 'supported';
And since this code runs in the initial evaluation of the module, I know it runs before I register my service worker (which I postpone until after the load event.
One concern that comes to mind is, if SW gets killed in between and gives us wrong info about the state.
That's not a problem. When a service worker is in control of a page, navigator.serviceWorker.controller will always return a reference to that service worker, even if it's not currently running.
As per the doc, it mentions workerTime will always be zero if SW is not there. Can there be a scenario when SW is installed and yet workerTime is zero?
Yes there can. Looking at my own analytics data over the last 30 days, I see a 0 value for workerStart in 14.5% of cases where I knew (using the above technique) that the service worker was in control at page load time. And of 14.5%, almost all of them were either Firefox or Safari, so it looks like those browsers don't correctly set the workerStart value.
We are currently designing a web service based process, in which we will be using the web-service invoke and receive steps to communicate with a Microsoft biz-talk server.
Our main concern is that a task on the receive step can wait for some time (up to one week) until the biz-talk responds to us, which (we think) would incur a performance penalty on the workflow system as it will be polling for response.
My question is, is there any known performance considerations for the receive step, specially for leaving work items for extended periods?
No, I don't think there will be any undue "overhead". Yes, internally the process engine "polls". For just about anything. Including invoking components, or executing timers. But from a system perspective, you're just waiting for a request.
It sounds like a "receive" step is exactly the right solution here.
I'm trying to get to grips with service fabric and I'm struggling a little bit. Some questions:
are all service fabric service instances single-threaded? I created a stateless web api, one instance, with a method that did a Task.Delay, then returned a string. Two requests to this service were served one after the other, not concurrently. So am I right in thinking then that the number of concurrent requests that can be served is purely a function of the service instance count in the application manifest? Edit Thinking about this, it is probably to do with the set up of OWIN Wep Api. Could it be it is blocking by session? I assumed there is no session by default?
I have long-running operations that I need to perform in service fabric (that can take several hours). Is there a recommended pattern that I can use for this in service fabric? These are currently handled using a storage queue that triggers a webjob. Maybe something with Reliable Queues and a RunAsync loop?
It seems you handled the first part so I will comment on the second part: "long-running operations".
We can see long running operations / workflows being handled far before service fabric came about. For this reason, we can build on the shoulders of giants by looking on the design patterns that software experts have been using for decades. For example, the famous and all inclusive Process Manager. Mind you that this pattern is sometimes an overkill. If it is in your case, just check out the rest of the related patterns in the Enterprise Integration Patterns book (by Gregor Hohpe).
As for the use of reliable collections, those are implementation details when choosing a data structure supporting the chosen design pattern.
I hope that helps
With regards to your second point - It really depends on the nature of your long running task.
Is your long running task the kind of workload that runs on an isolated thread that depends on local OS/VM level resources and eventually comes back with a result (A)? or is it the kind of long running task that goes through stages and builds up a model of the result through a series of persisted state changes (B)?
From what I understand of Service Fabric, it isn't really designed for running long running workloads (A), but more for writing horizontally-scalable, highly-available systems.
If you were absolutely keen on using service fabric (and your kind of workload tends to be more like B than A) I would definitely find a way to break down those long running tasks that could be processed in parallel across the cluster. But even then, there is probably more appropriate technologies designed for this such as Azure Batch?
P.s. If you are going to put a long running process in the RunAsync method, you should design the workload so it is interruptable and its state can be persisted in a way that can be resumed from another node in the cluster
In a stateful service, only the primary replica has write access to
state and thus is generally when the service is performing actual
work. The RunAsync method in a stateful service is executed only when
the stateful service replica is primary. The RunAsync method is
cancelled when a primary replica's role changes away from primary, as
well as during the close and abort events.
P.s.s Long running operations are the devil when trying to write scalable systems. Try and tackle that now and save yourself the future pain if possibe.
To the first point - this is purely a client issue. Chrome saw my requests as indentical and so delayed the 2nd request until the 1st got a response. Varying the parameter of the requests allowed them to be served concurrently.
I have a Windows Service which has several timer jobs in it. In the OnStart method, I am starting all those timers. In effect, it takes a lot of time starting the service from the Services console and most of the time, it reports this:
Windows could not start the 'servicename' service on Local Computer.
The service did not repond to the start or control request in a timely fashion.
I know I could create a timer which encapsulatesthe other timers so I could start them outside the OnStart event. However, I'm looking for other possible solutions as I don't like the idea of having a separate timer which does nothing other than start the other processes.
It isn't generally safe to perform external operations such as web or database calls during OnStart, because you can't predict how long they will take, and an excessively long service startup time can interfere with the system:
The Service Control Manager (SCM) waits until the service reports a status of SERVICE_RUNNING. It is recommended that the service reports this status as quickly as possible, as other components in the system that require interaction with SCM will be blocked during this time. Some functions may require interaction with the SCM either directly or indirectly.
So the proper solution is to move this initialization into a separate thread.
However, if you don't want to do this, you can call RequestAdditionalTime:
The RequestAdditionalTime method is intended to be called by the overridden OnContinue, OnPause, OnStart, or OnStop methods to request additional time for a pending operation, to prevent the Service Control Manager (SCM) from marking the service as not responding.
I find that WCF service will take 8-10 seconds to load the first hit. After that it will take less than a second.
Any thoughts?
Probably due to .NET's cold start. Have you looked at setting up the IIS Warmup Module which initializes dependancies before an initial request?
From the Learn IIS website
Decrease the response time for first requests by pre-loading worker processes. The IIS Application Warm-Up module lets you configure the Web application to be pre-loaded before the first request arrives so that the worker process responds to the first Web request more quickly.
Increase reliability by pre-loading worker processes when overlapped recycling occurs. Because the recycled worker process in an overlapped recycling scenario only communicates its readiness and starts accepting requests after it finishes loading and initializing the resources as specified by the configuration, pre-loading the dependencies reduces the response times for the first requests.
Customize the pre-loading of applications. You can configure the IIS Application Warm-Up module to initialize Web applications by using specific Web pages and user identities. This makes it possible to create specific initialization processes that can be executed synchronously or asynchronously, depending on the initialization logic. In addition, these procedures can use specific identities to ensure a proper initialization.