Can you explain me what mean:
"Essentially, there are two basic strategies which Hystrix supports: offload the work somewhere else (using dedicated thread pool) or do the work in the current thread (relying on semaphores). Using dedicated thread pools, also known as the bulkhead pattern, is the right strategy to use in most use cases: the calling thread is unblocked, plus the timeout expectations could be set as well. With semaphores, the current thread are going to be busy till the work is completed, successfully or not (timeouts are claimed to be also supported since 1.4.x release branch but there are certain side effects)."
My questions:
1) Why the timeout expectations could be NOT set as well in semaphore? I think semaphore has method "tryacquire" that Waits for timeout : https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Semaphore.html#tryAcquire(long,%20java.util.concurrent.TimeUnit)
"timeouts are claimed to be also supported since 1.4.x release branch but there are certain side effects" , what's the effects?
2) Using dedicated pool meaning that there is a thread pool for each service?
thanks
Related
I have a simple rest endpoint that executes Postgres procedure.
This procedure returns the current state of device.
For example:
20 devices.
Client app connect to API and make 20 responses to that endpoint every second.
For x clients there are x*20 requests.
For 2 clients 40 requests.
It causes a big cpu load on Postgres server only if there are many clients and/or many devices.
I didn’t create it but I need to redesign it.
How to limit concurrent queries to db only for it? It would be a hot fix.
My second idea is to create background worker that executes queries only one in the same time. Then the endpoint fetches data from memory.
I would try the simple way first. Try to reduce
the amount of database connections in the pool OR
the amount of working threads in the build-in Tomcat.
More flexible option would be to put the logic behind a thread pool limiting the amount of working threads. It is not trivial, if the Spring context and database is used inside a worker. Take a look on a Spring annotation #Async.
Offtopic: The solution we are discussing here looks like a workaround. The discussed solution alone will most probably increase the throughput only by factor 2 maybe 3. It is not JEE conform and it will be most probably not very stable. It is better to refactor the application avoiding such a problem. Another option would be to buy a new database server.
Update: JEE compliant solution would be to implement some sort of bulkhead pattern. It will limit the amount of concurrent running requests and reject it, if the some critical number is reached. The server application answers with "503 Service Unavailable". The client application catches this status and retries a second later (see "exponential backoff").
Reading Spring in action 5th edition chapter 11, last paragraph in section 11.1.2
By accepting a Mono as input, the method is invoked immediately
without waiting for the Taco to be resolved from the request body. And
because the repository is also reactive, it’ll accept a Mono and
immediately return a Flux, from which you call next() and return
the resulting Mono … all before the request is even processed!
How the service will immediately return before the request is even processed? Isn't that counter-intuitive?
I mean should the request be processed first before returning a response?
The book has everything you need. It is a well-written book, just make sure to read carefully while actually (make sure to download the source code from Manning) running the code. It will help you understand better.
From the book (https://livebook.manning.com/book/spring-in-action-fifth-edition/chapter-11/v-7/6):
11.1 Working with Spring WebFlux
Typical Servlet-based web frameworks, such as Spring MVC, are blocking
and multithreaded in nature, using a single thread per connection. As
requests are handled, a worker thread is pulled from a thread pool to
process the request. Meanwhile, the request thread is blocked until it
is notified by the worker thread that it is finished.
Consequently, blocking web frameworks do not scale effectively under
heavy request volume. Latency in slow worker threads makes things even
worse, because it will take longer for the worker thread to be
returned to the pool to be ready to handle another request.
In some use cases, this arrangement is perfectly acceptable. In fact,
this is largely how most web applications have been developed for well
over a decade. But times are changing and the clients of these web
applications have grown from people occasionally viewing websites on
the web browser to people frequently consuming content and using
applications that consume APIs. And these days the so-called "Internet
of Things" where humans aren’t even involved while cars, jet engines,
and other non-traditional clients are constantly exchanging data with
our APIs. With an increasing number of clients consuming our web
applications, scalability is more important than ever.
Asynchronous web frameworks, in contrast, achieve higher scalability
with fewer threads—generally one per CPU core. By applying a technique
known as event looping (as illustrated in Figure 11.1), these
frameworks are able to handle many requests per thread, making the
per-connection cost much cheaper.
In an event loop, everything is handled as an event, including
requests and callbacks from intensive operations (such as database and
network operations). When a costly operation is needed, the event loop
registers a callback for that operation to be performed in parallel
while the event loop moves on to handle other events. When the
operation is complete, the completion is treated as an event by the
event loop the same as requests. As a result, asynchronous web
frameworks are able to scale better under heavy request volume with
fewer threads (and thus reduced overhead for thread management).
Read the rest of this section and it will clarify any other concern.
Also, check Reactor https://github.com/reactor/reactor-core
For a complete example if you are still having difficulties https://www.baeldung.com/spring-webflux
What is the advantage of using threadpool in Hystrix?
Suppose we are calling a third party service ,when we call a service or DB that thread goes into waiting state than what is the use of keep creating thread for each call?
So,I mean how short circuited(Threadpooled) method is batter then normal(non short circuited) method?
Lets say when a remote service(any service) is started to respond slowly, but a typical application(service which is making call to remote service) will still continue to call that remote service. So short circuited(Threadpooled) method helps you build a Defensive system in this particular case.
As calling service does not know if the remote service is healthy or not and new threads are spawned every time a request comes in. This will cause threads on an already struggling server to be used.
We don’t want this to happen as we need these threads for other remote calls or processes running on our server and we also want to avoid CPU utilization spiking up. so this prevents resources from becoming blocked if latency occurs. Also Bounded thread pool also gives some breathing room for downstream services to recover.
For detail : ThreadPool in Hystrix
I want to use the spring state machine as the main processor of my application. I want to start the application, do the bootstrapping as an action of the initial state and tear down as an action of the end state. In the middle the application should wait for events.
So, I started by doing as shown in
http://docs.spring.io/spring-statemachine/docs/current/reference/html/developing-your-first-spring-statemachine-application.html
Everything works as described except that after exiting the run method the entire application stops and does not listen to further events.
How can this behavior be achieved? Is there a blueprint/template available? I didn't found one. Similar to a web component, listening for request, I want the state machine to wait for configured events. My application runs on a Raspberry Pi and those events are triggered by external actions like "button pressed", "a connected device delivers a measurement result".
Next to my main question I asked myself, whether spring state mechanine will work correct in my environment: I use Pi4J for hardware interaction. This framework usually uses its own threads for watching for hardware events. How will concurrent events be treated. Are actions always run synchroniously in the thread triggering the event or is there a separate thread pool?
Thanks,
Steve
This is a normal spring boot question as app will exit if nothing is keeping it alive. With boot apps you usually have a web layer and a thread from there keeps app alive.
statemachine docs have more info on how to configure executor to be threaded. On default execution happens in a same thread.
Pi4J is a good question as I'm not that familiar with its threading. I know that many bugs has been fixed as it used to create a lot of threads user didn't have no control and it's probably still a case. There's been some development on Pi4J to allow user to define thread factories which in theory could also passed to Spring TaskExecutor used by statemachine.
I'm just about to use the new EJB3 TimerService (as part of Java EE 6), and as usual, I'm impressed by the brevity of JavaDoc :)
Do you know what is the effect of the persistent property of the TimerConfig object?
JavaDoc TimerConfig says: The persistent property determines whether the corresponding timer has a lifetime that spans the JVM in which it was created. It is optional and defaults to true.
The persistent property means that the container is required to persist the timer state to a database. This is important if you need to guarantee that the timer will fire even if the server is taken offline (intentionally or crash). When the server comes back online, it is required to execute missed timers. Setting a timer as persistent also has the side-effect of ensuring that the timer only executes in one server JVM (but not necessarily the one that created it), whatever that means for your product. For example, in a clustered server environment, this typically means that even if EJB module is running on 3 JVMs, exactly one JVM will execute the timer.
persistent=true was the only option available prior to EJB 3.1. Some timer operations are not critical enough to warrant this level of reliability, so the option was added to allow non-persistent timers. Setting a timer as non-persistent also has the side-effect of ensuring it runs in the JVM in which it was created. This can be useful for updating an in-memory cache or static HTML.