I am working with Async Http Client in Java with this example . I just want to know about the performance issues when we are calling multiple services at a time asynchronously . I am concerned about the number of CPU cores and the number of threads used in this example. I want to know more about this like :
Is this example uses multiple threads for each request.
If suppose I am making n number of calls at that time CPU cores will not support this to run , how I come to know how many threads it will support based in CPU ?
Is this example is Thread Safe.
Is this example uses multiple threads for each request.
No, it does not
how many threads it will support based in CPU
By default the underlying I/O reactor starts one I/O dispatch thread per CPU core
Is this example is Thread Safe
This question is just too vague. Exactly what class instances are you talking about?
Thread-safety rules that apply to both blocking HttpClient and non-blocking HttpAsyncClient are
clients are thread-safe
connection managers are thread-safe
request / response messages are not thread safe
contexts are not thread safe
As far as HttpAsyncClient is concerned, as long as you do not use additional threads to process / generate content, HttpAsyncClient ensures proper access synchronization of all components involved.
Related
The project I am working for is using Spring WebFlux. I came across a very odd issue.
The detail is that some of pieces of code are purely wrote in Reactor style (couples of Flux/Mono pipelines), however, in a inner publishers, I have to call a method where there is "Mono.block()" inside.
The weird thing I aware is that the whole service would become totally stuck, and when I captured a thread dump, I saw all those "nioEventLoopGroup-*" threads were hung.
A fun fact is that if I leverage a "simple" thread (new Thread(..)) to call the method (there is .block inside), everything works fine.
So my question is that, are those "nioEventLoopGroup-*" threads not allowed to call any blocking code.
Sorry for asking a dumb question, but it's blocking issue for now, so I am looking forward your insight.
Reactor, by default, uses a fixed size thread pool. When you use block(), the actual work needs to be done in some thread or another, which depends on the nature of the subscription and the Mono/Flux. Most likely a set of new tasks will be scheduled on the same scheduler, but block() will suspend its thread, waiting for those tasks to complete, so there is one fewer thread for those other tasks to be scheduled on. Evidently you have enough of these calls to exhauast the entire thread pool. All your block() calls are waiting for other tasks to complete, but there are no threads available for them to be run on.
There's no reason to call block() inside a mapping in a reactive stream. There are always other ways of achieving the same goal without blocking - flatMap(), zip() etc etc.
I'm trying to learn writing custom Nifi Processor and from the documentation, the processor should be thread-safe. What I wanted to understand is, if, say - I have 100 flow file records connected to my custom processor, would my processor's onTrigger method ( assume that I haven't enabled #TriggerSerially on this method ) be triggered 100 times and in 100 separate threads ( irrespective of concurrently or not ), or is there a possibility that one flow file is used as input to more than one thread of onTrigger method on my processor.
I apologize if I didn't articulate the question correctly, but essentially, is is possible that the number of times my processor's onTrigger method is triggered, is more than the number of flow files that are connected as input to the processor?
The number of threads executing a processor is based on the number of concurrent tasks on the scheduling tab, which defaults to 1. If you increase this to 2, then 2 threads are concurrently executing the onTrigger method. A single flow file will only be processed by one of these threads.
The #TriggerSerially annotation prevents you from being able to increase the conccurent tasks, so it forces there to never be concurrent execution. A common use case for this would be a source processor that is pulling data from somewhere, typically you wouldn't to concurrently be pulling the same data twice.
I have a system which starts a new thread with each request to the application.
if application received hundreds of requests there may be not enough memory available to start a new thread so it will throw an exception.
I would like to know an ideal mechanism to handle this kind of a situation.
like, if application is receiving lots of request then if there is not enough memory or number of active threads reached the max then i will delay processing other requests.
but i have no idea how to implement this .
Easy solution: Increase thread-pool limits. This is actually viable although out of fashion these days.
More thorough solution: Use a SemaphoreSlim to limit the number of concurrently asynchronously active requests. Make sure to wait asynchronously. If you wait synchronously you'll again burn a thread while waiting. After having waited asynchronously you can resume normal synchronous blocking processing. This requires only small code changes.
Most thorough solution: Implement your processing fully async. That way you never run out of threads.
In my current project I have a structure like this:
Main Thread (GUI):
->Parser Thread
->Healer Thread
->Scripts Thread
the problem is that the Healer & Scripts Threads have to create childthreads with their appropiate timer, it would look like this:
->Parser Thread
->Healer Thread:
-->Healer 1
-->Healer 2
--> (...)
->Scripts Thread:
-->Script 1
--> (...)
For doing this I have thought about coding a dynamically Timer which would be created at runtime when a new Heal/Script is added.
Now the problem/question is:
maybe I have like 20 timers runing at the same time because of this, wouldn't this be a problem to my program performance (CPU consuming, etc)?
Is this the best way to achieve what I'm looking for?
Thanks in advance
There's no problem with having up to 20 timers active at one time in an application. Modern hardware is more than capable of handling that.
Remember also that timer messages are low priority messages and so are only synthesised when the message queue is empty. So, you need to keep the message queues of your threads serviced promptly in order for the messages to be delivered in a timely manner.
A bigger problem for you is that you cannot create TTimer instances outside the GUI/VCL thread. That's because the timer component calls AllocateHWnd which is not thread safe and can only be called from the GUI/VCL thread. So, you'll need to interact with the raw Win32 timer API directly and not use the VCL TTimer wrapper.
While using the disruptor, there may be a consumer(s) that is lagging behind, and because of that slow consumer, the whole application is affected.
Keeping in mind that every producer(Publisher) and consumer(EventProcessor) is running on a single thread each, what can be the solution to the slow consumer problem?
Can we use multiple threads on a single consumer? If not, what is a better alternative?
Generally speaking use a WorkerPool to allow multiple pooled worker threads to work on a single consumer, which is good if you have tasks that are independent and of a potentially variable duration (eg: some short tasks, some longer).
The other option is to have multiple independent workers parallel process over the events, but each worker only handle modulo N workers (eg 2 threads, and one thread processes odd, one thread processes even event IDs). This works great if you have consistent duration processing tasks, and allows batching to work very efficiently too.
Another thing to consider is that the consumer can do "batching", which is especially useful for example in auditing. If your consumer has 10 events waiting, rather than write 10 events to an audit log independently, you can collect all 10 events and write them at the same time. In my experience this more than covers the need to run multiple threads.
Try to separate slow part to other thread (I/O, not O(1) or O(log) calculations, etc.), or to apply some kind of back pressure when the consumer is overloaded (by yielding or temporary parking producers, replying with 503 or 429 status codes, etc.):
http://mechanical-sympathy.blogspot.com/2012/05/apply-back-pressure-when-overloaded.html
Use a set of identical eventHandlers. To avoid more than 1 eventHandler acting upon a single event, I use the following approach.
Create a thread pool of size Number of cores in the system
Executor executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors()); // a thread pool to which we can assign tasks
Then create a handler array
HttpEventHandler [] handlers = new HttpEventHandler[Runtime.getRuntime().availableProcessors()];
for(int i = 0; i<Runtime.getRuntime().availableProcessors();i++){
handlers[i] = new HttpEventHandler(i);
}
disruptor.handleEventsWith(handlers);
In the EventHandler
public void onEvent(HttpEvent event, long sequence, boolean endOfBatch) throws InterruptedException
{
if( sequence % Runtime.getRuntime().availableProcessors()==id){
System.out.println("-----On event Triggered on thread "+Thread.currentThread().getName()+" on sequence "+sequence+" -----");
//your event handler logic
}