We use masstransit 2.9.9 and have more than 100 servicebus instances in a single process. The problem is that masstransit consumes more than 600 threads, is it possible to limit thread count that masstransit uses?
Related
I am using Spring Kafka Consumer. I have set the concurrency to 10 and have 5 consumers created (for 5 topics). So there are 50 Spring Kafka Consumer threads.
What is the maximum number of threads I can have for Kafka Consumers? How can I increase the size of this thread pool? I have gone over the spring documentation but didn't find anything relavent.
There is no limit; by default, each container uses a SimpleAsyncTaskExecutor for the threads; there is no pool involved.
I have around 10 different rabbit mq queues in 10 different virtual hosts to connect to. For each queue, a separate SimpleMessageListenerContainer bean is defined in my spring boot application and a separate Spring Integration flow is created using each specific SimpleMessageListenerContainer.
The concurrency for SimpleMessageListenerContainer is set to 1-3. Each of the SimpleMessageListenerContainer bean is using separate CachingConnectoryFacory beans. The Connection Factory mode is set as CHANNEL.
We also have another IntegrationFlow to publish messages to an outbound queue that is using a different connection factory. I am not setting any ThreadPool Task Executors in the ConnectionFactory, so it using the default one. While doing the Load test we are noticing that the multiple thread pool (prefixed with pool-) are getting crated and after a certain point application crashes may due to the high number of threads.
It looks like the default thread pool executor is having max value of Integer unbounded which may spinning up threads based on the demand. I tried setting custom Thread Pool task executors for each connection factory and I noticed that the threads are not growing like previously but from the java profiler it shows the SimpleMessageListenerContainer threads are getting BLOCKED frequently.
I want to know if there any best practices or to be followed while setting the custom thread pool task executors in the connection factory like a ratio between Lisneter threads and connection factory threads etc?
I have done some debugging; ...-1 gets renamed to, for example AMQP Connection 127.0.0.1:5672.
That thread is not from the pool, but it is created by the same thread factory.
Similarly, the scheduled executor (for heartbeats) uses the same thread factory, and gets ...-2.
Hence the pool starts at ...-3. So indeed, you have a fixed pool of 8 threads, an I/O thread, and a heartbeat thread for each factory.
With a large number of factories like that, you probably don't need so many threads; I would suggest a single pooled executor with sufficient threads to satisfy your workload; experimentation is probably the only way to determine the number, but I would guess it's something less than 88 (11x8).
We are using activemq for micro services message broker. My query is about how to build failover mechanism for consumer service. If the consumer service is down , how to proceed further. Can we do somethign like hystrix or any other failover/circuit breaker mechanism for such scenarios.
Short answer is 'yes'. The details on how to implement depend on your use case. Recommended architecture:
Publish to virtual topic
Option A: Multi-threaded consumers
2. Multiple competing consumers reading from the queue that is used as the subscriber from the virtual topic
Option B: Single-threaded HA consumers
2. Multiple competing consumers reading from the queue that is used as the subscriber from the virtual topic using an exclusive consumer to ensure only one thread is processing data and the other consumers are hot-standby
I've done some tests with ActiveMQ and Spring JMS. I've configured DefaultMessageListenerContainer (DMLC) with WorkManagerTaskExecutor and CommonJ to manage the threads. I want to have control over the threads are running in the server.
I have made the decision of using DMLC based on a post written by Juergen Holler in Spring's forum: http://forum.spring.io/forum/other-spring-related/remoting/24208-what-s-the-best-practice-for-using-jms-in-spring?p=256420#post256420
In this post, he says "DMLC is the only listener container that does not impose the thread management onto the JMS provider, that is, does not use/block JMS provider threads." So I thought that all the threads would be managed by the server and there wouldn't be threads of ActiveMQ.
However, analysing the server's threads using JConsole, I've seen some threads of ActiveMQ that I didn't expect.
As you can see in the image, there are ActimeMQ threads (inactivity threads, transport threads, etc).
When I'm executing the tests, I see in the logs that the JMS messages are processed by CommonJ threads and not by ActiveMQ threads, so that's fine. However, I don't understand why are ActiveMQ threads are created if they are not used. Specially "ActiveMQ Transport" threads because for every queue I use I've got one thread "ActiveMQ Transport". Therefore if I'm consuming 50 queues, I've got 50 threads of "ActiveMQ Transport". Is it to keep the socket opened? Are these threads mandatory?
Details about the configuration:
ConnectionFactory used: org.apache.activemq.ActiveMQConnectionFactory
ActiveMQ version: 5.11.1
Transport protocol: TCP
Using DefaultMessageListenerContainer the client is in a loop invoking MessageConsumer.receive() method.
It seems a silly question and surely I've misunderstood some basic concepts.
Thanks to Rob Davies' explanation in this thread http://activemq.2283324.n4.nabble.com/ActimeMQ-Client-s-thread-management-td4705885.html, I've understood that all the interaction with ActiveMQ require a TCP thread per connection.
However, the amount of transport threads can be minimised replacing ActiveMQConnectionFactory with one of these ConnectionFactories:
PooledConnectionFactory (ActiveMQ)
SingleConnectionFactory (spring-jms)
CachingConnectionFactory (spring-jms)
I've load testing different JMS implementations for our notification service.
No one of ActiveMQ, HornetQ and OpenMQ behave as expected (issues with reliability and message prioritization). But as now I've best results with OpenMQ. Expect two issues that's probably just missconfiguration (I hope). One with JDBC storage
Test scenario:
2 producers with one queue send messages with different priority. 1 consumer consuming from queue with constant speed that is slightly lower than producers produce. OpenMQ is running standalone and uses PostgreSQL as persistence storage. All messages are sended and consumed from Apache Camel route and it's all persistent.
Issues:
After about 50000 messages I see warnings and errors in OpenMQ log about low memory (default cinfiguration with 256Mb Heap size). Producing is throutled by broker and after some time broker stop dispatching at all. Broker JVM memory usage is on maximum.
How I must configure broker to achieve that goal:
Broker do not depend on queue size (up to 1 000 000 msgs) and on memory limit. Performance is not issue - only reliability.
Thats possible?
I can not help with OpenMQ but perhaps with Camel and ActiveMQ. What issues do you face with ActiveMQ? Can you post your camel route and eventually spring context and the activemq config?