I am using spring, dsl, solace integration with spring boot. My application with one subscriber is able to poll approx 80 messages per second and 4 subscribers able to poll approx 125 messages per seconds. I want to process at least 500 messages per second. My JMS flow is stated below-
public #Bean IntegrationFlow defaultJmsFlow()
{
return IntegrationFlows.from(
//read JMS topic
Jms.messageDrivenChannelAdapter(this.connectionFactory).destination(this.config.getInputQueueName()).errorChannel(errorChannel()).configureListenerContainer(c ->
{
final DefaultMessageListenerContainer container = c.get();
container.setSessionTransacted(true);
container.setMaxConcurrentConsumers(10);
container.setConcurrentConsumers(4);
}).get())
.channel(messageProcessingChannel()).get();
}
After reading messages I'm sending those to DirectChannel. Any special configuration need to make to increase performance of my application so that at least 500 messages per seconds gets processed.
It is strongly recommended to cache consumers when integrating Spring JMS with Solace. This way, the connection stays persistent and messages will be delivered to the consumer quickly without the overhead of unbinding, disconnecting, and reconnecting. This can be set in the DefaultMessageListenerContainer with container.setCacheLevel(3).
If you are using concurrent consumers to read messages from the same Solace queue, ensure that the Solace queue is non-exclusive. Exclusive queues will deliver messages to one active consumer and non-exclusive queues will deliver messages round robin to all consumers.
Related
I wonder whether spring kafka Producer within spring boot guarantee delivery or not.
Does anybody know what happens if some random listener fails to receive message? Would spring kafka retry to send the message?
There are some concepts here:
Producer will produce events and send them to kafka server. You must be aware on the producer side for retries and things like that if Kafka will have downtime or other error scenarios that are specific to your context.
Consumers will have assign partitions by Kafka, each partition will deliver events and each event will have an offset. Consumers will poll for data from kafka (they will request for data, kafka will not push data to consumers, but consumers will go to kafka and require data). Every event that is delivered with success by Kafka to the consumers will produce and Acknowledgment and Kafka will commit the offset of the event. So the next event, with a higher offeset will be delivered to the consumer. If a consumer goes down, partitions will be reasigned to other consumers, so you won't lose your data. If you have only one consumer, the data will be stored in Kafka and when the consumer will be back, it will go and request data from the latest/earliest offset.
I have spring kafka consumer application. The application acts as pass through which polls the messages from kafka broker and send to IBM MQ. What would be a best/simplistic approach to store the offset in case of failure?
The simplest approach is to use the default mechanism of storing the offsets in kafka itself.
If you add a SeekToCurrentErrorHandler, the container will keep redelivering records that are failed in the listener, up to 10 times by default but it can be configured for infinite retries.
If you add stateful retry, the listener adapter can add a delay between each delivery attempt.
See Stateful Retry.
ackOnError should be set to false.
I am running multiple instances of the same Spring Boot 2.0.4 Application, for scaling purposes, that consume messages from an ActiveMQ queue using the following:
#JmsListener(destination = "myQ")
Only the first consumer receives messages and if I stop the first consumer the second instance starts receiving the messages. I want each consumer to consume a message, not the same message, in a round robin fashion. But only the first consumer consumes messages.
It sounds like you want a JMS Topic rather than a Queue. You should also research durable subscriptions, shared subscriptions, and durable topics before you settle on the configuration you need for your setup.
See:
JMS API Programming Model (Search for JMS Message Consumers)
Queues vs Topics
Durable Queues and Topics
An exclusive consumer in Activemq is one that is sent every message from a broker until the consumer dies or goes away, at which time the broker switches consumer.
What is it that defines when the switchover takes place? How do you configure this in Spring JMS/ActiveMQ?
It's not Spring JMS doing the checking; it's the JMS provider, ActiveMQ.
JMS is an API specification; an empty framework, essentially. ActiveMQ provides the implementation backing for managing connections, message brokering, load-balancing, fail-over, etc.
The ActiveMQ broker handles switching-over consumers based on queue properties (you don't need to do anything special in your code):
queue = new ActiveMQQueue("TEST.QUEUE?consumer.exclusive=true");
The switch-over takes place when either the consumer disconnects gracefully or the broker determines that the consumer has disappeared (via the wireFormat.maxInactivityDuration elapsing without any messages or keep-alives being received). You don't have to configure anything if you're happy with the default value of wireFormat.maxInactivityDuration (30 seconds), but you can tweak that if you want to change how long it takes before the broker gives up on a client.
How to configure cluster of Consumers in ActiveMQ?
I created a simple embedded ActiveMQ application with two consumers of one Queue, consumers are working in separate threads. But when I send a message to the Queue, JMS delivers it to first consumer no matter how long it sleeps after receiving.
I think you're trying to explain that the first consumer is receiving all the messages. There is a FAQ entry for this type of problem available here:
http://activemq.apache.org/i-do-not-receive-messages-in-my-second-consumer.html
Bruce