JMS Connection delivering messages sent to the queue while the connection was stopped - jms

I am facing an issue with JMS Connection stop() and start(). A simple java program illustrating the same is:
public class Test {
static Connection producerConn = null;
static BufferedWriter consumerLog = null;
static BufferedWriter producerLog = null;
public static final void main(String[] args) throws Exception {
ConnectionFactory cf = new ActiveMQConnectionFactory("failover:(tcp://localhost:61616)");
producerConn = cf.createConnection();
producerLog = new BufferedWriter(new FileWriter("produced.log"));
consumerLog = new BufferedWriter(new FileWriter("consumed.log"));
new Thread(new Runnable() {
public void run() {
try {
producerConn.start();
Session session = producerConn.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue("SampleQ1");
MessageProducer producer = session.createProducer(queue);
Random random = new Random();
byte[] messageBytes = new byte[1024];
for (int i = 0; i < 100; i++) {
random.nextBytes(messageBytes);
Message message = session.createObjectMessage(messageBytes);
producer.send(message);
Thread.currentThread().sleep(10);
producerLog.write(message.getJMSMessageID());
producerLog.newLine();
producerLog.flush();
}
System.out.println("Produced 100000 messages...");
producerLog.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
}).start();
System.out.println("Started producer...");
new Thread(new Runnable() {
public void run() {
int count = 0;
try {
producerConn.start();
Session session = producerConn.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue("SampleQ1");
MessageConsumer consumer = session.createConsumer(queue);
consumer.setMessageListener(new Test().new MyListener());
}
catch (Exception e) {
e.printStackTrace();
}
}
}).start();
System.out.println("Started consumer...");
}
private class MyListener implements MessageListener{
private int count = 0;
public void onMessage(Message message) {
try {
message.acknowledge();
System.out.println("count is " +count++);
if(count == 5){
producerConn.stop();
System.out.println("Sleeping Now for 5 seconds. . ." +System.currentTimeMillis());
Thread.currentThread().sleep(5000);
producerConn.start();
}
System.out.println("Waking up . . ." +System.currentTimeMillis());
consumerLog.write(message.getJMSMessageID());
consumerLog.newLine();
consumerLog.flush();
}
catch (Exception e) {
e.printStackTrace();
}
}
}
}
My idea is to simulate the connection stop() and start(). Therefore, in the consumer thread after calling stop(), I have placed a sleep of 5 seconds. However, in the mean time the producer thread continues its job of sending message to the queue.
I expected the test to just consume only the message delivered before the consumer calls stop() and after it calls start() again after waking up from the sleep. But what's happening here is, when consumer wakes up it reads all the messages from the server even those that were sent to the queue when consumer's message reception was stopped.
Am I doing anything wrong here ?

There is nothing wrong there, it's the correct behavior. In asynchronous messaging producer and consumer are loosely decoupled. A producer does not care whether a consumer is consuming messages or not. It keeps putting messages to a queue while the consumer may be down, or stopped consuming messages or actively consuming messages.
The connection.stop() method has no effect on producer. It affects only consumer, stop() method pauses the delivery of messages from JMS provider to a consumer while start() method starts/resumes message delivery.
Hope this helped.

Related

Springboot Kafka #Listener consumer pause/resume not working

I have a springboot Kafka Consumer & Producer. The consumer is expected to read data from topic 1 by 1, process(time consuming) it & write it to another topic and then manually commit the offset.
In order to avoid rebalancing, I have tried to call pause() and resume() on KafkaContainer but the consumer is always running & never responds to pause() call, tried it even with a while loop and faced no success(unable to pause the consumer). KafkaListenerEndpointRegistry is Autowired.
Springboot version = 2.6.9, spring-kafka version = 2.8.7
#KafkaListener(id = "c1", topics = "${app.topics.topic1}", containerFactory = "listenerContainerFactory1")
public void poll(ConsumerRecord<String, String> record, Acknowledgment ack) {
log.info("Received Message by consumer of topic1: " + value);
String result = process(record.value());
producer.sendMessage(result + " topic2");
log.info("Message sent from " + topicIn + " to " + topicOut);
ack.acknowledge();
log.info("Offset committed by consumer 1");
}
private String process(String value) {
try {
pauseConsumer();
// Perform time intensive network IO operations
resumeConsumer();
} catch (InterruptedException e) {
log.error(e.getMessage());
}
return value;
}
private void pauseConsumer() throws InterruptedException {
if (registry.getListenerContainer("c1").isRunning()) {
log.info("Attempting to pause consumer");
Objects.requireNonNull(registry.getListenerContainer("c1")).pause();
Thread.sleep(5000);
log.info("kafkalistener container state - " + registry.getListenerContainer("c1").isRunning());
}
}
private void resumeConsumer() throws InterruptedException {
if (registry.getListenerContainer("c1").isContainerPaused() || registry.getListenerContainer("c1").isPauseRequested()) {
log.info("Attempting to resume consumer");
Objects.requireNonNull(registry.getListenerContainer("c1")).resume();
Thread.sleep(5000);
log.info("kafkalistener container state - " + registry.getListenerContainer("c1").isRunning());
}
}
Am I missing something? Could someone please guide me with the right way of achieving the required behaviour?
You are running the process() method on the listener thread so pause/resume will not have any effect; the pause only takes place when the listener thread exits the listener method (and after it has processed all the records received by the previous poll).
The next version (2.9), due later this month, has a new property pauseImmediate, which causes the pause to take effect after the current record is processed.
You can try like this. This work for me
public class kafkaConsumer {
public void run(String topicName) {
try {
Consumer<String, String> consumer = new KafkaConsumer<>(config);
consumer.subscribe(Collections.singleton(topicName));
while (true) {
try {
ConsumerRecords<String, String> consumerRecords = consumer.poll(Duration.ofMillis(80000));
for (TopicPartition partition : consumerRecords.partitions()) {
List<ConsumerRecord<String, String>> partitionRecords = consumerRecords.records(partition);
for (ConsumerRecord<String, String> record : partitionRecords) {
kafkaEvent = record.value();
consumer.pause(consumer.assignment());
/** Implement Your Business Logic Here **/
Once your processing done
consumer.resume(consumer.assignment());
try {
consumer.commitSync();
} catch (CommitFailedException e) {
}
}
}
} catch (Exception e) {
continue;
}
}
} catch (Exception e) {
}
}

Does DefaultMessageListenerContainer stop , shutdown close database connections?

I have a DefaultMessageListenerContainer which is processing a message from the queue.
While the message is being processed -- stop , shutdown methods are called on DefaultMessageListenerContainer. Does this close database connections?
Looks like it is closing the database connections and hence the message being processed is getting interrupted from completely processing.
I see these errors :
o.s.jdbc.support.SQLErrorCodesFactory : Error while extracting database name
Closed Connection; nested exception is java.sql.SQLRecoverableException: Closed Connection
could these be because the DefaultMessageListenerContainer was stopped and shutdown ?
My code is as follows . startStopContainer is where I am trying to stop and shutdown container. I want to shutdown container only if listener completed processing the current message. I added logic to figure out if listener completed processing .
Is the below logic the only way or is there a better way to figure out if listener completed processing. Please suggest. Thank you.
public class MyMessageConsumerFacade {
private ConnectionFactory connectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(url);
connectionFactory.setUserName(userName);
connectionFactory.setPassword(password);
return connectionFactory;
}
#Bean
public MessageListenerContainer listenerContainer() {
DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.setDestinationName(queueName);
container.setMessageListener(new MyJmsListener());
return container;
}
}
public class MyJmsListener implements MessageListener {
public boolean onMessageCompleted;
public void onMessage(Message message) {
onMessageCompleted = false;
processMessage(message);
onMessageCompleted = true;
}
}
private String startStopContainer(ExecutionContext etk) {
String response = "success";
AnnotationConfigApplicationContext context = null;
DefaultMessageListenerContainer myNewContainer = null;
if (context == null) {
context = new AnnotationConfigApplicationContext(MyMessageConsumerFacade.class);
}
if (myNewContainer == null) {
myNewContainer = context.getBean(DefaultMessageListenerContainer.class);
}
MyCaseMessageJmsListener messageJmsListener = (MyCaseMessageJmsListener) myNewContainer.getMessageListener();
if (!myNewContainer.isRunning()) {// container not running
myNewContainer.start();
}
//due to some business logic we need to stop listener every 5 minutes, so sleep for 5 minutes and then stop
try {
Thread.sleep(300000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
if (myNewContainer.isRunning()) {
myNewContainer.stop();
}
//Before shutting down container , make sure listener processed all msgs completely
if(!messageJmsListener.isOnMessageCompleted) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
if(messageJmsListener.isOnMessageCompleted) {
myNewContainer.shutdown();
}
if (context != null) {
context.destroy();
}
return response;
}
Is there a better way than this?
No; the container knows nothing about JDBC or any connections thereto.
Stopping the container only stops the JMS consumer(s) the consumers from receiving messages; shutDown() on the container closes the consumer(s).
Something else is closing your JDBC connection.

JMS Configuring backoff/retry without blocking onMessage()

javax.JMS version 2.0.1
Provider : ibm.mq v9.0
Framework : Java Spring boot
From what I know, onMessage() is asynchronous. I am successfully retrying the message send. However, the re-sending of messages happens instantaneously after a message failure. Ideally I want the retry to happen in a sliding window style eg. First retry after 20 seconds, second retry after 40 etc.
How can I achieve this without a Thread.Sleep() which, I presume, will block the entire Java thread and is not something I want at all ?
Code is something like this
final int TIME_TO_WAIT = 20;
public void onMessage(Message , message)
{
:
:
int t = message.getIntProperty("JMSXDeliveryCount");
if(t > 1)
{
// Figure out a way to wait for (TIME_TO_WAIT * t)
}
}
catch(Exception e)
{
// Do some logging/cleanup etc.
throw new RunimeException(e);// this causes a message retry
}
I would suggest you use exponential backoff in the retry logic, but you would need to use the Delivery Delay feature.
Define a custom JmsTemplate that will use delay property from the message, you should add retry count in the message property as well so that you can delay as per your need like 20, 40, 80, 160, etc
public class DelayedJmsTemplate extends JmsTemplate {
public static String DELAY_PROPERTY_NAME = "deliveryDelay";
#Override
protected void doSend(MessageProducer producer, Message message) throws JMSException {
long delay = -1;
if (message.propertyExists(DELAY_PROPERTY_NAME)) {
delay = message.getLongProperty(DELAY_PROPERTY_NAME);
}
if (delay >= 0) {
producer.setDeliveryDelay(delay);
}
if (isExplicitQosEnabled()) {
producer.send(message, getDeliveryMode(), getPriority(), getTimeToLive());
} else {
producer.send(message);
}
}
}
Define Components, that will have the capability fo re-enqueue of the message, you can define this interface in the base message listener. The handleException method should do all the tasks of enqueue and computing delay etc. You may not always interested in enqueuing, in some cases, you would discard messages as well.
You can see a similar post-processing logic here
https://github.com/sonus21/rqueue/blob/4c9c5c88f02e5cf0ac4b16129fe5b880411d7afc/rqueue-core/src/main/java/com/github/sonus21/rqueue/listener/PostProcessingHandler.java
#Component
#Sl4j
public class MessageListener {
private final JmsTemplate jmsTemplate;
#Autowired
public MessageListener(JmsTemplate jmsTemplate) {
this.jmsTemplate = jmsTemplate;
}
#JmsListener(destination = "myDestination")
public void onMessage(Message message) throws JMSException {
try {
// do something
} catch (Exception e) {
handleException("myDestination", message, e);
}
}
// Decide whether the message should be ignored due to many retries etc
private boolean shouldBeIgnored(String destination, Message message) {
return false;
}
// add logic to compute delay
private long getDelay(String destination, Message message, int deliveryCount) {
return 100L;
}
private void handleException(String destination, Message message, Exception e) throws JMSException {
if (shouldBeIgnored(destination, message)) {
log.info("destination: {}, message: {} is ignored ", destination, message, e);
return;
}
if (message.propertyExists("JMSXDeliveryCount")) {
int t = message.getIntProperty("JMSXDeliveryCount");
long delay = getDelay(destination, message, t + 1);
message.setLongProperty(DELAY_PROPERTY_NAME, delay);
message.setIntProperty("JMSXDeliveryCount", t + 1);
jmsTemplate.send(destination, session -> message);
} else {
// no delivery count, is this the first message or should be ignored?
}
}
}

How to fix weird "channelMax" error rabbitmq rpc?

I created simple client and server. Client sends rpc requests:
RabbitTemplate template.convertSendAndReceive(...) ;
Server receive it, and answers back:
#RabbitListener(queues = "#{queue.getName()}")
public Object handler(#Payload String key)...
Then I make client send rpc requests asynchronously, simultaneously(which produces lot of concurrent rpc requests).
And unexpectedly receive an error:
org.springframework.amqp.AmqpResourceNotAvailableException: The channelMax limit is reached. Try later.
at org.springframework.amqp.rabbit.connection.SimpleConnection.createChannel(SimpleConnection.java:59)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$ChannelCachingConnectionProxy.createBareChannel(CachingConnectionFactory.java:1208)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$ChannelCachingConnectionProxy.access$200(CachingConnectionFactory.java:1196)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.doCreateBareChannel(CachingConnectionFactory.java:599)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createBareChannel(CachingConnectionFactory.java:582)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.getCachedChannelProxy(CachingConnectionFactory.java:552)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.getChannel(CachingConnectionFactory.java:534)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.access$1400(CachingConnectionFactory.java:99)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$ChannelCachingConnectionProxy.createChannel
Rabbitmq client seems create too many channels. How to fix it?
And why my client create them so many?
Channels are cached so there should only be as many channels as there are actual RPC calls in process.
You may need to increase the channel max setting on the broker.
EDIT
If your RPC calls are long-lived, you can reduce the time the channel is used by using the AsyncRabbitTemplate with an explicit reply queue, and avoid using the direct reply-to feature.
See the documentation.
EDIT2
Here is an example using the AsyncRabbitTemplate; it sends 1000 messages on 100 threads (and the consumer has 100 threads).
The total number of channels used was 107 - 100 for the consumers and only 7 were used for sending.
#SpringBootApplication
public class So56126654Application {
public static void main(String[] args) {
SpringApplication.run(So56126654Application.class, args);
}
#RabbitListener(queues = "so56126654", concurrency = "100")
public String slowService(String in) throws InterruptedException {
Thread.sleep(5_000L);
return in.toUpperCase();
}
#Bean
public ApplicationRunner runner(AsyncRabbitTemplate asyncTemplate) {
ExecutorService exec = Executors.newFixedThreadPool(100);
return args -> {
System.out.println(asyncTemplate.convertSendAndReceive("foo").get());
for (int i = 0; i < 1000; i++) {
int n = i;
exec.execute(() -> {
RabbitConverterFuture<Object> future = asyncTemplate.convertSendAndReceive("foo" + n);
try {
System.out.println(future.get(10, TimeUnit.SECONDS));
}
catch (InterruptedException e) {
Thread.currentThread().interrupt();
e.printStackTrace();
}
catch (ExecutionException e) {
e.printStackTrace();
}
catch (TimeoutException e) {
e.printStackTrace();
}
});
}
};
}
#Bean
public AsyncRabbitTemplate asyncTemplate(ConnectionFactory connectionFactory) {
return new AsyncRabbitTemplate(connectionFactory, "", "so56126654", "so56126654-replies");
}
#Bean
public Queue queue() {
return new Queue("so56126654");
}
#Bean
public Queue reeplyQueue() {
return new Queue("so56126654-replies");
}
}

Apache camel and Jms example program

I am using one program to publish messages to activemq using jms and apache camel..
public final class CamelJmsTofileExample {
private CamelJmsTofileExample() {}
public static void main(String args[]) throws Exception {
CamelContext context = new DefaultCamelContext();
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(
"vm://localhost?broker.persistent=false");
context.addComponent("test-jms",
JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));
context.addRoutes(new RouteBuilder() {
public void configure() {
from("test-jms:queue:test.queue").to("file://test");
}
});
ProducerTemplate template = context.createProducerTemplate();
context.start();
for (int i = 0; i < 100; i++) {
template.sendBody("test-jms:queue:test.queue", "Test Message: " + i);
}
Thread.sleep(1000);
context.stop();
}
}
It is putting 10 message correctly...But the problem is when increasing the count of "i" to 100,500 or something i am not able to find that many messages in test folder..Help me in resolving this problem....Thanks in advance..
If you send so many messages to a queue, then you likely need to wait longer in your thread sleep before you stop Camel and the application.
eg you need to give it more time to process all the messages on the queue.

Resources