How to configure ActiveMQ Artemis or my Spring JMS client to not consume any other message on error?
I've configured my broker as follows:
<addresses>
<address name="SimplePointToPoint">
<anycast>
<queue name="SimplePointToPoint"/>
</anycast>
</address>
</addresses>
<address-settings>
<address-setting match="SimplePointToPoint">
<max-delivery-attempts>-1</max-delivery-attempts>
</address-setting>
</address-settings>
This is the code of my consumer:
#JmsListener(destination = "SimplePointToPoint")
public void consumeMessage(Message<String> message) {
if (message.getPayload().contains("error"))
throw new RuntimeException();
log.info("received message: {}", message.getPayload());
}
When sending message1, message2 with error and message3 I want the consumer to process message1 and retry message2 with error forever. message3 should not be processed until message2 with error is successfully consumed. What actually happens is that message3 gets consumed between retries. How to change this behavior? Any ideas?
ActiveMQ prefetch is 1000 by default.
See the documentation.
You can specify an instance of the ActiveMQPrefetchPolicy on an ActiveMQConnectionFactory or ActiveMQConnection. This allows you to configure all the individual prefetch values; as each different quality of service has a different value. e.g.
persistent queues (default value: 1000)
...
Related
I'm working with ActiveMQ Artemis 2.17 and Spring Boot 2.5.7.
I'm publishing messages on topic and queue and consuming it. All is done through JMS.
All queues (anycast or multicast) are durables. My topic (multicast address) has 2 durable queues in order to have 2 separate consumers.
For my topic, the 2 consumers use durable and shared subscriptions (JMS 2.0).
All processing is transactional, managed through Atomikos transaction manager (I need it for a 2 phases commit with the database).
I have a problem with the redelivery policy and DLQ. When I throw an exception during the processing of the message, the redelivery policy applies correctly for the queue (anycast queue) and a DLQ is created with the message. However, for the topic (multicast queue), the redelivery policy does not apply and the message is not sent into a DLQ.
Here is my ActiveMQ Artemis broker configuration:
<?xml version='1.0'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<configuration xmlns="urn:activemq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
<core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:activemq:core ">
<name>0.0.0.0</name>
<!-- Codec and key used to encode the passwords -->
<!-- TODO : set master-password configuration into the Java code -->
<password-codec>org.apache.activemq.artemis.utils.DefaultSensitiveStringCodec;key=UBNTd0dS9w6f8HDIyGW9
</password-codec>
<!-- Configure the persistence into a database (postgresql) -->
<persistence-enabled>true</persistence-enabled>
<store>
<database-store>
<bindings-table-name>BINDINGS_TABLE</bindings-table-name>
<message-table-name>MESSAGE_TABLE</message-table-name>
<page-store-table-name>PAGE_TABLE</page-store-table-name>
<large-message-table-name>LARGE_MESSAGES_TABLE</large-message-table-name>
<node-manager-store-table-name>NODE_MANAGER_TABLE</node-manager-store-table-name>
<jdbc-lock-renew-period>2000</jdbc-lock-renew-period>
<jdbc-lock-expiration>20000</jdbc-lock-expiration>
<jdbc-journal-sync-period>5</jdbc-journal-sync-period>
<!-- Configure a connection pool -->
<data-source-properties>
<data-source-property key="driverClassName" value="org.postgresql.Driver"/>
<data-source-property key="url" value="jdbc:postgresql://localhost/artemis"/>
<data-source-property key="username" value="postgres"/>
<data-source-property key="password" value="ENC(-3eddbe9664c85ec7ed63588b000486cb)"/>
<data-source-property key="poolPreparedStatements" value="true"/>
<data-source-property key="initialSize" value="2"/>
<data-source-property key="minIdle" value="1"/>
</data-source-properties>
</database-store>
</store>
<!-- Configure the addresses, queues and topics default behaviour -->
<!-- See: https://activemq.apache.org/components/artemis/documentation/latest/address-model.html -->
<address-settings>
<address-setting match="#">
<dead-letter-address>DLA</dead-letter-address>
<auto-create-dead-letter-resources>true</auto-create-dead-letter-resources>
<dead-letter-queue-prefix/>
<dead-letter-queue-suffix>.DLQ</dead-letter-queue-suffix>
<expiry-address>ExpiryQueue</expiry-address>
<auto-create-expiry-resources>false</auto-create-expiry-resources>
<expiry-queue-prefix/>
<expiry-queue-suffix>.EXP</expiry-queue-suffix>
<expiry-delay>-1</expiry-delay>
<max-delivery-attempts>5</max-delivery-attempts>
<redelivery-delay>250</redelivery-delay>
<redelivery-delay-multiplier>2.0</redelivery-delay-multiplier>
<redelivery-collision-avoidance-factor>0.5</redelivery-collision-avoidance-factor>
<max-redelivery-delay>10000</max-redelivery-delay>
<max-size-bytes>100000</max-size-bytes>
<page-size-bytes>20000</page-size-bytes>
<page-max-cache-size>5</page-max-cache-size>
<max-size-bytes-reject-threshold>-1</max-size-bytes-reject-threshold>
<address-full-policy>PAGE</address-full-policy>
<message-counter-history-day-limit>0</message-counter-history-day-limit>
<default-last-value-queue>false</default-last-value-queue>
<default-non-destructive>false</default-non-destructive>
<default-exclusive-queue>false</default-exclusive-queue>
<default-consumers-before-dispatch>0</default-consumers-before-dispatch>
<default-delay-before-dispatch>-1</default-delay-before-dispatch>
<redistribution-delay>0</redistribution-delay>
<send-to-dla-on-no-route>true</send-to-dla-on-no-route>
<slow-consumer-threshold>-1</slow-consumer-threshold>
<slow-consumer-policy>NOTIFY</slow-consumer-policy>
<slow-consumer-check-period>5</slow-consumer-check-period>
<!-- We disable the automatic creation of queue or topic -->
<auto-create-queues>false</auto-create-queues>
<auto-delete-queues>true</auto-delete-queues>
<auto-delete-created-queues>false</auto-delete-created-queues>
<auto-delete-queues-delay>30000</auto-delete-queues-delay>
<auto-delete-queues-message-count>0</auto-delete-queues-message-count>
<config-delete-queues>OFF</config-delete-queues>
<!-- We disable the automatic creation of address -->
<auto-create-addresses>false</auto-create-addresses>
<auto-delete-addresses>true</auto-delete-addresses>
<auto-delete-addresses-delay>30000</auto-delete-addresses-delay>
<config-delete-addresses>OFF</config-delete-addresses>
<management-browse-page-size>200</management-browse-page-size>
<default-purge-on-no-consumers>false</default-purge-on-no-consumers>
<default-max-consumers>-1</default-max-consumers>
<default-queue-routing-type>ANYCAST</default-queue-routing-type>
<default-address-routing-type>ANYCAST</default-address-routing-type>
<default-ring-size>-1</default-ring-size>
<retroactive-message-count>0</retroactive-message-count>
<enable-metrics>true</enable-metrics>
<!-- We automatically force group rebalance and a dispatch pause during group rebalance -->
<default-group-rebalance>true</default-group-rebalance>
<default-group-rebalance-pause-dispatch>true</default-group-rebalance-pause-dispatch>
<default-group-buckets>1024</default-group-buckets>
</address-setting>
</address-settings>
<!-- Define the protocols accepted -->
<!-- See: https://activemq.apache.org/components/artemis/documentation/latest/protocols-interoperability.html -->
<acceptors>
<!-- Acceptor for only CORE protocol -->
<!-- We enable the cache destination as recommended into the documentation. See: https://activemq.apache.org/components/artemis/documentation/latest/using-jms.html -->
<acceptor name="artemis">
tcp://0.0.0.0:61616?protocols=CORE,tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;useEpoll=true,cacheDestinations=true
</acceptor>
</acceptors>
<!-- Define how to connect to another broker -->
<!-- TODO : est-ce utile ? -->
<connectors>
<connector name="netty-connector">tcp://localhost:61616</connector>
<connector name="broker1-connector">tcp://localhost:61616</connector>
<connector name="broker2-connector">tcp://localhost:61617</connector>
</connectors>
<!-- Configure the High-Availability and broker cluster for the high-availability -->
<ha-policy>
<shared-store>
<master>
<failover-on-shutdown>true</failover-on-shutdown>
</master>
</shared-store>
</ha-policy>
<cluster-connections>
<cluster-connection name="gerico-cluster">
<connector-ref>netty-connector</connector-ref>
<static-connectors>
<connector-ref>broker1-connector</connector-ref>
<connector-ref>broker2-connector</connector-ref>
</static-connectors>
</cluster-connection>
</cluster-connections>
<!-- <cluster-user>cluster_user</cluster-user>-->
<!-- <cluster-password>cluster_user_password</cluster-password>-->
<!-- should the broker detect dead locks and other issues -->
<critical-analyzer>true</critical-analyzer>
<critical-analyzer-timeout>120000</critical-analyzer-timeout>
<critical-analyzer-check-period>60000</critical-analyzer-check-period>
<critical-analyzer-policy>HALT</critical-analyzer-policy>
<page-sync-timeout>72000</page-sync-timeout>
<!-- the system will enter into page mode once you hit this limit.
This is an estimate in bytes of how much the messages are using in memory
The system will use half of the available memory (-Xmx) by default for the global-max-size.
You may specify a different value here if you need to customize it to your needs.
<global-max-size>100Mb</global-max-size>
-->
<!-- Security configuration -->
<security-enabled>false</security-enabled>
<!-- Addresses and queues configuration -->
<!-- !!! DON'T FORGET TO UPDATE 'slave-broker.xml' FILE !!! -->
<addresses>
<address name="topic.test_rde">
<multicast>
<queue name="rde_receiver_1">
<durable>true</durable>
</queue>
<queue name="rde_receiver_2">
<durable>true</durable>
</queue>
</multicast>
</address>
<address name="queue.test_rde">
<anycast>
<queue name="queue.test_rde">
<durable>true</durable>
</queue>
</anycast>
</address>
</addresses>
</core>
</configuration>
The JMS configuration into the Spring Boot is the following:
#Bean
public DynamicDestinationResolver destinationResolver() {
return new DynamicDestinationResolver() {
#Override
public Destination resolveDestinationName(Session session, String destinationName, boolean pubSubDomain) throws JMSException {
if (destinationName.startsWith("topic.")) {
pubSubDomain = true;
}
else
pubSubDomain =false;
return super.resolveDestinationName(session, destinationName, pubSubDomain);
}
};
}
#Bean
public JmsListenerContainerFactory<?> queueConnectionFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer,
JmsErrorHandler jmsErrorHandler) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setPubSubDomain(false);
factory.setSessionTransacted(true);
factory.setErrorHandler(jmsErrorHandler);
return factory;
}
#Bean
public JmsListenerContainerFactory<?> topicConnectionFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer,
JmsErrorHandler jmsErrorHandler) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
// This provides all boot's default to this factory, including the message converter
configurer.configure(factory, connectionFactory);
factory.setPubSubDomain(true);
factory.setSessionTransacted(true);
factory.setSubscriptionDurable(true);
factory.setSubscriptionShared(true);
factory.setErrorHandler(jmsErrorHandler);
return factory;
}
The messages publisher:
#GetMapping("api/send")
public void sendDataToJms() throws InterruptedException {
OrderRest currentService = this.applicationContext.getBean(OrderRest.class);
for (Long i = 0L; i < 1L; i++) {
currentService.sendTopicMessage(i);
currentService.sendQueueMessage(i);
Thread.sleep(200L);
}
}
#Transactional
public void sendTopicMessage(Long id) {
Order myMessage = new Order("--" + id.toString() + "--", new Date());
jmsTemplate.convertAndSend("topic.test_rde", myMessage);
}
#Transactional
public void sendQueueMessage(Long id) {
Order myMessage = new Order("--" + id.toString() + "--", new Date());
jmsTemplate.convertAndSend("queue.test_rde", myMessage);
}
And the listeners:
#Transactional
#JmsListener(destination = "topic.test_rde", containerFactory = "topicConnectionFactory", subscription = "rde_receiver_1")
public void receiveMessage_rde_1(#Payload Order order, #Headers MessageHeaders headers, Message message, Session session) {
log.info("---> Consumer1 - rde_receiver_1 - " + order.getContent());
throw new ValidationException("Voluntary exception", "entity", List.of(), List.of());
}
#Transactional
#JmsListener(destination = "queue.test_rde", containerFactory = "queueConnectionFactory")
public void receiveMessage_rde_queue(#Payload Order order, #Headers MessageHeaders headers, Message message, Session session) {
log.info("---> Consumer1 - rde_receiver_queue - " + order.getContent());
throw new ValidationException("Voluntary exception", "entity", List.of(), List.of());
}
Is it normal that the redelivery policy and the DLQ mechanismd do only apply for a queue (anycat queue)?
Is it possible to also apply it for topic (multicast queue) and shared durable subscriptions?
If not, how can I have a topic behaviour but with redelivery and DLQ mechanisms? Should I use the divert solution of ActiveMQ?
Thank a lot for your help.
Regards.
I have found the problem. It comes from Atomikos that does not support the JMS v2.0 but only JMS 1.1.
So, it's not possible to get a shared durable subscription on a topic that support at the same time the 2-PC and the redelivery policy.
There is a channel :
<integration:channel id="sampleChannel">
</integration:channel>
And there is a kafka outbound channel with sampleChannel value for channel property :
<int-kafka:outbound-channel-adapter id="kafkaOutboundChannelAdapter"
kafka-template="template"
auto-startup="false"
channel="sampleChannel"
topic="foo"
sync="false"
send-failure-channel="errorChannel"
partition-id-expression="1">
</int-kafka:outbound-channel-adapter>
When message sends and get to sampleChannel, this exception throws :
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'org.springframework.context.support.ClassPathXmlApplicationContext#1623b78d.sampleChannel'
Why kafka outbound can't take message from the sampleChannel ?
Your auto-startup is set to false, so at the time when Application Context is started there is no subscriber to the sampleChannel.
Set auto-startup to true. Or change your sampleChannel to be pollable of publish-subscribe channel.
I am using Spring Integration with JTA support through Atomikos and JMS bound to different Webshpere MQ and a Oracle Database.
Obviosly for others it seems to be the same thread as in Spring Integration JMS with JTA rollback when message goes to errorChannel but it isn't not at all.
The flow is the following:
message-driven-channel-adapter receive the message within transaction
some transformations
ServiceActivator with deeper business logic
DataBase Update
Everything works really fine expect for one szenario:
If an unchecked exception occurs (maybe due to inkonsistent data which shouldn't be) within the ServiceActivator the message is rethrown and handled within an ErrorHandler (via ErrorChannel).
There - in some cases - the orgininal incoming message should be send to a DeadLetter Queue (Webshere MQ). This is done with outbound-channel-adapter.
See enclosed configuration:
<jms:message-driven-channel-adapter id="midxMessageDrivenAdapter"
channel="midxReplyChannel"
acknowledge="auto"
transaction-manager="transactionManager"
connection-factory="connectionFactory"
concurrent-consumers="1"
error-channel="errorChannel"
max-concurrent-consumers="${cmab.integration.midx.max.consumer}"
idle-task-execution-limit="${cmab.integration.midx.idleTaskExecutionLimit}"
receive-timeout="${cmab.integration.midx.receiveTimeout}"
destination="midxReplyQueue"/>
................
<int:service-activator input-channel="midxReplyProcessChannel" ref="processMidxReplyDbWriter" />
<int:service-activator input-channel="errorChannel" ref="inputErrorHandler" />
<jms:outbound-channel-adapter id="deadLetterOutboundChannelAdapter"
channel="errorDlChannel" destination="deadLetterQueue"
delivery-persistent="true" connection-factory="nonXAConnectionFactory"/>
Some important hints:
message-driven-channel-adapter:
connectionFactory is a MQXAQueueConnectionFactory wihtin an AtomikosConnectionFactoryBean
transaction-manager is Spring JtaTransactionManager
outbound-channel-adapter:
connection-factory is a nonXAConnectionFactory.
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory();
MQQueueConnectionFactory mqQueueConnectionFactory = new MQQueueConnectionFactory();
.....
cachingConnectionFactory.setTargetConnectionFactory(mqQueueConnectionFactory);
return cachingConnectionFactory;
And now the error which I observed:
Although I handled the unchecked exception in my ErrorHandler (ref="inputErrorHandler"; send the message to DeadLetter-Queue) an rollback is initiated and the message-driven-channel-adapter receives the message again and again.
Curious is the fact that indeed the message is delivered to the DeadLetterQueue via the outbound-channel-adapter. The destination (deadLetterQueue) contains the failed messages.
Question: What I'm doing wrong? The failed original incoming message is rolled back although I handled the exception in my ErrorHandler.
Any help really appreciate. Many thanks in advance.
concerning to my comment see enclose code of my InputErrorHandler:
if (throwable instanceof MessageHandlingException) {
MessageHandlingException messageHandlingException = (MessageHandlingException) throwable;
if (messageHandlingException.getCause() != null
&& (messageHandlingException.getCause() instanceof CmabProcessDbException
|| messageHandlingException.getCause() instanceof CmabReplyFormatException)) {
String appMessage = ((CmabException) messageHandlingException.getCause()).getAppMessagesAsString();
LOGGER.error(String.format("cmab rollback exception occured: %s", appMessage));
LOGGER.error("******** initiate rollback *********");
throw throwable.getCause();
} else {
ErrorDto payload = fillMessagePayload(messageHandlingException, throwableClassName);
sendMessageToTargetQueue(payload);
}
As I mentioned the "business" ServiceActivator throws an unchecked exception so in this case the ELSE-Statements are calling.
Within that I build up a Message with MessagBuilder and sends it ti the errorDlChannel (s. above the outbound-channel-adapter!).
private void sendMessageToDeadLetterQueue(Message<?> message, String description, String cause) {
.......
Message<?> mb =
MessageBuilder.withPayload(message.getPayload())
.copyHeaders(message.getHeaders())
.setHeaderIfAbsent("senderObject", this.getClass().getName())
.setHeaderIfAbsent("errorText", description)
.setHeaderIfAbsent("errorCause", errorCause).build();
errorDlChannel.send(mb);
}
That's all. This is for this case the last statement. There is nothing anything else in the main method of my ErrorHandler. No rethrow or other stuff.
So this is the reason of my confusion. For me the exception is handled by sending it to the errorDlChannel (outbound-channel-adapter -> DeadLetterQueue).
I saw the message on the DeadLetter Queue but nevertheless a jta rollback occurs...and IMO this shouldn't bee.
I am using Spring AMQP for processing the messages in RabbitMQ. Below is the issue:
1. (say) there are 3 messages in ready state inside RabbitMQ
2. First one is picked up by MessageListener and starts processing. (say) It ends up throwing an exception
3. In this case, the container remains up but the remaining 2 messages are not processed until i restart the container. Also the first messages stays in unacknowledged state.
Is it the expected behavior? If not, how to make sure that other 2 messages will be processed irrespective first one failed processing?
MQ configuraion:
<rabbit:connection-factory id="connectionFactory" host="localhost" username="guest" password="guest" />
<rabbit:admin connection-factory="connectionFactory" />
<rabbit:listener-container
connection-factory="connectionFactory"
concurrency="1"
acknowledge="auto">
<rabbit:listener queue-names="testQueue" ref="myProcessorListener " />
</rabbit:listener-container>
MessageListener class:
public class MyProcessorListener implements MessageListener{
....
#Override
public void onMessage(Message message) {
try{
...Some logic...
} catch (Exception e) {
throw new RuntimeException(e.getMessage(), e);
}
The message is redelivered over and over again; in order to reject it (and discard or route to a dead letter queue), you need to throw AmqpRejectAndDontRequeueException or set the container's requeue-rejected property to false. When configuring with Java it's defaultRequeueRejected.
You can also use a custom error handler.
This is all explained in the reference manual.
I have a simple publisher, which sends messages to a queue.
<int:channel id="publishChannel"/>
<int-jms:outbound-channel-adapter channel="publishChannel" destination="testQueue" session-transacted="true"/>
#Publisher(channel = "publishChannel")
public String sendMessage (String text) {
return text;
}
If the broker crashes, the publisher throws an MessageHandlingException.
Is it possible to block the publisher, till the broker is available again or to make a periodic retry?
You can add a retry advice to the outbound adapter.
<int-jms:outbound-channel-adapter channel="publishChannel" destination="testQueue" session-transacted="true>
<int:request-handler-advice-chain>
<ref bean="myRetryAdvice" />
</request-handler-advice-chain>
</int-jms:outbound-channel-adapter>
You can configure the advice with a backoff policy (e.g. exponential) and to take some action when retries are exhausted (rather than throwing the final exception).