I'm just starting with ActiveMQ Artemis and have Artemis 2.17.0 installed on my machine. Created SpringBoot test app where both JMS and MQTT publishers and receivers exist. Created also small RestController so I can send messages using both JMS and MQTT producers. Receivers are quite simple and just create a log message to console. Now when I create a message using MQTT producer, both JMS and MQTT receivers get and log message to console. But when I send a message using JMS producer, the message is being received only in JMS receiver, no MQTT message in console. Tried several times. Implementation is ok I think as MQTT producer example is working fine. Is there any limitation for routing messages among protocols in Artemis in this way? Or what kind of problem it can be?
Code info about JMS implementation: https://dmarko.tcl-digitrade.com/post/2021/activemq-artemis-spring-boot/
Code info about MQTT implementation: https://dmarko.tcl-digitrade.com/post/2021/activemq-artemis-mqtt/
Apache ActiveMQ Artemis has a flexible addressing model that supports both Point-to-Point and Publish-Subscribe patterns.
By default, Spring Boot creates a JmsTemplate configured to transmit Point-to-Point while MQTT uses a Publish-Subscribe pattern, so the JMS and MQTT receivers are using different messaging patterns and this is causing your issue.
To configure a JmsTemplate for the Publish-Subscribe pattern set spring.jms.pub-sub-domain=true through Boot’s application.properties or set the JmsTemplate pubSubDomain to true, ie:
jmsTemplate.setPubSubDomain(true);
To configure a JmsListener for the Publish-Subscribe pattern set spring.jms.pub-sub-domain=true through Boot’s application.properties or set the JmsListenerContainerFactory pubSubDomain to true, ie:
#Bean
public JmsListenerContainerFactory<?> topicConnectionFactory(ConnectionFactory connectionFactory, DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setPubSubDomain(true);
return factory;
}
#JmsListener(destination = "${prices.mqtt.east}", containerFactory = "topicConnectionFactory")
public void receiveFromTopic(String message) {
...
}
Related
I am trying to listen message through Spring boot application using IBM MQ topic subscription.
Available info (Provided by MQ Admin):
Topic name
Host
Port
QueueManager
BrokerDurableSubscriptionQueue
I am trying to set BrokerDurableSubscriptionQueue property in MQConnectionFactory.
I can find mqConnectionFactory.setBrokerSubQueue(queueName) which I guess can be used for Non-Durable Subscription.
But I cannot find similar property for Durable subscription.
However I can see MQTopic class has setBrokerDurSubQueue property but I am not sure how can I make use of MQTopic object in my case.
I am using below code:
MQConnectionFactory:
#Bean
public MQTopicConnectionFactory topicConnectionFactory(){
MQTopicConnectionFactory mqTopicConnectionFactory= new MQConnectionFactory();
mqTopicConnectionFactory.setHostName(); //mq host name
mqTopicConnectionFactory.setPort(); // mq port
mqTopicConnectionFactory.setQueueManager(); //mq queue manager
mqTopicConnectionFactory.setChannel(); //mq channel name
mqTopicConnectionFactory.setTransportType(1);
mqTopicConnectionFactory.setSSLCipherSuite(); //tls cipher suite name
return mqTopicConnectionFactory;
}
#Bean
public JmsListenerContainerFactory<?> topicListenerFactory(MQTopicConnectionFactory mqtopicConnectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer)
{
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
configurer.configure(factory, mqtopicConnectionFactory);
factory.setPubSubDomain(true);
factory.setSubscriptionDurable(true);
return factory;
}
Listener:
#JmsListener(
destination = "someTopic",
subscription = "someTopic",
containerFactory = "topicListenerFactory"
)
public void receiveMessage(String msg) {
repository.save(msg);
}
Background:
When you provide a specific queue for IBM MQ to use when you subscribe to a topic it is called a unmanaged subscription because MQ is not managing the underlying queue since you have provided it.
When a queue is not provided it is called a managed subscription, in this case MQ creates a queue for you to hold the published messages.
If it is a non-durable subscription the queue created is a temporary dynamic queue with a name like:
SYSTEM.MANAGED.NDURABLE.<8 hex characters>
If it is a durable subscription the queue created is a permanent dynamic queue with a name like:
SYSTEM.MANAGED.DURABLE.<8 hex characters>
What you have uncovered is that the IBM MQ classes for JMS API only supports managed subscriptions.
Suggestions:
I can suggest two options if you want to use IBM MQ classes for JMS API to receive messages published to a topic on a specific queue:
Have a MQ admin setup an administrative subscription on the queue manager. You can do this a few different ways. The example below would be using a MQSC command.
DEFINE SUB('XYZ') TOPICSTR('SOME/TOPIC') DEST(SOME.QUEUE)
Create a utility app using IBM MQ classes for Java that can open a queue and create a durable subscription with a provided queue, the only purpose of this app would be to subscribe and unsubscribe a provided queue, it would not be used to consume any of the published messages.
For both options above you would have the IBM MQ classes for JMS API application open the queue to consume the published message, for all purposes it would not know or need to know the messages were published to a topic. The message will still contain JMS headers showing the topic string where the message was published, so you can inquire this if required. You could also subscribe multiple topics to a single queue if you like.
I am trying to listen to a Solace End Point using Sping Boot and when ran my app i am getting the Error:
2018-09-28 03:16:57.446 WARN 27305 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'TEST1.OUT' - trying to recover. Cause: Error creating session - operation not supported on router (Capability Mismatch: Router does not support transacted sessions.)
Is there a config argument that i can set to not to use transaction sessions.
Thanks
You will need to create a JmsListenerContainerFactory that does not make use of transactions. For example:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(
ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory listenerFactory =
new DefaultJmsListenerContainerFactory();
configurer.configure(listenerFactory, connectionFactory);
listenerFactory.setTransactionManager(null);
listenerFactory.setSessionTransacted(false);
return listenerFactory;
}
Full details can be found in the spring boot docs.
Do note that the Solace message broker supports transactions(local and XA).
To enable local transactions:
Enable allow‑transacted‑sessions in the client-profile used by your username.
Disable direct transport in your JMS connection factory.
Full details can be found in the Solace documentation.
Excellent answer.
To complement Russell answer, in the method which will handle the consume, in the annotation, we must specify the container factory bean created in the last step.
#JmsListener(destination = "TOPIC.TRX_PAYMENT", containerFactory = "jmsListenerContainerFactory")
I use Kafka and Spring Boot with Spring Kafka. After abnormal application termination and then restart, my application started receiving the old, already processed messages from Kafka queue.
What may be the reason for that and how to find and resolve the issue?
my Kafka properties:
spring.kafka.bootstrap-servers=${kafka.host}:${kafka.port}
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.group-id=postfenix
spring.kafka.consumer.enable-auto-commit=false
My Spring Kafka factory and listener:
#Bean
public ConcurrentKafkaListenerContainerFactory<String, Post> postKafkaListenerContainerFactory(KafkaProperties kafkaProperties) {
ConcurrentKafkaListenerContainerFactory<String, Post> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.getContainerProperties().setAckMode(AckMode.MANUAL);
factory.setConsumerFactory(postConsumerFactory(kafkaProperties));
return factory;
}
#KafkaListener(topics = "${kafka.topic.post.send}", containerFactory = "postKafkaListenerContainerFactory")
public void sendPost(ConsumerRecord<String, Post> consumerRecord, Acknowledgment ack) {
Post post = consumerRecord.value();
// do some logic here
ack.acknowledge();
}
When using Kafka, the clients need to commit offsets themselves. This is in contrast to other message brokers, such as AMQP brokers, where the broker keeps track of messages a client did already receive.
In your case, you do not commit offsets automatically and therefore Kafka expects you to commit them manually (because of this setting: spring.kafka.consumer.enable-auto-commit=false). If you do not commit offsets manually in your program, the behaviour you describe is pretty much the expected one. Kafka simply does not know what messages your program did process successfully. Each time you restart your program, Kafka will see that your program did not commit any offsets yet and will apply the strategy you provide in spring.kafka.consumer.auto-offset-reset=earliest, which means the first message in the queue.
If this is all new to you, I suggest reading up this documentation on Kafka and this Spring documentation, because Kafka is quite different than other message brokers.
My environment: spring 4.1, JBoss EAP 6.4, IBM MQ 8.0:
Messages are not redelivered in the case where Listener throws RuntimeException.
I have the following in JmsConfig:
#Bean
DefaultMessageListenerContainer defaultMessageListenerContainer(QueueConnectionFactory connectionFactory, JndiDestinationResolver dr, MessageListener ml) {
DefaultMessageListenerContainer mlc = new DefaultMessageListenerContainer();
mlc.setConnectionFactory(connectionFactory);
mlc.setMessageListener(ml);
mlc.setDestinationName(jndiInQueue);
mlc.setDestinationResolver(dr);
mlc.setSessionTransacted(true);
mlc.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
return mlc;
}
If I use a JmsTransactionManager and pass it to the above method and use like so:
mlc.setTransactionManager(tm)
Following warnings are written to the log:
It is not valid to commit a non-transacted session, and the behavior is the same, no redelivery.
ConnectionFactory is obtained via JNDI, I wonder if sourcing the ConnectionFactory through jndi has something to do with this?
From the AbstractMessageListenerContainer Javadocs:
In order to consistently arrange for redelivery with any container variant, consider "CLIENT_ACKNOWLEDGE" mode or - preferably - setting "sessionTransacted" to "true" instead
There is a similar question on SO.
Flip your ack mode to Session.SESSION_TRANSACTED instead of CLIENT_ACKNOWLEDGE.
Client Ack mode doesn't work as most folks want it.. and is a common "gotcha" in JMS. It acknowledges current message AND all previous messages in the session. It is not per-message acknowledgement.
Edit:
Also check related post-- IBM MQ may require you to use the "XA" versions of the connection factory class.
ref: Websphere Liberty profile - transacted Websphere MQ connection factory
Below is an interesting example of sending messages over MQTT with the standard outbound-channel-adapter (not the MQTT outbound adapter):
https://github.com/joshlong/spring-integration-mqtt
The authors implement their own message handler, and pass it to the adapter.
Now my question is: Is it possible to implement a custom message handler using the MQTT outbound adapter? Or is it only possible with the general outbound-channel-adapter of Spring Integration?
My objective is to obtain logs and handle callbacks from the Paho client, so I can for example handle connection errors, timeouts, etc...
Spring Integration 4.0 provides the MQTT module with MqttPahoMessageHandler as default implementation of AbstractMqttMessageHandler.
I'd say that you can extend from MqttPahoMessageHandler to achieve your MqttCallback wishes, but yes, you can use that custom MessageHandler implementation only from <int:outbound-channel-adapter ref="">.
The out-of-the-box <int-mqtt:outbound-channel-adapter> is just for population a bean for MqttPahoMessageHandler and you can't change that behaviour.
From other side, when you will start to do Spring Integration from JavaConfig you will get deal just only with classes, so there is no boundaries to restict you with custom tags:
#ServiceActivator(inputChannel = "sendToMqttChannel")
#Bean
public MessageHandler mqttHandler() {
return new MyMqttPahoMessageHandler();
}