Spring JMS - activemq - individualDLQ not used - spring-boot

I'm trying to set up spring JMS for activemq, and I'd like individual DLQs for easier monitoring rather than everything being lumped on one DLQ.
However my bean for this doesn't seem to be picked up. Could anyone point me out what I'm doing wrong as the documentation's pretty vague on how to do this programatically?
My Queue config:
#Bean
public MessageConverter jacksonJmsMessageConverter() {
MappingJackson2MessageConverter converter = new MappingJackson2MessageConverter();
converter.setTargetType(MessageType.TEXT);
converter.setTypeIdPropertyName("_type");
return converter;
}
#Bean
public DeadLetterStrategy deadLetterStrategy() {
IndividualDeadLetterStrategy deadLetterStrategy = new IndividualDeadLetterStrategy();
deadLetterStrategy.setQueueSuffix(".dlq");
return deadLetterStrategy;
}
#Bean
public RedeliveryPolicy redeliveryPolicy() {
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setInitialRedeliveryDelay(5000);
redeliveryPolicy.setBackOffMultiplier(2);
redeliveryPolicy.setUseExponentialBackOff(true);
redeliveryPolicy.setMaximumRedeliveries(5);
return redeliveryPolicy;
}
#Bean
public Queue myQueue() {
ActiveMQQueue queue = new ActiveMQQueue("myQueue");
return queue;
}

You can apply Individual Dead Letter Strategy using configurations something like this
#Bean
DeadLetterStrategy deadLetterStrategy(){
IndividualDeadLetterStrategy dlq = new IndividualDeadLetterStrategy(); //Messages of each will get to their respective Dead Letter Queues. if Original queue = 'x', its DLQ = 'prefix + x'
dlq.setQueueSuffix(".dlq");
dlq.setUseQueueForQueueMessages(true);
return dlq;
}
#Bean
public BrokerService brokerService(#Autowired DeadLetterStrategy strategy) throws Exception {
BrokerService broker = new BrokerService();
TransportConnector connector = new TransportConnector();
connector.setUri(new URI("your broker url")); //default/embedded broker url: vm://localhost?broker.persistent=true
broker.addConnector(connector);
PolicyEntry entry = new PolicyEntry();
entry.setDestination(new ActiveMQQueue("*")); //given DeadLetterStrategy will be applied to all types of Queues; ',' can also be used
entry.setDeadLetterStrategy(strategy);
PolicyMap map = new PolicyMap();
map.setPolicyEntries(Arrays.asList(entry));
broker.setDestinationPolicy(map);
return broker;
}
And finally your queue should look like this:
#JmsListener(destination = "main_queue_name" + ".dlq")
protected void processFailedItem(YourCustomPojo data) {
//do whatever you want
}

Related

RabbitMQ Spring "Cannot determine target ConnectionFactory for lookup key" when using Java lambda parallelStream

We have a Spring Java application using RabbitMQ, and here is the scenario:
There is a consumer receiving messages from a queue and sending them to another one. We are using "SimpleRabbitListenerContainerFactory" as the container factory, but when sending the messages to the other queue inside a "parallelStream" we've got an IllegalStateException "Cannot determine target ConnectionFactory for lookup key" Exception
When we remove the "parallelStream" it works flawlessly.
public void sendMessage(final StagingMessage stagingMessage, final Long timestamp, final String country) {
final List<TransformedMessage> messages = processMessageList(stagingMessage);
messages.parallelStream().forEach(message -> {
final TransformedMessage transformedMessage = buildMessage(timestamp, ApiConstants.POST_METHOD, country);
myMessageSender.sendQueue(country, transformedMessage);
});
}
Connectio Facotory, where the lookup key is set:
#Configuration
#EnableRabbit
public class RabbitBaseConfig {
#Autowired
private QueueProperties queueProperties;
#Bean
#Primary
public ConnectionFactory connectionFactory(final ConnectionFactory connectionFactoryA, final ConnectionFactory connectionFactoryB) {
final SimpleRoutingConnectionFactory simpleRoutingConnectionFactory = new SimpleRoutingConnectionFactory();
final Map<Object, ConnectionFactory> map = new HashMap<>();
for (final String queue : queueProperties.getAQueueMap().values()) {
map.put("[" + queue + "]", connectionFactoryA);
}
for (final String queue : queueProperties.getBQueueMap().values()) {
map.put("[" + queue + "]", connectionFactoryB);
}
simpleRoutingConnectionFactory.setTargetConnectionFactories(map);
return simpleRoutingConnectionFactory;
}
#Bean
public Jackson2JsonMessageConverter jackson2JsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
Welcome to stack overflow!
You should always show the pertinent code and configuration beans when asking questions like this.
I assume you are using the RoutingConnectionFactory.
It uses a ThreadLocal to store the lookup key so the send has to happen on the same thread that set the key.
You generally should never go asynchronous in a listener anyway; you risk message loss. To increase concurrency, use the concurrency properties on the container.
EDIT
One technique would be to convey the lookup key in a message header:
#Bean
public RabbitTemplate template(ConnectionFactory rcf) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(rcf);
Expression expression = new SpelExpressionParser().parseExpression("messageProperties.headers['cfSelector']");
rabbitTemplate.setSendConnectionFactorySelectorExpression(expression);
return rabbitTemplate;
}
#RabbitListener(queues = "foo")
public void listen1(String in) {
IntStream.range(0, 10)
.parallel()
.mapToObj(i -> in + i)
.forEach(val -> {
this.template.convertAndSend("bar", val.toUpperCase(), msg -> {
msg.getMessageProperties().setHeader("cfSelector", "[bar]");
return msg;
});
});
}

Using a Connection Name List in a JMS Load Balanced Environment

Our JMS infrastructure is load balanced. As a result of this, I am attempting to use a connectionNameList when configuring the connection factory. The idea here is that any JMS message that arrives on either of the primary or secondary queue manager will get picked up and processed. However, it only appears that messages are being picked up by the primary.
Here is my listener annotation:
#JmsListener(destination = "${request-queue}", containerFactory = "DefaultJmsListenerContainerFactory")
public void onMessage(Message msg) {
System.out.println(msg.toString());
}
Here is the JMS listener container factory:
#Bean(name = "DefaultJmsListenerContainerFactory")
public DefaultJmsListenerContainerFactory createJmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(buildConnectionFactory());
factory.setConcurrency(numberOfListeners);
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
factory.setSessionTransacted(false);
factory.setErrorHandler(queueErrorHandler);
factory.setBackOff(getBackOffStrategy());
return factory;
}
And here is the connection factory:
#Bean(name = "MQConnectionFactory")
public ConnectionFactory buildConnectionFactory() {
try {
MQConnectionFactory mqcf = new MQConnectionFactory();
mqcf.setConnectionNameList(mq1.daluga.com(2171),mq2.daluga.com(2171));
mqcf.setChannel(channel);
mqcf.setTransportType(WMQConstants.WMQ_CM_CLIENT);
return mqcf;
} catch (Exception e) {
throw new RuntimeException(message, e);
}
}
I suspect something in my configuration is just not right. Is there anything obvious that folks see that might cause messages not to be picked up from the secondary queue manager?
Thanks!

spring integrationflow does not set the message listener on receivercontainer

We use spring-integration (4.3.12) together with spring-amqp(1.7.4) to send and receive messages between micro services.
To keep out the integration/amqp configuration stuff out of the micro services, we want to use a library containing integration/amqp factories for creation of the objects required.
What I expect:
I create an IntegrationFlow instance with a messageHandler/messageHandler method (see below for the code) and a SimpleMessageHandlerContainer. When I send a message to the bound exchange then IO expect the messageHandler gets called with the message.
What do I get:
An exception: "MessageDispatchingException: Dispatcher has no subscribers"
If I use the MessageListenerContainer directly (set the messageHandler direct at the container) then I get the message as expected.
I guess, the problem lies within the programmatically initialization of the integrationFlow, but I cant find any information what I´m doing wrong.
Can anybody give me a hint?
Thanx
Now the code used:
public IntegrationFlow createMessageNotifierIntegrationFlow(//
String brokerNameSpace, String messageHandlerNameSpace, //
Object messageHandler, String methodName) {
ConnectionFactory cf = createConnectionFactory(brokerNameSpace);
AmqpAdmin amqpAdmin = createAmqpAdmin(brokerNameSpace, cf);
Inbound inbound = createInbound(messageHandlerNameSpace);
Queue queue = createQueue(messageHandlerNameSpace, amqpAdmin, inbound);
MessageNotifierIntegrationFlowBuilder builder = MessageNotifierIntegrationFlowBuilder
.newBuilder(messageHandlerNameSpace, this);
IntegrationFlow integrationFlow = builder//
.withConnectionFactory(cf)//
.withMessageHandler(messageHandler)//
.withMessageHandlerMethod(methodName)//
.withFlowExceptionHandler(new FlowExceptionHandler())//
.withInbound(inbound)//
.withAmqpAdmin(amqpAdmin)//
.withInboundQueue(queue)//
.build();
String beanName = brokerNameSpace + "-" + messageHandlerNameSpace + "-" + inbound.getQueue();
return integrationFlow;
}
public IntegrationFlow build() {
LOGGER.info("creating IntegrationFlow for {}", inbound.getQueue());
validateBuilder();
SimpleMessageListenerContainer receiverContainer = amqpObjectFactory.createMessageListenerContainer(//
inbound, connectionFactory, //
flowExceptionHandler, inboundQueue, messageHandlerNameSpace);
final AmqpInboundChannelAdapterSpec adapter = (AmqpInboundChannelAdapterSpec) Amqp.inboundAdapter(receiverContainer);
StandardIntegrationFlow flow = IntegrationFlows //
.from(adapter) //
.log("receiveData")//
.transform(TO_STRING_TRANSFORMER) //
.handle(messageHandler, messageHandlerMethod) //
.log("to message handler").get();
// flow.start() maybe later?
flow.start();
return flow;
}
public SimpleMessageListenerContainer createMessageListenerContainer(//
final Inbound inbound, //
final ConnectionFactory connectionFactory, //
final FlowExceptionHandler flowExceptionHandler, //
final Queue inboundQueue, String messageHandlerNameSpace) {
final String beanName = messageHandlerNameSpace + "-container-" + inbound.getQueue();
SimpleMessageListenerContainer container = null;
container = new SimpleMessageListenerContainer(connectionFactory);
container.setMaxConcurrentConsumers(inbound.getMaxconsumers());
container.setConcurrentConsumers(inbound.getMinconsumers());
container.setStartConsumerMinInterval(inbound.getMininterval());
container.addQueues(inboundQueue);
container.setAcknowledgeMode(inbound.getAckmode());
container.setDefaultRequeueRejected(inbound.getRequeueRejected());
container.setErrorHandler(flowExceptionHandler);
container.setRecoveryInterval(RECOVERY_INTERVAL);
container.setAutoStartup(false);
return container;
}

Spring amqp delay messaging with rabbitMQ

I am struggling hard to find out the way for scheduled/Delaying messages in Spring AMQP/Rabbit MQ and found solution in here.But i still with a prolem
about Spring AMQP/Rabbit MQ which can not received any message.
My source as the following:
#Configuration
public class AmqpConfig {
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setAddresses("172.16.101.14:5672");
connectionFactory.setUsername("admin");
connectionFactory.setPassword("admin");
connectionFactory.setPublisherConfirms(true);
return connectionFactory;
}
#Bean
#Scope("prototype")
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
return template;
}
#Bean
CustomExchange delayExchange() {
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
return new CustomExchange("my-exchange", "x-delayed-message", true, false, args);
}
#Bean
public Queue queue() {
return new Queue("spring-boot-queue", true);
}
#Bean
Binding binding(Queue queue, Exchange delayExchange) {
return BindingBuilder.bind(queue).to(delayExchange).with("spring-boot-queue").noargs();
}
#Bean
public SimpleMessageListenerContainer messageContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory());
container.setQueues(queue());
container.setExposeListenerChannel(true);
container.setMaxConcurrentConsumers(1);
container.setConcurrentConsumers(1);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
container.setMessageListener(new ChannelAwareMessageListener() {
public void onMessage(Message message, Channel channel) throws Exception {
byte[] body = message.getBody();
System.err.println("receive msg : " + new String(body));
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false); //确认消息成功消费
}
});
return container;
}
}
#Component
public class Send implements RabbitTemplate.ConfirmCallback{
private RabbitTemplate rabbitTemplate;
#Autowired
public Send(RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
this.rabbitTemplate.setConfirmCallback(this);
rabbitTemplate.setMandatory(true);
}
public void sendMsg(String content) {
CorrelationData correlationId = new CorrelationData(UUID.randomUUID().toString());
rabbitTemplate.convertAndSend("my-exchange", "", content, new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setHeader("x-delay", 6000);
return message;
}
},correlationId);
System.err.println("delay message send ................");
}
/**
* 回调
*/
#Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
System.err.println(" callback id :" + correlationData);
if (ack) {
System.err.println("ok");
} else {
System.err.println("fail:" + cause);
}
}
}
Is there someone could give a help.
Thanks all.
Delay messaging is nothing to do with Spring amqp, it's a library which will reside with your code, so the library can't hold any message as such. There are two approaches you can try:
Old Approach:
Set the TTL(time to live) header in each message/queue(policy) and then introduce a DLQ to handle it. once the ttl expired your messages will move from DLQ to main queue so that your listener can process it.
Latest Approach:
Recently RabbitMQ came up with RabbitMQ Delayed Message Plugin , using which you can achieve the same and this plugin support available since RabbitMQ-3.5.8.
You can declare an exchange with the type x-delayed-message and then publish messages with the custom header x-delay expressing in milliseconds a delay time for the message. The message will be delivered to the respective queues after x-delay milliseconds
Details:
To use the delayed-messaging feature, declare an exchange with the type x-delayed-message:
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
channel.exchangeDeclare("my-exchange", "x-delayed-message", true, false, args);
Note that we pass an extra header called x-delayed-type, more on it under the Routing section.
Once we have the exchange declared we can publish messages providing a header telling the plugin for how long to delay our messages:
byte[] messageBodyBytes = "delayed payload".getBytes("UTF-8");
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("x-delay", 5000);
AMQP.BasicProperties.Builder props = new AMQP.BasicProperties.Builder().headers(headers);
channel.basicPublish("my-exchange", "", props.build(), messageBodyBytes);
byte[] messageBodyBytes2 = "more delayed payload".getBytes("UTF-8");
Map<String, Object> headers2 = new HashMap<String, Object>();
headers2.put("x-delay", 1000);
AMQP.BasicProperties.Builder props2 = new AMQP.BasicProperties.Builder().headers(headers2);
channel.basicPublish("my-exchange", "", props2.build(), messageBodyBytes2);
In the above example we publish two messages, specifying the delay time with the x-delay header. For this example, the plugin will deliver to our queues first the message with the body "more delayed payload" and then the one with the body "delayed payload".
If the x-delay header is not present, then the plugin will proceed to route the message without delay.
More here: git

Spring-Boot MQTT Configuration

I have a requirement to send payload to a lot of devices whose names are picked from Database. Then, i have to send to different topics, which will be like settings/{put devicename here}.
Below is the configuration i was using which i got from spring-boot reference documents.
MQTTConfiguration.java
#Configuration
#IntegrationComponentScan
public class MQTTConfiguration {
#Autowired
private Settings settings;
#Autowired
private DevMqttMessageListener messageListener;
#Bean
MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory clientFactory = new DefaultMqttPahoClientFactory();
clientFactory.setServerURIs(settings.getMqttBrokerUrl());
clientFactory.setUserName(settings.getMqttBrokerUser());
clientFactory.setPassword(settings.getMqttBrokerPassword());
return clientFactory;
}
#Bean
MessageChannel mqttOutboundChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "mqttOutboundChannel")
public MessageHandler mqttOutbound() {
MqttPahoMessageHandler messageHandler = new MqttPahoMessageHandler("dev-client-outbound",
mqttClientFactory());
messageHandler.setAsync(true);
messageHandler.setDefaultTopic(settings.getMqttPublishTopic());
return messageHandler;
}
#MessagingGateway(defaultRequestChannel = "mqttOutboundChannel")
public interface DeviceGateway {
void sendToMqtt(String payload);
}
}
Here, i am sending to only 1 topic. So i added the bean like below to send to multiple number of topics;
#Bean
public MqttClient mqttClient() throws MqttException {
MqttClient mqttClient = new MqttClient(settings.getMqttBrokerUrl(), "dev-client-outbound");
MqttConnectOptions connOptions = new MqttConnectOptions();
connOptions.setUserName(settings.getMqttBrokerUser());
connOptions.setPassword(settings.getMqttBrokerPassword().toCharArray());
mqttClient.connect(connOptions);
return mqttClient;
}
and i send using,
try {
mqttClient.publish(settings.getMqttPublishTopic()+device.getName(), mqttMessage);
} catch (MqttException e) {
LOGGER.error("Error While Sending Mqtt Messages", e);
}
Which works.
But my question is, Can i achieve the same, using output channel for better performance? If yes, any help is greatly appreciated. Thank You.
MqttClient is synchronous.
The MqttPahoMessageHandler uses an MqttAsyncClient and can be configured (set async to true) to not wait for the confirmation, but publish the confirmation later as an application event.
If you are using your own code and sending multiple messages in a loop, it will probably be faster to use an async client, and wait for the IMqttDeliveryToken completions later.

Resources