I have a HornetQ standalone cluster with 2 nodes and Mule ESB is configured to connect HornetQ Cluster discovery group.
I wrote a Mule Callback Listener which is consuming messages from HornetQ Cluster. But 2nd node in cluster is behaving like a backup or failover server. When i send multiple messages its always delivered to node1 only. When i stop node1 then node2 is picking the messages. JConsole always shows number of consumers as 0 for node2. Whereas node1 has 6 consumer as defined in Mule JMS connector->numberOfConsumers property as shown in below configuration.
<jms:connector name="hornetq-connector" username="guest"
maxRedelivery="5" password="guest" specification="1.1"
connectionFactory-ref="connectionFactory" numberOfConsumers="6" >
<spring:property name="retryPolicyTemplate" ref="ThreadingPolicyTemplate" />
</jms:connector>
I have only one consumer that is defined in mule flow as shown below.
<flow name="Flow2" doc:name="Flow2">
<jms:inbound-endpoint queue="InboundQueue"
connector-ref="hornetq-connector">
<jms:transaction action="ALWAYS_BEGIN" timeout="10000" />
</jms:inbound-endpoint>
<component class="com.test.Consumer" />
</flow>
Mule Callback java class is added as component as shown in above code (). Please find the code below.
public class Consumer implements org.mule.api.lifecycle.Callable{
private static DateFormat df1 = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss");
#Override
public Object onCall(MuleEventContext eventContext) throws Exception {
// TODO Auto-generated method stub
System.out.println(df1.format(new Date().getTime())+"Consumer:Message Received,"+ Thread.currentThread()+ "," + eventContext.getMessageAsString());
Thread.sleep(5000);
System.out.println(df1.format(new Date().getTime())+"Consumer: Message process complete, "+ Thread.currentThread()+ "," + eventContext.getMessageAsString());
return "Success";
}
}
How to distribute this one consumer across all the HornetQ clustered nodes?
Related
We are using WSO2 with RabbitMQ in our project. One requirement is that the consumer of RabbitMQ should generate the queue in case it did not previously exist.
We make the following proxy (without creating the "queue" queue previously in the Rabbit broker):
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="AMQPProxy3" startOnLoad="true" trace="enable" transports="rabbitmq" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<log level="full"/>
<property name="OUT_ONLY" scope="default" type="STRING" value="true"/>
<property name="FORCE_SC_ACCEPTED" scope="axis2" type="STRING" value="true"/>
<send>
<endpoint>
<address uri="http://localhost:8080/greeting"/>
</endpoint>
</send>
</inSequence>
<outSequence/>
<faultSequence/>
</target>
<parameter name="rabbitmq.queue.name">queue</parameter>
<parameter name="rabbitmq.connection.factory">AMQPConnectionFactory</parameter>
</proxy>
The ESB throws the following exception:
TID: [-1] [] [2017-08-25 13:23:06,139] ERROR {org.apache.axis2.transport.rabbitmq.ServiceTaskManager} - Error, Connection already closed AMQPProxy3, Listner id - 302 {org.apache.axis2.transport.rabbitmq.ServiceTaskManager}
com.rabbitmq.client.AlreadyClosedException: channel is already closed due to channel error; protocol method: #method(reply-code=404, reply-text=NOT_FOUND - no queue 'queue' in vhost '/', class-id=50, method-id=10)
In this case, the RabbitMQ consumer using the ESB does not create the queue in case it does not exist.
However, if we use a Java project that uses amqp-client-4.0.2.jar (Java library that gives us the official page of Rabbit) that consumer is able to create the queue.
package com.ing.rabbitmq;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class Send2 {
private final static String QUEUE_NAME = "queue";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = "{\"id\":100,\"content\":\"Manolito\"}";
channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
System.out.println(" [x] Sent '" + message + "'");
channel.close();
connection.close();
}
}
Could we get the ESB to create the queue just like if we used the java client (amqp-client-4.0.2.jar)?
I think it should be possible with WSO2 ESB.
Try setting out the following property in your proxy configuration.
<parameter name="rabbitmq.queue.autodeclare">true</parameter>
If that not works, please let me know the WSO2 ESB version you are using.
I am using Spring AMQP for processing the messages in RabbitMQ. Below is the issue:
1. (say) there are 3 messages in ready state inside RabbitMQ
2. First one is picked up by MessageListener and starts processing. (say) It ends up throwing an exception
3. In this case, the container remains up but the remaining 2 messages are not processed until i restart the container. Also the first messages stays in unacknowledged state.
Is it the expected behavior? If not, how to make sure that other 2 messages will be processed irrespective first one failed processing?
MQ configuraion:
<rabbit:connection-factory id="connectionFactory" host="localhost" username="guest" password="guest" />
<rabbit:admin connection-factory="connectionFactory" />
<rabbit:listener-container
connection-factory="connectionFactory"
concurrency="1"
acknowledge="auto">
<rabbit:listener queue-names="testQueue" ref="myProcessorListener " />
</rabbit:listener-container>
MessageListener class:
public class MyProcessorListener implements MessageListener{
....
#Override
public void onMessage(Message message) {
try{
...Some logic...
} catch (Exception e) {
throw new RuntimeException(e.getMessage(), e);
}
The message is redelivered over and over again; in order to reject it (and discard or route to a dead letter queue), you need to throw AmqpRejectAndDontRequeueException or set the container's requeue-rejected property to false. When configuring with Java it's defaultRequeueRejected.
You can also use a custom error handler.
This is all explained in the reference manual.
I have a simple publisher, which sends messages to a queue.
<int:channel id="publishChannel"/>
<int-jms:outbound-channel-adapter channel="publishChannel" destination="testQueue" session-transacted="true"/>
#Publisher(channel = "publishChannel")
public String sendMessage (String text) {
return text;
}
If the broker crashes, the publisher throws an MessageHandlingException.
Is it possible to block the publisher, till the broker is available again or to make a periodic retry?
You can add a retry advice to the outbound adapter.
<int-jms:outbound-channel-adapter channel="publishChannel" destination="testQueue" session-transacted="true>
<int:request-handler-advice-chain>
<ref bean="myRetryAdvice" />
</request-handler-advice-chain>
</int-jms:outbound-channel-adapter>
You can configure the advice with a backoff policy (e.g. exponential) and to take some action when retries are exhausted (rather than throwing the final exception).
I have this requirement to push messages to a single queue but receive them at more than one place (a java class).
For that I used a topic-exchange and bound it to a queue that receives messages based on different patterns. Now when I try to receive them at the other end the listener-container send the messages to both the listeners rather than bifurcating them based on the pattern decided.
I have attached the configuration and the code below for a quick look
<?xml version="1.0" encoding="UTF-8" ?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:rabbit="http://www.springframework.org/schema/rabbit"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit.xsd">
<rabbit:connection-factory id="connectionFactory" host="localhost" username="guest" password="guest" />
<rabbit:admin connection-factory="connectionFactory" />
<rabbit:template id="pgTemplate" connection-factory="connectionFactory" exchange="PG-EXCHANGE"/>
<rabbit:queue id="pgQueue" />
<rabbit:topic-exchange id="pgExchange" name="PG-EXCHANGE">
<rabbit:bindings>
<rabbit:binding queue="pgQueue" pattern="pg"></rabbit:binding>
<rabbit:binding queue="pgQueue" pattern="txn"></rabbit:binding>
</rabbit:bindings>
</rabbit:topic-exchange>
<bean id="pgReceiver" class="com.pg.mq.service.impl.MessageReceiver"/>
<bean id="txnReceiver" class="com.pg.mq.service.impl.MessageReceiver"/>
<rabbit:listener-container id="ListenerContainer" connection-factory="connectionFactory">
<rabbit:listener ref="pgReceiver" queues="pgQueue" method="handleMessage"/>
<rabbit:listener ref="txnReceiver" queues="pgQueue" method="handleMessage"/>
</rabbit:listener-container>
</beans>
Message Sender
public String pushMessagePG(Object object) {
if(object != null && object instanceof Rabbit)
pgTemplate.convertAndSend("pg", object); // send
return null;
}
Message Receivers
PG RECEIVER
public void handleMessage(Rabbit message) {
System.out.println("inside Message Receiver");
System.out.println("Listener received message----->" + message);
}
TXN RECEIVER
public void handleMessage(Rabbit message) {
System.out.println("inside TXN Message Receiver");
System.out.println("Listener received message----->" + message);
}
Calling Code
Sender sender = (Sender) context.getBean("messageSender");
for(int i = 0; i < 30; i++) sender.pushMessagePG(new Rabbit(5));
Thread.sleep(2000);
for(int i = 0; i < 30; i++) sender.pushMessageTXN(new Rabbit(2));
Sorry, but you didn't understand the AMQP specification a bit.
Messages are sent to the exchange by the routingKey
Queues are bound to the exchange by the routingKey or pattern for topic-exchange
Only single consumer can receive a message from queue.
There is no mechanism like distribution in the AMQP. I mean something similar like Topic concept in JMS.
topic-exchange puts messages to all queues which routingKey matches to patterns of the bindings.
This article describes that in the best way.
Since you bind the same queue for both patterns all messages are placed to the same queue.
From other consumers are listen on the queues and just only queues. They knows nothing about the routingKey.
Since you have two listeners for the same queue it isn't surprise that both of them can handle messages from that queue and it is independently of the routingKey.
You really should use different queues for different patterns. Or consider to use Spring Integration with its Message Routing power. You can use the standard AmqpHeaders.RECEIVED_ROUTING_KEY (Spring Integration) header for that case.
I wonder if anyone can help. My starter for 10 is that I know very little (next to nothing) about JMS and messaging in general - so please go easy with me with any answers/comments :)
Given that this is a learning exerise for me, I'm trying to put together a very basic Spring JMS config, and then write some integration tests to help me understand how it all works.
This is my current Spring context config with its JMS components:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:p="http://www.springframework.org/schema/p"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:jms="http://www.springframework.org/schema/jms"
xmlns:amq="http://activemq.apache.org/schema/core"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/jms
http://www.springframework.org/schema/jms/spring-jms.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd">
<bean class="com.lv.gi.jmstest.ApplicationContextProvider" />
<!-- Embedded ActiveMQ Broker -->
<amq:broker id="broker" useJmx="false" persistent="false">
<amq:transportConnectors>
<amq:transportConnector uri="tcp://localhost:0" />
</amq:transportConnectors>
</amq:broker>
<!-- ActiveMQ Destination -->
<amq:queue id="destination" physicalName="myQueueName" />
<!-- JMS ConnectionFactory to use, configuring the embedded broker using XML -->
<amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost" />
<!-- JMS Producer Configuration -->
<bean id="jmsProducerConnectionFactory"
class="org.springframework.jms.connection.SingleConnectionFactory"
depends-on="broker"
p:targetConnectionFactory-ref="jmsFactory" />
<bean id="jmsProducerTemplate" class="org.springframework.jms.core.JmsTemplate"
p:connectionFactory-ref="jmsProducerConnectionFactory"
p:defaultDestination-ref="destination" />
<bean class="com.lv.gi.jmstest.JmsMessageProducer">
<constructor-arg index="0" ref="jmsProducerTemplate" />
</bean>
<!-- JMS Consumer Configuration -->
<bean id="jmsConsumerConnectionFactory"
class="org.springframework.jms.connection.SingleConnectionFactory"
depends-on="broker"
p:targetConnectionFactory-ref="jmsFactory" />
<jms:listener-container container-type="default"
connection-factory="jmsConsumerConnectionFactory"
acknowledge="auto">
<jms:listener destination="myQueueName" ref="jmsMessageListener" />
</jms:listener-container>
<bean id="jmsMessageListener" class="com.lv.gi.jmstest.JmsMessageListener" />
</beans>
My JmsMessageProducer class has a postMessage method as follows:
public void postMessage(final String message) {
template.send(new MessageCreator() {
#Override
public Message createMessage(final Session session) throws JMSException {
final TextMessage textMessage = session.createTextMessage(message);
LOGGER.info("Sending message: " + message);
return textMessage;
}
});
}
And my JmsMessageListener (implements MessageListener) has an onMessage method as follows:
public void onMessage(final Message message) {
try {
if (message instanceof TextMessage) {
final TextMessage tm = (TextMessage)message;
final String msg = tm.getText();
LOGGER.info("Received message '{}'", msg);
}
} catch (final JMSException e) {
LOGGER.error(e.getMessage(), e);
}
}
In my test class, I can fire up the Spring context, get the JmsMessageProducer bean, and call postMessage; and I see the message on the console as expected:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = { "classpath:/com/lv/gi/jmstest/JmsMessageListenerTest-context.xml" })
public class TestJms {
private JmsMessageProducer jmsMessageProducer;
#Before
public void setup() {
jmsMessageProducer = ApplicationContextProvider.getApplicationContext().getBean(JmsMessageProducer.class);
}
#Test
public void doStuff() throws InterruptedException {
jmsMessageProducer.postMessage("message 1");
}
}
Whilst this works, its not really much of a test, because other than me visually seeing the message received on the console, I can't assert the message has been received.
We use mockito, so I'm wondering if there is a way within my test that I can replace the MessageListener bean with a mock, then call verify on it. I guess I could do it by providing a different Spring context file for the purpose of these tests, but that might not work well with my next requirement ...
My end goal with all of this is to create a Topic, where my message producer can fire a message onto the queue, and 1 of more MessageListeners will read the message off the queue, and when all registered listeners have read the message, the message is deleted from the queue. (I think Topic is the right terminology for this!)
To prove that this system would work, I would want a test where I can fire up the Spring context. The first thing I'd want to do is replace the listener with 3 mocks all wired to the same destination so that I can use verify on each of them. I'd post a message, then verify that each mock has received. Then I'd want to remove/disable 2 of the listeners, call postMessage, and verify the onMessage method was called on the one remaining mock listener. Then perhaps wait a while, and re-establish the 2 mocks, and verify their onMessage method was called. And finally, check the message is no longer on the queue (by virtue of the fact that the 3 registered listeners had all received the message)
With the above in mind, I think what I'm trying to do is register and de-register (or disable) listeners against the destination at runtime, and if I can do that, then I can register mocks.
Phew!, that's complicated!, but I hope it makes sense as to what I'm trying to do?
Any pointers as to how to achieve this? Many thanks in advance,
Nathan
In my opinion, as soon as you are doing integration testing, you should not try to mock anything.
On one hand you write unit tests. For example, you could test the behaviours of your consumer, by calling the onMessage method of jmsMessageListener directly from your tests. You typically don't use SpringJUnit4ClassRunner for this kind of test, and you typically use Mockito to mock the dependencies of the object you are testing.
On the other hand you have integration tests. These are testing the behaviour of your entire application. It would make sense to use SpringJUnit4ClassRunner.class here, but not Mockito. You should test that whatever your jmsListener was supposed to do has been done. For example, if your application was supposed to write logs about the incoming message, open the log file and check it.
Your example here is very simple, maybe this is why you are confused (in my opinion). With a more complex listener, it would be more natural to it isolated from the rest of the system.