Given the fact that the only requirement I have is to use an instance of ActiveMQ, how would I make my ActiveMQ to use a JDBC connection without creating a embedded one with VM transport.
This is my factory bean:
#Bean
public ActiveMQConnectionFactory connectionFactory() {
logger.info("ActiveMQConnectionFactory");
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory(brokerUrl);
activeMQConnectionFactory.setTrustAllPackages(true);
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setRedeliveryDelay(15000);
redeliveryPolicy.setMaximumRedeliveries(-1);
activeMQConnectionFactory.setRedeliveryPolicy(redeliveryPolicy);
return activeMQConnectionFactory;
}
I have an image of activemq exposed in the URL: tcp//0.0.0.0:61616 but even with the JDBC adapter configured as shown below I'm not able to persist messages in the SQL server, activemq is ingoring this and use the KahaDB as default. The only way found to use the jdbc is to change from tcp to vm:localhost but this creates an embedded activemq.
#Bean
public BrokerService broker(DataSource dataSource, ActiveMQConnectionFactory activeMQConnectionFactory) throws Exception {
logger.info("BrokerService");
final BrokerService broker = new BrokerService();
JDBCPersistenceAdapter jdbc = new JDBCPersistenceAdapter(dataSource, new OpenWireFormat());
jdbc.setUseLock(true);
Statements statements = jdbc.getStatements();
statements.setBinaryDataType(BINARY_DATA_TYPE);
broker.setUseJmx(true);
broker.setPersistent(true);
broker.setPersistenceAdapter(jdbc);
broker.addConnector(format("vm:(broker:(tcp://localhost:61616,network:static:%s)?persistent=true)", brokerUrl));
logger.info("BrokerService URL: " + broker.getTransportConnectors().get(0).getConnectUri().toString());
return broker;
}
Recommend using xml file to configure the broker. Easier to manage vs coding up a broker.
ActiveMQ JDBC info:
ref: https://activemq.apache.org/jdbc-support
Persistence Adapter info:
ref: https://activemq.apache.org/persistence
Starting an embedded broker referencing an xml file:
ref: https://activemq.apache.org/vm-transport-reference
specifically:
vm://localhost?brokerConfig=xbean:activemq.xml
Related
I am listening to the queue in my application like this
#JmsListener(containerFactory = "MessageListener", destination = "${mq.destination-name}")
My connection factory Bean looks like this
#Bean(name = "MessageListener")
public JmsListenerContainerFactory<?> mqMessageListenerContainer(
ConnectionFactory connectionFactory, QueueErrorHandler QueueErrorHandler)
{
SimpleJmsListenerContainerFactory simpleJmsListenerContainerFactory = new SimpleJmsListenerContainerFactory();
simpleJmsListenerContainerFactory.setConnectionFactory(connectionFactory);
simpleJmsListenerContainerFactory.setErrorHandler(QueueErrorHandler);
return simpleJmsListenerContainerFactory;
}
I want to understand when I don't have
simpleJmsListenerContainerFactory.setSessionTransacted(true);
simpleJmsListenerContainerFactory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
these set in factory configuration when is my application starts a transaction on reading a message and when does it commits ?
I'm not sure if what I'm trying to do is possible but basically I'm trying to update the current implementation from HornetQ to ActiveMQ making use of Artemis.
My system is a JMS consumer from the HornetQ.
The current implementation works if I use "HornetQJMSConnectionFactory" but when I change to ActiveMQJMSConnectionFactory it is not able to connect.
In order to test the new implementation, I've spun up a local instance of ActiveMQ and works with the new implementation.
So I've tried multiple different things including forcing protocol =HORNETQ and nothing works.
No compilation error, "only":
ERROR o.s.j.l.DefaultMessageListenerContainer.refreshConnectionUntilSuccessful - Could not refresh JMS Connection for destination 'QueueX' - retrying using FixedBackOff{interval=5000, currentAttempts=1, maxAttempts=unlimited}. Cause: Failed to create session factory; nested exception is ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=AMQ219013: Timed out waiting to receive cluster topology. Group:null]
Old Implementation
private ConnectionFactory createConnectionFactory(SyncProperties.SmpJmsServer jmsServer) {
final String className = "org.hornetq.core.remoting.impl.netty.NettyConnectorFactory";
Map<String, Object> params = new HashMap<String, Object>();
params.put("host", getJmsHost());
params.put("port", getJmsPort());
TransportConfiguration transportConfiguration = new TransportConfiguration(className, params);
HornetQJMSConnectionFactory hornetQJMSConnectionFactory = new HornetQJMSConnectionFactory(false, transportConfiguration);
hornetQJMSConnectionFactory.setConnectionTTL(300000);
hornetQJMSConnectionFactory.setConsumerWindowSize(0);
UserCredentialsConnectionFactoryAdapter adapter = new UserCredentialsConnectionFactoryAdapter();
adapter.setTargetConnectionFactory(hornetQJMSConnectionFactory);
adapter.setUsername(getJmsUsername());
adapter.setPassword(getJmsPassword());
CachingConnectionFactory smpCachingConnectionFactory = new CachingConnectionFactory(adapter);
return smpCachingConnectionFactory;
}
New Implementation
public ConnectionFactory createActiveMQJMSConnectionFactory() {
ActiveMQJMSConnectionFactory activeMQJMSConnectionFactory = new ActiveMQJMSConnectionFactory(false, amqTransportConfiguration());
activeMQJMSConnectionFactory.setConnectionTTL(300000);
activeMQJMSConnectionFactory.setConsumerWindowSize(0);
UserCredentialsConnectionFactoryAdapter adapter = new UserCredentialsConnectionFactoryAdapter();
adapter.setTargetConnectionFactory(activeMQJMSConnectionFactory);
adapter.setUsername(getJmsUsername());
adapter.setPassword(getJmsPassword());
CachingConnectionFactory smpCachingConnectionFactory = new CachingConnectionFactory(adapter);
return smpCachingConnectionFactory;
}
#Bean("amqTransportConfiguration")
public TransportConfiguration amqTransportConfiguration() {
return new TransportConfiguration("org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory", getParams());
}
static Map<String, Object> getParams() {
Map<String, Object> params = new HashMap<String, Object>();
params.put("host", getJmsHost());
params.put("port", getJmsPort());
return params;
}
Thanks for the help.
Although ActiveMQ Artemis is based on the HornetQ code-base an ActiveMQ Artemis client won't be able to talk with a HornetQ broker. Each client sends a protocol/client identifier when it connects. This ID is different between ActiveMQ Artemis clients and HornetQ clients. A HornetQ broker will not recognize the ID sent by the ActiveMQ Artemis client and therefore will not complete a handshake.
That said, we have worked to ensure that HornetQ clients can still talk to an ActiveMQ Artemis broker. That's what the org.apache.activemq.artemis.core.protocol.hornetq.HornetQProtocolManager provides.
In any case, upgrading your client won't really do much for you anyway. If you want to upgrade anything I recommend you upgrade EAP or even move to a standalone version of ActiveMQ Artemis so you can get the latest fixes & features.
I am designing a simple system where the flow is going to be like this:
Message Producer Microservice --> Active MQ --> Message Consumer Microservice --> Mongo DB
I need to design a queuing strategy in a way so that if MongoDB is down, I should not lose the message (because Message consumer will dequeue the message).
My consumer is written like this:
#JmsListener(destination = "Consumer.myconsumer.VirtualTopic.Tracking")
public void onReceiveFromQueueConsumer2(TrackingRequest trackingRequest) {
log.debug("Received tracking request from the queue by consumer 2");
log.debug(trackingRequest.toString());
}
How do you provide client acknowledgement?
You can use client acknowledge mode from your "Message Consumer Microservice." Since you're using a Spring JmsListener you can define the listener container using the containerFactory and then you can set the mode you want on your listener container using sessionAcknowledgeMode. See the Spring documentation for more details on what ack mode you might want to use here.
From the perspective of the ActiveMQ client you can configure redelivery semantics however you like in case of a failure. See the ActiveMQ documentation for more about that.
Alright, so I was able to solve this dilemma, here is what your config should be like (thanks to Justin for his valuable inputs):
#Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(brokerUrl);
connectionFactory.setPassword(userName);
connectionFactory.setUserName(password);
connectionFactory.setTrustAllPackages(true);
connectionFactory.setRedeliveryPolicy(redeliveryPolicy());
return connectionFactory;
}
#Bean
public JmsTemplate jmsTemplate() {
JmsTemplate template = new JmsTemplate();
template.setConnectionFactory(connectionFactory());
return template;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory listenerCF = new DefaultJmsListenerContainerFactory();
listenerCF.setConnectionFactory(connectionFactory());
listenerCF.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
listenerCF.setSessionTransacted(true);
return listenerCF;
}
#Bean
public RedeliveryPolicy redeliveryPolicy() {
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setRedeliveryDelay(600000L); //keep trying every 10 minutes
redeliveryPolicy.setMaximumRedeliveries(-1); //Keep trying till its successfully inserted
return redeliveryPolicy;
}
I've created a Spring JMS application using version 4.1.2.RELEASE, which is connected to a broker that is running ActiveMQ 5.11.0. The problem that I'm seeing is as follows. In the logs, I notice that every second, I'm seeing a connection being created as such.
2017-06-21 13:10:21,046 | level=INFO | thread=ActiveMQ Task-1 | class=org.apache.activemq.transport.failover.FailoverTransport | Successfully connected to tcp://localhost:61616
I know that it is creating a new ActiveMQ connection each time, because it says successfully "connected" and not "reconnected" as shown in the code located here: http://grepcode.com/file/repo1.maven.org/maven2/com.ning/metrics.collector/1.3.3/org/apache/activemq/transport/failover/FailoverTransport.java#891
I don't have a caching connection factory set for my consumer, but I'm wondering if the following is the culprit when it comes to why I'm seeing constant connections being created.
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
The following post states that consumers should not be cached, but I wonder if that applies to caching the connection + session. If the connection is cached, but the session is not, then I wonder if that creates a problem.
Why DefaultMessageListenerContainer should not use CachingConnectionFactory?
The following are the configurations that I'm using in my application. I am hoping that it is something that I've misconfigured, and would appreciate any insights that anyone has to offer.
Spring Configurations
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() throws Throwable {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
factory.setMaxMessagesPerTask(-1);
factory.setConcurrency(1);
factory.setSessionTransacted(true);
return factory;
}
#Bean
public CachingConnectionFactory cachingConnectionFactory(){
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(connectionFactory());
cachingConnectionFactory.setCacheConsumers(false);
cachingConnectionFactory.setSessionCacheSize(1);
return cachingConnectionFactory;
}
#Bean
public ActiveMQConnectionFactory connectionFactory(){
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setInitialRedeliveryDelay(1000L);
redeliveryPolicy.setRedeliveryDelay(1000L);
redeliveryPolicy.setMaximumRedeliveries(6);
redeliveryPolicy.setUseExponentialBackOff(true);
redeliveryPolicy.setBackOffMultiplier(5);
ActiveMQConnectionFactory activeMQ = new ActiveMQConnectionFactory("admin", "admin", "tcp://localhost:61616");
activeMQ.setRedeliveryPolicy(redeliveryPolicy);
activeMQ.setPrefetchPolicy(prefetchPolicy());
return activeMQ;
}
#Bean
public JmsMessagingTemplate jmsMessagingTemplate(){
ActiveMQTopic activeMQ = new ActiveMQTopic("topic.out");
JmsMessagingTemplate jmsMessagingTemplate = new JmsMessagingTemplate(cachingConnectionFactory());
jmsMessagingTemplate.setDefaultDestination(activeMQ);
return jmsMessagingTemplate;
}
protected ActiveMQPrefetchPolicy prefetchPolicy(){
ActiveMQPrefetchPolicy prefetchPolicy = new ActiveMQPrefetchPolicy();
int prefetchValue = 1000;
prefetchPolicy.setQueuePrefetch(prefetchValue);
return prefetchPolicy;
}
Thanks,
Juan
The issue was indeed the following code.
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
The moment that I removed it, the rapid connection creation stopped.
Im having a SpringBoot application which consume my custom serializable message from ActiveMQ Queue. So far it is worked, however, the consume rate is very poor, only 1 - 20 msg/sec.
#JmsListener(destination = "${channel.consumer.destination}", concurrency="${channel.consumer.maxConcurrency}")
public void receive(IMessage message) {
processor.process(message);
}
The above is my channel consumer class's snippet, it has a processor instance (injected, autowired and inside it i have #Async service, so i can assume the main thread will be released as soon as message entering #Async method) and also it uses springboot activemq default conn factory which i set from application properties
# ACTIVEMQ (ActiveMQProperties)
spring.activemq.broker-url= tcp://localhost:61616?keepAlive=true
spring.activemq.in-memory=true
spring.activemq.pool.enabled=true
spring.activemq.pool.expiry-timeout=1
spring.activemq.pool.idle-timeout=30000
spring.activemq.pool.max-connections=50
Few things worth to inform:
1. I run everything (Eclipse, ActiveMQ, MYSQL) in my local laptop
2. Before this, i also tried using custom connection factory (default AMQ, pooling, and caching) equipped with custom threadpool task executor, but still getting same result. Below is a snapshot performance capture which i took and updating every 1 sec
3. I also notive in JVM Monitor that the used heap keep incrementing
I want to know:
1. Is there something wrong/missing from my steps?I can't even touch hundreds in my message rate
2. Annotated #JmsListener method will execute process async or sync?
3. If possible and supported, how to use traditional sync receive() with SpringBoot properly and ellegantly?
Thank You
I'm just checking something similar. I have defined DefaultJmsListenerContainerFactory in my JMSConfiguration class (Spring configuration) like this:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(CachingConnectionFactory connectionFactory) {
// settings made based on https://bsnyderblog.blogspot.sk/2010/05/tuning-jms-message-consumption-in.html
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory(){
#Override
protected void initializeContainer(DefaultMessageListenerContainer container) {
super.initializeContainer(container);
container.setIdleConsumerLimit(5);
container.setIdleTaskExecutionLimit(10);
}
};
factory.setConnectionFactory(connectionFactory);
factory.setConcurrency("10-50");
factory.setCacheLevel(CACHE_CONSUMER);
factory.setReceiveTimeout(5000L);
factory.setDestinationResolver(new BeanFactoryDestinationResolver(beanFactory));
return factory;
}
As you can see, I took those values from https://bsnyderblog.blogspot.sk/2010/05/tuning-jms-message-consumption-in.html. It's from 2010 but I could not find anything newer / better so far.
I have also defined Spring's CachingConnectionFactory Bean as a ConnectionFactory:
#Bean
public CachingConnectionFactory buildCachingConnectionFactory(#Value("${activemq.url}") String brokerUrl) {
// settings based on https://bsnyderblog.blogspot.sk/2010/02/using-spring-jmstemplate-to-send-jms.html
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(brokerUrl);
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(activeMQConnectionFactory);
cachingConnectionFactory.setSessionCacheSize(10);
return cachingConnectionFactory;
}
This setting will help JmsTemplate with sending.
So my answer to you is set the values of your connection pool like described in the link. Also I guess you can delete spring.activemq.in-memory=true because (based on documentation) in case you specify custom broker URL, "in-memory" property is ignored.
Let me know if this helped.
G.