JMS queue static snapshot - jms

I need to browse a JMS queue and filter it based on how many message of particular criteria exists.
But the problem is in JBoss EAP, while browsing the queue, if new messages comes it is also considered in browse which make the process to run so long because this application is continuously getting a lot of messages.
Basically need to understand whether I can get static snapshot of the queue so that I can scan through message without considering the new & upcoming messages.
PS: This was working fine in Weblogic server.
Here's the browser code:
Context namingContext = null;
try {
String userName = System.getProperty("username", DEFAULT_USERNAME);
String password = System.getProperty("password", DEFAULT_PASSWORD);
// Set up the namingContext for the JNDI lookup
final Properties env = new Properties();
env.put(Context.INITIAL_CONTEXT_FACTORY, INITIAL_CONTEXT_FACTORY);
env.put(Context.PROVIDER_URL, System.getProperty(Context.PROVIDER_URL, PROVIDER_URL));
env.put(Context.SECURITY_PRINCIPAL, userName);
env.put(Context.SECURITY_CREDENTIALS, password);
namingContext = new InitialContext(env);
// Perform the JNDI lookups
String connectionFactoryString = System.getProperty("connection.factory", DEFAULT_CONNECTION_FACTORY);
ConnectionFactory connectionFactory = (ConnectionFactory) namingContext.lookup(connectionFactoryString);
try (JMSContext context = connectionFactory.createContext(userName, password)) {
Queue queue = (Queue) namingContext.lookup("jms/ubsexecute");
QueueBrowser browser = context.createBrowser(queue);
Enumeration enumeration = browser.getEnumeration();
int i =1;
while (enumeration.hasMoreElements()) {
Object nextElement = enumeration.nextElement();
System.out.println("Read a message " + i++);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} catch (NamingException e) {
log.severe(e.getMessage());
e.printStackTrace();
} finally {
if (namingContext != null) {
try {
namingContext.close();
} catch (NamingException e) {
log.severe(e.getMessage());
}
}
}
} catch (Exception e) {
// TODO: handle exception
}

As noted in the JavaDoc for javax.jms.QueueBrowser:
Messages may be arriving and expiring while the scan is done. The JMS API does not require the content of an enumeration to be a static snapshot of queue content. Whether these changes are visible or not depends on the JMS provider.
The content of the enumeration provided by the queue browser from the JMS provider in JBoss EAP is not static and there is no way to force it to be.
Since the behavior your looking for is not guaranteed by JMS I recommend you adjust your application so that it doesn't rely upon such behavior.
A couple of alternatives come to mind:
Set an upper limit on how many messages the browser will inspect.
Use a provider-specific management call to get the number of messages in the queue before the browser is created and then only browse through that number of messages.

Related

Replay/synchronous messages memory not released for messages sent by producers to queue in SpringBoot JMS with ActiveMQ

1. Context:
A two-modules/microservice application developed with SpringBoot 2.3.0 and ActiveMQ.
Also we use ActiveMQ 5.15.13 server/broker.
Broker is defined in both modules with application properties.
Also broker connection pool is defined in both modules as well with application properties and added in both modules the pooled-jms artifact dependency (with maven):
spring.activemq.broker-url=xxx
spring.activemq.user=xxx
spring.activemq.password=xx
spring.activemq.non-blocking-redelivery=true
spring.activemq.pool.enabled=true
spring.activemq.pool.time-between-expiration-check=5s
spring.activemq.pool.max-connections=10
spring.activemq.pool.max-sessions-per-connection=10
spring.activemq.pool.idle-timeout=60s
Other configurations for JMS I done are:
spring.jms.listener.acknowledge-mode=auto
spring.jms.listener.auto-startup=true
spring.jms.listener.concurrency=5
spring.jms.listener.max-concurrency=10
spring.jms.pub-sub-domain=false
spring.jms.template.priority=100
spring.jms.template.qos-enabled=true
spring.jms.template.delivery-mode=persistent
In module 1 the JmsTemplate is used to send synchronous messages (or we can name replay-messages as well). I've opted out for a proper queue instead of a temporary queue as I understand that if there are lots of messages sent than a temporary queue is not recommended to be used for replays - so that's what I did.
2. Code samples:
MODULE 1:
#Value("${app.request-video.jms.queue.name}")
private String requestVideoQueueNameAppProperty;
#Bean
public Queue requestVideoJmsQueue() {
logger.info("Initializing requestVideoJmsQueue using application property value for " +
"app.request-video.jms.queue.name=" + requestVideoQueueNameAppProperty);
return new ActiveMQQueue(requestVideoQueueNameAppProperty);
}
#Value("${app.request-video-replay.jms.queue.name}")
private String requestVideoReplayQueueNameAppProperty;
#Bean
public Queue requestVideoReplayJmsQueue() {
logger.info("Initializing requestVideoReplayJmsQueue using application property value for " +
"app.request-video-replay.jms.queue.name=" + requestVideoReplayQueueNameAppProperty);
return new ActiveMQQueue(requestVideoReplayQueueNameAppProperty);
}
#Autowired
private JmsTemplate jmsTemplate;
public Message callSendAndReceive(TextJMSMessageDTO messageDTO, Destination jmsDestination, Destination jmsReplay) {
return jmsTemplate.sendAndReceive(jmsDestination, jmsSession -> {
try {
TextMessage textMessage = jmsSession.createTextMessage();
textMessage.setText(messageDTO.getText());
textMessage.setJMSReplyTo(jmsReplay);
textMessage.setJMSCorrelationID(UUID.randomUUID().toString());
textMessage.setJMSDeliveryMode(DeliveryMode.NON_PERSISTENT);
return textMessage;
} catch (IOException e) {
logger.error("Error sending JMS message to destination: " + jmsDestination, e);
throw new JMSException("Error sending JMS message to destination: " + jmsDestination);
}
});
}
MODULE 2:
#JmsListener(destination = "${app.backend-get-request-video.jms.queue.name}")
public void onBackendGetRequestsVideoMessage(TextMessage message, Session session) throws JMSException, IOException {
logger.info("Get requests video file message consumed!");
try {
Object replayObject = handleReplayAction(message);
JMSMessageDTO messageDTO = messageDTOFactory.getJMSMessageDTO(replayObject);
Message replayMessage = messageFactory.getJMSMessage(messageDTO, session);
BytesMessage replayBytesMessage = jmsSession.createBytesMessage();
fillByteMessageFromMediaDTO(replayBytesMessage, mediaMessageDTO);
replayBytesMessage.setJMSCorrelationID(message.getJMSCorrelationID());
final MessageProducer producer = session.createProducer(message.getJMSReplyTo());
producer.send(replayBytesMessage);
JmsUtils.closeMessageProducer(producer);
} catch (JMSException | IOException e) {
logger.error("onBackendGetRequestsVideoMessage()JMSException: " + e.getMessage(), e);
throw e;
}
}
private void fillByteMessageFromMediaDTO(BytesMessage bytesMessage, MediaJMSMessageDTO mediaMessageDTO)
throws IOException, JMSException {
String filePath = fileStorageConfiguration.getMediaFilePath(mediaMessageDTO);
FileInputStream fileInputStream = null;
try (FileInputStream fileInputStream = new FileInputStream(filePath)) {
byte[] byteBuffer = new byte[1024];
int bytes_read = 0;
while ((bytes_read = fileInputStream.read(byteBuffer)) != -1) {
bytesMessage.writeBytes(byteBuffer, 0, bytes_read);
}
} catch (JMSException e) {
logger.error("Can not write data in JMS ByteMessage from file: " + fileName, e);
} catch (FileNotFoundException e) {
logger.error("Can not open stream to file: " + fileName, e);
} catch (IOException e) {
logger.error("Can not read data from file: " + fileName, e);
}
}
3. The problem:
As I send many messages and receive many corresponding replays through producer/comsumer/JmsTamplate both application modules 1 and 2 are fast-filling the heap memory allocated until an out-of-memory error is thrown, but the memory leak appears only when using synchronous messages with replay as shown above.
I've debugged my code and all instances (session, producers, consumers, jmsTamplate, etc) are pooled and have instances of the right classes from pooled-jms library; so pool should - apparently - work properly.
I've made a heap dump of the second module and looks like producers messages (ActiveMQBytesMessage) are still in memory even long time after have been successfully consumed by the right consumer.
I have asynchronous messages sent as well in my modules and seams that those messages producer-consumer works well; the problem is present only for the synch/replay messages producer-consumer.
Sample heap dump files - taken after full night of application inactivity - as following:
module 1
module_1_dump
module 2
module_2_dump
activemq broker/server
activemq_dump
Anyone have any idea what I'm doing wrong?!

JMS Message Persistence on ActiveMQ

I need to ensure redelivery of JMS messages when the consumer fails
The way the producer is set up now - DefaultJmsListenerContainerFactory and Session.AUTO_ACKNOWLEDGE
I'm trying to build a jar and try in here to save the message into the server, once the app is able to consume, the producer in the jar will produce the message to the app.
Is that a good approach to do so?! any other way/recommendation to improve this?
public void handleMessagePersistence(final Object bean) {
ObjectMapper mapper = new ObjectMapper();
final String beanJson = mapper.writeValueAsString(bean); // I might need to convert to xml instead
// parameterize location of persistence folder
writeToDriver(beanJson);
try {
Producer.produceMessage(beanJson, beanJson, null, null, null);
} catch (final Exception e) {
LOG.error("Error producing message ");
}
}
here what I have to writ out the meesage:
private void writeToDriver(String beanJson) {
File filename = new File(JMS_LOCATION +
LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSS")) + ".xml");
try (final FileWriter fileOut = new FileWriter(filename)) {
try (final BufferedWriter out = new BufferedWriter(fileOut)) {
out.write(beanJson);
out.flush();
}
} catch (Exception e) {
LOG.error("Unable to write out : " + beanJson, e);
}
}

MQ MessageConsumer does not respond to receive() method

I have a java program I run to write messages to Mid-Tier IBM MQ's to test functionality before attaching our main programs to them. The write method looks like the following below:
private static void sendSingleMessage(ConnectionFactory connectionFactory,
String[] messages, String destination) throws Exception {
Connection connection = null;
try {
connection = connectionFactory.createConnection();
for (String payload : messages) {
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue(destination);
MessageProducer producer = session.createProducer(queue);
Message msg = session.createTextMessage(payload);
System.out.println("Sending text '" + payload + "'");
producer.send(msg);
session.close();
System.out.println("Message sent");
}
} finally {
if (connection != null) {
connection.close();
}
}
}
The connectionFactory is setup before this method executes, but within that method I set the MQConncetionFactory properties(host,port,channel,queuemanager, etc...) This send method works and I can see the queue depth increasing on my IBM MQ Explorer when I call it from my main method.
When I run a similar readSingleMessage method, the code gets stuck on the consumer.receive() and never finishes executing. See below:
private static void readSingleMessage(ConnectionFactory connectionFactory,
String[] messages, String destination) throws Exception {
Connection connection = null;
try {
connection = connectionFactory.createConnection();
for (String payload : messages) {
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue(destination);
MessageConsumer consumer = session.createConsumer(queue);
System.out.println("Recieving text '" + payload + "'");
consumer.receive();
session.close();
System.out.println("Received message");
}
} finally {
if (connection != null) {
connection.close();
}
}
}
Is there anyway I can further debug this, or find why I am able to write to the queue, but unable to read a message off of it?
You have to start the JMS Connection by calling the start() method on it. You cannot receive any messages until the connection is started. This is noted in the JMS Specification and Javadoc.
As an aside, if you use the JMS 2.0 "simplified" API and create a JMSContext object (an object which is essentially a combined Connection and Session) you do not need to call start to receive messages. A consumer crated from it can be used to receive messages without being explicitly started.

activemq consumer does not return data even when queue not empty

I wrote a sample code to add elements to activemq, and then retrieve them. I was successfully able to add around 1000 elements, but while retrieving the elements, somehow code gets stuck after retrieving around 50 - 200 elements, even when the queue has a lot of elements.
Following is the code i used for adding elements to the queue
#POST
#Path("/addelementtoqueue")
#Consumes(MediaType.APPLICATION_FORM_URLENCODED)
public void addElementToQeueue(#FormParam("count") int count) throws Exception {
IntStream.range(0, count)
.forEach(e -> {
try {
addElement(e);
}catch(Exception e1) {
throw new RuntimeException(e1);
}
});
}
private void addElement(int i) throws Exception {
Connection conn = GlobalConfiguration.getJMSConnectionFactory().createConnection();
conn.start();
Session session = conn.createSession(false, Session.CLIENT_ACKNOWLEDGE);
MessageProducer prod = session.createProducer(queue);
prod.send(queue, session.createTextMessage("message "+ i), DeliveryMode.PERSISTENT, 4, 0);
prod.close();
session.close();
conn.close();
}
and this is the snippet i am using for retrieving elements from the queue
#POST
#Path("/removeelementfromqueue")
#Consumes(MediaType.APPLICATION_FORM_URLENCODED)
public void removeElementToQeueue(#FormParam("count") int count) throws Exception {
IntStream.range(0, count)
.forEach(e -> {
try {
extractElement();
}catch(Exception e1) {
throw new RuntimeException(e1);
}
});
}
private void extractElement() throws Exception {
Connection conn = GlobalConfiguration.getJMSConnectionFactory().createConnection();
conn.start();
Session session = conn.createSession(false, Session.CLIENT_ACKNOWLEDGE);
queue = session.createQueue("walkin.testing");
MessageConsumer consumer = session.createConsumer(queue);
TextMessage msg = (TextMessage)consumer.receive();
System.out.println(msg.getText());
msg.acknowledge();
consumer.close();
session.close();
conn.close();
}
I am getting the connection factory via resource.xml, the snippet for the same is
<resources>
<Resource id="MyJmsResourceAdapter" type="ActiveMQResourceAdapter">
BrokerXmlConfig = jdbcBroker:(tcp://0.0.0.0:61616)
ServerUrl = tcp://0.0.0.0:61616?jms.prefetchPolicy.queuePrefetch=0
</Resource>
<Resource id="MyJmsConnectionFactory" type="javax.jms.ConnectionFactory">
ResourceAdapter = MyJmsResourceAdapter
</Resource></resources>
I am using activeMQ 5.13.1, with apache-tomee-plus-1.7.2 and Java 8, jdbc store as mysql. I have configured activemq-jdbc-performance.xml as the configuration file for apache activemq.
I have tried to a lot of research on this one, but i am unable to identify the root cause of this issue. It would be highly helpful, if any one can suggest me what i am doing wrong
I recommend against opening/closing connections/sessions/queues for every operation, but instead can use a pool to minimize how many of each resource is needed. Pretty sure that connections are thread-safe but that sessions are not and you need to create/use/dedicate a session per each active thread. By pooling, you can minimize the number of sessions created to however active threads are currently running and reuse them later.
So I'm assuming that you're having a resource problem, that even though it appears that everything's getting closed/released correctly, something isn't (perhaps outside the code I see here). Have you checked the activemq logs? Have you debugged through this and made sure that it's not hanging when trying to create the n'th connection or session?

Correct usage of JMS-Topic communication

I want to use JMS (Topic) in my JavaEE 6 project. I have one class which acts as a publisher and subscriber of a topic at once. The following code shows the most important parts of the class.
public class MessageHandler implements MessageListener {
private static TopicConnectionFactory factory;
private static Topic topic;
private TopicSubscriber subscriber;
private TopicPublisher publisher;
public MessageHandler() throws NamingException, JMSException {
if (factory == null) {
Context context = new InitialContext();
factory = (TopicConnectionFactory) new InitialContext()
.lookup("jms/myfactory");
topic = (Topic) context.lookup("jms/mytopic");
}
TopicConnection connection = factory.createTopicConnection();
connection.start();
TopicSession session = connection
.createTopicSession(false, Session.AUTO_ACKNOWLEDGE);
subscriber = session.createSubscriber(topic);
}
#Override
public void onMessage(Message message) {
try {
ObjectMessage msg = (ObjectMessage) message;
Object someO= msg.getObject();
System.out.println(this + " receives "+someO);
} catch (JMSException e) {
e.printStackTrace();
}
}
public void sendMessage(Object someO) {
try {
ObjectMessage msg = session.createObjectMessage();
msg.setObject(someO);
publisher = session.createPublisher(topic);
publisher.publish(msg);
publisher.close();
} catch (JMSException e) {
e.printStackTrace();
}
}
}
My question is, if this is a good way to design such a class. My idea was to share one connection and session for both subscribing and publishing. But I'm scared that this could lead to some overhead or blocking because I'm not closing the connection, session, subscriber and publisher until the object is not needed anymore. All examples I found online directly close everything after a message was sent or received...
Thanks in advance!
Why do you want the class to be subscriber and publisher at once?
Whenever using a messaging system, you may well act as both, but why would you do it for the same topic, you surely don't want to receive your own messages?
So, the purpose of a topic, is to be used among several parts within an application or among several applications - one is placing a message into the topic and others receive the message they subscribed for.
And that also explains what you saw in the examples - the message processing is a one time thing, thus the connection can be closed afterwards.
By the way, since you ask this question within the "java-ee 6" area - can't you use a message driven bean, annotate your topic configuration and let the application server do the infrastructure part for you?

Resources