I am using weblogic 10.3 .
I am trying to configure a durable subscription with persistent messaged backed by a jdbc store (in Oracle DB). I have a topic to which an MDB is listening as a durable subscriber.
Under scenario-1 : If I send the message, it hits the MDB.
Under scenario-2 : I suspend the MDB hoping that the messages send to the topic will stay around as long as they don’t get consumed by the MDB (which is the only registered durable subscriber). But when I do send the message to the topic, it briefly shows up there and then it goes away (i see it using HermesJMS).
I was under the impression that the messages since they are not being consumed by the MDB will get logged into the JDBC store in this case but the the WLSTORE table in oracle db does not get any messages persisted in it either.
I later found out that the messages do show up in the topic > durable subscribers > Show messages in the admin console.
So apparantly what’s happening is topic doesn’t keep the message, but the durable subscription that’s registered under it, keeps it until the message does get consumed.
Question -1) But the fact that messages are not going to the Oracle based jdbc store, still doesn’t make sense ? What am I doing wrong here ?
Question -2) Even when I do resume the MDB to start listening for the messages, the topic keeps showing all the message still intact under the durable subscribers (in admin console) - I was hoping they will get removed from there as they got processed by the only registered durable subscriber.
Messages do not show up in the topic, since topics and queues are different communication models. Assume you have a durable topic with 2 (durable) subscribers: A and B. You want both of them to get the message. To ensure that, A and B both have to confirm they received the message.
This is also the reason why you get all message redelivered, after reconnecting, your MDB has to call commit() on the message, to tell the server it is done processing.
This also explains why the topic itself does not store messages, they are stored per durable consumer. Because A might commit the message, but B will not (might be "offline"). So you need a copy for each consumer.
I found a couple of interesting things in further testing -
for Question 1) _ Even if I don't configure the jdbc store for the JMS server in weblogic 10, it has its own default file store which is always working without any configuration. This file store is used for storing the persistent messages for durable subscriptions and that store will keep the messages around even across server restarts.
For more reading - http://docs.oracle.com/cd/E17904_01/web.1111/e13701/store.htm#i1130575
for Question 2)_ My MDB was expecting XML messages in certain format, during my testing to keep things simple, i started sending small text messages and forgot that they will not get processed successfully in the MDB. so the MDB was failing on all those messages, and rolling back the transaction which is why the messages were never getting removed when I resumed the MDB.
This pretty much answers both my questions.
Related
I have 2 systems where System A has to send messages to System B. I am new to JMS so I don't have a big idea on how to implement this. I was thinking of using a message broker (ActiveMQ) to send messages. So, System A will send messages to a queue and the Message Listener in B will consume those messages. There are many users in System B and I want these messages to be shown whenever the user log into the system. So my problem is, if System B keeps consuming messages even when the users are not logged in how can they see the messages which are already consumed?
Should I store the consumed messages in a database? I don't understand how this works.
You could let SystemB read all the messages and act as a gateway between JMS and the system itself, i.e. store each user message in the database and when a user logs in, read those messages from the database and display them. If the user has to acknowledge they've read each message that might be a better solution as you can then track if they've read them all and delete each one from the database as they acknowledge they've read it.
Another solution might be virtual topics and queues. A single topic that messages are sent to that is split into queues, one per user. When a user logs in, SystemB reads from that user's queue. This is separate from the application's domain (it's JMS at this point) so the message is marked as consumed and is taken off the queue by ActiveMQ. If the user doesn't read it and needs to see it the next time they login then you need the database solution.
It's essentially where two domains meet. The information the user needs to see comes in on JMS, which has its own rules (message consumed, remove from queue etc). The information then enters your application's domain which might have different rules (must read, save for next login etc).
If a user doesn't login for a long period of time, their queue might fill up and not be able to receive any more messages, whereas, if the messages are always read and stored in a database it doesn't matter how frequently they login as the database should be better at holding large amounts of messages.
Another option is one topic per user and messages are sent to those topics but other systems would need to know which users are in your system, which probably isn't a good idea. Or you could use Apache Camel to route incoming messages on the main topic to user topics. The messages would need to be durable and transacted in case the broker went down. When a user logs in, read from the topic to get all their messages. You can route based on content or headers.
Your problem is one of message persistence and or re-delivery. There are a couple of approaches:
JMS Durable subscription: you could make a durable subscription on a topic from system B and only consume messages while users are logged in. When you don't receive() messages, your messages will be held at the broker for you until you call receive() again. In case A sent messages persistently, all of this is saved by the ActiveMQ broker on disk.
JMS Queue: system A puts messages into a queue and System B doesn't pick up the messages unless users are logged in. The queue will get bigger until you call receive() again from system B. Similar to durable subscriptions, but with a queue you can only have one consumer for each message. With durable subscriptions it's a easier to configure a fault-tolerant version of system B...
Add a 'replay server' for n-times delivery: system A publishes to a topic (could be non-persistent) and a third component (C) would also subscribe to every message and persist to disk. When system B needs to see a message again, it could ask system C for those messages, ideally supporting from_time, or similar.
Is it possible to keep a history of messages (with message content would be perfect) that have already been retrieved and are no longer on a queue?
In the application I can see when the sender attempts to put the message in the queue and when the receiver attempts to pick the messages up, but I'd like to see when the message really arrived into the queue and when the messages were really received.
Does MQ Explorer have this function? How would I use it?
What you are looking for is a message tracking/auditing software for IBM MQ. You can find a list of what is available here.
It is possible to use an API exit to make copies of messages in a queue or to audit both PUT and GET operations.
It is also possible to put messages to a topic, then create as many administrative subscriptions to destination queues as required. Something can then GET and log messages from one of those destination queues. The problem with this is that MQ changes the message ID between publication and consumption whereas in a queue it remains static.
There is no native MQ function to capture messages. It's possible to use linear logs and later scrape the logs but these do not necessarily capture all messages due to optimization. (A message PUT to a waiting getter outside of syncpoint for example.) However there is at least one commercial product to scrape linear transaction logs to audit message activity.
The philosophy of MQ in general is that it is the delivery mechanism and deals with envelope data to route and deliver but does not deal with payload data. WAS, IIB and other broker/transformation engines are where IBM has put all of the functions that deal with message payloads.
I have got started with ActiveMQ and able to move forward, but I am not able to figure out how I can view all messages that are sent to a particular queue.
I can see the message in the queue as long as it is not consumed by a consumer but as soon as it is consumed by a consumer, I can no longer see the message in the queue.
In my project I used to use "MQJ Explorer" and connect with a queue manager and there I could view all the messages that were sent to a particular queue, so I am expecting similar thing with ActiveMQ.
I know while sending I can have the message marked as "persistent" but still it will be there in the queue only till it is not consumed, once it is consumed I cannot see it any more.
Please let me know if things work differently with ActiveMQ.
The Queue holds messages until they are consumed at which point they are discarded. So you cannot browse all messages that were sent to the queue and then consumed. The only solution to this would be to use a Camel route or similar to mirror the Queue to some other Audit type Queue but you'd need to use a lot of care as this could easily lead to filling the message store as the mirrored messages will continue to grow and take space unless they are purged periodically.
I am using ActiveMQ v5.10.0 and having an issue almost every weekend where my ActiveMQ instance stops delivering persistent messages sent on queues to the consumers. I have not been able to figure out what could be causing this.
While the issue was happening I tried following things:
I added a new consumer on the affected queue but it didn't receive
any messages.
I restarted the original consumer but it didn't receive any messages after the restart.
I purged the messages that were held on the queue but then messages started accumulating again and broker didn't deliver any of the new messages. When I purged the expiry count didn't increase neither the dequeue and dispatch counters.
I sent 100 non-persistent messages on the affected queue, surprisingly it received those messages.
I tried sending 100 persistent messages on that queue, it didn't deliver anyone of them, all the messages were held by broker.
I created a fresh new queue and sent 100 persistent messages and none of them was delivered to the consumer whereas all the non-persistent messages were delivered.
The same things happen if I send persistent or non-persistent messages from STOMP producers. Surprisingly all this happened only for queues, topic consumers were able to receive persistent as well as non-persistent messages.
I have already posted this on ActiveMQ user forum: http://activemq.2283324.n4.nabble.com/Broker-not-delivering-persistent-messages-to-consumer-on-queue-td4691245.html but no one from ActiveMQ has suggested anything.
The jstack output also isn't very helping.
More details:
1. I am not using any selectors, message groups feature
2. I have disabled producer flow control in my setup
I want some suggestions as to what configuration values might cause this issue- memory limits, message TTL etc.
I am still learning about this activemq and jms stuff.
I already tried some example and now I can produce and consuming message from the queue/topic.
Now I have a problem, when my client/consumer lost the connection, the message in queue/topic still send out that message, that message become lost and not kept in the queue/topic. So my question is how I can keep that failed message and how to make the broker resend that message again?
thanks
You are mixing up terminology a bit.
Queues will hold messages until consumed or the broker is restarted, unless the message has been marked as persistent, in which case they will stick around even after a broker restart.
Topics only deliver the current message to any current subscriber. However there are several methods you can use to persist messages published to a topic:
Durable subscribers.
Virtual Destinations .
Virtual Topics tend to be popular for many reasons over durable subscribers, but it really depends on the use-case.
How you create a durable subscriber depends on what you are using to create the subscriber (Spring, POJO, some other API?). All methods will at some point call the Session.createDurableSubscriber method, but I suggest reading up on how they behave before choosing this over Virtual Topic or Composite Queues.
The thing which you are looking for might be Durable subscription
You can find documentation for same at http://activemq.apache.org/how-do-durable-queues-and-topics-work.html