I'm using Weblogic JMS. What I'd like to do is:
a) producer A produces JMS messages and put them on the queue ( groupA )
b) when processing each message from groupA I want to generate another messages ( groupB )
I've got 16 workers to process this messages.
Now, how I can ensure, that all messages from groupA will be processed before any message from groupB ?
A bit late here but hopefully this will help anyone with the same issue. GroupA is effectively an Intermediate Destination. The JMS queue should have the default value of Pass-Through in the Unit-of-Work (UOW) Message Handling Policy setting. As your MDB processes these messages it needs to get the Unit of Work jms properties and reset them on the new message being sent for GroupB. This jms queue should have Unit-of-Work (UOW) Message Handling Policy set to Single Message Delivery. When the messages are received on this jms queue they will not be processed until all messages for a unit of work group are present i.e. all sequence numbers 1,2,3 etc and an end of message identifier. Once they are all present the mdb will consume them as 1 object message and the individual messages will be contained in a List. It is then your job to code to iterate the list and process as you need.
Weblogic Docs here
Related
I am in charge maintaining a production software written in Golang which uses RabbitMq as its message queue.
Consider the following situation:
A number of goroutines are publishing to a queue name logs.
Another set goroutines read from the queue and write the messages to a MongoDB collection.
Each publisher or consumer has its Own connection, and its own channel respectively, they are working in an infinite loop and never die. (The connections and channels are established when the program starts.)
autoAck, exclusive and noWait are all set to false and prefetch is set to 20 with global set to false for all
channels. All queues are durable with autoDelete, exclusive
and noWait all set to false.
The basic assumption was that each message in the queue will be delivered to one and only one consumer, so each message would be inserted in the database exactly once.
The problem is that there are duplicate messages in the MongoDB collection.
I would like to know if it is possible that more than one consumer gets the same message causing them to insert duplicates?
The one case I could see with your setup where a message would be processed more than once is if one of the consumers has an issue at some point.
The situation would follow such a scenario:
Consumer gets a bunch of messages from the queue
Consumer starts processing a message
Consumer commits the message to mongodb
either due to rabbitmq channel/connection issue, or other type of issue consumer side, the consumer never acknowledges the message
the message as it hasn't been acknowledged is requeued at the top of the queue
same message is processed again, causing the duplication
Such cases should show some errors in your consumers logs.
We are using IBM WebSphere MQ as JMS provider with Spring MDP (Message Driven POJO).
Is there any way in JMS where we can configure time related properties in message so that message can be consumed at particular defined time only?
For example, if I am sending three messages into queue M1, M2 and M3. Where, I can configure M2 message property let say 3 AM. And consumer side, consumer can only pick this message # 3 AM only. If time is not defined, messages should be consumed in a way that JMS Receiver does.
JMS 2.0 specification has defines Delivery Delay. With this feature a message producer can specify that a message must not be delivered until after a specified time interval. The message will be available for delivery after the specified time. But this may not help you as you want to a message to be consumed at a specified time. Typically messaging applications are designed to consume messages as soon as they are made available by the messaging provider.
If you want to process messages at a specified time only, you could create another queue "queue_3am", and schedule a reader to run exactly at 3am.
A variation is to set the timestamp as a message property. So one queue could contain messages to be processed at different points in time. The reader could use message selectors to get relevant messages only.
But you should use a "message pickup timeframe" by adding two timestamps as message properties, for eaxmple set the window to 1 or 5 minutes.
The receiver can use a message selector: A selector is a condition using message properties.
Have a look at this
When a local queue manager receives the following message in it's AMQ error log:
09/13/12 08:00:19 - Process(3017.20) User(mqm) Program(amqrmppa_nd)
AMQ9544: Messages not put to destination queue.
EXPLANATION: During the processing of channel 'TO_QM_QD2T1_C1' one or
more messages could not be put to the destination queue and attempts
were made to put them to a dead-letter queue. The location of the
queue is 2, where 1 is the local dead-letter queue and 2 is the remote
dead-letter queue.
... what is the mechanism by which MQ exchanges such information? Is there a built in facility within the channel program API itself, or is the info exchanges as discrete messages placed on the SYSTEM.CLUSTER.COMMAND.QUEUE (in the case of a cluster)? Given that this could occur in a remote queue definition situation, with only simple sender/receiver channel pairs, and no corresponding COMMAND QUEUE necessarily, I could imagine that it would be handled via the channel process communications... just wondering...
The channel agents have a bi-directional communication between them, even though messages flow in only one direction. When a message fails to find the destination at the remote end there are several possibilities for what happens next. The channel will only continue to run if the message can be successfully put somewhere and the first place to try is the remote DLQ. If that fails, the local MCA must either relocate the message or stop the channel. Therefore, the two message channel agents work out between them what happens and what the status of the channel should be.
The peculiar wording of the error message reflects that the different dispositions of the message derive from within the same code path and exception handling and the optimization of WMQ. The MCA knows the message was put to a DLQ at that point and rather than having two different error messages or logic to work out the wording on which DLQ was used, it just drops a number into a template. Thus a single error message and streamlined logic are used for both possibilities.
I was wondering what is the difference between a JMS Queue and JMS Topic.
ActiveMQ page says
Topics
In JMS a Topic implements publish and subscribe semantics. When you publish a message it goes to all the subscribers who are
interested - so zero to many subscribers will receive a copy of the
message. Only subscribers who had an active subscription at the time
the broker receives the message will get a copy of the message.
Queues
A JMS Queue implements load balancer semantics. A single message will be received by exactly one consumer. If there are no
consumers available at the time the message is sent it will be kept
until a consumer is available that can process the message. If a
consumer receives a message and does not acknowledge it before closing
then the message will be redelivered to another consumer. A queue can
have many consumers with messages load balanced across the available
consumers.
I want to have 'something' what will send a copy of the message to each subscriber in the same sequence as that in which the message was received by the ActiveMQ broker.
Any thoughts?
That means a topic is appropriate. A queue means a message goes to one and only one possible subscriber. A topic goes to each and every subscriber.
It is simple as that:
Queues = Insert > Withdraw (send to single subscriber) 1:1
Topics = Insert > Broadcast (send to all subscribers) 1:n
Topics are for the publisher-subscriber model, while queues are for point-to-point.
A JMS topic is the type of destination in a 1-to-many model of distribution.
The same published message is received by all consuming subscribers. You can also call this the 'broadcast' model. You can think of a topic as the equivalent of a Subject in an Observer design pattern for distributed computing. Some JMS providers efficiently choose to implement this as UDP instead of TCP. For topic's the message delivery is 'fire-and-forget' - if no one listens, the message just disappears. If that's not what you want, you can use 'durable subscriptions'.
A JMS queue is a 1-to-1 destination of messages. The message is received by only one of the consuming receivers (please note: consistently using subscribers for 'topic client's and receivers for queue client's avoids confusion). Messages sent to a queue are stored on disk or memory until someone picks it up or it expires. So queues (and durable subscriptions) need some active storage management, you need to think about slow consumers.
In most environments, I would argue, topics are the better choice because you can always add additional components without having to change the architecture. Added components could be monitoring, logging, analytics, etc.
You never know at the beginning of the project what the requirements will be like in 1 year, 5 years, 10 years. Change is inevitable, embrace it :-)
Queues
Pros
Simple messaging pattern with a transparent communication flow
Messages can be recovered by putting them back on the queue
Cons
Only one consumer can get the message
Implies a coupling between producer and consumer as it’s an one-to-one relation
Topics
Pros
Multiple consumers can get a message
Decoupling between producer and consumers (publish-and-subscribe pattern)
Cons
More complicated communication flow
A message cannot be recovered for a single listener
As for the order preservation, see this ActiveMQ page. In short: order is preserved for single consumers, but with multiple consumers order of delivery is not guaranteed.
If you have N consumers then:
JMS Topics deliver messages to N of N
JMS Queues deliver messages to 1 of N
You said you are "looking to have a 'thing' that will send a copy of the message to each subscriber in the same sequence as that in which the message was received by the ActiveMQ broker."
So you want to use a Topic in order that all N subscribers get a copy of the message.
TOPIC:: topic is one to many communication... (multipoint or publish/subscribe)
EX:-imagine a publisher publishes the movie in the youtub then all its subscribers will gets notification....
QUEVE::queve is one-to-one communication ...
Ex:-When publish a request for recharge it will go to only one qreciever ...
always remember if request goto all qreceivers then multiple recharge happened so while developing analyze which is fit for a application
Queue is JMS managed object used for holding messages waiting for subscribers to consume. When all subscribers consumed the message , message will be removed from queue.
Topic is that all subscribers to a topic receive the same message when the message is published.
I'm new to Oracle AQ.
I have created a table and a queue like so:
EXEC dbms_aqadm.create_queue_table(queue_table=>'MY_QUEUE_TABLE',
queue_payload_type=>'sys.aq$_jms_text_message',
multiple_consumers=>TRUE);
EXEC dbms_aqadm.create_queue(queue_name=>'CONTACT_INFO_QUEUE',
queue_table=>'MY_QUEUE_TABLE',
max_retries=>24,
retry_delay=>60,
retention_time=>3600);
Then I wrote a Listener to the queue in Java. When I start the Listener, it waits 6 minutes and then collects all the messages from the queue.
But I can't tell in MY_QUEUE_TABLE which messages have been consumed. Because I want a multiple consumer queue, I think the messages should stick around. However, how does Oracle AQ keep track of which messages each listener has consumed?
Each queue will keep track and ensure that all consumers have dequeued. You can look at the actual queue table to see how many consumers have consumed a message. Check aq$_my_queue_table and aq$_my_queue_table_I to see the status of messages.