I would like to understand what is the maximum number of re-deliver in CLIENT_ACKNOWLEDGE if you are not doing the acknowledgment.
Do we have any maximum number configured, if so what is that
property and can we override it?
If we don't have any maximum
number, then the message will always stay in the queue? is there any
way to clear it.
It's not part of the JMS specification; some vendors have a mechanism to deliver to a dead letter queue after some number of (configurable) attempts.
Recent brokers provide the delivery count in the JMSXDeliveryCount header so you can decide to throw it away when the count reaches some number.
If you are using CLIENT_ACKNOWLEDGE and don't do so, it won't be redelivered at all (unless you close the consumer/connection).
Related
This is a general question. Let me say I have a queue manager locally. I have a transmission queue/remote queue definition setup through which I connect to destination queue manager queue. If destination queue manager queue's maximum message length capacity is 1000 and if I put a message length more than that it automatically moves to destination queue manager dead letter queue provided that my transmission queue max message length is more than what I input. This is the expected behaviour. But is there any way on MQ world to handle this and not move it to dead queue? Or is it the sole responsibility of the application that puts this message to not put over the max length?
Thanks in advance.
Changing the default Maximum Message Length (i.e. MAXMSGL) from 4MB to a small value is a bad idea.
Myth: MQ does NOT allocate space based on the value in maximum message length field. Setting it to a very small value or to a very large value has not bearing on the disk space. MQ ONLY writes the size/amount of the real message.
Secondly, the application team should tell the MQAdmin what the largest message will be that the application will send. If they say 10MB then the MQAdmin can increase max. message length to 10MB or something a little larger i.e. 12MB.
The largest value that can be used is 100MB.
Note: The MQAdmin will need to increase the max. message length for: channel, XMITQ, local queue and the dead letter queue for any message larger than the default size of 4MB or it will not flow.
Thanks Roger and JoshMc. Infact I tried both options, client to QM and between QM and QM. Client to QM is fine as the client receives the error code and basically nothing happens. But the concern is only between sender QM and receiver QM. What I have seen mostly is that there will be only one transmission queue with maximum message length to connect to a particular queue manager. All the different remote connection/queues use that transmission queue. So if the sender commits a mistake by sending a large message than the destination queue cannot accept, it usually end up passing through the transmission queue but failing in destination and reaching the destination's dead queue. Now the destination owner is alerted/or need to take some remediation for a mistake that he didn't commit. That's the whole reason for me asking this question. Thanks a lot for you both on shedding more light and spending your time for me.
I think Morag Hughson has given out something for me to try out but still it will have its own negative impact. But I was looking for something like that where we can control at the MQ level to not allow the message to go to destination dead queue.
According to the SQS documentation, the maximum number of inflight messages is set to 120,000 for standard queues. However, sometimes I see my queues maxing out at lower numbers, such as here:
Does anyone know why this might be the case? I have code that dynamically changes the number of SQS listeners depending on the number of messages in the queue, but I don't want to do anything if I've hit the maximum. My problem is now that the max limit doesn't seem to be consistent. Some queues go to 120K, but this one is stuck at 100K instead, and as far as I can tell there is no setting that allows me to set this limit.
approximateNumberOfMessagesNotVisible indicates the number of messages in-flight, as you are rightly said. It depends on how many consumers you have, and what is througput of each consumer.
If the actual number is caping at 100k, then your consumers are swamped and have no more receiving capacity.
Anyways, it's better if you provide more info on the use-case as 100k in-flight messages look out of ordinary and you may be not using correct solution for your problem.
I want to understand how ApacheMQ's prefetch limit works. Are all the messages sent in one burst? What if there are concurrent consumers, what happens then?
What is the difference between prefetch limit of 0 and 1?
Read the link recommended by #Tim Bish -- the quotes I offer are from that page.
So ActiveMQ uses a prefetch limit on how many messages can be streamed
to a consumer at any point in time. Once the prefetch limit is
reached, no more messages are dispatched to the consumer until the
consumer starts sending back acknowledgements of messages (to indicate
that the message has been processed). The actual prefetch limit value
can be specified on a per consumer basis.
Specifically on the 0 versus 1 prefetch limit difference:
If you have very few messages and each message takes a very long time
to process you might want to set the prefetch value to 1 so that a
consumer is given one message at a time. Specifying a prefetch limit
of zero means the consumer will poll for more messages, one at a time,
instead of the message being pushed to the consumer.
I have a queue that is loaded with high priority JMS messages throughout the day, I want to get them out the door quickly. The queue is also being loaded periodically with lower priority messages in large batches. The problem that I see on busy days, is that there are always enough high priority messages at the front of the queue that none of the lower priority messages get selected until that volume drops off. Often they will sit on the queue until they middle of the night. The app is distributed over a number of servers, but the CPUs are not even breathing hard, the JMS seems to be the choak point.
My hunch is to implement some sort of aging algorithm that increases priority for messages that have been on the queue for a very long time, but of course, that is what middleware is supposed to do for me. I can't imagine that the JMS provider (IBM WebsphereMQ) or the application server (TIBCO BusinessWorks) doesn't have some sort of facility to cope with this. So before I go write some code, I thought I would ask, is there any way to get either of these technologies to help me out with this problem?
The BusinessWorks activity that is reading the queue is a JMS SOAP Event Source, but I could turn it into a JMS Queue Receiver activity or whatever.
All thoughts on how to solve this are welcome :-) TIA
That's like tying 1 hand behind your back and then complaining that you cannot swim properly. D'oh! First off, who's bright idea was it to mix messages. Just because you can do something does not mean you should.
The app is distributed over a number of servers, but the CPUs are not
even breathing hard, the JMS seems to be the choak point.
Well then, the solution is easy. Put high priority messages into queue "A" (the existing queue) and low priority messages into a new queue "B". Next, startup another instance of your JMS application to read the messages off queue "B".
Also, JMS is probably not the choke-point. It is what the application is doing with the message data after the JMS layer picks up the message that is taking a long time (i.e. backend work).
Finally, how many instances of your JMS application is running against the existing queue? If you are only running 1 instance, why? If you have lots of CPU capacity then why don't you run 10 instances of your JMS application. Do some true parallel processing of messages.
If you really want to keep you messages mixed on the same queue and have the high priority messages processed first, and yet your volume of messages is such that you cannot work through all the volume sometimes until the middle of the night, then you quite simply do not have enough processing applications. MQ is a parallel processing system, it is designed to allow many applications to put or get from a queue at once. Make use of this by running more of your getting applications at the same time. They will work through your high priority messages quicker and then get back to processing the lower priority ones.
From your description it's clear that you want the high priority messages to processed first. In such a case lower priority messages will have to wait.
MQ will not increase the priority of messages if they are sitting in queue for long time. How will it know that it has to change property of a message :)?. You will need to develop an application to do that.
I would think segregating messages based on priority, for example, high priority messages are put to one queue and lower priority messages to another queue could be one option you could look at.
Second option would be to look at the changing the delivery sequence (MSGDLVSQ) to FIFO. This makes to messages to be delivered to consumers in the order they arrived into queue. But note this will ignore the message priority, meaning if there is a lower priority message followed by a higher priority message, then higher priority message will wait till the lower priority message is delivered.
I am wondering what the fields on the oracle table GV$PERSISTENT_QUEUES really mean.
The Documentation:
ENQUEUED_MSGS NUMBER Number of messages enqueued
DEQUEUED_MSGS NUMBER Number of messages dequeued
Note: This column will not be incremented until all the subscribers of the message have dequeued the message and its retention time has elapsed.
...
ENQUEUED_EXPIRY_MSGS NUMBER Number of messages enqueued with expiry
ENQUEUED_DELAY_MSGS NUMBER Number of messages enqueued with delay
MSGS_MADE_EXPIRED NUMBER Number of messages expired by time manager
MSGS_MADE_READY NUMBER Number of messages made ready by time manager
...
ENQUEUE_TRANSACTIONS NUMBER Number of enqueue transactions
DEQUEUE_TRANSACTIONS NUMBER Number of dequeue transactions
Oracle Documentation (11.2)
My Questions:
How can the number of dequeued messages be larger than the number of enqueued messages?
If messages with a certain delay get added to the queue, do they get counted at ENQUEUED_MSGS and ENQUEUED_DELAY_MSGS?
If a message with a certain delay gets delivered after the delay, will it get counted at DEQUEUED_MSGS and MSGS_MADE_READY?
If so, how can MSGS_MADE_READY be larger than ENQUEUED_DELAY_MSGS?
What do the fields ENQUEUED_EXPIRY_MSGS and MSGS_MADE_EXPIRED mean?
What's the difference between ENQUEUED_MSGS and ENQUEUE_TRANSACTIONS, same with dequeueing?
Thank you in advance for help!
I am pretty sure of having found the solution to most of the above questions.
DEQUEUED_MSGS can be greater than ENQUEUED_MSGS in case of reboot of a database. Queue Entries that are still in the Queue Table will remain there. After database reboot, the entries will get dequeued and added to the number of dequeued messages, but they won't get added to the number of enqueued messages.
The Field ENQUEUED_MSGS is the sum of all messages that got enqueued into the Queue.
The Field ENQUEUED_DELAY_MSGS is the sum of all messages enqueued with delay.
ENQUEUED_MSGS - ENQUEUED_DELAY_MSGS = All messages that got enqueued without delay
The same is for DEQUEUED_MSGS (all) and MSGS_MADE_READY (only with delay).
I don't know yet what ENQUEUE_TRANSACTIONS and DEQUEUE_TRANSACTIONS mean (maybe DEQUEUE_TRANSATIONS describes the number of dequeues of one message in a multi consumer queue), but I won't use those fields.