I have some limitation that I can only have 1 partition of kafka topic which I can listen and still need to improve throughput of message processing and send that message MQ in end during that processing
So once I receive message in kafkaListener, I have used Asycn for further processing (that is for storing message in Db and later post to MQ).
Issue I see Session cache is not working , as once I read from kafkalistner it opens up new thread to do further work and once it reaches to JMS send method, after certain point I end with MQ error "max connections reach channel capacity" I thin MQRC 2537
Not sure what would be issue and I am using com.ibm.mq /mq-jms-spring-boot-starter as dependency
I have set Session cache to 20 and Async to 30 , does this means it will still try to create 10 more JMS connection, if all 30 threads task comes at almost same time
My understanding to SessionCache is that , only that many max session connection will be created only ..and out of 30 ..10 threads need to wait for JMS session to be available
Please assist , we are using Spring boot
I think the problem here is that you end up with too many connections being opened. All the pooling is done in the background for you by spring-jms, and it appears that your pool size exceeds the number of connections that the channel is configured for.
You will need to apply a throttle, by reducing the connection pool. There are caching properties that you can set for mq-jms-spring-boot-starter. See https://github.com/ibm-messaging/mq-jms-spring
eg.
ibm.mq.pool.enabled=true
ibm.mq.pool.maxConnections=5
ibm.mq.pool.blockIfFull=true
ibm.mq.pool.blockIfFullTimeout=60
You will need to determine sensible values for your envrionment.
Related
Little bit of backgroud: I need to improve the performance of one of our batch framework. There, batch inputs are sent to a JMS queue. Further, at the queue endpoint, we have a MDB, which is consuming the messages. Now, what i suspect here that if there are large number of messages, there is no MDB instance available to consume the messages as all of them are held up in processing the previous messages. To improve this, i am thinking of implementing a threadpool in the MDB business logic so that once the MDB has received the message and deliver it to the thread, it gets free for consuming another message.
Now before implementing this, i want to monitor my JMS queues to check if the messages are really waiting in the queues or not. So i need to know if this monitoring can be done via some WAS admin console or some JMX application. My main purpose is to check the waiting time of each jms message in the queue.
First, you can set the number of processes (MDB instances) that will consumes the Q in parallel. The default is 10 (Per member of the cluster..).
With the console: Resources -> JMS -> activation specifications, Set "Maximum concurrent MDB invocations per endpoint" which is defined as `"The maximum number of endpoints to which messages are delivered concurrently."``
As for monitoring the Q and generating some load, you can have a look at JMSToolBox on sourceforge
In the "Destination information" dialog in JMSToolBox, you will also be able to see the number of concurrent consumers on the Q
Also if you want to measure the time spend by a message in the Q, just compute the difference between the current time and the JMSTimestamp JMS standard property from the message it is process by the MDB in the onMessage() method
I used the Spring JMS cachingconnectionfactory to improve the performance of the my application based on Spring Integration and IBM MQ. I put sessioncachesize as 10 as we have the max of 10 concurrent threads working (ThreadPoolTaskExecutor) on consume/sending messages.
When I looked at the number of connections opened in MQ explorer (open output count for queue), it shows 10 and it stays on for days and never getting closed.
Is there a way to programatically to detect connections which are
potentially stale - say idle for half a day - I checked the
resetConnection() but not sure how to get the last used time for the
session.
Does Spring provides any connection time out parameter for
cacheconnection factory? or How to release these idle connections?
Also, the heartbeat/keepalive mechanism will not work for us as we want to physical close the cached connections based on last used time.
If the timeout is a property of the Session object returned by IBM, you could subclass the connection factory, override createSession(); call super.createSession(...) then set the property before returning it.
You might also have to override getSession(...) and keep calling it until you a get a session that is not closed. I don't see any logic to check the session state in the standard factory. (getSession() calls createSession() when the cache is empty).
What are the best practices regarding sessions in an application that is designed to fetch messages from a MQ server every 5 seconds?
Should I keep one session open for the whole time (could be weeks or longer), or better open a session, fetch the messages, and then close the session again?
I am using the .net IBM XMS v8 client library.
Adding to what #Attila Repasi's response, I would go for a consumer with message listener attached. The message listener would get called whenever a message needs to be delivered to application. This avoids application explicitly calling receive() to retrieve messages from queue and waste CPU cycles if there are no messages on the queue.
Check the XMS.NET best practices
Keep the connection and session open for a longer period if your application sends or receive message continuously. Creation of connection or session is a time consuming operation and consumes lot of resources and involves network flow (for client connections).
I'm not sure what you are calling a session, but typically applications connect to the queue manager serving them once at start, and keep that connection up while running.
I don't see a reason to disconnect just to reconnect 5 seconds later.
As for keeping the queues open, it depends on your environment.
If there are no special circumstances, I would keep the queue open.
I think the most worth thinking about is how you issue the GETs to read the messages.
I have a MDB that receives messages to process a batch (BatchProcessor). Instead of working on one item in the batch at a time, the BatchProcessor will create a BatchItemProcessor and using a Semaphore will allow a max number of items to be processed concurrently at a time. The BatchItemProcessors are stateless session beans and injected by the Glassfish container.
We need to send a SOAP message to another service whenever we finish processing a batch item with its status. The BatchItemProcessor creates/sends a message to a JMS topic. We have another MDB, StatusSender, which listens for this messages and creates/sends the SOAP messge to the other service.
The problem is that the SOAP messages aren't being sent until all the batch items have been dispatched by the BatchProcessor.
Looking at the logs, the BatchItemProcessors are being created in an EJB thread pool (e.g. name from logs: __ejb-thread-pool10) and the BatchProccessor and StatusSender are created in, what I am assuming is, the Glassfish MDB thread pool (e.g. name from log: p: thread-pool-1; w: 39).
I am assuming (because I can't find documentation to confirm/deny) that p: thread-pool-1; identifies the pool and w: 39 the actual thread. If this is true, then the BatchProcessor uses a completely different thread than the StatusSender since the thread identifiers are different (one for BatchProcess and multiple ones for StatusSender).
Which means I'm really confused as to why the BatchProcessor onMessage must complete before the StatusSender onMessage calls can be executed.
Looking at my Glassfish EJB Container/MDB Settings configuration (both default-config and server-config) I have the following (which I believe are defaults):
Initial and Min Pool Size = 0
Max Pool Size = 32
Pool Resize Quantity = 8
Idle Timeout = 600
If someone call help me with this issue it would be very appreciated. Also, any recommondations for a good resource (book, website, etc.) would also be appreciated.
UPDATE
My issue was related to some code logic which was causing the delay of StatusSender onMessage being executed. However, I would still appreciate any recommendation for a good reference on Glassfish and threads/pools.
We are using IBM MQ and we are facing some serious problems regarding controlling its asynchronous delivery to its recipient.We are having some java listeners configured, now the problem is that we need to control the messages coming towards listener, because the messages coming to server are in millions count and server machine dont have that much capacity t process so many threads at a time, so is there any way like throttling on IBM MQ side where we can configure preetch limit like Apache MQ does?
or is there any other way to achieve this?
Currently we are closing connection with IBM MQ when some X limit has reached on listener, but doesen't seems to be efficient way.
Please guys help us out to solve this issue.
Generally with message queueing technologies like MQ the point of the queue is that the sender is decoupled from the receiver. If you're having trouble with message volumes then the answer is to let them queue up on the receiver queue and process them as best you can, not to throttle the sender.
The obvious answer is to limit the maximum number of threads that your listeners are allowed to take up. I'm assuming you're using some sort of MQ threadpool? What platform are you using that provides unlimited listener threads?
From your description, it almost sounds like you have some process running that - as soon as it detects a message in the queue - it reads the message, starts up a new thread and goes back and looks at the queue again. This is the WRONG approach.
You should have a defined number of process threads running (start with one and scale up as required, and within limits of your server) which read from the queue themselves. They would each open the queue in shared mode and either get-with-wait or do immediate get with a sleep if you get a MQRC 2033 (no messages in queue).
Hope that helps.
If you are running in the application server environment, then the maxPoolDepth property on the activationSpec will define the maximum ServerSessionPool size for the MDB - decreasing this will throttle the number messages being delivered concurrently.
Of course, if your MDB (or javax.jms.MessageListener in the JSE environment) does nothing but hand the message to something else (or, worse, just spawn an unmanaged Thread and start it) onMessage will spin rapidly and you can still encounter problems. So in that case you need to limit other resources too, e.g. via threadpool configuration.
Closing the connection to the QM is never an efficient way, as the MQCONN/MQDISC cycle is expensive.