I want to perform MQ load testing using JMETER for JMS Point-to-Point Messaging. I am able to connect and send request message for single connection to single remote queue. Can we establish multiple channel connection using same connection factory and send message to different queues. I have establish approximately 1500 channel connection with 1500 dedicated remote queues. I am using JMETER version 2.11
If you mean using a different uniquely named SVRCONN channels then no. The Channel specified in the connection factory can't be changed. To simulate 1 channel per connection you would need to create a connection factory for each channel.
However, there is no technical reason you cannot use the same channel for multiple queues simply by referencing the same connection factory for each test. Performance wise, there really won't be a difference between using 1500 instances of the same channel and 1500 individually named channels.
You may need to adjust the number of instances that that given channel and/or be started from a single client if you expect all 1500 to go at the same time.
Related
I have a Spring Boot based messaging app sending/receiving JMS messages to/from IBM MQ queue manager.
Basically, it uses MQConnectionFactory to organize connection to IBM MQ and a JmsPoolConnectionFactory from messaginghub:pooledjms to enable JMS connection pool, which is removed from MQConnectionFactory in IBM MQ 7.x
The app uses two different appoach to work with JMS. A "correct" one runs a JMSListener to receive messages and then sends a response on each message using JmsTemplate.send(). And there is a second "troubling" approach, where the app sends requests using JmsTemplate.send() and waits for response using JmsTemplate.readByCorrelId() until received or timed out.
I say troubling because this makes JMS sessions last longer if the response is delayed and could easily exhaust IBM MQ connection limit. Unfortunately, I cannot rewrite the app at the moment to the first approach to resolve the issue.
Now I want to restrict the number of connections in the pool. Of course, the delayed requests will fail but IBM MQ connection limit is more important at the moment, so this is kind of appropriate. The problem is that even if I disable the JmsPoolConnectionFactory, it seems that MQConnectionFactory still opens multiple connections to the query manager.
While profiling the app I see multiple threads RvcThread: com.ibm.mq.jmmqi.remote.impl.RemoteTCPConnection#12433875[...] created by JMSCCMasterThreadPool and corresponding connections to the query manager in MQ Explorer. I wonder why there are many of them in spite of the connection pooling is removed from MQConnectionFactory? I suppose it should open and reuse a single connection then but it is not true in my test.
Disabling "troubling" JmsTemplate.readByCorrelId() and leaving only "correct" way in the app removes these multiple connections (and the waiting threads of course).
Replacing JmsPoolConnectionFactory with SingleConnectionFactory has not effect on the issue.
Is there any way to limit those connections? Is it possible to control max threads in the JMSCCMasterThreadPool as a workaround?
Because it affects other applications your MQ admins probably want you to not exhaust the overall Queue Manager's connection limit (MaxChannels and MaxActiveChannels parameters in qm.ini). They can help you by defining an MQ channel exclusively used by your application. By this, they can limit the number of connections of your application with the MAXINST / MAXINSTC channel parameter. You will get an exception when this number is exhausted which is appropriate as you say. Other applications won’t be affected anymore.
I have to stress test (load test) an MQTT broker on my machine (or a machine in my network) for getting say the number of clients that can simultaneously remain connected to the broker on that machine, the number of messages that can be simultaneously published by different clients to registered topics and things like these.
How can I achieve this?
I'm quite new at this and have seen some videos about Jmeter but where can I get answers to the questions mentioned above in Jmeter? Just a side note, Jmeter doesn't have inbuilt MQTT support and people have made some plugins for this, namely, https://github.com/xmeter-net/mqtt-jmeter
p.s. Jmeter is not a necessity. I just mentioned it because I found some videos and resources on it.
MQTT Connect sampler opens a connection (one connection per JMeter thread - virtual user) so if you have 2 threads defined in the Thread Group - 2 connections will be open.
MQTT Pub Sampler sends message to the defined topic over the existing connection. If you want to send messages to different topics from the same user - you can parameterize the topic names using i.e. CSV Data Set Config, the same applies to the message payload
When you're done you can call MQTT DisConnect
See Testing the MQTT Messaging Broker for IoT - A Guide article for more detailed information if needed.
I am looking for detailed description on how IBM MQ Client or listener get the messages from MQ Server when new messages are placed on MQ Queue or Topic.
How the connection between MQ Client and MQ Server are created?
Does MQ Client initiates the connection with Server or does server initiates the connection to its consumer?
In case we have connection pool defined on MQ Client, how client knows that it has to create more connections with Server as the messages are increasing on Server? How Client know about the messages on Server?
Is there a communication from Server to Client which tells client that new messages have arrived?
I am looking for these details not the details on how this is setup or how to setup MQ Channels or Listeners. I am looking for how it works behind the scene.
If someone can point me to the right direction or documentation, it would be great.
It's hard to speak definitively about how the IBM WebSphereMQ client & server work since they're closed source, but based on my experience with other messaging implementations I can provide a general explanation.
A JMS connection is initiated by a client to a server. A JMS client uses a javax.jms.ConnectionFactory to create a javax.jms.Connection which is the connection between the client and server.
Typically when a client uses a pool the pool is either filled "eagerly" (which means a certain number of connections are created when the pool is initialized to fill it to a certain level) or "lazily" (which means the pool is filled with connections one-by-one as clients request them from the pool). If a client requests a connection from the pool and if all the connections in the pool are being used and the maximum size of connections allowed by the pool hasn't been reached then another connection will be created. If the pool has reached its maximum allowed size (i.e no more connections can be created) then the client requesting a connection will have to wait for another client to return their connection to the pool at which point the pool will then give it to the waiting client.
A JMS client can find out about messages on the server in a few different ways.
If the JMS client wants to ask the server occasionally about the messages it has on a particular queue it can create a javax.jms.Consumer and use the receive() method. This method can wait forever for a message to arrive on the queue or it can take a timeout parameter so that if a message doesn't arrive in the specified timeout the call to receive() will return.
If the JMS client wants to receive a message from a particular queue as soon as the message arrives on the queue then it can create a javax.jms.MessageListener implementation and register it on the queue. When such a listener is registered on a queue then when a message arrives on the queue the server will send the message to the listener. This is sometimes referred to as a "callback" since the server is "calling back" to the client.
The first thing you should do is take a course on JMS/IBM MQ or go to the new IBM conference called: Integration Technical Conference
Ok, now to your questions:
How the connection between MQ Client and MQ Server are created?
You simply issue the createQueueConnection method of QueueConnectionFactory class and specify the credentials.
QueueConnection conn = cf.createQueueConnection("myUserId", "myPwd");
Does MQ Client initiates the connection with Server or does server initiates the connection to its consumer?
MQ client application starts the connection - always.
In case we have connection pool defined on MQ Client, how client knows that it has to create more connections with Server as the messages are increasing on Server? How Client know about the messages on Server?
It is up to the team's architect or lead developer to understand message flow and message patterns. Hence, they will know what to set the pool count at. Also, lots and lots of testing too. Some client applications will only require a pool count of 10 whereas other applications may need a pool count of 50 because it is a heavy flow.
Is there a communication from Server to Client which tells client that new messages have arrived?
You use the createReceiver method of the QueueSession class to retrieve a message. Set a timeout value for the createReceiver method rather than continuously polling the queue manager.
Again, some training on the use of JMS/IBM MQ is strongly recommended.
Can there more than one listener to a queue manager ? I have used one listener/queue manager combination so far and wonder if this possible. This is because we have 2 applications connecting to same queue manager and seems to have problem with that.
There are a couple meanings for the term listener in an MQ context. Let's see if we can clear up some confusion over the terminology and then answer the question as it relates to each.
As defined in the spec, a JMS listener is an object that implements a callback mechanism. It listens on destinations for messages and calls onMessage when they arrive. The destinations may be queues or topics hosted by any JMS-compliant transport provider.
In IBM MQ terms, a listener is a process (runmqlsr) that handles inbound connection requests on a server. Although these can handle a variety of protocols, in practice they are almost exclusively TCP listeners that bind a port (1414 by default) and negotiate connection requests on sockets.
TCP Ports
Tim's answer applies to the second of these contexts. MQ can listen for sockets on multiple ports and indeed it is quite common to do so. Each listener listens on one and only one port. It may listen on that port across all network interfaces or can be bound to a specific network interface. No two listeners can bind to the same combination of interface and port though.
In a B2B context the best practice is to run a dedicated listener for each external business partner to isolate each of their connections across dedicated access paths. Internally I usually recommend separate ports for QMgr-to-QMgr, app-to-QMgr and interactive user connections.
In that sense it is possible to run multiple listeners on a given QMgr. Each of those listeners can accept many connections. Their job is to negotiate the connection then hand the socket off to a Message Channel Agent which talks to the QMgr on behalf of the remotely connected client or QMgr.
JMS Listeners
Based on comments, Ulab refers to JMS listeners. These objects establish a connection to a queue manager and then wait in GET mode for new messages arriving on a destination. On arrival of a message, they call the onMessage method which is an asynchronous callback routine.
As to the question "can there more than one (JMS) listener to a queue manager?" the answer is definitely yes. A multi-threaded application can have multiple listeners connected, multiple application instances can connect at the same time, and many thousands of application connections can be handled by a single queue manager with sufficient memory, disk and CPU available.
Of course, each of these applications is ultimately connected to one or more queues so then the question becomes one of whether they can connect to the same queue.
Many listeners can listen on the same queue so long as they do not get exclusive access to it. Each will receive a portion of the messages arriving.
Listeners on QMgr-managed subscriptions are exclusively attached to a dynamic queue but multiple instances on the same topic will all receive the same messages.
If the queue is clustered and there is more than one instance of it multiple listeners will be required to get all the messages since they will normally be distributed by MQ workload distribution across those instances.
Yes, you can create as many listeners as you wish:
http://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.explorer.doc/e_listener.htm
However, there is no reason why two applications can't connect to the queue manager via the same listener (on the same port). What problem have you run into?
i kind of don't understand when to use the MQ client connection channel. From my understanding, when client trying to connect MQ server, it can be completed by defining the channel object with server connection channel value directly in application code. Therefore, if so, then why do we need to make use of such client connection channel?
Please help explain to me in detail. Thanks very much
A Server Connection Channel is used by clients to connect to a queue manager.
You don't really use a client connection channel to connect to queue manager. A client connection channel defines the connection parameters required to connect to a queue manager for example queue manager name, connection name, SSL etc. These channel definitions are stored in channel definition table (CCDT) files. CCDT files are used by client applications through MQCHLLIB and MQCHLTAB environment variables.
This link and another has little more details.
In older versions of WebSphere MQ, a Client Channel Definition Table was used to specify SSL parameters and for failover so the application could select from several equivalent queue managers at connection time. The CCDT file is a compiled artifact and the DEFINE CHL(channel name) CHLTYPE(CLNTCONN) command is what generates the entries in the CCDT file. So you would only use the CLNTCONN channel type if you wanted to create a CCDT file.
Newer versions of WebSphere MQ expose the CCDT fields in the MQCONNX API and the reconnection parameters are in the CONNAME parameter and the client.ini file. Although these have made the CCDT file obsolete for newer applications, the functionality is still required for commercial and legacy applications. IBM has not announced that CCDT functionality is deprecated and it is in V7.5 which was just released so that functionality will remain for the foreseeable future.
What is a channel?
A channel is a logical communication link between a WebSphere® MQ client and a WebSphere MQ server, or between two WebSphere MQ servers. A channel has two definitions: one at each end of the connection. The same channel name must be used at each end of the connection, and the channel type used must be compatible.
WebSphere® MQ uses two different types of channels:
Message Channel
MQI Channel
A message channel, which is a unidirectional communications link between two queue managers. WebSphere MQ uses message channels to transfer messages between the queue managers. To send messages in both directions, you must define a channel for each direction.
A message channel is a one-way link. It connects two queue managers by using message channel agents (MCAs). Its purpose is to transfer messages from one queue manager to another. Message channels are not required by the client server environment.
An MQI channel, which is bidirectional and connects an application (MQI client) to a queue manager on a server machine. WebSphere MQ uses MQI channels to transfer MQI calls and responses between MQI clients and queue managers
Source