How to get Websphere MQ Queue depth in case of Multi Instanced environment.
For a single instance we are getting a queue depth using the MQManager like this:
#SuppressWarnings("unchecked")
private MQQueueManager createQueueManager() throws MQException {
MQEnvironment.channel = channel;
MQEnvironment.port = port;
MQEnvironment.hostname = host;
MQEnvironment.properties.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES);
return new MQQueueManager(manager);
}
For Websphere MQ in multi-instancing environment how to perform the same ?
You can do it either of these ways:
Use a load balancer which is capable of routing the TCP connections based on the availability of the queue manager instances behind it, and connect to the address of the load balancer instead of directly to the queue manager.
Use a Client Channel Definition Table to specify the parameters for the queue manager connection. You will need to configure a queue manager group containing the instances of your queue manager and connect using the CCDT:
https://www-01.ibm.com/support/knowledgecenter/SSFKSJ_7.1.0/com.ibm.mq.doc/ja11090_.htm
Related
I am very new to JMS & JNDI, and I'm trying to receive message from a queue programatically, but I am not sure how I can connect to the server. I have Websphere console UI where I login to manage all my queues. This console UI is hosted at linux-server:7276. Below is the UI link
https://my-server:9043/ibm/console
I have referred to the sample classes from IBM MQ JmsJndiConsumer:
String contextFactory = "com.sun.jndi.ldap.LdapCtxFactory";
String initialContextUrl = "ldap://my-server:9043"
Hashtable<String, String> environment = new Hashtable<String, String>();
environment.put(Context.INITIAL_CONTEXT_FACTORY, contextFactory);
environment.put(Context.PROVIDER_URL, initialContextUrl);
environment.put(Context.SECURITY_PRINCIPAL, userName);
environment.put(Context.SECURITY_CREDENTIALS, password);
InitialContext context = new InitialContext(environment);
It always throws an error in the context Connection or outbound has closed.
The queue I am trying to connect to has these details:
Connection factories : jms/atConnectionFactory
queue-manager : appit-node.Sit-TBus
Bus name : TBus
queue name : jms/appitone-event
Bootstrap Member : linux-server:7276
You're using my-server:9043 for both your HTTPS and LDAP server. Both of these can't be using the same port on the same machine so one of them must be wrong which is almost certainly why it's failing.
I have a Spring Boot app (Jhipster) that uses STOMP over WebSockets to communicate information from the server to users.
I recently added an ActiveMQ server to handle scaling the app horizontally, with an Amazon auto-scaling group / load-balancer.
I make use the convertAndSendToUser() method, which works on single instances of the app to locate the authenticated users' "individual queue" so only they receive the message.
However, when I launch the app behind the load balancer, I am finding that messages are only being sent to the user if the event is generated on the server that their websocket-proxy connection (to the broker) is established on?
How do I ensure the message goes through ActiveMQ to whichever instance of the app that the user is actually "connected too" regardless of which instance receives, say an HTTP Request that executes the convertAndSendToUser() event?
For reference here is my StompBrokerRelayMessageHandler:
#Bean
public AbstractBrokerMessageHandler stompBrokerRelayMessageHandler() {
StompBrokerRelayMessageHandler handler = (StompBrokerRelayMessageHandler) super.stompBrokerRelayMessageHandler();
handler.setTcpClient(new Reactor2TcpClient<>(
new StompTcpFactory(orgProperties.getAws().getAmazonMq().getStompRelayHost(),
orgProperties.getAws().getAmazonMq().getStompRelayPort(), orgProperties.getAws().getAmazonMq
().getSsl())
));
return handler;
}
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
config.enableStompBrokerRelay("/queue", "/topic")
.setSystemLogin(orgProperties.getAws().getAmazonMq().getStompRelayHostUser())
.setSystemPasscode(orgProperties.getAws().getAmazonMq().getStompRelayHostPass())
.setClientLogin(orgProperties.getAws().getAmazonMq().getStompRelayHostUser())
.setClientPasscode(orgProperties.getAws().getAmazonMq().getStompRelayHostPass());
config.setApplicationDestinationPrefixes("/app");
}
I have found the name corresponding to the queue that is generated on ActiveMQ by examining the headers in the SessionSubscribeEvent, that is generated in the listener when a user subscribes to a user-queue, as simpSessionId.
#Override
#EventListener({SessionSubscribeEvent.class})
public void onSessionSubscribeEvent(SessionSubscribeEvent event) {
log.debug("Session Subscribe Event:" +
"{}", event.getMessage().getHeaders().toString());
}
Corresponding queues' can be found in ActiveMQ, in the format: {simpDestination}-user{simpSessionId}
Could I save the sessionId in a key-value pair and just push messages onto that topic channel?
I also found some possibilities of setting ActiveMQ specific STOMP properties in the CONNECT/SUBSCRIBE frame to create durable subscribers if I set these properties will Spring than understand the routing?
client-id & subcriptionName
Modifying the MessageBrokerReigstry config resolved the issue:
config.enableStompBrokerRelay("/queue", "/topic")
.setUserDestinationBroadcast("/topic/registry.broadcast")
Based on this paragraph in the documentation section 4.4.13:
In a multi-application server scenario a user destination may remain
unresolved because the user is connected to a different server. In
such cases you can configure a destination to broadcast unresolved
messages to so that other servers have a chance to try. This can be
done through the userDestinationBroadcast property of the
MessageBrokerRegistry in Java config and the
user-destination-broadcast attribute of the message-broker element in
XML
I did not see any documentation on "why" /topic/registry.broadcast was the correct "topic" destination, but I am finding various iterations of it:
websocket sessions sample doesn't cluster.. spring-session-1.2.2
What is MultiServerUserRegistry in spring websocket?
Spring websocket - sendToUser from a cluster does not work from backup server
I have a (legacy) TCP service that has multiple processes. Each process runs on the same host, but on a different port. The service is single threaded, so the way to increase throughput is to round-robin each request across each of the ports.
I am providing an AMQP exposure to this legacy application. Its very simple - take a string off the AMQP queue, pass it to the application, and return the response string to the AMQP reply queue.
This works great on a single port. However, i'd like to fan out the requests across all the ports.
Spring Integration seems to only provide AbstractClientConnectionFactory implementations that either connect directly to a single host/port (TcpNetClientConnectionFactory) or maintain a pool of connections to a single host/port (CachingClientConnectionFactory). There arent any that pool connections between a single host and multiple ports.
I have attempted to write my own AbstractClientConnectionFactory that maintains a pool of AbstractClientConnectionFactory objects and round-robins between them. However, I have struck several issues to do with handing the TCP connections when the target service goes away or the network is interrupted that I have not been able to solve.
There is also the approach taken by this question: Spring Integration 4 - configuring a LoadBalancingStrategy in Java DSL but the solution to that was to hardcode the number of endpoints. In my case, the number of endpoints is only known at runtime and is a user-configurable setting.
So, basically I need to create a TcpOutboundGateway per port dynamically at runtime and somehow register it in my IntegrationFlow. I have attempted the following:
#Bean
public IntegrationFlow xmlQueryWorkerIntegrationFlow() {
SimpleMessageListenerContainer inboundQueue = getMessageListenerContainer();
DirectChannel rabbitReplyChannel = MessageChannels.direct().get();
IntegrationFlowBuilder builder = IntegrationFlows
.from(Amqp.inboundGateway(inboundQueue)
.replyChannel(rabbitReplyChannel))
/* SOMEHOW DO THE ROUND ROBIN HERE */
//I have tried:
.channel(handlerChannel()) //doesnt work, the gateways dont get started and the message doesnt get sent to the gateway
//and I have also tried:
.handle(gateway1)
.handle(gateway2) //doesnt work, it chains the handlers instead of round-robining between them
//
.transform(new ObjectToStringTransformer())
.channel(rabbitReplyChannel);
return builder.get();
}
#Bean
//my attempt at dynamically adding handlers to the same channel and load balancing between them
public DirectChannel handlerChannel() {
DirectChannel channel = MessageChannels.direct().loadBalancer(new RoundRobinLoadBalancingStrategy()).get();
for (AbstractClientConnectionFactory factory : generateConnections()) {
channel.subscribe(generateTcpOutboundGateway(factory));
}
return channel;
}
Does anyone know how I can solve this problem?
See the dynamic ftp sample - in essence each outbound gateway goes in its own application context and the dynamic router routes to the appropriate channel (for which the outbound adapter is created on demand if necessary).
Although the sample uses XML, you can do the same thing with java configuration, or even with the Java DSL.
See my answer to a similar question for multiple IMAP mail adapters using Java configuration and then a follow-up question.
I'm unable to process large messages from IBM MQ and get the below error:
JMSCMQ0001: WebSphere MQ call failed with compcode '1' ('MQCC_WARNING') reason '2080' ('MQRC_TRUNCATED_MSG_FAILED')
I'm using the DefaultListenerContainer and not consuming via a MessageConsumer using IBM MQ Java API classes directly. I believe by using IBM MQ JMS API you can specific options before retrieving the message from the queue. But how do I do that with DefaultListenerContainer, is there a system property I can set for these?
If using IBM MQ JMS API(I'm not consuming message like this, pasted just for reference):
MQGetMessageOptions mqGetMessageOptions = new MQGetMessageOptions();
mqGetMessageOptions.waitInterval = ipreoProperties.getMqReceiveWaitTime();
mqGetMessageOptions.options = MQC.MQGMO_WAIT | MQC.MQPMO_SYNCPOINT | MQC.MQGMO_ACCEPT_TRUNCATED_MSG;
Below is my Java Config for the IBM MQ Connection:
#Bean
public CachingConnectionFactory ipreoMQCachingConnectionFactory() {
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory();
//Not defining MQQueueConnectionFactory as separate bean as Spring boot's auto-configuration finds two instances
//of ConnectionFactory and throws ambiguous implementation exception
//One implementation is CachingConnectionFactory and other one would be MQQueueConnectionFactory if defined separately
MQQueueConnectionFactory mqConnectionFactory = new MQQueueConnectionFactory();
try {
mqConnectionFactory.setHostName(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_MQ_HOSTNAME));
mqConnectionFactory.setQueueManager(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_MQ_QUEUE_MGR));
mqConnectionFactory.setPort(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_MQ_PORT, Integer.class));
mqConnectionFactory.setChannel(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_MQ_CHANNEL));
//mqConnectionFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
//Setting connection mode as Client so it doesn't complain for native IBM MQ libraries
mqConnectionFactory.setIntProperty(CommonConstants.WMQ_CONNECTION_MODE, CommonConstants.WMQ_CM_CLIENT);
} catch (JMSException exception) {
exception.printStackTrace();
}
cachingConnectionFactory.setTargetConnectionFactory(mqConnectionFactory);
//Setting session caching size as 10, don't think we need more
cachingConnectionFactory.setSessionCacheSize(10);
cachingConnectionFactory.setReconnectOnException(true);
return cachingConnectionFactory;
}
public DefaultMessageListenerContainer ipreoDealActivityListenerContainer() {
DefaultMessageListenerContainer factory = new DefaultMessageListenerContainer();
factory.setConnectionFactory(ipreoMQCachingConnectionFactory());
factory.setDestinationName(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_DEAL_QUEUE_NAME));
factory.setMessageListener(ipreoDealActivityListener());
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
return factory;
}
#Bean
public MessageListener ipreoDealActivityListener() {
return new IpreoDealActivityListener();
}
Appreciate your help, thanks.
Adding a late response as it might be useful to someone.
In my case, when the java client had this exception, we noticed the actual message size was larger than the default 4 MB buffer size.
The Java API does not provide a hook to change buffer size. Hence, the buffer size has to be updated at the MQ server level.
First, we increased the message size in queue properties - It did not work.
Then, we increased the message size property at the MQ channel level as well, which finally resolved the issue.
To summarise, increase the buffer size at the MQ server for queue & the channel both.
On a client connection to a queue manager you can limit the size of messages on both the server and client side. I've seen this error before when the client side limit was smaller then the size of the message.
I don't know how you can set the message size limit directly in the JMS client, but you could use a Client Channel Definition Table. It's a file containing the details for connecting to queue managers, created on a queue manager and then copied to the client host. You need to reference the file by issuing setCCDTURL on the connection factory (setting the host, port and channel is not required when using a CCDT, the CCDT will specify those).
When the CCDT is created on the queue manager the appropriate message size limit needs to be set on the client channel.
The server side limit is set on the server connection channel.
Within the JMS client code handling of the receive buffer us handled automatically; the theory is that specific error should never be received by a JMS Application.
The first snippet of code is the Java Classes API and this could get that error.
How big actually are these messages? What level of the JMS client code are you using - make sure that it is the latest version. And certainly one of the 7.5 or 8 releases.
This answer also has some more information on this.
I want to send message to a remote IBM MQ using Jmeter for performance testing. I went through this link. But it requires the JNDI specific details like, QueueConnection Factory, JNDI Name Request queue, Initial Context Factory & Provider URL. Whereas the queu details i have are Qmanager, Qname, hostname, channel, port as given in the code shared in this link. Do these properties have any relation? Can i configure the Jmeter JMS test using the queue details i have?
Thanks in advance.
The first link you gave has a description using Java JMS/MQ and the second shows Java MQ (non-JMS).
JMS is just an abstraction layer. In simple terms, JMS is like giving everything a nick-name. A QCF (QueueConnectionFactory) is simply an object that has all of the information to connect to a queue manager.
i.e.
DEFINE QCF(myQCF) QMANAGER(MQWT1) CHANNEL(TEST.CHL) HOSTNAME(127.0.0.1) PORT(1415) TRANSPORT(CLIENT) FAILIFQUIESCE(YES)
A JMS queue is just a nick-name to an MQ queue.
DEFINE Q(test.q) QUEUE(TEST.Q1) QMANAGER(MQWT1) TARGCLIENT(JMS) FAILIFQUIESCE(YES)
Therefore, in your JMS code you simply reference your QCF (i.e. myQCF) and the JMS queue (i.e. test.q) and you are good to go.
Hashtable env = new Hashtable();
env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.fscontext.RefFSContextFactory");
env.put(Context.PROVIDER_URL, ""file:/C:/JNDI-Directory");
try
{
Context ctx = new InitialContext(env);
QueueConnectionFactory cf = (QueueConnectionFactory) ctx.lookup("myQCF");
Queue q = (Queue) ctx.lookup("test.q");
}
catch (NamingException e)
{
System.err.println(e.getLocalizedMessage());
e.printStackTrace();
}
It can be done via the beanshell as well. You can directly access the queue manager via the api, or via exposing the queue via a jms binding. The first is more simple and does not require the MQ client installation.