How to purge/delete message from weblogic JMS queue - weblogic-10.x

Is there and way (apart from consuming the message) I can purge/delete message programmatically from JMS queue. Even if it is possible by wlst command line tool, it will be of much help.

Here is an example in WLST for a Managed Server running on port 7005:
connect('weblogic', 'weblogic', 't3://localhost:7005')
serverRuntime()
cd('/JMSRuntime/ManagedSrv1.jms/JMSServers/MyAppJMSServer/Destinations/MyAppJMSModule!QueueNameToClear')
cmo.deleteMessages('')
The last command should return the number of messages it deleted.

You can use JMX to purge the queue, either from Java or from WLST (Python). You can find the MBean definitions for WLS 10.0 on http://download.oracle.com/docs/cd/E11035_01/wls100/wlsmbeanref/core/index.html.
Here is a basic Java example (don't forget to put weblogic.jar in the CLASSPATH):
import java.util.Hashtable;
import javax.management.MBeanServerConnection;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;
import javax.management.ObjectName;
import javax.naming.Context;
import weblogic.management.mbeanservers.runtime.RuntimeServiceMBean;
public class PurgeWLSQueue {
private static final String WLS_USERNAME = "weblogic";
private static final String WLS_PASSWORD = "weblogic";
private static final String WLS_HOST = "localhost";
private static final int WLS_PORT = 7001;
private static final String JMS_SERVER = "wlsbJMSServer";
private static final String JMS_DESTINATION = "test.q";
private static JMXConnector getMBeanServerConnector(String jndiName) throws Exception {
Hashtable<String,String> h = new Hashtable<String,String>();
JMXServiceURL serviceURL = new JMXServiceURL("t3", WLS_HOST, WLS_PORT, jndiName);
h.put(Context.SECURITY_PRINCIPAL, WLS_USERNAME);
h.put(Context.SECURITY_CREDENTIALS, WLS_PASSWORD);
h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES, "weblogic.management.remote");
JMXConnector connector = JMXConnectorFactory.connect(serviceURL, h);
return connector;
}
public static void main(String[] args) {
try {
JMXConnector connector =
getMBeanServerConnector("/jndi/"+RuntimeServiceMBean.MBEANSERVER_JNDI_NAME);
MBeanServerConnection mbeanServerConnection =
connector.getMBeanServerConnection();
ObjectName service = new ObjectName("com.bea:Name=RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean");
ObjectName serverRuntime = (ObjectName) mbeanServerConnection.getAttribute(service, "ServerRuntime");
ObjectName jmsRuntime = (ObjectName) mbeanServerConnection.getAttribute(serverRuntime, "JMSRuntime");
ObjectName[] jmsServers = (ObjectName[]) mbeanServerConnection.getAttribute(jmsRuntime, "JMSServers");
for (ObjectName jmsServer: jmsServers) {
if (JMS_SERVER.equals(jmsServer.getKeyProperty("Name"))) {
ObjectName[] destinations = (ObjectName[]) mbeanServerConnection.getAttribute(jmsServer, "Destinations");
for (ObjectName destination: destinations) {
if (destination.getKeyProperty("Name").endsWith("!"+JMS_DESTINATION)) {
Object o = mbeanServerConnection.invoke(
destination,
"deleteMessages",
new Object[] {""}, // selector expression
new String[] {"java.lang.String"});
System.out.println("Result: "+o);
break;
}
}
break;
}
}
connector.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}

Works great on a single node environment, but what happens if you are on an clustered environment with ONE migratable JMSServer (currently on node #1) and this code is executing on node #2. Then there will be no JMSServer available and no message will be deleted.
This is the problem I'm facing right now...
Is there a way to connect to the JMSQueue without having the JMSServer available?
[edit]
Found a solution: Use the domain runtime service instead:
ObjectName service = new ObjectName("com.bea:Name=DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean");
and be sure to access the admin port on the WLS-cluster.

if this is one time, the easiest would be to do it through the console...

the program in below link helps you to clear only pending messages from queue based on redelivered message parameter
http://techytalks.blogspot.in/2016/02/deletepurge-pending-messages-from-jms.html

Related

How to integrate a Spring RMI server with a pure Java RMI client which is a non-spring Swing GUI?

I'm migrating a J2EE EJB application to Spring services. It's a desktop application which has a Swing GUI and to communicate to the J2EE server it uses RMI. I have created a simple spring service with spring boot which exports a service by using spring remoting, RMIServiceExporter. The client is a rich client and have a complicated architecture so i'm trying make minimum changes to it to call the spring rmi service.
So in summary I have a plain RMI client and a spring RMI server. I have learned that spring rmi abstracts pure java rmi so in my case they don't interoperate.
I will show the code below but the current error is this. Note that my current project uses "remote://". So after I have got this error I have also tried "rmi://". But, in both cases it gives this error.
javax.naming.CommunicationException: Failed to connect to any server. Servers tried: [rmi://yyy:1099 (No connection provider for URI scheme "rmi" is installed)]
at org.jboss.naming.remote.client.HaRemoteNamingStore.failOverSequence(HaRemoteNamingStore.java:244)
at org.jboss.naming.remote.client.HaRemoteNamingStore.namingStore(HaRemoteNamingStore.java:149)
at org.jboss.naming.remote.client.HaRemoteNamingStore.namingOperation(HaRemoteNamingStore.java:130)
at org.jboss.naming.remote.client.HaRemoteNamingStore.lookup(HaRemoteNamingStore.java:272)
at org.jboss.naming.remote.client.RemoteContext.lookupInternal(RemoteContext.java:104)
at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:93)
at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:146)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at com.xxx.ui.common.communication.JbossRemotingInvocationFactory.getRemoteObject(JbossRemotingInvocationFactory.java:63)
at com.xxx.gui.comm.CommManager.initializeSpringEJBz(CommManager.java:806)
at com.xxx.gui.comm.CommManager.initializeEJBz(CommManager.java:816)
at com.xxx.gui.comm.CommManager.initializeAndLogin(CommManager.java:373)
at com.xxx.gui.comm.CommManager$2.doInBackground(CommManager.java:273)
at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at javax.swing.SwingWorker.run(SwingWorker.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
I have searched for how we can interoperate spring rmi and plain/pure java rmi and i read several answers from similar questions at stackoverflow and web but i couldn't find anything useful or fits my case because even the best matched answer says only that it doesn't interoperate.
I thought that maybe i need to turn my swing gui client to spring by using spring boot but i couldn't be sure about application context since i don't want to break existing client code. So i have looked for maybe there is something like partial spring context so that maybe i can put only my CommManager.java client code to it and spring only manages this file.
And then I thought that maybe I need to change my RMI server to force spring to create some kind of plain/pure Java RMI instead of default spring RMI thing. I say thing because I read something about spring rmi that explains it's an abstraction over rmi and we can force it to create standard RMI stub.
While I'm searching for a solution i have encountered the Spring Integration but I couldn't understand it really since it looks like an other abstraction but it also tell something about adapters. Since I have seen "adapter" maybe it is used for this kind of integration/legacy code migration cases. But I couldn't go further.
Client Side:
CommManager.java
private boolean initializeEJBz(String userName, String password) throws Exception {
...
ri = RemoteInvocationFactory.getRemoteInvocation(user, pass);
if (ri != null) {
return initializeEJBz(ri);
} else {
return false;
}
}
RemoteInvocationFactory.java
package com.xxx.ui.common.communication;
import javax.naming.NamingException;
public final class RemoteInvocationFactory {
private static final CommunicationProperties cp = new CommunicationProperties();
public static synchronized RemoteInvocation getRemoteInvocation(
byte[] userName, byte[] password) throws NamingException {
String url = System.getProperty("rmi://xxx.com:1099");
if (url != null) {
return new JbossRemotingInvocationFactory(userName, password, url);
}
return null;
}
...
JbossRemotingInvocationFactory.java
package com.xxx.ui.common.communication;
...
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
...
import java.util.Hashtable;
import java.util.concurrent.TimeUnit;
public class JbossRemotingInvocationFactory implements RemoteInvocation {
private final byte[] userName, password;
private final String providerURL;
private volatile InitialContext initialContext;
private final SecretKey secretKey;
private static final String SSL_ENABLED = "jboss.naming.client.connect.options.org.xnio.Options.SSL_ENABLED";
private static final String SSL_STARTTLS = "jboss.naming.client.connect.options.org.xnio.Options.SSL_STARTTLS";
private static final String TIMEOUT = "jboss.naming.client.connect.timeout";
private long timeoutValue;
private final boolean startSsl;
#SuppressWarnings("unchecked")
public JbossRemotingInvocationFactory(byte[] userName, byte[] password, String providerURL) {
try {
KeyGenerator keyGenerator = KeyGenerator.getInstance("AES");
keyGenerator.init(128);
secretKey = keyGenerator.generateKey();
this.providerURL = providerURL;
startSsl = Boolean.valueOf(System.getProperty(SSL_ENABLED));
String property = System.getProperty("myproject.connect.timeout");
if (property != null) {
try {
timeoutValue = TimeUnit.MILLISECONDS.convert(Long.parseLong(property), TimeUnit.SECONDS);
} catch (Exception e) {
timeoutValue = TimeUnit.MILLISECONDS.convert(10, TimeUnit.SECONDS);
}
}
Hashtable jndiProperties = new Hashtable();
this.userName = encrypt(userName);
addOptions(jndiProperties);
jndiProperties.put(Context.SECURITY_CREDENTIALS, new String(password, UTF_8));
initialContext = new InitialContext(jndiProperties);
this.password = encrypt(password);
} catch (NamingException | NoSuchAlgorithmException ne) {
throw new RuntimeException(ne);
}
}
#Override
#SuppressWarnings("unchecked")
public <T> T getRemoteObject(Class<T> object, String jndiName) throws NamingException {
if (initialContext != null) {
T value = (T) initialContext.lookup(jndiName);
initialContext.removeFromEnvironment(Context.SECURITY_CREDENTIALS);
initialContext.removeFromEnvironment(Context.SECURITY_PRINCIPAL);
return value;
} else {
throw new IllegalStateException();
}
}
#Override
public <T> T getRemoteObject(Class<T> object) throws NamingException {
throw new IllegalAccessError();
}
...
private void addOptions(Hashtable jndiProperties) {
jndiProperties.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
jndiProperties.put("jboss.naming.client.ejb.context", "true");
jndiProperties.put("jboss.naming.client.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");
jndiProperties.put("jboss.naming.client.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT", "false");
jndiProperties.put(SSL_STARTTLS, "false");
jndiProperties.put(TIMEOUT, Long.toString(timeoutValue));
if (startSsl) {
jndiProperties.put("jboss.naming.client.remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "true");
jndiProperties.put(SSL_ENABLED, "true");
}
jndiProperties.put("jboss.naming.client.connect.options.org.xnio.Options.SASL_DISALLOWED_MECHANISMS", "JBOSS-LOCAL-USER");
jndiProperties.put(Context.PROVIDER_URL, providerURL);
jndiProperties.put(Context.SECURITY_PRINCIPAL, new String(decrypt(userName), UTF_8));
}
#Override
public void reconnect() {
try {
Hashtable jndiProperties = new Hashtable();
addOptions(jndiProperties);
jndiProperties.put(Context.SECURITY_CREDENTIALS, new String(decrypt(password), UTF_8));
initialContext = new InitialContext(jndiProperties);
} catch (NamingException ignore) {
}
}
}
CommManager.java
private boolean initializeEJBz(RemoteInvocation remoteInvocation) throws Exception {
cs = remoteInvocation.getRemoteObject(CustomerService.class, JNDINames.CUSTOMER_SERVICE_REMOTE);
...
// here is the integration point. try to get RMI service exported.
myService = remoteInvocation.getRemoteObject(HelloWorldRMI.class, JNDINames.HELLO_WORLD_REMOTE);
return true;
}
public static final String CUSTOMER_SERVICE_REMOTE = getRemoteBean("CustomerServiceBean", CustomerService.class.getName());
public static final string HELLO_WORLD_REMOTE = getRemoteBean("HelloWorldRMI", HelloWorldRMI.class.getName());
...
private static final String APPLICATION_NAME = "XXX";
private static final String MODULE_NAME = "YYYY";
...
protected static String getRemoteBean(String beanName, String interfaceName) {
return String.format("%s/%s/%s!%s", APPLICATION_NAME, MODULE_NAME, beanName, interfaceName);
}
Server Side:
HelloWorldRMI.java:
package com.example.springrmiserver.service;
public interface HelloWorldRMI {
public String sayHelloRmi(String msg);
}
HelloWorldRMIImpl:
package com.example.springrmiserver.service;
import java.util.Date;
public class HelloWorldRMIimpl implements HelloWorldRMI {
#Override
public String sayHelloRmi(String msg) {
System.out.println("================Server Side ========================");
System.out.println("Inside Rmi IMPL - Incoming msg : " + msg);
return "Hello " + msg + " :: Response time - > " + new Date();
}
}
Config.java:
package com.example.springrmiserver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.remoting.rmi.RmiServiceExporter;
import org.springframework.remoting.support.RemoteExporter;
import com.example.springrmiserver.service.HelloWorldRMI;
import com.example.springrmiserver.service.HelloWorldRMIimpl;
#Configuration
public class Config {
#Bean
RemoteExporter registerRMIExporter() {
RmiServiceExporter exporter = new RmiServiceExporter();
exporter.setServiceName("helloworldrmi");
//exporter.setRegistryPort(1190);
exporter.setServiceInterface(HelloWorldRMI.class);
exporter.setService(new HelloWorldRMIimpl());
return exporter;
}
}
SpringServerApplication.java:
package com.example.springrmiserver;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import java.util.Collections;
#SpringBootApplication
public class SpringRmiServerApplication {
public static void main(String[] args)
{
//SpringApplication.run(SpringRmiServerApplication.class, args);
SpringApplication app = new SpringApplication(SpringRmiServerApplication.class);
app.setDefaultProperties(Collections.singletonMap("server.port", "8084"));
app.run(args);
}
}
So, my problem is how to interoperate pure/plain/standard java rmi client which is in a swing GUI with spring rmi server?
Edit #1:
By the way if you can provide further explanations or links about internal details of spring RMI stub creation and why they don't interoperate i will be happy. Thanks indeed.
And also, if you look at my getRemoteBean method which is from legacy code, how does this lookup string works? I mean where does rmi registry file or something resides at server or is this the default format or can i customize it?
Edit #2:
I have also tried this kind of lookup in the client:
private void initializeSpringEJBz(RemoteInvocation remoteInvocation) throws Exception {
HelloWorldRMI helloWorldService = (HelloWorldRMI) Naming.lookup("rmi://xxx:1099/helloworldrmi");
System.out.println("Output" + helloWorldService.sayHelloRmi("hello "));
//hw = remoteInvocation.getRemoteObject(HelloWorldRMI.class, "helloworldrmi");
}
Edit #3:
While I'm searching i found that someone in a spring forum suggested that to force spring to create plain java rmi stub we have to make some changes on the server side so i have tried this:
import java.rmi.server.RemoteObject;
public interface HelloWorldRMI extends **Remote** {
public String sayHelloRmi(String msg) throws **RemoteException**;
...
}
...
public class HelloWorldRMIimpl extends **RemoteObject** implements HelloWorldRMI {
...
}
Is the code above on the right path to solve the problem?
Beside that the first problem is the connection setup as you can see in the beginning of the question. Why i'm getting this error? What is the difference between "rmi://" and "remote://" ?
While I was trying to figure out, I could be able to find a solution. It's true that Spring RMI and Java RMI do not interoperate but currently i don't have enough knowledge to explain its cause. I couldn't find any complete explanation about internals of this mismatch yet.
The solution is using plain Java RMI in Spring backend by using java.rmi.*(Remote, RemoteException and server.UnicastRemoteObject).
java.rmi.server.UnicastRemoteObject is used for exporting a remote object with Java Remote Method Protocol (JRMP) and obtaining a stub that communicates to the remote object.
Edit:
I think this post is closely related to this interoperability issue: Java Spring RMI Activation
Spring doesn't support RMI activation. Spring includes an RmiServiceExporter for calling remote objects that contains nice improvements over standard RMI, such as not requiring that services extend java.rmi.Remote.
Solution:
This is the interface that server exports:
package com.xxx.ejb.interf;
import java.rmi.Remote;
import java.rmi.RemoteException;
public interface HelloWorldRMI extends Remote {
public String sayHelloRmi(String msg) throws RemoteException;
}
and this is the implementation of exported class:
package com.xxx.proxyserver.service;
import com.xxx.ejb.interf.HelloWorldRMI;
import java.rmi.RemoteException;
import java.rmi.server.UnicastRemoteObject;
import java.util.Date;
public class HelloWorldRMIimpl extends UnicastRemoteObject implements HelloWorldRMI {
public HelloWorldRMIimpl() throws RemoteException{
super();
}
#Override
public String sayHelloRmi(String msg) {
System.out.println("================Server Side ========================");
System.out.println("Inside Rmi IMPL - Incoming msg : " + msg);
return "Hello " + msg + " :: Response time - > " + new Date();
}
}
and the RMI Registry is:
package com.xxx.proxyserver;
import com.xxx.proxyserver.service.CustomerServiceImpl;
import com.xxx.proxyserver.service.HelloWorldRMIimpl;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
import java.util.Collections;
#SpringBootApplication
public class ProxyServerApplication {
public static void main(String[] args) throws Exception
{
Registry registry = LocateRegistry.createRegistry(1200); // this line of code automatic creates a new RMI-Registry. Existing one can be also reused.
System.out.println("Registry created !");
registry.rebind("just_an_alias",new HelloWorldRMIimpl());
registry.rebind("path/to/service_as_registry_key/CustomerService", new CustomerServiceImpl());
SpringApplication app = new SpringApplication(ProxyServerApplication.class);
app.setDefaultProperties(Collections.singletonMap("server.port", "8084")); // Service port
app.run(args);
}
}
Client:
...
HelloWorldRMI helloWorldService = (HelloWorldRMI)Naming.lookup("rmi://st-spotfixapp1:1200/just_an_alias");
System.out.println("Output" + helloWorldService.sayHelloRmi("hello from client ... "));
...

InstanceNotFoundException when trying to get Activemq MBean

I have the following configuration:
#Configuration
public class ConfigureRMI {
#Value("${jmx.rmi.host:localhost}")
private String rmiHost;
#Value("${jmx.rmi.port:1099}")
private Integer rmiPort;
#Bean
public RmiRegistryFactoryBean rmiRegistry() {
final RmiRegistryFactoryBean rmiRegistryFactoryBean = new RmiRegistryFactoryBean();
rmiRegistryFactoryBean.setPort(rmiPort);
rmiRegistryFactoryBean.setAlwaysCreate(true);
return rmiRegistryFactoryBean;
}
#Bean
#DependsOn("rmiRegistry")
public ConnectorServerFactoryBean connectorServerFactoryBean() throws Exception {
final ConnectorServerFactoryBean connectorServerFactoryBean = new ConnectorServerFactoryBean();
connectorServerFactoryBean.setObjectName("connector:name=rmi");
connectorServerFactoryBean.setServiceUrl(String.format("service:jmx:rmi://%s:%s/jndi/rmi://%s:%s/jmxrmi", rmiHost, rmiPort, rmiHost, rmiPort));
return connectorServerFactoryBean;
}
#Bean
#DependsOn("connectorServerFactoryBean")
public DestinationViewMBean queueMonitor() {
JMXConnectorServer connector = null;
MBeanServerConnection connection;
ObjectName nameConsumers;
try {
connector = connectorServerFactoryBean().getObject();
connection = connector.getMBeanServer();
nameConsumers = new ObjectName("org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Queue,destinationName=tasks");
} catch (Exception e) {
e.printStackTrace();
return null;
}
DestinationViewMBean mbView = MBeanServerInvocationHandler.newProxyInstance(connection, nameConsumers, DestinationViewMBean.class, true);
return mbView;
}
}
It configures and instantiates DestinationViewMBean that I try to use later in code like this:
Long queueSize = queueMonitor.getQueueSize();
But it throws an exception javax.management.InstanceNotFoundException: org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Queue,destinationName=tasks
I'm sure the names are as I typed. I can see the brocker name and tasks queue in ActiveMQ web console, elements are queued and dequeued as intended. But I cant't monitor the queue size. The method I used (the one I provided) was made from many answers here on SO and man pages on JMX and ActiveMQ.
I'm wondering if I'm missing something obvious. I turned firewall down, I'm on localhost. Why can't DestinationViewMBean find the queue?
UPD: I used JConsole to check the MBean name. I managed to fix InstanceNotFoundException but now I can't get any attribute from the bean. I've tried a lot of them in debugger (just run throught the attributes I could find in DestinationViewMBean interface). But on every try of attribute getter I get javax.management.AttributeNotFoundException: getAttribute failed: ModelMBeanAttributeInfo not found for QueueSize (or any other attribute).

WMQ connection socket constantly closed between v9_client and v6_server

We have a standalone java application using third-party tool to manage connection pooling, which worked for us in v6_client + v6_server setup for a long time.
Now we are trying to migrate from v6 to v9 (yes, we are pretty late to the party.....), and found v9_client connection to v6_server connection is constantly interrupted, meaning:
Socket created by XAQueueConnectionFactory#createXAConnection() is always closed immediately, and the created XAConnection seems to be unaware of it.
Due to socket close mentioned above, XASession created from the XAConnection.createXASession() always creates a new socket and close the socket after XASession.close().
We went throught the complete list of tunables for v9_client (XAQCF
column in https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.ref.dev.doc/q111800_.html) and only spot two potential new configs we haven't used in v6_client, SHARECONVALLOWED and PROVIDERVERSION. Unfortunately neither helps us out..... Specifically:
We tried setShareConvAllowed(WMQConstants.WMQ_SHARE_CONV_ALLOWED_[YES/NO]) Considering there's no SHARECNV property in v6_server side, not a surprise.
We tried "Migration/Restricted/Normal Mode" by setProviderVersion("[6/7/8") ([7/8] throws exceptions, as expected....).
Just wondering if anybody else had similar experience and could share some insights. We tried v9_server+v9_client and haven't seen any similar problem, so that could also be our eventual solution.....
Btw, the WMQ is hosted on linux (RedHat), and we only use products of MQXAQueueConnectionFactory on client side (ibm mq class for jms).
Thanks.
Additional details/updates.
[update-1]
--[playgrond setup]
v9_client jars:
javax.jms-api-2.0.jar
com.ibm.mq.allclient(-9.0.0.[1/5]).jar
v6_client jars:
in addition to v9_client jars, introduced the following jars in eclipse classpath
com.ibm.dhbcore-1.0.jar
com.ibm.mq.pcf-6.0.3.jar
com.ibm.mqjms-6.0.2.2.jar
com.ibm.mq-6.0.2.2.jar
com.ibm.mqetclient-6.0.2.2.jar
connector.jar
jta-1.1.jar
Testing code - single thread:
import javax.jms.*;
import com.ibm.mq.jms.*;
import com.ibm.msg.client.wmq.WMQConstants;
public class MQSeries_simpleAuditQ {
private static String queueManager = "QM.RCTQ.ALL.01";
private static String host = "localhost";
private static int port = 26005;
public static void main(String[] args) throws JMSException {
MQXAQueueConnectionFactory queueFactory= new MQXAQueueConnectionFactory();
System.out.println("\n\n\n*******************\nqueueFactory implementation version: " +
queueFactory.getClass().getPackage().getImplementationVersion() + "*****************\n\n\n");
queueFactory.setHostName(host);
queueFactory.setPort(port);
queueFactory.setQueueManager(queueManager);
queueFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
if (queueFactory.getClass().getPackage().getImplementationVersion().split("\\.")[0].equals("9")) {
queueFactory.setProviderVersion("6");
//queueFactory.setShareConvAllowed(WMQConstants.WMQ_SHARE_CONV_ALLOWED_YES);
}
XASession xaSession;
javax.jms.QueueConnection xaQueueConnection;
try {
// Obtain a QueueConnection
System.out.println("Creating Connection...");
xaQueueConnection = (QueueConnection)queueFactory.createXAConnection(" ", "");
xaQueueConnection.start();
for (int counter=0; counter<2; counter++) {
try {
xaSession = ((XAConnection)xaQueueConnection).createXASession();
xaSession.close();
} catch (Exception ex) {
System.out.println(ex);
}
}
System.out.println("Closing connection.... ");
xaQueueConnection.close();
} catch (Exception e) {
System.out.println("Error processing " + e.getMessage());
}
}
}
--[observations]
v6_client only created and close a single socket, while v9_client (both 9.0.0.[1/5]):
socket created and closed right after xaQueueConnection = (QueueConnection)queueFactory.createXAConnection(" ", "");
in the inner for loop, socket created right after xaSession = ((XAConnection)xaQueueConnection).createXASession();, and closed after xaSession.close();
Naively i was expecting socket remains open until xaQueueConnection.close().
[update-2]
Using queueFactory.setProviderVersion("9"); and queueFactory.setShareConvAllowed(WMQConstants.WMQ_SHARE_CONV_ALLOWED_YES); for v9_server+v9_client, we don't see the same constant socket close issue in v6_server+v9_client, which is a good news.
[update-3] MCAUSER on attribute for all SVRCONN channel on v6_server. Same on v9_server (which doesn't have the same socket close problem when connected with the same v9_client).
display channel (SYSTEM.ADMIN.SVRCONN)
MCAUSER(mqm)
display channel (SYSTEM.AUTO.SVRCONN)
MCAUSER( )
display channel (SYSTEM.DEF.SVRCONN)
MCAUSER( )
[update-4]
I tried setting MCAUSER() to mqm, then using both (blank) and mqm from client side, both can create connections, but still seeing the same unexpected socket close using v9_client+v6_user. After updating MCAUSER(), i always added refresh security, and restart the qmgr.
I also tried setting system variable to blank in eclipse before creating the connection using blank user, didn't help either.
[update-5]
Limiting our discussion to v9_client+v9_server. The async testing code below generates a ton of xasession create/close request, using limited number of existing connections. Using SHARECNV(1) we would also end up with unaffordable high TIME_WAIT count, but using larger than 1 SHARECNV (eg. 10) might introduce extra performance penalty......
Async testing code
import java.util.ArrayList;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.FutureTask;
import javax.jms.*;
import com.ibm.mq.jms.*;
import com.ibm.msg.client.wmq.WMQConstants;
public class MQSeries_simpleAuditQ_Async_v9 {
private static String queueManager = "QM.ALPQ.ALL.01";
private static int port = 26010;
private static String host = "localhost";
private static int connCount = 20;
private static int amp = 100;
private static ExecutorService amplifier = Executors.newFixedThreadPool(amp);
public static void main(String[] args) throws JMSException {
MQXAQueueConnectionFactory queueFactory= new MQXAQueueConnectionFactory();
System.out.println("\n\n\n*******************\nqueueFactory implementation version: " +
queueFactory.getClass().getPackage().getImplementationVersion() + "*****************\n\n\n");
queueFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
if (queueFactory.getClass().getPackage().getImplementationVersion().split("\\.")[0].equals("9")) {
queueFactory.setProviderVersion("9");
queueFactory.setShareConvAllowed(WMQConstants.WMQ_SHARE_CONV_ALLOWED_YES);
}
queueFactory.setHostName(host);
queueFactory.setPort(port);
queueFactory.setQueueManager(queueManager);
//queueFactory.setChannel("");
ArrayList<QueueConnection> xaQueueConnections = new ArrayList<QueueConnection>();
try {
// Obtain a QueueConnection
System.out.println("Creating Connection...");
//System.setProperty("user.name", "mqm");
//System.out.println("system username: " + System.getProperty("user.name"));
for (int ct=0; ct<connCount; ct++) {
// xaQueueConnection instance of MQXAQueueConnection
QueueConnection xaQueueConnection = (QueueConnection)queueFactory.createXAConnection(" ", "");
xaQueueConnection.start();
xaQueueConnections.add(xaQueueConnection);
}
ArrayList<Double> totalElapsedTimeRecord = new ArrayList<Double>();
ArrayList<FutureTask<Double>> taskBuffer = new ArrayList<FutureTask<Double>>();
for (int loop=0; loop <= 10; loop++) {
try {
for (int i=0; i<amp; i++) {
int idx = (int)(Math.random()*((connCount)));
System.out.println("Using connection: " + idx);
FutureTask<Double> xaSessionPoker = new FutureTask<Double>(new XASessionPoker(xaQueueConnections.get(idx)));
amplifier.submit(xaSessionPoker);
taskBuffer.add(xaSessionPoker);
}
System.out.println("loop " + loop + " completed");
} catch (Exception ex) {
System.out.println(ex);
}
}
for (FutureTask<Double> xaSessionPoker : taskBuffer) {
totalElapsedTimeRecord.add(xaSessionPoker.get());
}
System.out.println("Average xaSession poking time: " + calcAverage(totalElapsedTimeRecord));
System.out.println("Closing connections.... ");
for (QueueConnection xaQueueConnection : xaQueueConnections) {
xaQueueConnection.close();
}
} catch (Exception e) {
System.out.println("Error processing " + e.getMessage());
}
amplifier.shutdown();
}
private static double calcAverage(ArrayList<Double> myArr) {
double sum = 0;
for (Double val : myArr) {
sum += val;
}
return sum/myArr.size();
}
// create and close session through QueueConnection object passed in.
private static class XASessionPoker implements Callable<Double> {
// conn instance of MQXAQueueConnection. ref. QueueProviderService
private QueueConnection conn;
XASessionPoker(QueueConnection conn) {
this.conn = conn;
}
#Override
public Double call() throws Exception {
XASession xaSession;
double elapsed = 0;
try {
final long start = System.currentTimeMillis();
// ref. DualSessionWrapper
// xaSession instance of MQXAQueueSession
xaSession = ((XAConnection) conn).createXASession();
xaSession.close();
final long end = System.currentTimeMillis();
elapsed = (end - start) / 1000.0;
} catch (Exception e) {
// TODO Auto-generated catch block
System.out.println(e);
}
return elapsed;
}
}
}
We found the root cause is a combination of no more session pooling + bitronix TM doesn't offer session pooling across TX. Specifically (in our case), bitronix manages JmsPooledConnection pooling, but everytime a (xa)session is used (under JmsPooledConnection), a new socket is created (createXASession()) and closed (xaSession.close()).
One solution, is to wrap the jms connection with (xa)session pool, similar to what's been done in https://github.com/messaginghub/pooled-jms/tree/master/pooled-jms/src/main/java/org/messaginghub/pooled/jms
http://bjansen.github.io/java/2018/03/04/high-performance-mq-jms.html also suggests Spring CachingConnectionFactory works well, which sounds like a speical case of the first solution.

ClassNotFound in HazelcastMembers for ReplicatedMaps, but ok for Maps

Our Problem might be similar to that one:
Hazelcast ClassNotFoundException for replicated maps
Since the description of the environment is not given in detail I describe our problematic enironment here:
We have a dedicated Hazelcast Server(Member), out of the box with some config. No additional classes added (The ones from our project).
Then we got two Hazelcast Clients using this Member with several of our own classes.
The Clients intend to use Replicated Maps, so at some point in our software they do "hazelcastInstance.getReplicatedMap("MyName")" and then do some put operations.
Doing this, the dedicated hazelcast server throws a ClassNotFound for our classes we want to put into the replicated map. I understand this. How should he know about the classes.
Then I change to a Map insteadof replicatedMap.
"hazelcastInstance.getMap("MyName")"
With no other change it works. And this is what makes me wonder how that can be?
Does this have to do with different InMemory Storage? Does replicatedMap here behave differently ?
Hazelcast Version is: 3.9.2
One info might be important: the Client configures a NearCache for all the maps used:
EvictionConfig evictionConfig = new EvictionConfig()
.setMaximumSizePolicy(EvictionConfig.MaxSizePolicy.ENTRY_COUNT)
.setSize(eapCacheId.getMaxAmountOfValues());
new NearCacheConfig()
.setName(eapCacheId.buildFullName())
.setInMemoryFormat(InMemoryFormat.OBJECT)
.setInvalidateOnChange(true)
.setEvictionConfig(evictionConfig);
}
I changed the InMemoryFormat to BINARY. Still the same ClassNotFound
The Start of the stacktrace is:
at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:224)
at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:185)
at com.hazelcast.spi.impl.NodeEngineImpl.toObject(NodeEngineImpl.java:339)
EDIT: wrote a little Test do demonstrate my problem:
package de.empic.hazelclient.client;
import com.hazelcast.client.HazelcastClient;
import com.hazelcast.client.config.ClientConfig;
import com.hazelcast.config.EvictionConfig;
import com.hazelcast.config.InMemoryFormat;
import com.hazelcast.config.NearCacheConfig;
import com.hazelcast.core.HazelcastInstance;
import java.util.Map;
import java.util.Random;
public class HazelClient {
private static final String[] MAP_KEYS = {"Mike", "Ben", "Luis", "Adria", "Lena"};
private static final String MAP_NAME = "Regular Map" ;
private static final String REPLICATED_MAP_NAME = "Replicated Map" ;
private static final String CACHE_MEMBERS = "192.168.56.101:5701" ;
private static final String MNGT_CENTER = "192.168.56.101:5701" ;
HazelcastInstance hazelClientInstance = null ;
private static Random rand = new Random(System.currentTimeMillis());
public static void main(String[] args) {
new HazelClient(true).loop();
}
private HazelClient(boolean useNearCache)
{
ClientConfig cfg = prepareClientConfig(useNearCache) ;
hazelClientInstance = HazelcastClient.newHazelcastClient(cfg);
}
private void loop()
{
Map<String, SampleSerializable> testMap = hazelClientInstance.getMap(MAP_NAME);
Map<String, SampleSerializable> testReplicatedMap = hazelClientInstance.getReplicatedMap(REPLICATED_MAP_NAME);
int count = 0 ;
while ( true )
{
// do a random write to map
testMap.put(MAP_KEYS[rand.nextInt(MAP_KEYS.length)], new SampleSerializable());
// do a random write to replicated map
testReplicatedMap.put(MAP_KEYS[rand.nextInt(MAP_KEYS.length)], new SampleSerializable());
if ( ++count == 10)
{
// after a while we print the map contents
System.out.println("MAP Content -------------------------");
printMapContent(testMap) ;
System.out.println("REPLIACTED MAP Content --------------");
printMapContent(testReplicatedMap) ;
count = 0 ;
}
// we do not want to drown in system outs....
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private void printMapContent(Map<String, SampleSerializable> map)
{
for ( String currKey : map.keySet())
{
System.out.println(String.format(" - %s -> %s", currKey, map.get(currKey)));
}
}
private ClientConfig prepareClientConfig(boolean useNearCache)
{
ClientConfig cfg = new ClientConfig();
cfg.setInstanceName("SampleInstance");
cfg.getProperties().put("hazelcast.client.statistics.enabled", "true");
cfg.getProperties().put("hazelcast.client.statistics.period.seconds", "5");
if ( useNearCache )
{
cfg.addNearCacheConfig(defineNearCache(MAP_NAME));
cfg.addNearCacheConfig(defineNearCache(REPLICATED_MAP_NAME));
}
// we use a single member for demo
String[] members = {CACHE_MEMBERS} ;
cfg.getNetworkConfig().addAddress(members);
return cfg ;
}
private NearCacheConfig defineNearCache(String name)
{
EvictionConfig evictionConfig = new EvictionConfig()
.setMaximumSizePolicy(EvictionConfig.MaxSizePolicy.ENTRY_COUNT)
.setSize(Integer.MAX_VALUE);
NearCacheConfig nearCacheConfig = new NearCacheConfig()
.setName(name)
.setInMemoryFormat(InMemoryFormat.OBJECT)
.setInvalidateOnChange(true)
.setEvictionConfig(evictionConfig) ;
return nearCacheConfig;
}
}
To have the full info, the Hazelcast member is started with the following xml:
<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config https://hazelcast.com/schema/config/hazelcast-config-3.8.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<instance-name>server-cache</instance-name>
<network>
<port>5701</port>
<join>
<multicast enabled="false"/>
<tcp-ip enabled="true">
<members>192.168.56.101:5701</members>
</tcp-ip>
</join>
<public-address>192.168.56.101:5701</public-address>
</network>
<management-center enabled="true">http://192.168.56.101:6679/mancenter</management-center>
</hazelcast>
The fact that the Hazelcast Member is running in docker while the clients are not is not important I think.
Can you post your configurations for both mapConfig and replicatedMapConfig? I will try to reproduce this.
I’m thinking this has to do with where the serialization happens. A couple things to keep in mind, there are two different configurations for map and replicatedmap. When you changed your getReplicatedMap("MyName") to .getMap("MyName"), if you don’t have a map config for “MyName”, then it will use the default config.
By default, Replicated Maps store in object memory format for performance.
I found my mistake. I configured the near cache to be of memory type "BINARY"
The server itself I did not configure. After having the replicated maps defined as "BINARY" in the server it works.

Dummy TIBCO queue for testing

I have written a piece of code to connect to TIBCO queue and fetch customer data. But I cannot find a way to test it before moving it to production. Can we create a dummy queue somehow so that I can ensure that my program is functional before packaging it for production?
Here is my code..
package GI;
import javax.jms.JMSSecurityException;
import javax.naming.InitialContext;
import javax.jms.Queue;
import javax.jms.TextMessage;
import javax.jms.QueueSender;
import javax.jms.DeliveryMode;
import javax.jms.QueueSession;
import javax.jms.QueueConnection;
import javax.jms.QueueConnectionFactory;
import javax.jms.QueueReceiver;
public class Queue
{
public Queue()
{
System.err.println("\n------------------------------------------------------------------------");
System.err.println("tibjmsTEQueue SAMPLE");
System.err.println("------------------------------------------------------------------------");
}
public String QueueProcess(String XMLRequest, String serverUrl, String userName, String password, String InQueueName)
{
String [] args = new String[4];
args[0]=serverUrl;
args[1]=userName;
args[2]=password;
args[3]=InQueueName;
System.out.println("Server....................... "+serverUrl);
System.out.println("User......................... "+userName);
System.out.println("InQueue...................... "+InQueueName);
System.out.println("------------------------------------------------------------------------\n");
try
{
tibjmsUtilities.initSSLParams(serverUrl,args);
}
catch (JMSSecurityException e)
{
System.err.println("JMSSecurityException: "+e.getMessage()+", provider="+e.getErrorCode());
e.printStackTrace();
System.exit(0);
}
System.err.println("Starting");
QueueConnection connectionIn = null;
try
{
InitialContext context = new InitialContext();
Queue tibcoQueue = (Queue) context.lookup("queue/queue0");
QueueConnectionFactory factory = new com.tibco.tibjms.TibjmsQueueConnectionFactory(serverUrl);
connectionIn = factory.createQueueConnection(userName, password);
QueueSession sessionIn = connectionIn.createQueueSession(false,javax.jms.Session.AUTO_ACKNOWLEDGE);
QueueSender queueSender = sessionIn.createSender(tibcoQueue);
queueSender.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
QueueReceiver queueReceiver = sessionIn.createReceiver(tibcoQueue);
connectionIn.start();
System.out.println("Connected to Queue...");
System.out.println("XMLRequest: " + XMLRequest);
TextMessage sendXMLRequest = sessionIn.createTextMessage(XMLRequest);
queueSender.send(sendXMLRequest);
System.out.println("sent: " + sendXMLRequest.getText());
TextMessage XMLResponse = (TextMessage) queueReceiver.receive();
System.out.println("received: " + XMLResponse.getText());
return((String) XMLResponse.getText());
}
catch(Exception e)
{
System.err.println("Exitting with Error");
e.printStackTrace();
System.exit(0);
}
finally
{
System.err.println("Exitting");
try {connectionIn.close();}
catch(Exception e1) {}
}
return("Failure");
}
public static void main(String args[])
{
Queue t = new Queue();
}
}
It is never suggested to mix production and non-production traffic on the same EMS messaging instance. That is a very risky practice since it introduces the possibility of inadvertently sending non-production traffic to production applications. I would suggest first checking with your EMS operations team as they can likely point you to a development EMS instance that you can use.
For unit and/or integration testing where you are not looking to stress-test the environment, many developers install an instance of EMS on their local development workstation. Again your EMS operations team may be able to provide you with a license and installation binary to install locally.

Resources