We have an existing J2EE application that uses WebSphere MQ to retrieve data from IMS.
The J2EE application sends the IMS transaction name to MQ, which retrieves IMS data. The returned data is then parsed for further processing.
Recently we migrated the application to WebSphere 7. The application worked fine on a windows box. However, when we ported the application to a zLinux (Linux on System z) box - we were able to talk to IMS and data was getting returned from IMS to J2EE application. The parsing process however raises an ArrayIndexOutofBoundsException.
The inputs are the same in both environments and with the operational code being the same(same java build) there is a significant difference observed in the behaviour. Is this anything to do with CharacterCodeSet not being accepted by zLinux environment ? We use a hard-coded value for CCSID from within the J2EE application.
Is it that zLinux environment does not support the existing CCSID requires a different CCSID?
Incidentally the answer to the above question lies in the BIG ENDIAN / LITTLE ENDIAN problem. Linux being little endian stores byte information differently compared to AIX / windows boxes. This was causing a parsing failure - i.e. the piece of code that was parsing the message returned from MQ successfully was not able to parse it in Linux when the format is different.
Related
I have a problem with a server that called server A:
Server A: Red Hat Enterprise Linux Server release 7.2 (Maipo)
Server B: Red Hat Enterprise Linux Server release 7.7 (Maipo)
jdk-8u231 installed on all of servers.
I have an Spring Boot application running on 2 servers.
Whenever i use Jmeter to send 100 concurrency request to application running on each servers, the application running on Server B have no problem.
But in Server A, the application will be not responding, that mean the Process (PID) still running but I can't visit actuator endpoint, cannot visit Swagger page, cannot send new request ... log file show nothing since that time.
Thread dump and heap dump have no significant difference.
Could anyone show me how to analysis that problem?
I still have no idea why the problem occur.
Well, I can only speculate here, but here some ideas that can help:
There are two possible sources of issue here Java Application and Linux (+its network policies, firewalls and so forth).
Since You don't know for sure, what happens, try working by "elimination".
Create a script that will run 100 concurrent requests. Place the script at the Server A (the problematic one) and run The script will run against "localhost" (obviously). If you see that it works, then the issue is not in Java at all. Probably some network policies or linux setup, who knows.
Place a log message in the controller of the java application and examine the log. The log should print the request number among other things, so that you'll be able to understand whether you get stuck after a well defined number of requests or its always a different number.
Check the configurations of Spring Boot application. Maybe there is a difference in the number of threads allocated to serve the request by the embedded web server that runs inside the spring boot application (assuming you're not using a reactive stack) and this number differs. In this case you won't be able to call rest endpoints, actuator, etc.
If JMX connection is available to the setup, connect via the JMX and check the MBean of Tomcat (again, assuming there is a tomcat under the hood) to check pretty much the same information as in 4.
You've mentioned thread dumps. Try to take more than one thread dump but one before you're running JMeter test, one during the running (when everything still works), one when everything is stuck.
In the thread dumps check the actual stacktraces, maybe all the threads are stuck working with Database or something and can't serve requests like I've explained in "4"
Examine GC logs, maybe GC works so hard that you can't really interact with the application.
We are migrating some apps from WAS full profile to WAS Liberty profile.
Some apps have MDBs and need JMS Activation Specs definitions connected to MQ.
In order to enforce strict FIFO ordering of messages in a cluster, we set the "WAS_EndpointInitialState" property to "INACTIVE" on those Activation Specs to tell WAS full profile to not start the Activation Spec on startup. When the cluster starts, we start (ie "resume") the activation on one server only.
Q: How to achieve this with Liberty (v16.0.x) ?
I don't see an equivalent parameter within the "properties.wmqJms" stanza.
Thanks
Liberty doesn't have an equivalent parameter/capability for activation specs.
You can open a request for enhancement here:
https://www.ibm.com/developerworks/rfe/?PROD_ID=544
In case it helps during the meantime, a crude way of simulating the capability is to start the server with the jmsActivationSpec elements commented out, and make configuration updates to uncomment as you want them activated.
Unfortunately as-is (with v16.0.0.3 and the current beta version), it is not possible to deploy an application with MDBs in production due to a serious lack on functionalities in Liberty profile (JMS Activations).
When using the jmsActivationSpec+ properties.wmqJms stanzas, it is impossible:
to configure the activation to stop after x failed tentatives to consume the message. Liberty tries to consume the message forever without any notification!!
to start the activation in an inactive state on startup., so it is impossible to enforce the FIFO paradigm on a Q when deployed in a cluster (or collective or other form of cluster)
Those are already captured in the following RFE:
https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=95885
https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=95794
https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=88543
For us it's a clear no-go to move to WebSphere Liberty profile for those reasons
This is way too late for the OP, but in case someone comes here looking for a current answer.
Liberty / Open Liberty now offer (as of 18.0.0.1) such a function, which you can enable via the autoStart attribute, e.g.:
<jmsActivationSpec autoStart="false" id="myJMSActSpec"/>
See here for a quick example of how you would use the EndpointControl MBean and/or the server resume CLI command to start message delivery into the server.
We are having some issues with an application of ours which we are attempting to diagnose. While taking a close look at things, we think we may be having some DBCP connection pool issues.
Among a few things we noticed, we discovered something via a secondary support application (small JDBC based sqlclient for monitoring the DB) using the same driver the main application uses. That discovery was entropy exhaustion. After applying the fix noted in Oracle JDBC intermittent Connection Issue to this small utility, the issue went away.
At that time, we suspected the main application could be suffering from the same problem. We did not apply the same fix at this point, but rather we've started monitoring available entropy via /proc/sys/kernel/random/entropy_avail every 5 seconds to validate.
After reviewing the data for a 24 hour period, we do not see the same drop in available entropy as with the jdbc utility prior to the use of /dev/urandom. Rather, we noticed that the entropy never drops below 128 bytes nor climbs above 191 bytes. We have searched the application configuration files and can't find anything related to specifying the random number source.
OS: Red Hat Enterprise Linux Server release 6.3 (Santiago)
JDBC Driver: ojdbc6-11.2.0.3
Pooling Method: Hibernate DBCP
So, my questions are:
1) If we've not knowingly told the application/driver to use /dev/urandom vs /dev/random, what would possibly explain why we don't see the same entropy drop when new pool connections are created?
2) Why would the minimum and maximum available entropy be so rigid at 128/191? I would expect a little more, pardon the pun, randomness in these values.
I hesitate to go posting a bunch of configuration files not knowing which may be relevant. If there is something particular you'd like to see, please let me know and I will share.
Does your application use JDBC connection pooling or does it make authentication attempts as frequently as your test application did/does?
Keep in mind that each authentication attempt consumes the random pool.
I am using WebSphere 6.2 and my requirement is I admin have to get an alert when the application on the server stopped or server down .. How to achieve this ? In higher versions this feature is there ? please help me
Thanks in Advance ,
Raj
I'm assuming you mean WebSphere application server processes (rather than the physical server on which WebSphere is running) and individual applications running on those processes. I'm also assuming you mean when those elements have stopped unexpectedly rather than when somebody has deliberately stopped them.
If so, you're going to have to use external monitoring software to detect most of those conditions. We use a combination of scripts that scan for processes and specific error messages in logs and external site-monitoring software that checks for application responsiveness. Such scripts can be standalone, handwritten scripts, or run under generic monitoring tools from IBM (Tivoli) or 3rd-parties.
Alternatively, I think you should also be able to write something that uses JMX to read specific things about WebSphere state, and there is at least one sophisticated monitoring tool you could purchase, IBM Tivoli Composite Application Manager (ITCAM) for Application Diagnostics, which can monitor WebSphere internals.
In Websphere MQ series , command level for a queue manager is 701. What does it actually specify ?
WebSphere products use a "[version].[release].[modification].[Fix Pack]" naming convention. For example 7.0.1.6 is the current release specified down to the Fix Pack level.
Fix packs are limited to bug fixes and very limited non-disruptive functionality enhancements.
Modifications can include functionality enhancements but no API changes. For example the Multi-Instance Queue Manager was introduced in 7.0.1.
Releases may introduce significant new function and limited API changes but are highly forward and backward compatible withing the same version.
Versions encapsulate a core set of functionality. Changes at this level may sacrifice some backward compatibility in trade for significant new functionality. For example, WMQ Pub/Sub was moved from Message Broker to base MQ in the V7 release.
Since administrative functionality does not change in Fix Packs but may change at the Modification level, compatibility with administrative tools is based on the queue manager's Command Level.
There is an old but still useful TechNote which described this when the numbering conventions were adopted for WMQ.
It displays the major version number of WMQ - e.g. 530,600,700,701. Despite being 'only' a .0.1 increment, WMQ 7.0.1 gets a new major version number due to a number of internal changes (e.g. multi-instance QMs), although WMQ 6.0.1.x and 6.0.2.x were both CMDLEVEL 600
Command level, although similar to the V.R.M.F, it not exactly the same thing. The Command level is used to allow configuration applications to know what commands (and attributes within those commands) will be understood by the command server.
The first thing any configuration application should do is discover the PLATFORM and CMDLEVEL of the queue manager. Then that application can determine which commands/attributes it would be acceptable to send to that queue manager.
It is possible that CMDLEVEL could be increased in the service stream. Then the V.R.M.F. would not necessarily match the CMDLEVEL. This would happen if some new external attributes were added in the service stream, so queue managers without that patch would not understand them, but queue managers with the patch would. How does an application determine what to send? Well, the CMDLEVEL would determine that and so would have to be upped by the patch.