I am using JBoss 7x, and have the following use case.
I am going to do load testing of messaging queues with Jboss. The queues are external to JBoss.
I will push a lot of message in the queue, around 1000 message. When around 100+ message has been pushed I want to crash JBoss. Later I want to re-start the Jboss the verify the message processing.
I had earlier made use of Byteman to crash the JVM using the following
JAVA_OPTS="-javaagent:/BYTEMAN_HOME/lib/byteman.jar=script:/QUICKSTART_HOME/jta-crash-rec/src/main/scripts/xa.btm ${JAVA_OPTS}"
Details are here: https://github.com/Naresh-Chaurasia/jboss-eap-quickstarts/tree/7.3.x/jta-crash-rec
In the above case when ever XA Transaction is happening the JVM is being crashed using byteman, but in my case I want to only crash the JVM/Jboss lets say after 100+ messages. i.e not for each transaction but after processing some messages.
I have also tried a few examples from here, to get ideas of how to achieve it, but did not succeed. https://developer.jboss.org/docs/DOC-17213#top
Question: How can I crash JBoss/ running JVM using byteman or some other way.
See the Programmers Guide that comes bundled with the distribution.
Sections headed "CountDowns" and "Aborting Execution" provide what's necessary. These are built-in features of the Rule Language.
Related
We use Spring Kafka Client on a project. Recently we noticed that if a Kafka Consumer dies due to OutOfMemoryError service continues operating normally and no new consumers get created. The only way to fix this is to monitor OOM in logs and restart the service manually.
We are looking for a way to make consumer recreation automatic, e.g.:
Force Spring (somehow) to detect died consumers and in runtime create new ones.
In case of OOM in consumer thread kill the entire service, so that AWS auto-scaling group can create a new instance of the service.
Any suggestions or ideas are appreciated.
Thank you!
I have found a solution related to option 2 mentioned above.
Starting java version "1.8.0_92" there are a few JVM options allowing to kill entire JVM in case of OOME:
1. -XX:+ExitOnOutOfMemoryError
2. -XX:+CrashOnOutOfMemoryError
were added, see the release notes.
ExitOnOutOfMemoryError When you enable this option, the JVM exits on the first occurrence of an out-of-memory error. It can be used if
you prefer restarting an instance of the JVM rather than handling out
of memory errors.
CrashOnOutOfMemoryError If this option is enabled, when an out-of-memory error occurs, the JVM crashes and produces text and
binary crash files.
I am using WebSphere 6.2 and my requirement is I admin have to get an alert when the application on the server stopped or server down .. How to achieve this ? In higher versions this feature is there ? please help me
Thanks in Advance ,
Raj
I'm assuming you mean WebSphere application server processes (rather than the physical server on which WebSphere is running) and individual applications running on those processes. I'm also assuming you mean when those elements have stopped unexpectedly rather than when somebody has deliberately stopped them.
If so, you're going to have to use external monitoring software to detect most of those conditions. We use a combination of scripts that scan for processes and specific error messages in logs and external site-monitoring software that checks for application responsiveness. Such scripts can be standalone, handwritten scripts, or run under generic monitoring tools from IBM (Tivoli) or 3rd-parties.
Alternatively, I think you should also be able to write something that uses JMX to read specific things about WebSphere state, and there is at least one sophisticated monitoring tool you could purchase, IBM Tivoli Composite Application Manager (ITCAM) for Application Diagnostics, which can monitor WebSphere internals.
I am running a listener program on a JMS queue hosted in Sun Java System Application Server 9.1_02 (build b08-p03)
After receiving a message I will extract and log some details in to a log file.
I observed that when running listener for really long hours, it stops receiving messages. I had to manually stop the program and start it again. Then it receives some 200-300 message and then stops again. I had to restart it to retrieve another set of messages.
Why is this weird behavior? Can someone throw light on this?
Thank you
Chaitanya
I found the issue myself. I am running the listener as Load Runner Java Vuser script. One negative thing about LR is that it does not show any exceptions. The script appears to be running but it actually does nothing. I found this when I ported the script to Eclipse and found that the script is throwing exceptions. This is essentially because I am not checking for a certain condition.
Happy that I demystified at last!
Thanks.
I have a top-down EJB Webservice (deployed on Websphere 6.01) that hangs the first time I call it after starting the JVM. If I see the logs, it seems to do everything normal that I expect it to (retrieving data from database) and I see all sysouts printed till the very last statement in EJB method. But client never gets the response back and eventually times out. Something is going on in the container (or in WS router application) after the control goes out of EJB but I don’t know how to troubleshoot it further. If it had something to do with startup, I would see slow down in the beginning of service invocation, not at the end.
From second call onwards it all works normal. Any suggestions would be appreciated.
Two possibile directions:
1). Try stubbing out all the implementation of the EJB, no database calls, no work. See whether you still get the problem. If you don't, then presumably something interesting is happening in your code. If you do, then there's something weird happening in the infrastructure - probably PMR time.
2). WebSphere has plenty of trace you can turn on. Try enabling trace relating to Web container. This may let you see where it's getting stuck.
I would recommend opening a PMR anyway.
I have a peculiar situation here. I have installed JBoss 5.1.0 as a service in Wintel box.
The service will restart itself if the JBoss instance fails.
However I could not find a way to test this scenario. I killed the JVM that was running the JBoss, but it did not restart the service. I need to make the JBoss service end abnormally so that I can ensure it is restarts again.
In a nutsehll, I need a way to make JBoss end abnormally.
Please help.
Write a JSP that calls System.exit(1)? That might fall foul of the Security Manager, though, and JBoss might not permit it.
In my experience, JBoss nodes (and app servers in general) tend not to crash in such a way as the process exists. Instead, they're more likely to consume increasing resources (e.g. memory) until they stop responding, and need explicit restarting. That's certainly easier to reproduce, but it's harder to handle automatically.