While tring to start websphere server it is throwing the error:
thread server startup:2'' (0000000a) has been active for 786590 milliseconds and may be hung.
How to resolve this WSVR0605W error?
That error will always be displayed when a thread has been running for a certain time. It is not for certain that this actually indicates an error, it depends on what your applications do on your server. If you have an operation that runs within the same thread for more than 10 minutes, this error will eventually pop up even though it is within the normal operation of your application. If this is the case you will later get a message of the style [10/29/13 8:23:11:008 CET] 00004709 ThreadMonitor W WSVR0606W: Thread "Thread_ID" (00004709) was previously reported to be hung but has completed. It was active for approximately 727968 milliseconds. There is/are 0 thread(s) in total in the server that still may be hung.
If you do not have anything that should be running for that long, the thread may indeed be hung and you need to investigate what operation exactly is hung. Start by examining the logs using the same thread ID as the one reported as possibly hung.
Related
This was Ultimate Thread Group in Jmeter.
I used Blazemeter extension to create .jmx file then replace with ultimate thread group
Although i put 30 second shut down time.
When it reach the maximum thread and Hold Load was finished it seem doesnt shut down thread (Very slow 1 or 2 thread every 20 seconds). Is there anything that i could do to solve this problems?
Most probably your application gets overloaded and starts responding slowly.
JMeter politely "asks" threads to stop when they will be free and threads in their turn kindly wait for the system under test to respond (in order to avoid potential Socket Closed errors). Once it responds they're immediately stopped.
So the solution is to set reasonable timeouts using i.e. HTTP Request Defaults
as JMeter doesn't have any pre-defined timeout and if server won't respond or cut the connection from its end - JMeter will wait for the response forever.
Also consider removing Listeners from your test plan, they don't add any value and just consume resources
I have a setup with a MQ Queue and a Liberty application taking messages from MQ an puts them into a database.
After random number of messages going nice and clean through I get a timeout exception (see below) When this happens it seems like the thread hangs utilizing 100% CPU, the more this happens the higher load on the server. Any suggestions?
com.ibm.tx.jta.embeddable.impl.EmbeddableTimeoutManager I WTRN0124I: When the timeout occurred the thread with which the transaction is, or was most recently, associated was Thread[Default Executor-thread-37,5,Default Executor Thread Group]. The stack trace of this thread when the timeout occurred was: com.ibm.mq.jmqi.remote.impl.RemoteSession.receiveOneTSH(RemoteSession.java:884) com.ibm.mq.jmqi.remote.impl.RemoteSession.receive1stGetReplyTSH(RemoteSession.java:1509) com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiGetMessageWithProps(RemoteFAP.java:9820) com.ibm.mq.jmqi.remote.api.RemoteFAP.MQGET(RemoteFAP.java:9511) com.ibm.mq.ese.jmqi.InterceptedJmqiImpl.MQGET(InterceptedJmqiImpl.java:1213) com.ibm.mq.ese.jmqi.ESEJMQI.MQGET(ESEJMQI.java:430) com.ibm.msg.client.wmq.compat.base.internal.MQSESSION.MQGET(MQSESSION.java:975) com.ibm.msg.client.wmq.compat.base.internal.MQQueue.getMsg2Int(MQQueue.java:1355) com.ibm.msg.client.wmq.compat.base.internal.MQQueue.getMsg2NoExc(MQQueue.java:1445) com.ibm.msg.client.wmq.compat.jms.internal.MQSession.consume(MQSession.java:1865) com.ibm.msg.client.wmq.compat.jms.internal.MQSession.loadMessageReference(MQSession.java:4224) com.ibm.msg.client.jms.internal.JmsSessionImpl.consume(JmsSessionImpl.java:3639) com.ibm.msg.client.jms.internal.JmsSessionImpl.run(JmsSessionImpl.java:3227) com.ibm.mq.jms.MQSession.run(MQSession.java:937) com.ibm.mq.connector.inbound.ASFWorkImpl.doDelivery(ASFWorkImpl.java:110) com.ibm.mq.connector.inbound.AbstractWorkImpl.run(AbstractWorkImpl.java:229) com.ibm.ws.jca.inbound.security.JCASecurityContextService.runInInboundSecurityContext(JCASecurityContextService.java:49) com.ibm.ws.jca.internal.WorkProxy.run(WorkProxy.java:354) com.ibm.ws.context.service.serializable.ContextualRunnable.run(ContextualRunnable.java:79) com.ibm.ws.jca.internal.WorkProxy.call(WorkProxy.java:285) com.ibm.ws.jca.internal.WorkProxy.call(WorkProxy.java:58) java.util.concurrent.FutureTask.run(FutureTask.java:277) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) java.lang.Thread.run(Thread.java:811)
I am getting the stuck thread error often while trying to send the JMS message to another manager server within the domain in our production environment.
Initially, we felt that it might be due to Load on the server but issue occurred randomly Even at the time of less load and system processing well some high volume time
We are not able to find the reason for the same.
Error Information:
weblogic.jms.client.JMSConnectionFactory.createQueueConnection(JMSConnectionFact
ory.java:199)
The term stuck thread is commonly used in Oracle WebLogic Server.
What is a stuck thread and why is thread diagnosed as a stuck thread?
When does stuck thread occur?
What kind of impact does it happen to
the running application?
What are the prevention mechanisms?
Stuck Threads are threads that are blocked, and can't return to the threadpool for a certain amout of time. By Default, the WLS comes with 600 secs. If some thread doesn't return in 600 secs, It gets a flag 'stuck thread'.
– > Stuck Threads are only flags, there to warn you that this thread is taking too long.
Searching a bit on google I found this site: http://www.munzandmore.com/2012/ora/weblogic-stuck-threads-howto
It explains what are stuck threads, as well some methods to work around them.
I see there are threads in hang in the websphere application server. How can I troubleshoot this problem? What documentation should I send to the application developer?
Thanks.
The most important thing is the thread stack - that should show up with the message indicating the hung thread, and it'll tell you what that thread was doing.
That, on its own, might not be enough, particularly if that thread is waiting on some other thread. In that case, you might need a thread dump. That can be triggered with "kill -3" against the process ID on non-Windows systems (I'd have to do more research to tell you the equivalent process on Windows, although there are tools that can simulate "kill -3"), and the server also can be configured to do that when it detects a hung thread, using the JVM system property com.ibm.websphere.threadmonitor.dump.java (set to either "true" or an integer value representing the maximum number of thread dumps you want).
The thread dump will go to a file called "javacore...txt" (the "..." will be a long string representing stuff like the timestamp), except on Solaris, where it will go to the server's native_stdout.log. The javacore has a lot more than just thread stacks, so you can search for "Thread Details" to find that section quickly. You'll need to search on the thread name/stack from the server log to figure out which thread is the hung one and go from there.
If you are experiencing performance, hang, or high CPU issues with WebSphere Application Server, there is a procedure documented by IBM support team to collect the data necessary to diagnose and resolve the kind of issues. This procedure is based essentially on
enabling Application Server verboseGC
running a script, at the time of the problem, which collects 3 javacores for the problematic JVM
At the end of this procedure, you need to collect:
*.tar.gz file generated by the script
javacores generated by the script
server logs (SystemOut.log, native_stderr.log,...)
and send the results to IBM Support.
To get the script and for additional information about this procedure, I suggest to give a look to the following articles:
WebSphere MustGather procedure on Linux
WebSphere MustGather procedure on Windows
A similar document exists also for AIX platform.