Can't start Liberty server in debug mode - "agent library failed to init: jdwp" - websphere-liberty

When I try to start a WebSphere Liberty server in debug mode (to connect the Eclipse Java debugger) via:
$ ./wlp/bin/server debug MyServer
I get the following error:
Error occurred during initialization of VM
agent library failed to init: jdwp
ERROR: Cannot load this JVM TI agent twice, check your java command line for duplicate jdwp options.
and the server doesn't start.

The jvm.options contents can collide with the 'debug' subcommand
If the jvm.options file already includes options enabling the debugger then use the
liberty start <server>
subcommand rather than:
liberty debug <server>
E.g. in my case my server had file .../usr/servers/MyServer/jvm.options with contents:
#Generated by liberty-maven-plugin
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=7777
so these options collided with the options triggered by the debug subcommand.
OR
You can remove the jvm.options contents or file and use the debug subcommand instead.
NOTE: Reading this explanation, the error message becomes clear, but this Q&A is hopefully useful in case it didn't occur to someone that this is the solution.

In jvm.options put these entries:
-Dwas.debug.mode=true
-Dcom.ibm.websphere.ras.inject.at.transform=true
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=7777

Just to add an update, jvm.options can also reside in /usr/shared so the same debug options span multiple servers. This would be a bad approach if you're running multiple servers and need to debug them all individually.

Related

Jmeter perfmon :EXCEPTION_ACCESS_VIOLATION

I am trying to run a performance test of my server using jmeter perfmon plugin. But I am getting an error as in the following image. I have searched a lot for a solution but couldn't find an answer. I tried setting JVM arguments but it didn't work. Link to error image.How can I get it worked?
INFO 2019-12-30 14:11:25.114 [kg.apc.p] (): Binding UDP to 4444
INFO 2019-12-30 14:11:25.155 [kg.apc.p] (): Binding TCP to 4444
INFO 2019-12-30 14:11:25.161 [kg.apc.p] (): JP#GC Agent v2.2.3 started
INFO 2019-12-30 14:11:37.974 [kg.apc.p] (): Accepting new TCP connection
INFO 2019-12-30 14:11:37.978 [kg.apc.p] (): Yep, we received the 'test' command
INFO 2019-12-30 14:11:37.979 [kg.apc.p] (): Starting measures: memory: cpu:
A fatal error has been detected by the Java Runtime Environment:
EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000010014ed4, pid=12236, tid=6844
JRE version: Java(TM) SE Runtime Environment (11.0.4+10) (build 11.0.4+10-LTS)
Java VM: Java HotSpot(TM) 64-Bit Server VM (11.0.4+10-LTS, mixed mode, tiered, compressed oops, g1
gc, windows-amd64)
Problematic frame:
C [sigar-amd64-winnt.dll+0x14ed4]
No core dump will be written. Minidumps are not enabled by default on client versions of Windows
An error report file with more information is saved as:
E:\\hs_err_pid12236.log
If you would like to submit a bug report, please visit:
http://bugreport.java.com/bugreport/crash.jsp
The crash happened outside the Java Virtual Machine in native code.
See problematic frame for where to report the bug.
I downloaded another sigar 1.6.4 jar (1.6.4.129 (17.1.2016) release) from below link and replaced sigar jar with this in specific tool lib path (ServerAgent-2.2.3\lib).., then it worked for me.
https://javalibs.com/artifact/org.hyperic/sigar
Looking into JVM crashes occassionally with access violation when using Sigar.getProcMem with disabled Windows performance counters #77 issue it might be the case that Windows don't have performance counters enabled hence Sigar fails to query the CPU status from Windows PerfMon
Check if CPU (and other stats) are exposed in PerfMon
Try running Powershell and Server Agent as Administrator
Try using cmd.exe interpreter instead of powershell
Try running ServerAgent.bat --sysinfo to see if there are any errors there
Inspect e:\hs_err_pid12236.log file for any clues
If nothing helps be aware that you can use SSHMon plugin for server monitoring, it doesn't require server-side software installation
I faced the same issue. Error code as follows:
"JRE version: OpenJDK Runtime Environment (11.0+28) (build 11+28)
Java VM: OpenJDK 64-Bit Server VM (11+28, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
Problematic frame:
C [sigar-amd64-winnt.dll+0x14ed4]"
I downgraded the JDK and JRE to the version : 8u241
Go to the Oracle download page here and search for the below JRE & JDK and install it in the Windows server and then start the serverAgent.bat
jdk-8u241-windows-x64.exe
jre-8u241-windows-x64.exe
This should fix the issue 100% sure. It worked for me.
Follow the below step.
Disable firewall setting
if still having same issue
Download the below .dll file and replace it(lib folder)
https://github.com/cnstar9988/sigar/raw/master/sigar-amd64-winnt.dll
It was solved at my end using the below steps:
Download ‘jre 1.8’ from this link https://java.com/en/download/win10.jsp
Once the jre is installed, copy the folder named ‘jre1.8.0_261’.
Image
Visit the ServerAgent folder that was downloaded >> Paste the ‘jre1.8.0_261’ folder inside the ‘ServerAgent-2.2.1’ folder.
Open ‘startAgent.bat’ file in Notepad ++ >>Initially it would look like this
Image
Replace text ‘java’ with the path of java.exe that is within the jre folder placed inside the ServerAgent-2.2.1 folder
Open command prompt in the ServerAgent-2.2.1 folder path and enter ‘startAgent.bat’ file to run the file
Execute JMeter script and now error should not happen.

How do I start a Liberty server in debug mode in the background with the liberty-maven-plugin?

Use Case
The liberty-maven-plugin has a debug goal which allows you to start the server in debug mode, but in the foreground.
Often when wanting to attach the debugger, this is all you'd need. You can execute tests separately somehow or via other goals launched from another shell.
But the debug goal, like the run goal, runs in the foreground and blocks until the server process completes.
If I want to continue to the next phase of my build, e.g. on to run integration tests, it could help to configure a start goal to start the server in debug mode, in the background, and continue on.
Use: src/test/resources/jvm.options
In src/test/resources/jvm.options, for example:
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=7777
This is the default value for the <jvmOptionsFile> configuration parameter (described here), so of course you could configure a different location if desired.
Note:
Though an alternative might appear to be to use the <jvmOptions> configuration, it does not work well with the commas (',') embedded in this value.
Warning:
You may be expecting a message from the starting JVM:
Listening for transport dt_socket at address: 7777
However you will not see this from your Maven execution log, it actually appears in the console.log file of the configured server. So the Maven execution will just pause at the point of logging message:
[INFO] Starting server defaultServer.
If you fail to connect in time you'll see:
[INFO] Server defaultServer start failed. Check server logs for details.

Running remote JBoss in debug mode, Bootup Time Issue

I am trying to run remote jboss in debug mode to debug my application and i have set
set JAVA_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=y %JAVA_OPTS%
in run.bat file, and am trying to start the jboss server using ant jboss.start but it is taking some time and then timing out, am not sure why this is happening, is there a way out?
Also as jboss is timing out, I would like to increase bootup time for the server, how can i do it from the command line?
Side note, server.log file does not show any messages which jboss is trying to bootup in debug mode (not sure if this would help but thought so sharing this info.)
Any clue or suggestions?
You should set:
suspend=n
in debugger startup settings.
suspend - True if the target VM is to be suspended immediately before the main class is loaded; false otherwise.
Your JBoss waits infinitely for debugger connection, this is useful when you want to troubleshoot startup issues, otherwise do not suspend your VM and let the debugger be connected any time you want.
I was frustated to get jboss started in debug mode through command line
Finally following seemed to work
1.Below should be the last line in /bin/run.conf.bat
rem # Sample JPDA settings for remote socket debugging
set JAVA_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n %JAVA_OPTS%
If there is anything below this, delete it.
2.Also when you are running jboss from command prompt please check whether -Xdebug is at the start of JAVA_OPTS.
Then it should display following message
Listening for transport dt_socket at address: 8787
3.In eclipse use 127.0.0.1 instaed of localhost.
I don't know the reason behind it...but when I changed localhost to 127.0.0.1 it worked
(Make sure you replace localhost with 127.0.0.1 at 2 locations 1.Eclipse remote debug and 2.URL while running your application in browser)
Also check project name.
Sometimes we need to change the port number.
I never had such issues with tomcat.

How to enable remote debugging in JBOSS 5?

How do you enable remote debugging in JBoss 5? The usual line from JBoss 4:
set JAVA_OPTS= -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n %JAVA_OPTS%
As can be found in other answers like this do not seem to work. The server never suspends. Googling "jboss 5 remote debug" doesn't bring up anything in the JBoss docs. Did something change from JBoss 4? How can I remotely debug my JBoss 5 server from eclipse?
You have suspend=n in your options. Change this to suspend=y.
Or is the line you posted not the one you're using with JBoss?
We have to do 2 changes to debug remote java application that is running in JBoss
Configure in JBoss
Creating remote debugger in in eclipse
Open run.conf.bat(windows) or run.conf(Linux) file in JBoss bin folder.
go to the below lines
`# Sample JPDA settings for remote socket debugging`
#JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n
and remove hash in JAVA_OPTS
# Sample JPDA settings for remote socket debugging
JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=y
Restart the server if it is started.
2. In Eclipse, Run -> Debug configuration -> Remote Java Application and create a new.
Note that you have to give port that is mentioned in run.conf.bat file

Tomcat failed to shutdown

When I add the following Java options to enable debugging:
JAVA_OPTS="$JAVA_OPTS -noverify -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005"
I get the following error whenever I try to shutdown the tomcat:
ERROR: transport error 202: bind failed: Address already in use ["transport.c",L41]
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) ["debugInit.c",L500]
JDWP exit error JVMTI_ERROR_INTERNAL(113): No transports initializedFATAL ERROR in native method: JDWP No transports initialized, jvmtiError=JVMTI_ERROR_INTERNAL(113)
Thank you for a nice short explanation, PHeath! Following your advice, I found the best way to solve the problem is simply to use CATALINA_OPTS instead of JAVA_OPTS.
Looking into catalina.sh, one can see CATALINA_OPTS is only used by the "start" and "start-security" commands, whereas JAVA_OPTS is also used by the "stop" command (at least with Tomcat 6.0.33 on openSUSE 12.1).
At least if you have Tomcat installed on Linux using a package manager then modifying the CATALINA_OPTS variable in /etc/tomcat6/tomcat6.conf (or whatever path in your distribution) is cleaner than changing the catalina.sh script directly, for the package manager assumes that the user changes only configuration files and breaking this assumption may cause problems when upgrading the Tomcat packages (e. g. lost settings because the catalina.sh file is overwritten).
I think one should prefer CATALINA_OPTS over JAVA_OPTS not only for JDWP but for many other options as well: e. g. if one uses the heap size option -Xmx... then it would be reasonable to put it into CATALINA_OPTS, as the "stop" command does not need much heap.
You are trying to debug tomcat on startup, so it binds to port 5005 when the jvm starts.
When you run catalina.sh stop, it starts up another jvm which also tries to bind to port 5005.
You need to move the debug args to the run and start arguments (in catalina.sh) of tomcat, putting them straight into the JAVA_OPTS is the cause of the issue you're having.
The problem is your tomcat is still running on the debug port(5005) or some other service running on the same port(5005).
If tomcat still running, you can kill it
if it in linux environment ps -ef|grep java, and identify the process id of it. and kill the process using sudo kill -9 .
If it in windows environment got to task manager and kill the tomcat and java process.
Now you should be able to start the server in debug mood without any prob.
This can happen on debugging unit test through the tool(eclipse) which has been executed through the maven. To sole this you can flow the same process.
First close the Eclipse and kill the java process as well and start it again.
This is due to both applications are listening the same port number i.e 8000 while running in debug mode.
One quick solution is change the debug port to 8001 in startup.bat
SET DEBUGPORT=8001
It seems that the port 5005 is already in use. Check open ports with netstat command.
This may be because you already opened tomcat. Check your processes.
It appears you are starting Tomcat with the Debugger enabled, This causes the JVM to attach to the Process for Debugging, However in the catalina.sh there is a case statement for start, stop, restart, so on and so forth. Issuing the stop command still adds this in as it is part of your Global JAVA_OPTS and tries to start the debugger listening on the same port for the shutdown command. If you remove the address=50005 from your JAVA_OPTS or use the start jdpa commands to start the VM with the debugger this will fix your problem.
Look at the default catalina.sh in the latest Tomcat distribution if you need a clean copy. It sounds like someone has made changes inside yours that are invalid and causing JDPA to run on start, stop, any command issued.
set JPDA_ADDRESS=8001 in catalina.bat i.e debug port
and change all 3 ports in server.xml
In my case (Tomcat installed form tarball) I had those debug options unintentionally set in my env. This fixed the error:
$ unset JAVA_OPTS

Resources