I am trying to deploy a web application on WAS 9.0.5.5.
It is a standalone node without management node.
After creating an empty profile and starting it using startServer.sh, everything was fine and the WAS console (9043 port) was available.
But after the application deployment, found that the WAS console was inaccessible after a long SSL handshake.
Not sure why this happened since I did not change any SSL setting.
Could see the SSL_TLSv2 was set both on the console and in the ssl.client.props
com.ibm.ssl.alias=DefaultSSLSettings
com.ibm.ssl.protocol=SSL_TLSv2
com.ibm.ssl.securityLevel=HIGH
com.ibm.ssl.trustManager=IbmPKIX
com.ibm.ssl.keyManager=IbmX509
com.ibm.ssl.contextProvider=IBMJSSE2
com.ibm.ssl.enableSignerExchangePrompt=gui
Also the stopServer.sh failed with the error below. Any advice for me to get out here?
************ Start Display Current Environment ************
Host Operating System is Linux, version 3.10.0-1062.el7.x86_64
Java version = 1.8.0_261, Java Runtime Version = 8.0.6.16 - pxa6480sr6fp16-20200902_01(SR6 FP16), Java Compiler = j9jit29, Java VM name = IBM J9 VM
was.install.root = /opt/IBM/WebSphere/AppServer
user.install.root = /opt/IBM/WebSphere/AppServer/profiles/MPA
Java Home = /opt/IBM/WebSphere/AppServer/java/8.0/jre
ws.ext.dirs = /opt/IBM/WebSphere/AppServer/java/8.0/lib:/opt/IBM/WebSphere/AppServer/classes:/opt/IBM/WebSphere/AppServer/lib:/opt/IBM/WebSphere/AppServer/installedChannels:/opt/IBM/WebSphere/AppServer/lib/ext:/opt/IBM/WebSphere/AppServer/web/help:/opt/IBM/WebSphere/AppServer/deploytool/itp/plugins/com.ibm.etools.ejbdeploy/runtime
Classpath = /opt/IBM/WebSphere/AppServer/profiles/MPA/properties:/opt/IBM/WebSphere/AppServer/properties:/opt/IBM/WebSphere/AppServer/lib/startup.jar:/opt/IBM/WebSphere/AppServer/lib/bootstrap.jar:/opt/IBM/WebSphere/AppServer/java/8.0/lib/tools.jar:/opt/IBM/WebSphere/AppServer/lib/lmproxy.jar:/opt/IBM/WebSphere/AppServer/lib/urlprotocols.jar
Java Library path = /opt/IBM/WebSphere/AppServer/java/8.0/jre/lib/amd64/compressedrefs:/opt/IBM/WebSphere/AppServer/java/8.0/jre/lib/amd64:/opt/IBM/WebSphere/AppServer/lib/native/linux/x86_64/:/opt/IBM/WebSphere/AppServer/bin:/opt/IBM/WebSphere/AppServer/nulldllsdir:/usr/lib64:/usr/lib
Orb Version = IBM Java ORB build orbdev-userlvl-20200106.1711
Max file descriptor count = 50000
Current trace specification = *=info
************* End Display Current Environment *************
[12/16/20 1:48:06:032 PST] 00000001 ManagerAdmin I TRAS0017I: The startup trace state is *=info.
[12/16/20 1:48:06:132 PST] 00000001 AdminTool A ADMU0128I: Starting tool with the MPA profile
[12/16/20 1:48:06:133 PST] 00000001 AdminTool A ADMU3100I: Reading configuration for server: mpa_server
[12/16/20 1:48:06:169 PST] 00000001 ModelMgr I WSVR0801I: Initializing all server configuration models
[12/16/20 1:48:08:657 PST] 00000001 ProviderTrack I com.ibm.ffdc.osgi.ProviderTracker AddingService FFDC1007I: FFDC Provider Installed: com.ibm.ffdc.util.provider.FfdcOnDirProvider#5896a505
[12/16/20 1:48:08:921 PST] 00000001 SSLConfig W CWPKI0041W: One or more key stores are using the default password.
[12/16/20 1:48:08:929 PST] 00000001 SSLConfigMana I CWPKI0051I: The process has the java security property jdk.certpath.disabledAlgorithms set to [MD2, MD5, SHA1 jdkCA & usage TLSServer, RSA keySize < 1024, DSA keySize < 1024, EC keySize < 224]. The WebSphere Application server is setting the java security property jdk.certpath.disabledAlgorithms to [MD2, RSA keySize < 1024, MD5].
[12/16/20 1:48:08:930 PST] 00000001 SSLConfigMana I CWPKI0051I: The process has the java security property jdk.tls.disabledAlgorithms set to [SSLv3, RC4, DES, MD5withRSA, DH keySize < 1024, DESede, EC keySize < 224, 3DES_EDE_CBC, anon, NULL]. The WebSphere Application server is setting the java security property jdk.tls.disabledAlgorithms to [SSLv3, RC4, DH keySize < 768, MD5withRSA].
[12/16/20 1:48:08:933 PST] 00000001 SSLConfigMana I CWPKI0027I: Disabling default hostname verification for HTTPS URL connections.
[12/16/20 1:48:08:962 PST] 00000001 SecurityObjec I CWSCF0002I: The client code is attempting to load the security configuration the server and this operation is not allowed.
[12/16/20 1:51:15:474 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.1445219360461837085.txt com.ibm.ws.ssl.channel.impls.SSLUtils 1052
[12/16/20 1:51:15:601 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.5712207576305752668.txt com.ibm.ws.ssl.channel.impl.SSLConnectionLink 540
[12/16/20 1:51:15:919 PST] 00000001 SecurityObjec I CWSCF0002I: The client code is attempting to load the security configuration the server and this operation is not allowed.
[12/16/20 1:51:15:920 PST] 00000001 SecurityConne A JSAS0006I: Security connection interceptor init
ialized.
[12/16/20 1:51:15:935 PST] 00000001 ServiceInit A JSAS0001I: Security configuration initialized.
[12/16/20 1:51:15:936 PST] 00000001 ServiceInit A JSAS0003I: Authentication mechanism: BASICAUTH
[12/16/20 1:51:15:974 PST] 00000001 CSIClientRI A JSAS0007I: Client request interceptor registered.
[12/16/20 1:51:16:639 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.7382828071492998187.txt com.ibm.ws.security.orbssl.WSSSLClientSocketFactoryImpl.createSSLSocket 222
[12/16/20 1:51:16:828 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.5725476233661856283.txt com.ibm.ws.orbimpl.transport.WSSSLTransportConnection.createSocket 270
[12/16/20 1:51:16:884 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.2654931705161208931.txt com.ibm.ws.naming.util.CommonHelpers.retry 876
[12/16/20 1:51:16:984 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.1815847588980124586.txt com.ibm.ws.naming.jndicos.CNContextImpl.lookupExt 1443
[12/16/20 1:51:16:996 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.1476667816130282325.txt com.ibm.ws.naming.util.Helpers.logJndiContextException 1636
[12/16/20 1:51:17:080 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.6880057812892354780.txt com.ibm.ws.management.connector.rmi.RMIConnectorClient.reconnect 237
[12/16/20 1:51:17:260 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.5125431433737765534.txt com.ibm.websphere.management.AdminClientFactory.createAdminClient 267
[12/16/20 1:51:17:563 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.3289464480553651737.txt com.ibm.ws.management.connector.jsr160rmi.JSR160RMIConnectorClient.reconnect 179
[12/16/20 1:51:17:668 PST] 00000001 Ffdc I com.ibm.ffdc.util.provider.FfdcOnDirProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/ffdc/ffdc.2859402585966834209.txt com.ibm.websphere.management.AdminClientFactory.createAdminClient 275
[12/16/20 1:51:17:674 PST] 00000001 WsServerStop E ADMU3002E: Exception attempting to process server mpa_server
[12/16/20 1:51:17:675 PST] 00000001 WsServerStop E ADMU3007E: Exception com.ibm.websphere.management.exception.ConnectorException: com.ibm.websphere.management.exception.ConnectorException: ADMC0016E: The system cannot create a SOAP connector to connect to host dyn-23.datadomain.com at port 8880.
[12/16/20 1:51:17:676 PST] 00000001 WsServerStop A ADMU3007E: Exception com.ibm.websphere.management.exception.ConnectorException: com.ibm.websphere.management.exception.ConnectorException: ADMC0016E: The system cannot create a SOAP connector to connect to host dyn-23.datadomain.com at port 8880.
at com.ibm.ws.management.connector.ConnectorHelper.createConnector(ConnectorHelper.java:628)
at com.ibm.ws.management.tools.WsServerStop.runTool(WsServerStop.java:373)
at com.ibm.ws.management.tools.AdminTool.executeUtility(AdminTool.java:271)
at com.ibm.ws.management.tools.WsServerStop.main(WsServerStop.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.ibm.wsspi.bootstrap.WSLauncher.launchMain(WSLauncher.java:234)
at com.ibm.wsspi.bootstrap.WSLauncher.main(WSLauncher.java:101)
at com.ibm.wsspi.bootstrap.WSLauncher.run(WSLauncher.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:354)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:181)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.eclipse.core.launcher.Main.invokeFramework(Main.java:340)
at org.eclipse.core.launcher.Main.basicRun(Main.java:282)
at org.eclipse.core.launcher.Main.run(Main.java:981)
at com.ibm.wsspi.bootstrap.WSPreLauncher.launchEclipse(WSPreLauncher.java:422)
at com.ibm.wsspi.bootstrap.WSPreLauncher.main(WSPreLauncher.java:179)
Caused by: com.ibm.websphere.management.exception.ConnectorException: ADMC0016E: The system cannot create a SOAP connector to connect to host dyn-23.datadomain.com at port 8880.
at com.ibm.websphere.management.AdminClientFactory.createAdminClientPrivileged(AdminClientFactory.java:635)
at com.ibm.websphere.management.AdminClientFactory.access$000(AdminClientFactory.java:127)
at com.ibm.websphere.management.AdminClientFactory$1.run(AdminClientFactory.java:210)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:63)
at com.ibm.websphere.management.AdminClientFactory.createAdminClient(AdminClientFactory.java:206)
at com.ibm.ws.management.connector.ConnectorHelper.tryProtocol(ConnectorHelper.java:665)
at com.ibm.ws.management.connector.ConnectorHelper.createConnector(ConnectorHelper.java:597)
... 29 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:83)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:57)
at java.lang.reflect.Constructor.newInstance(Constructor.java:437)
at com.ibm.websphere.management.AdminClientFactory.createAdminClientPrivileged(AdminClientFactory.java:457)
... 35 more
Caused by: com.ibm.websphere.management.exception.ConnectorNotAvailableException: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: internal_error; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: internal_error]
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.reconnect(SOAPConnectorClient.java:439)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.<init>(SOAPConnectorClient.java:238)
... 40 more
Caused by: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: internal_error; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: internal_error]
at org.apache.soap.transport.http.SOAPHTTPConnection.send(SOAPHTTPConnection.java:475)
at org.apache.soap.rpc.Call.WASinvoke(Call.java:510)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient$4.run(SOAPConnectorClient.java:397)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.reconnect(SOAPConnectorClient.java:382)
... 41 more
[12/16/20 1:51:17:692 PST] 00000001 AdminTool A ADMU0509I: The server "mpa_server" cannot be reached. It appears to be stopped.
[12/16/20 1:51:17:693 PST] 00000001 AdminTool A ADMU0211I: Error details may be seen in the file: /opt/IBM/WebSphere/AppServer/profiles/MPA/logs/mpa_server/stopServer.log
LE: I managed to fix the issue. In my case, the only supported protocol is "SSL_TLS", more about the protocols at https://www.ibm.com/docs/en/imdmcdhsa/11.6.0?topic=server-setting-ssl-protocol-version
I also updated the security.xml file, although is mentioned that's required only in TLSv1.2 protocol case.
Related
My application need to call some API with https TLS 1.2 endpoint. The application is located in Window Server 2008 R2.
It work fine with Postman but cannot work in Java Spring Boot (JDK 11 + RestTemplate) and also not work with SoapUI and cUrl.
With Java, it throw exeption
javax.net.ssl.SSLException: Connection reset
With curl
C:\Users\devadmin>curl -v -k https://api.visa.com/
* Trying 198.217.139.29:443...
* Connected to api.visa.com (198.217.139.29) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* OpenSSL SSL_connect: Connection was reset in connection to api.visa.com:443
* Closing connection 0
curl: (35) OpenSSL SSL_connect: Connection was reset in connection to api.visa.com:443
With SoapUI 5.7.0
Fri May 20 08:30:44 ICT 2022:DEBUG:Attempt 1 to execute request
Fri May 20 08:30:44 ICT 2022:DEBUG:Sending request: GET /vmorc/offers/v1/all HTTP/1.0
Fri May 20 08:30:44 ICT 2022:DEBUG:I/O error closing connection
javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)
at sun.security.ssl.SSLSocketImpl.checkWrite(Unknown Source)
at sun.security.ssl.AppOutputStream.write(Unknown Source)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:131)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:138)
at org.apache.http.impl.conn.LoggingSessionOutputBuffer.flush(LoggingSessionOutputBuffer.java:95)
at org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:270)
at org.apache.http.impl.SocketHttpClientConnection.close(SocketHttpClientConnection.java:245)
at org.apache.http.impl.conn.DefaultClientConnection.close(DefaultClientConnection.java:164)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.close(AbstractPooledConnAdapter.java:152)
at org.apache.http.protocol.HttpRequestExecutor.closeConnection(HttpRequestExecutor.java:142)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:129)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:633)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:454)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport$Helper.execute(HttpClientSupport.java:233)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport.execute(HttpClientSupport.java:323)
at com.eviware.soapui.impl.wsdl.submit.transports.http.HttpClientRequestTransport.submitRequest(HttpClientRequestTransport.java:297)
at com.eviware.soapui.impl.wsdl.submit.transports.http.HttpClientRequestTransport.sendRequest(HttpClientRequestTransport.java:227)
at com.eviware.soapui.impl.wsdl.WsdlSubmit.run(WsdlSubmit.java:119)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.Alerts.getSSLException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
at sun.security.ssl.AppOutputStream.write(Unknown Source)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:131)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:138)
at org.apache.http.impl.conn.LoggingSessionOutputBuffer.flush(LoggingSessionOutputBuffer.java:95)
at org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:270)
at org.apache.http.impl.AbstractHttpClientConnection.flush(AbstractHttpClientConnection.java:275)
at org.apache.http.impl.conn.AbstractClientConnAdapter.flush(AbstractClientConnAdapter.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:258)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport$SoapUIHttpRequestExecutor.doSendRequest(HttpClientSupport.java:119)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
... 14 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.writeRecord(Unknown Source)
... 24 more
Fri May 20 08:30:44 ICT 2022:DEBUG:Closing the connection.
Fri May 20 08:30:44 ICT 2022:DEBUG:Connection closed
Fri May 20 08:30:44 ICT 2022:INFO:I/O exception (java.net.SocketException) caught when processing request: Connection reset
Fri May 20 08:30:44 ICT 2022:DEBUG:Connection reset
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.writeRecord(Unknown Source)
at sun.security.ssl.AppOutputStream.write(Unknown Source)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:131)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:138)
at org.apache.http.impl.conn.LoggingSessionOutputBuffer.flush(LoggingSessionOutputBuffer.java:95)
at org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:270)
at org.apache.http.impl.AbstractHttpClientConnection.flush(AbstractHttpClientConnection.java:275)
at org.apache.http.impl.conn.AbstractClientConnAdapter.flush(AbstractClientConnAdapter.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:258)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport$SoapUIHttpRequestExecutor.doSendRequest(HttpClientSupport.java:119)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:633)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:454)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport$Helper.execute(HttpClientSupport.java:233)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport.execute(HttpClientSupport.java:323)
at com.eviware.soapui.impl.wsdl.submit.transports.http.HttpClientRequestTransport.submitRequest(HttpClientRequestTransport.java:297)
at com.eviware.soapui.impl.wsdl.submit.transports.http.HttpClientRequestTransport.sendRequest(HttpClientRequestTransport.java:227)
at com.eviware.soapui.impl.wsdl.WsdlSubmit.run(WsdlSubmit.java:119)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Fri May 20 08:30:44 ICT 2022:INFO:Retrying request
Fri May 20 08:30:44 ICT 2022:DEBUG:Reopening the direct connection.
Fri May 20 08:30:44 ICT 2022:DEBUG:Attempt 2 to execute request
.......................
Fri May 20 08:30:45 ICT 2022:INFO:Retrying request
Fri May 20 08:30:45 ICT 2022:DEBUG:Reopening the direct connection.
Fri May 20 08:30:45 ICT 2022:DEBUG:Attempt 4 to execute request
Fri May 20 08:30:45 ICT 2022:DEBUG:Sending request: GET /vmorc/offers/v1/all HTTP/1.0
Fri May 20 08:30:45 ICT 2022:DEBUG:I/O error closing connection
javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)
at sun.security.ssl.SSLSocketImpl.checkWrite(Unknown Source)
at sun.security.ssl.AppOutputStream.write(Unknown Source)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:131)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:138)
at org.apache.http.impl.conn.LoggingSessionOutputBuffer.flush(LoggingSessionOutputBuffer.java:95)
at org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:270)
at org.apache.http.impl.SocketHttpClientConnection.close(SocketHttpClientConnection.java:245)
at org.apache.http.impl.conn.DefaultClientConnection.close(DefaultClientConnection.java:164)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.close(AbstractPooledConnAdapter.java:152)
at org.apache.http.protocol.HttpRequestExecutor.closeConnection(HttpRequestExecutor.java:142)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:129)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:633)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:454)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport$Helper.execute(HttpClientSupport.java:233)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport.execute(HttpClientSupport.java:323)
at com.eviware.soapui.impl.wsdl.submit.transports.http.HttpClientRequestTransport.submitRequest(HttpClientRequestTransport.java:297)
at com.eviware.soapui.impl.wsdl.submit.transports.http.HttpClientRequestTransport.sendRequest(HttpClientRequestTransport.java:227)
at com.eviware.soapui.impl.wsdl.WsdlSubmit.run(WsdlSubmit.java:119)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.Alerts.getSSLException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
at sun.security.ssl.AppOutputStream.write(Unknown Source)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:131)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:138)
at org.apache.http.impl.conn.LoggingSessionOutputBuffer.flush(LoggingSessionOutputBuffer.java:95)
at org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:270)
at org.apache.http.impl.AbstractHttpClientConnection.flush(AbstractHttpClientConnection.java:275)
at org.apache.http.impl.conn.AbstractClientConnAdapter.flush(AbstractClientConnAdapter.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:258)
at com.eviware.soapui.impl.wsdl.support.http.HttpClientSupport$SoapUIHttpRequestExecutor.doSendRequest(HttpClientSupport.java:119)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
... 14 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.writeRecord(Unknown Source)
... 24 more
Fri May 20 08:30:45 ICT 2022:DEBUG:Closing the connection.
Fri May 20 08:30:45 ICT 2022:DEBUG:Connection closed
Fri May 20 08:30:45 ICT 2022:DEBUG:Connection shut down
Fri May 20 08:30:45 ICT 2022:ERROR:Exception in request: java.net.SocketException: Connection reset
Fri May 20 08:30:45 ICT 2022:ERROR:An error occurred [Connection reset], see error log for details
Fri May 20 08:30:45 ICT 2022:INFO:Error getting response for [https://api.visa.com.All:Request 1]; java.net.SocketException: Connection reset
Openssl in same server
C:\openssl\bin>openssl s_client -state -connect api.visa.com:443
Loading 'screen' into random state - done
CONNECTED(0000017C)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL3 alert read:fatal:handshake failure
SSL_connect:error in SSLv2/v3 read server hello A
163092:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:./ssl/s23_clnt.c:580:
openssl in local machine
openssl s_client -state -connect api.visa.com:443
CONNECTED(00000084)
SSL_connect:before SSL initialization
SSL_connect:SSLv3/TLS write client hello
SSL_connect:SSLv3/TLS write client hello
SSL_connect:SSLv3/TLS read server hello
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify error:num=19:self signed certificate in certificate chain
verify return:1
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
verify return:1
depth=0 C = US, ST = California, L = Foster City, O = Visa International Service Association, CN = api.visa.com
verify return:1
SSL_connect:SSLv3/TLS read server certificate
SSL_connect:SSLv3/TLS read server key exchange
SSL_connect:SSLv3/TLS read server certificate request
SSL_connect:SSLv3/TLS read server done
SSL_connect:SSLv3/TLS write client certificate
SSL_connect:SSLv3/TLS write client key exchange
SSL_connect:SSLv3/TLS write change cipher spec
SSL_connect:SSLv3/TLS write finished
SSL_connect:SSLv3/TLS write finished
SSL_connect:SSLv3/TLS read change cipher spec
SSL_connect:SSLv3/TLS read finished
---
Certificate chain
0 s:C = US, ST = California, L = Foster City, O = Visa International Service Association, CN = api.visa.com
i:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
1 s:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
2 s:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGxjCCBa6gAwIBAgIQAgYl0jd+Dh9K+623wLZioTANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E
aWdpQ2VydCBTSEEyIFNlY3VyZSBTZXJ2ZXIgQ0EwHhcNMjIwMTI0MDAwMDAwWhcN
MjMwMjI0MjM1OTU5WjCBgDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3Ju
aWExFDASBgNVBAcTC0Zvc3RlciBDaXR5MS8wLQYDVQQKEyZWaXNhIEludGVybmF0
aW9uYWwgU2VydmljZSBBc3NvY2lhdGlvbjEVMBMGA1UEAxMMYXBpLnZpc2EuY29t
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxhTk91X6oabj19sIcpKs
g3Dt8xbhtxX7hj/gWU2rbblkGSJtC+juFfVYvKI/MdVEuHXQWF/KO/j5Nrv7uFIa
GF1ij2whZ0VzzfVIwZ6AORnEvipQPqoJrqWeDUbWH7+TWYw/JuVtUVDLb4WTCuhy
AGpXmOOeAHkdFOECGPbGR7Vnvn/C51+daE1NVR1/gcuOSuZDzrKS2GxllC3nAnPj
67ITR7dR0paN01Bugo6BMEpZE7nTnBQb005YPO9mGYaXg8XeKgNI+eiF0QR+9D1T
64mZ6Qoq3m9f6eqyul9AtSzE90JDqh+fEmwSQXGaRpkNGBJR0g557En9coy5NIGV
2wIDAQABo4IDbDCCA2gwHwYDVR0jBBgwFoAUD4BhHIIxYdUvKOeNRji0LOHG2eIw
HQYDVR0OBBYEFLq61prBmN9tRW1yPNtHkOuXtdKgMBcGA1UdEQQQMA6CDGFwaS52
aXNhLmNvbTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG
AQUFBwMCMIGNBgNVHR8EgYUwgYIwP6A9oDuGOWh0dHA6Ly9jcmwzLmRpZ2ljZXJ0
LmNvbS9EaWdpY2VydFNIQTJTZWN1cmVTZXJ2ZXJDQS0xLmNybDA/oD2gO4Y5aHR0
cDovL2NybDQuZGlnaWNlcnQuY29tL0RpZ2ljZXJ0U0hBMlNlY3VyZVNlcnZlckNB
LTEuY3JsMD4GA1UdIAQ3MDUwMwYGZ4EMAQICMCkwJwYIKwYBBQUHAgEWG2h0dHA6
Ly93d3cuZGlnaWNlcnQuY29tL0NQUzB+BggrBgEFBQcBAQRyMHAwJAYIKwYBBQUH
MAGGGGh0dHA6Ly9vY3NwLmRpZ2ljZXJ0LmNvbTBIBggrBgEFBQcwAoY8aHR0cDov
L2NhY2VydHMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0U0hBMlNlY3VyZVNlcnZlckNB
LTIuY3J0MAwGA1UdEwEB/wQCMAAwggF+BgorBgEEAdZ5AgQCBIIBbgSCAWoBaAB1
AK33vvp8/xDIi509nB4+GGq0Zyldz7EMJMqFhjTr3IKKAAABfo0+i04AAAQDAEYw
RAIgL7FC2dC/vrFIkWRW1ubiOPN8DpIiDdm8MzrhQggcP+YCIDGMRhq3SHURSWTH
6zunFmvFA72AxGuIalGANmYLSeGmAHYANc8ZG7+xbFe/D61MbULLu7YnICZR6j/h
Ku+oA8M71kwAAAF+jT6LZAAABAMARzBFAiEAoFhcUK6ZY2BdDLQmBcA9/KL8xUcp
w4Wvvs+tw4ormO4CIHYgvG4W1Umx1ffnHCHJHaDYygpA+9ombLdg7SU9TmSRAHcA
s3N3B+GEUPhjhtYFqdwRCUp5LbFnDAuH3PADDnk2pZoAAAF+jT6LggAABAMASDBG
AiEAvQ9b50UFsZnj+9Tq2v5iujUH1JhPb1Rapet2K+EfvQcCIQCmx51NigGVAAtG
lYNrAK2d3pdh3BT01cBUsrkrMH78GTANBgkqhkiG9w0BAQsFAAOCAQEAIpNyqUXg
yIs3OuKeoAqurrlWJikJIikKrO8ajGvL9HlSoWhV5DEbo32k4h7FSiMOvcwCxqzP
UvO1K2zaAxRyMiB4E+9OVNSWpzuxFkXPMRoruQy+mfj1sn7dt30eLCsPN2fHLs79
RbO1Brgn2eGvTybg+xl0q+7++m2//dQp0ASifbvjuloCQ/VG6qlKBcsysQ2bawH5
VJpLlzZFy9AeqLJFpHEDJfyEpXyqqSpKDomeyu+kwRdKeMSDO87ic3DxdfAwHxV3
PEj2e+UL0az9B5Q2/EfHvXS/jsdEykULvJkNDvF3lMOPqXoIlpp/HjTBympmn+Qn
SmLg3JNmt1PEYA==
-----END CERTIFICATE-----
subject=C = US, ST = California, L = Foster City, O = Visa International Service Association, CN = api.visa.com
issuer=C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
---
Acceptable client certificate CA names
C = US, O = VISA, OU = Visa International Service Association, CN = VI CA1
C = US, O = VISA, OU = Visa International Service Association, CN = Visa Information Delivery External CA
C = US, O = VISA, OU = Visa International Service Association, CN = Visa Information Delivery Root CA
CN = VDPCA, O = VDPVISACA, C = US
CN = VDPCA, O = VDPVISACA, C = US
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 4959 bytes and written 452 bytes
Verification error: self signed certificate in certificate chain
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: 32E83402862CA8E61BB1A07BED75506A42D3AC94963734710F312B79BE05A07F
Session-ID-ctx:
Master-Key: 19EE1A9B6873D35D1FDEA404B4E9A830F0E7E1E6AD82CFC700F4C4EBA4238916DF708A8762958977A3FC54350850F61D
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1653039821
Timeout : 7200 (sec)
Verify return code: 19 (self signed certificate in certificate chain)
Extended master secret: yes
---
Add http trace: http trace
Why it still work with Postman and not work with other ways? And How can I resolve this?
we have our java application deployed in WebSphere Application server(8.5.5.12) with IBM SDK 8.0.5.17 with backend database as oracle (12.1.0.2.0). we are now trying to encrypt the data between WebSphere Application and the database. We have followed the exact steps documented in the following linkhttps://www.ibm.com/developerworks/community/blogs/d89a3ddf-2acf-4cc8-b11b-14f33b5c653e/entry/Configuring_Secure_Socket_Layer_SSL_communication_between_the_OpenPages_application_server_WebSphere_and_the_Oracle_database?lang=en When we try to test the data source connection getting following error
java.sql.SQLRecoverableException: IO Error: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty DSRA0010E: SQL State = 08006, Error Code = 17,002.
same configuration works perfect with IBM SDK version 7 (7.0.4.1) but not with IBM SDK 8, does any one experienced similar issue or do you see any known issues configuring SSL with IBM SDK 8.Also copied the ssl debug log
Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at com.ibm.jsse2.k.a(k.java:24)
at com.ibm.jsse2.at.a(at.java:572)
at com.ibm.jsse2.at.a(at.java:387)
at com.ibm.jsse2.at.a(at.java:338)
at com.ibm.jsse2.at.a(at.java:733)
at com.ibm.jsse2.i.write(i.java:8)
at oracle.net.ns.Packet.send(Packet.java:419)
at oracle.net.ns.ConnectPacket.send(ConnectPacket.java:241)
at oracle.net.ns.NSProtocolStream.negotiateConnection(NSProtocolStream.java:157)
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:264)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1452)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:496)
... 105 more
Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at com.ibm.jsse2.util.f.<init>(f.java:50)
at com.ibm.jsse2.util.e.a(e.java:18)
at com.ibm.jsse2.aB.a(aB.java:21)
at com.ibm.jsse2.aB.a(aB.java:185)
at com.ibm.jsse2.aB.a(aB.java:137)
at com.ibm.jsse2.aB.checkServerTrusted(aB.java:49)
at com.ibm.jsse2.E.a(E.java:166)
at com.ibm.jsse2.E.a(E.java:121)
at com.ibm.jsse2.D.r(D.java:223)
at com.ibm.jsse2.D.a(D.java:198)
at com.ibm.jsse2.at.a(at.java:649)
at com.ibm.jsse2.at.i(at.java:627)
at com.ibm.jsse2.at.a(at.java:310)
at com.ibm.jsse2.i.write(i.java:3)
... 111 more
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:300)
at java.security.cert.PKIXParameters.<init>(PKIXParameters.java:142)
at java.security.cert.PKIXBuilderParameters.<init>(PKIXBuilderParameters.java:99)
at com.ibm.jsse2.util.f.<init>(f.java:106)
... 124 more
: {2}.
[6/12/19 6:24:54:097 EDT] 00000098 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /u01/IBM/WebSphere/AppServer/profiles/manuonsite1/logs/ffdc/manuonsite1was_srv_2293118f_19.06.12_06.24.54.0822519622081070003296.txt com.ibm.ws.management.AdminServiceImpl.invoke 679
[6/12/19 6:24:54:098 EDT] 00000098 MBeanHelper E Could not invoke an operation on object: WebSphere:name=DataSourceCfgHelper,process=manuonsite1was_srv,platform=dynamicproxy,node=awscentosNode03,version=8.5.5.12,type=DataSourceCfgHelper,mbeanIdentifier=DataSourceCfgHelper,cell=awscentosNode03Cell,spec=1.0 because of an mbean exception: java.sql.SQLRecoverableException: IO Error: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty DSRA0010E: SQL State = 08006, Error Code = 17,002
[6/12/19 6:24:54:389 EDT] 00000098 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:394 EDT] 00000098 SystemOut O WebContainer : 1, WRITE: TLSv1.2 Application Data, length = 16384
[6/12/19 6:24:54:394 EDT] 00000098 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:399 EDT] 00000098 SystemOut O WebContainer : 1, WRITE: TLSv1.2 Application Data, length = 16384
[6/12/19 6:24:54:399 EDT] 00000098 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:401 EDT] 00000098 SystemOut O WebContainer : 1, WRITE: TLSv1.2 Application Data, length = 323
[6/12/19 6:24:54:401 EDT] 00000098 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:406 EDT] 00000098 SystemOut O WebContainer : 1, WRITE: TLSv1.2 Application Data, length = 16384
[6/12/19 6:24:54:406 EDT] 00000098 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:408 EDT] 000000a0 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:409 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:410 EDT] 00000098 SystemOut O WebContainer : 1, WRITE: TLSv1.2 Application Data, length = 16384
[6/12/19 6:24:54:410 EDT] 00000098 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:412 EDT] 00000098 SystemOut O WebContainer : 1, WRITE: TLSv1.2 Application Data, length = 8
[6/12/19 6:24:54:412 EDT] 00000098 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:414 EDT] 00000098 SystemOut O WebContainer : 1, WRITE: TLSv1.2 Application Data, length = 5276
[6/12/19 6:24:54:416 EDT] 00000098 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:417 EDT] 00000098 SystemOut O WebContainer : 1, WRITE: TLSv1.2 Application Data, length = 5
[6/12/19 6:24:54:444 EDT] 000000a0 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:451 EDT] 000000a0 SystemOut O WebContainer : 6, WRITE: TLSv1.2 Application Data, length = 16384
[6/12/19 6:24:54:451 EDT] 000000a0 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:458 EDT] 000000a0 SystemOut O WebContainer : 6, WRITE: TLSv1.2 Application Data, length = 16384
[6/12/19 6:24:54:458 EDT] 000000a0 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:459 EDT] 000000a0 SystemOut O WebContainer : 6, WRITE: TLSv1.2 Application Data, length = 281
[6/12/19 6:24:54:460 EDT] 000000a0 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:466 EDT] 000000a0 SystemOut O WebContainer : 6, WRITE: TLSv1.2 Application Data, length = 15563
[6/12/19 6:24:54:467 EDT] 000000a0 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:468 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:469 EDT] 000000a0 SystemOut O WebContainer : 6, WRITE: TLSv1.2 Application Data, length = 5
[6/12/19 6:24:54:469 EDT] 000000a1 SystemOut O WebContainer : 7, WRITE: TLSv1.2 Application Data, length = 398
[6/12/19 6:24:54:475 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:488 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:491 EDT] 000000a1 SystemOut O WebContainer : 7, WRITE: TLSv1.2 Application Data, length = 9328
[6/12/19 6:24:54:494 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:506 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:511 EDT] 000000a1 SystemOut O WebContainer : 7, WRITE: TLSv1.2 Application Data, length = 16384
[6/12/19 6:24:54:511 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:513 EDT] 000000a1 SystemOut O WebContainer : 7, WRITE: TLSv1.2 Application Data, length = 4956
[6/12/19 6:24:54:516 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:529 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:531 EDT] 000000a1 SystemOut O WebContainer : 7, WRITE: TLSv1.2 Application Data, length = 5753
[6/12/19 6:24:54:552 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:578 EDT] 000000a1 ServletWrappe I com.ibm.ws.webcontainer.servlet.ServletWrapper init SRVE0242I: [isclite] [/ibm/console] [/secure/javascriptToSession.jsp]: Initialization successful.
[6/12/19 6:24:54:579 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:580 EDT] 000000a1 SystemOut O WebContainer : 7, WRITE: TLSv1.2 Application Data, length = 284
[6/12/19 6:24:54:581 EDT] 000000a1 SystemOut O CipherBox: Using cipher AES/GCM/NoPadding from provider from init IBMJCE version 1.8
[6/12/19 6:24:54:582 EDT] 000000a1 SystemOut O WebContainer : 7, WRITE: TLSv1.2 Application Data, length = 5
[6/12/19 6:24:54:637 EDT] 0000005a SystemOut O SSLv3 protocol was requested but was not enabled
[6/12/19 6:24:54:643 EDT] 0000005b SystemOut O SSLv3 protocol was requested but was not enabled
[6/12/19 6:24:59:643 EDT] 0000005a SystemOut O SSLv3 protocol was requested but was not enabled
[6/12/19 6:24:59:648 EDT] 0000005b SystemOut O SSLv3 protocol was requested but was not enabled
[6/12/19 6:25:04:648 EDT] 0000005a SystemOut O SSLv3 protocol was requested but was not enabled
[6/12/19 6:25:04:653 EDT] 0000005b SystemOut O SSLv3 protocol was requested but was not enabled
[6/12/19 6:25:09:654 EDT] 0000005a SystemOut O SSLv3 protocol was requested but was not enabled
[6/12/19 6:25:09:658 EDT] 0000005b SystemOut O SSLv3 protocol was requested but was not enabled
In short: I make a remote JNDI call for a stateless bean from one Websphere server to another (different profiles in different JVMs on the same machine) but after the call is done, the InitialContext gets changed and I can no longer access my local beans.
The situation is as follows:
A Websphere server Server1 has a complex application with many beans and also needs to access a stateless bean (BeanX) on Websphere Server2 that runs another application.
I have managed to access BeanX on Server2 using either of the two methods:
Connection via code:
private static Context;
static {
Hashtable env = new Hashtable();
env.put(Context.INITIAL_CONTEXT_FACTORY, com.ibm.websphere.naming.WsnInitialContextFactory");
env.put(Context.PROVIDER_URL, "iiop://localhost:9101");
try {
ctx = NamingManager.getInitialContext(env);
mgr = (RemoteBeanXManager) PortableRemoteObject.narrow(ctx.lookup("BeanXManagerBean/remote"), RemoteBeanXManager.class);
} catch (NamingException e) {//logging
}
}
Connection via Webshere configuration
Configured in Server1: Environment -> Naming -> Name Space binding -> New ... Indirect
And introduced the required values, corresponding to the exposed bean from Server2.
Provider URL: corbaloc:iiop:localhost:9101
Initial context factory name: com.ibm.websphere.naming.WsnInitialContextFactory
And then proceeded to obtain the bean in the code as if it was in my local context.
I removed security for RMI/IIOP for Outbound in Server1 and for Inbound in Server2, so that the servers can communicate.
The bean is exposed by Server2, the remote interface RemoteBeanXManager exists in a common package for both applications and the call goes through correctly.
The problem appears after the call, when subsequent EJB injections in Server1 for its local beans return a com.ibm.ejs.container.util.ExceptionUtil.NoSuchEJBException
Basically, at the next #EJB annotation encountered, Server1 will throw:
com.ibm.ejs.container.EJBNotFoundException: EJB named SomeOtherBean
not present in application My Enterprise Application. at
com.ibm.ejs.container.HomeOfHomes.resolveEJBLink(HomeOfHomes.java:751)
What I managed to understand as I investigate, is that the InitialContext on Server1 is changed by the connection to Server2, and cannot 'reset' back after the call through the remote interface. I tried re-obtaining the InitialContext with the default settings after the remote manager is obtained, but to no avail.
I tried to add the optional configuration parameters:
com.ibm.websphere.naming.jndicache.cachename providerURL
com.ibm.websphere.naming.jndicache.cacheobject none
Still the same exception.
If you have any ideas on how to isolate/cache the context on Server1 so that it can resume normal operation after a RMI-IIOP call, please let me know
Updated to add:
When I comment out the code that performs the remote call, the application starts correctly, and all #EJB injections work.
In Server2, at the call of the method from Server1 - no exception, just these messages:
[11/27/18 9:28:13:525 EET] 00000139 BeanXManagerB I com.my.app.manager.impl.BeanXManagerBean performTaskForServer1 End
[11/27/18 9:28:13:883 EET] 00000139 RegisteredRes E WTRN0064E: An illegal attempt to commit a one phase capable resource in a subordinate transaction branch has occurred.
[11/27/18 9:28:13:920 EET] 00000139 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile2\logs\ffdc\Server2_db3b4f88_18.11.27_09.28.13.8878526370681854823401.txt com.ibm.tx.jta.TransactionImpl.prepareResources 467
[11/27/18 9:28:13:932 EET] 00000139 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile2\logs\ffdc\Server2_db3b4f88_18.11.27_09.28.13.9214065828085434219239.txt com.ibm.tx.jta.impl.TransactionImpl.prepareResources 1505
The complete stacktrace of Server1's error is:
[I thought it would be too long to post all the incident reports. ]
[11/27/18 9:28:13:946 EET] 0000011a DMAdapter I com.ibm.ws.ffdc.impl.DMAdapter getAnalysisEngine FFDC1009I: Analysis Engine using data base: D:\programs\IBM\WebSphere\AppServer\properties\logbr\ffdc\adv\ffdcdb.xml
[11/27/18 9:28:13:967 EET] 0000011a FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile1\logs\ffdc\Server1_ede9c292_18.11.27_09.28.13.9411059231941807719848.txt com.ibm.tx.jta.impl.RegisteredResources.prepareResource 1216
[11/27/18 9:28:13:991 EET] 0000011a FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile1\logs\ffdc\Server1_ede9c292_18.11.27_09.28.13.9683456920548561326798.txt com.ibm.tx.jta.TransactionImpl.prepareResources 467
[11/27/18 9:28:14:014 EET] 0000011a FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile1\logs\ffdc\Server1_ede9c292_18.11.27_09.28.13.9913497264494769300154.txt com.ibm.tx.jta.impl.TransactionImpl.prepareResources 1505
[11/27/18 9:28:14:032 EET] 0000011a FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile1\logs\ffdc\Server1_ede9c292_18.11.27_09.28.14.0285054752089388843670.txt com.ibm.ejs.csi.TranStrategy.commit 294
[11/27/18 9:28:14:039 EET] 0000011a FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile1\logs\ffdc\Server1_ede9c292_18.11.27_09.28.14.036797394156202410033.txt com.ibm.ejs.container.EJSHome.createBeanO 1047
[11/27/18 9:28:14:044 EET] 0000011a FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile1\logs\ffdc\Server1_ede9c292_18.11.27_09.28.14.0403016172875256112876.txt com.ibm.ejs.csi.EJBApplicationMetaData.createStartupBeans 6921
[11/27/18 9:28:14:044 EET] 0000011a EJBApplicatio E CNTR0190E: The StarterBean startup singleton session bean in the app1.jar module failed initialization with exception:
javax.ejb.NoSuchEJBException: An error occurred during initialization of singleton session bean MY Enterprise Application#app1.jar#StarterBean, resulting in the discarding of the singleton instance.; nested exception is: javax.ejb.EJBTransactionRolledbackException: nested exception is: javax.transaction.RollbackException
at com.ibm.ejs.container.util.ExceptionUtil.NoSuchEJBException(ExceptionUtil.java:540)
at com.ibm.ejs.container.EJSHome.createSingletonBeanO(EJSHome.java:3752)
at com.ibm.ejs.csi.EJBApplicationMetaData.createStartupBeans(EJBApplicationMetaData.java:959)
at com.ibm.ejs.csi.EJBApplicationMetaData.startedModule(EJBApplicationMetaData.java:680)
at com.ibm.ws.runtime.component.EJBContainerImpl.stateChanged(EJBContainerImpl.java:4525)
at com.ibm.ws.runtime.component.ApplicationMgrImpl$ComparableDeployedObjectListener.stateChanged(ApplicationMgrImpl.java:2652)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stateChanged(ApplicationMgrImpl.java:1178)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectEvent(DeployedApplicationImpl.java:1558)
at com.ibm.ws.runtime.component.DeployedModuleImpl.setState(DeployedModuleImpl.java:252)
at com.ibm.ws.runtime.component.DeployedModuleImpl.setState(DeployedModuleImpl.java:248)
at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:707)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:1150)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:800)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplicationDynamically(ApplicationMgrImpl.java:1450)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2311)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:436)
at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:379)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:127)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$1.run(CompositionUnitMgrImpl.java:654)
at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5574)
at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5700)
at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.startCompositionUnit(CompositionUnitMgrImpl.java:668)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.startCompositionUnit(CompositionUnitMgrImpl.java:612)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:1340)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:83)
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:287)
at javax.management.modelmbean.RequiredModelMBean$4.run(RequiredModelMBean.java:1263)
at java.security.AccessController.doPrivileged(AccessController.java:666)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1257)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:1096)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:831)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:813)
at com.ibm.ws.management.AdminServiceImpl$1.run(AdminServiceImpl.java:1350)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.management.AdminServiceImpl.invoke(AdminServiceImpl.java:1243)
at com.ibm.ws.management.application.sync.StartDeploymentTask.startDeployment(StartDeploymentTask.java:249)
at com.ibm.ws.management.application.sync.StartDeploymentTask.fullAppUpdate(StartDeploymentTask.java:121)
at com.ibm.ws.management.application.sync.StartDeploymentTask.performTask(StartDeploymentTask.java:109)
at com.ibm.ws.management.application.sync.AppBinaryProcessor$ExpandApp.expand(AppBinaryProcessor.java:1770)
at com.ibm.ws.management.application.sync.AppBinaryProcessor.postProcessSynchronousExt(AppBinaryProcessor.java:811)
at com.ibm.ws.management.bla.sync.BLABinaryProcessor.postProcess(BLABinaryProcessor.java:599)
at com.ibm.ws.management.bla.sync.BLABinaryProcessor.onChangeCompletion(BLABinaryProcessor.java:476)
at com.ibm.ws.management.bla.sync.BinaryProcessorWrapper.onChangeCompletion(BinaryProcessorWrapper.java:109)
at com.ibm.ws.management.repository.FileRepository.postNotify(FileRepository.java:1938)
at com.ibm.ws.management.repository.FileRepository.update(FileRepository.java:1442)
at com.ibm.ws.management.repository.client.LocalConfigRepositoryClient.update(LocalConfigRepositoryClient.java:189)
at com.ibm.ws.sm.workspace.impl.WorkSpaceMasterRepositoryAdapter.update(WorkSpaceMasterRepositoryAdapter.java:667)
at com.ibm.ws.sm.workspace.impl.RepositoryContextImpl.update(RepositoryContextImpl.java:1998)
at com.ibm.ws.sm.workspace.impl.RepositoryContextImpl.synch(RepositoryContextImpl.java:1946)
at com.ibm.ws.sm.workspace.impl.WorkSpaceImpl.synch(WorkSpaceImpl.java:549)
at com.ibm.ws.console.core.action.SyncWorkSpaceAction$1.run(SyncWorkSpaceAction.java:284)
at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5574)
at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5700)
at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
at com.ibm.ws.console.core.action.SyncWorkSpaceAction.execute(SyncWorkSpaceAction.java:288)
at org.apache.struts.action.RequestProcessor.processActionPerform(Unknown Source)
at org.apache.struts.action.RequestProcessor.process(Unknown Source)
at org.apache.struts.action.ActionServlet.process(Unknown Source)
at org.apache.struts.action.ActionServlet.doGet(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1235)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:779)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:478)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:143)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:96)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.setUpCommandAssistance(WSCUrlFilter.java:971)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.continueStoringTaskState(WSCUrlFilter.java:518)
at com.ibm.ws.console.core.servlet.WSCUrlFilter.doFilter(WSCUrlFilter.java:339)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:197)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:90)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:969)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1109)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:4217)
at com.ibm.ws.webcontainer.webapp.WebAppImpl.handleRequest(WebAppImpl.java:2208)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:304)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:1030)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:382)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:465)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:532)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:318)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:88)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1833)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1909)
Caused by: javax.ejb.EJBTransactionRolledbackException: nested exception is: javax.transaction.RollbackException
at com.ibm.ejs.container.BusinessExceptionMappingStrategy.mapCSIException(BusinessExceptionMappingStrategy.java:152)
at com.ibm.ejs.container.BusinessExceptionMappingStrategy.mapCSITransactionRolledBackException(BusinessExceptionMappingStrategy.java:613)
at com.ibm.ejs.container.EJSDeployedSupport.mapCSITransactionRolledBackException(EJSDeployedSupport.java:609)
at com.ibm.ejs.container.EJSContainer.postInvokeRolledbackException(EJSContainer.java:4392)
at com.ibm.ejs.container.EJSContainer.postInvokeForLifecycleInterceptors(EJSContainer.java:4584)
at com.ibm.ejs.container.SingletonBeanO.callTransactionalLifecycleInterceptors(SingletonBeanO.java:248)
at com.ibm.ejs.container.SingletonBeanO.initialize(SingletonBeanO.java:330)
at com.ibm.ejs.container.BeanOFactory.create(BeanOFactory.java:105)
at com.ibm.ejs.container.EJSHome.createSingletonBeanO(EJSHome.java:3738)
... 101 more
Caused by: javax.transaction.RollbackException
at com.ibm.tx.jta.impl.TransactionImpl.stage3CommitProcessing(TransactionImpl.java:1279)
at com.ibm.tx.jta.impl.TransactionImpl.processCommit(TransactionImpl.java:1053)
at com.ibm.tx.jta.impl.TransactionImpl.commit(TransactionImpl.java:974)
at com.ibm.ws.tx.jta.TranManagerImpl.commit(TranManagerImpl.java:439)
at com.ibm.tx.jta.impl.TranManagerSet.commit(TranManagerSet.java:191)
at com.ibm.ejs.csi.TranStrategy.commit(TranStrategy.java:866)
at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:188)
at com.ibm.ejs.csi.RequiresNew.postInvoke(RequiresNew.java:118)
at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:482)
at com.ibm.ejs.container.EJSContainer.postInvokeForLifecycleInterceptors(EJSContainer.java:4576)
... 105 more
[11/27/18 9:28:14:051 EET] 0000011a FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\programs\IBM\WebSphere\AppServer\profiles\profile1\logs\ffdc\Server1_ede9c292_18.11.27_09.28.14.0481845564063299454983.txt com.ibm.ws.runtime.component.EJBContainerImpl.stateChanged 6637
HomeOfHomes.resolveEJBLink is normally used for #EJB annotations that are resolved locally; an InitialContext is not used, nor is a JNDI lookup performed. The code processing the #EJB skips JNDI entirely and locates the EJB directly in the EJB Container. The fact the failure occurs after a remote lookup could just be a coincidence.
I recommend looking at the beanName parameter of the #EJB and verifying that the beanName does in fact match a bean name in the same application. If the beanName parameter is of the format myModule.jar#SomeOtherBean or myModule/SomeOtherBean, then make sure the module name is correct as well. The names myModule and SomeOtherBean must match the names on one of the CNTR0167I messages in SystemOut.log, in the same application.
For WebSphere, resolving an EJB based on the beanName parameter only works within the same application. If the #EJB is referencing an EJB in a different application, even if that application is on the same server process, then you must provide a binding that identifies the JNDI name of the referenced EJB. The binding would be specified in one of the files, ibm-ejb-jar-bnd.xml/xmi or ibm-web-bnd.xml/xmi. A binding may also be provided during application deployment (and the deployment would add it to the appropriate *bnd.xml file).
If the beanName parameter does appear to be correct, and a JNDI lookup is occurring for the #EJB annotation, then you'll need to provide the full exception stacks for both the client and server side of the lookup, to better understand how the #EJB processes was re-directed to the wrong server. If this is occurring, then the EJBNotFoundException should appear in the Server2 logs, and then flow back to Server1, where it would be reported again; so exception stacks from both server1 and server2 would be needed to identify how the lookup was incorrectly routed.
I'm getting the below jdbc driver not found error, though I've included t4sqlmx-3.3.jar in <liberty_server>/lib folder. What additional changes are required?
<library id="sqlmx-lib">
<fileset dir="lib" includes="t4sqlmx-3.3.jar"/>
</library>
Error :
Caused by: java.sql.SQLNonTransientException: DSRA4000E: A valid JDBC driver implementation class was not found for the jdbcDriver dataSource[default-0]/jdbcDriver[default-0] using the library sqlmx-lib. []
at com.ibm.ws.jdbc.internal.JDBCDriverService.classNotFound(JDBCDriverService.java:207)
at com.ibm.ws.jdbc.internal.JDBCDriverService.createAnyDataSource(JDBCDriverService.java:368)
at com.ibm.ws.jdbc.DataSourceService.init(DataSourceService.java:592)
at com.ibm.ws.jca.cm.AbstractConnectionFactoryService$2.run(AbstractConnectionFactoryService.java:460)
at com.ibm.ws.jca.cm.AbstractConnectionFactoryService$2.run(AbstractConnectionFactoryService.java:457)
at java.security.AccessController.doPrivileged(Native Method)
at com.ibm.ws.jca.cm.AbstractConnectionFactoryService.initPrivileged(AbstractConnectionFactoryService.java:457)
at com.ibm.ws.jca.cm.AbstractConnectionFactoryService.createResource(AbstractConnectionFactoryService.java:144)
at com.ibm.ws.resource.internal.ResourceFactoryTrackerData$1.getService(ResourceFactoryTrackerData.java:120)
... 107 more
After datasource fix, some sqlmx specific error:
[12/05/18 14:39:26:473 EDT] 000003f8 SystemErr R Caused by: java.sql.SQLException: Pool requests blocked for dataSource[default-0]/connectionManager, connection pool is being shut down.
[12/05/18 14:39:26:473 EDT] 000003f8 SystemErr R at com.ibm.ws.rsadapter.AdapterUtil.toSQLException(AdapterUtil.java:822)
[12/05/18 14:39:26:473 EDT] 000003f8 SystemErr R at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:152)
[12/05/18 14:39:26:474 EDT] 000003f8 SystemErr R at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:116)
[12/05/18 14:39:26:474 EDT] 000003f8 SystemErr R at org.hibernate.connection.DatasourceConnectionProvider.getConnection(DatasourceConnectionProvider.java:92)
[12/05/18 14:39:26:474 EDT] 000003f8 SystemErr R at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)
[12/05/18 14:39:26:474 EDT] 000003f8 SystemErr R ... 95 more
[12/05/18 14:39:26:474 EDT] 000003f8 SystemErr R Caused by: javax.resource.spi.ResourceAllocationException: Pool requests blocked for dataSource[default-0]/connectionManager, connection pool is being shut down.
[12/05/18 14:39:26:475 EDT] 000003f8 SystemErr R at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:1275)
[12/05/18 14:39:26:475 EDT] 000003f8 SystemErr R at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:573)
[12/05/18 14:39:26:475 EDT] 000003f8 SystemErr R at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:309)
[12/05/18 14:39:26:475 EDT] 000003f8 SystemErr R at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:143)
[12/05/18 14:39:26:475 EDT] 000003f8 SystemErr R ... 98 more
Based on the stack trace, it appears that you have configured a <dataSource> element in your server.xml, but not included it in your question. Using the <library> configuration you did provide, your configuration might look like this:
<dataSource jndiName="jdbc/myDB" type="javax.sql.XADataSource">
<jdbcDriver libraryRef="sqlmx-lib" javax.sql.XADataSource="com.tandem.t4jdbc.SQLMXDataSource"/>
<properties databaseName="SAMPLEDB" hostName="localhost" port="12345"/>
</dataSource>
For more information on how to configure a DataSource in Liberty, see:
Configuring relational database connectivity in Liberty
NOTE: Based on googling for the SQL/MX JDBC driver documentation, I found that their XADataSource implementation class name is com.tandem.t4jdbc.SQLMXDataSource. If that class name is not correct for your JDBC driver jar, then use whatever class your jar has. Also, if your driver does not provide any implementation of javax.sql.XADataSource, then change the configuration to be javax.sql.ConnectionPoolDataSource instead.
We are using Spring in our Web application.
I want to concurrently listen the messages from MQ using using Spring's DefaultMessageListenerContainer(DMLC).
In websphere I didnt change any connection configurations.
Issue:
When DMLC listner tried to receive few(6 messages) messages concurrently from Queue, Websphere Application Server logged FFDC incident logs.
Can anyone help me If I missing the connection configurations?
Version:
Server: Websphere Application Server 8.5
MessageListener:
<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destinationName" ref="queue"/>
<property name="messageListener" ref="myMessageListener"/>
<property name ="concurrentConsumers" value ="5"/>
<property name ="maxConcurrentConsumers" value ="20"/>
<property name="idleTaskExecutionLimit" value="10" />
<property name="idleConsumerLimit" value="5" />
<property name="recoveryInterval" value="60000" />
</bean>
QueueConnectionfactory - Connection pools:
Logs in Server logs:
[8/1/17 12:47:54:071 ] FreePool E J2CA0045E: Connection not available while invoking method createOrWaitForConnection for resource JMS$ConnectionFactory$JMSManagedConnection#0.
[8/1/17 12:43:44:131 ] FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on WebSphere/path/path/logFileName.txt Max connections reached 869
[8/1/17 12:43:44:131 ] FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on WebSphere/path/path/logFileName.txt com.ibm.ejs.jms.JMSConnectionHandle.createSession 1051
[8/1/17 12:44:44:132 ] SystemOut O 12:44:44.132 [jmsContainer-11] WARN o.s.j.l.DefaultMessageListenerContainer - Setup of JMS message listener invoker failed for destination 'queue:' - trying to recover. Cause: Failed to create session; nested exception is com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException: CWTE_NORMAL_J2CA1009
[8/1/17 12:44:45:153 ] FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on WebSphere/path/path/logFileName.txt com.ibm.ejs.jms.JMSMessageConsumerHandle.receive 264
[8/1/17 12:44:55:304 ] SystemOut O 12:44:55.304 [jmsContainer-11] INFO o.s.j.l.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
[8/1/17 12:47:55:310 ] FreePool E J2CA0045E: Connection not available while invoking method createOrWaitForConnection for resource JMS$ConnectionFactory$JMSManagedConnection#0
FFDC log incident:
[8/1/17 13:41:03:046] FFDC Exception:com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException SourceId:Max connections reached ProbeId:869 Reporter:com.ibm.ejs.j2c.PoolManager#aad4897
com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException: CWTE_NORMAL_J2CA1009
at com.ibm.ejs.j2c.FreePool.createOrWaitForConnection(FreePool.java:1782)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:3816)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:3092)
at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:1548)
at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:1031)
at com.ibm.ejs.jms.JMSConnectionHandle.createSession(JMSConnectionHandle.java:1434)
at org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:196)
at org.springframework.jms.listener.DefaultMessageListenerContainer.access$12(DefaultMessageListenerContainer.java:1)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.initResourcesIfNecessary(DefaultMessageListenerContainer.java:1091)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1070)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1063)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:960)
at java.lang.Thread.run(Thread.java:863)
==> Performing default dump from com.ibm.ejs.j2c.DiagnosticModuleForJ2C :Tue Aug 01 13:41:03 2017
Maximum number of connections has been reached, and the connection request has been waiting longer than:ConnectionWaitTime. Two possible solutions : increase the max number of connections, or increase the:ConnectionWaitTime.:
Maximum Connections = :10
Current number of connections = :10
Connection Wait Timout = :180
If the current number of connections is not greater than or equal to max connections, :there has been a WebSphere internal error.:
This = BEGIN:com.ibm.ejs.j2c.PoolManager#aad4897
com.ibm.ejs.j2c.PoolManager::tc BEGIN:com.ibm.ejs.ras.TraceComponent#330b9221
com.ibm.ejs.ras.TraceElement::ivLevel:10
com.ibm.ejs.ras.TraceElement::ivName:com.ibm.ejs.j2c.PoolManager
com.ibm.ejs.ras.TraceElement::ivDebugEnabled:false
com.ibm.ejs.ras.TraceElement::ivEventEnabled:false
com.ibm.ejs.ras.TraceElement::ivEntryEnabled:false
com.ibm.ejs.ras.TraceElement::ivDetailEnabled:false
com.ibm.ejs.ras.TraceElement::ivConfigEnabled:false
com.ibm.ejs.ras.TraceElement::ivInfoEnabled:true
com.ibm.ejs.ras.TraceElement::ivServiceEnabled:true
com.ibm.ejs.ras.TraceElement::ivWarningEnabled:true
com.ibm.ejs.ras.TraceElement::ivErrorEnabled:true
com.ibm.ejs.ras.TraceElement::ivFatalEnabled:true
com.ibm.ejs.ras.TraceComponent::defaultMessageFile:com.ibm.ejs.resources.seriousMessages
com.ibm.ejs.ras.TraceComponent::EXTENSION_NAME_DPID:DiagnosticProvider
com.ibm.ejs.ras.TraceComponent::ivDumpEnabled:false
com.ibm.ejs.ras.TraceComponent::ivResourceBundleName:com.ibm.ws.j2c.resources.J2CAMessages
com.ibm.ejs.ras.TraceComponent::ivLogger:com.ibm.ws.logging.WsLogger#8273a12c
com.ibm.ejs.ras.TraceComponent::ivDiagnosticProviderID:null
com.ibm.ejs.ras.TraceComponent::anyTracingEnabled:null
END:com.ibm.ejs.ras.TraceComponent#330b9221
[8/1/17 13:42:03:108 ] FFDC Exception:com.ibm.msg.client.jms.DetailedIllegalStateException SourceId:com.ibm.ejs.jms.JMSMessageConsumerHandle.receive ProbeId:264 Reporter:com.ibm.ejs.jms.JMSMessageConsumerHandle#ab2dfd8d
com.ibm.msg.client.jms.DetailedIllegalStateException: JMSCC0032: This message consumer is closed.
An application called a method that must not be used after the message consumer is closed.
Ensure that the message consumer is not closed before calling the method.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:86)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:58)
at java.lang.reflect.Constructor.newInstance(Constructor.java:542)
at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:314)
at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:228)
at com.ibm.msg.client.jms.internal.JmsErrorUtils.createException(JmsErrorUtils.java:104)
at com.ibm.msg.client.jms.internal.State.checkNotClosed(State.java:145)
at com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.receive(JmsMessageConsumerImpl.java:413)
at com.ibm.mq.jms.MQMessageConsumer.receive(MQMessageConsumer.java:212)
at com.ibm.ejs.jms.JMSMessageConsumerHandle.receive(JMSMessageConsumerHandle.java:553)
at com.ibm.ejs.jms.JMSMessageConsumerHandle.receive(JMSMessageConsumerHandle.java:501)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:431)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:311)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:264)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1071)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1063)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:960)
at java.lang.Thread.run(Thread.java:863)
==> Performing default dump from com.ibm.ejs.jms.JMSDiagnosticModule :Tue Aug 01 13:42:03 2017
+Data for directive [defaultdefault] obtained.:
==> Dump complete for com.ibm.ejs.jms.JMSDiagnosticModule :Tue Aug 01 13:42:03 CEST 201
[8/1/17 13:50:13:445] FFDC Exception:javax.jms.IllegalStateException SourceId:com.ibm.ejs.jms.JMSConnectionHandle.createSession ProbeId:1044 Reporter:com.ibm.ejs.jms.JMSConnectionHandle#e8e11802
javax.jms.IllegalStateException: Connection closed
at com.ibm.ejs.jms.JMSConnectionHandle.checkOpen(JMSConnectionHandle.java:837)
at com.ibm.ejs.jms.JMSConnectionHandle.createSession(JMSConnectionHandle.java:1430)
at org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:196)
at org.springframework.jms.listener.DefaultMessageListenerContainer.access$12(DefaultMessageListenerContainer.java:1)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.initResourcesIfNecessary(DefaultMessageListenerContainer.java:1091)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1070)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1063)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:960)
at java.lang.Thread.run(Thread.java:863)
8/2/17 15:30:31:874 CEST] FFDC Exception:javax.jms.IllegalStateException SourceId:com.ibm.ejs.jms.JMSConnectionHandle.stop ProbeId:477 Reporter:com.ibm.ejs.jms.JMSConnectionHandle#5c5db824
javax.jms.IllegalStateException: Method stop not permitted
at com.ibm.ejs.jms.JMSCMUtils.methodNotPermittedException(JMSCMUtils.java:244)
at com.ibm.ejs.jms.JMSConnectionHandle.checkRestrictedMethod(JMSConnectionHandle.java:854)
at com.ibm.ejs.jms.JMSConnectionHandle.stop(JMSConnectionHandle.java:768)
at org.springframework.jms.listener.AbstractJmsListeningContainer.stopSharedConnection(AbstractJmsListeningContainer.java:460)
at org.springframework.jms.listener.DefaultMessageListenerContainer.stopSharedConnection(DefaultMessageListenerContainer.java:791)
at org.springframework.jms.listener.AbstractJmsListeningContainer.doStop(AbstractJmsListeningContainer.java:326)
at org.springframework.jms.listener.AbstractJmsListeningContainer.stop(AbstractJmsListeningContainer.java:302)
at org.springframework.jms.listener.DefaultMessageListenerContainer.stop(DefaultMessageListenerContainer.java:582)
at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:223)
at org.springframework.context.support.DefaultLifecycleProcessor.access$2(DefaultLifecycleProcessor.java:207)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.stop(DefaultLifecycleProcessor.java:356)
at org.springframework.context.support.DefaultLifecycleProcessor.stopBeans(DefaultLifecycleProcessor.java:196)
at org.springframework.context.support.DefaultLifecycleProcessor.onClose(DefaultLifecycleProcessor.java:113)
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1045)
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:1000)
at com.bnppf.ssc.imb.sf.fwk.mia.context.CloseableGenericWebApplicationContext.close(CloseableGenericWebApplicationContext.java:34)
at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:548)
at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:142)
at com.ibm.ws.webcontainer.webapp.WebApp.notifyServletContextDestroyed(WebApp.java:1765)
at com.ibm.ws.webcontainer.webapp.WebApp.destroy(WebApp.java:3131)
at com.ibm.ws.webcontainer.webapp.WebAppImpl.destroy(WebAppImpl.java:1551)
at com.ibm.ws.container.AbstractContainer.destroy(AbstractContainer.java:75)
at com.ibm.ws.webcontainer.webapp.WebGroup.destroy(WebGroup.java:228)
at com.ibm.ws.webcontainer.webapp.WebGroup.removeWebApplication(WebGroup.java:269)
at com.ibm.ws.webcontainer.VirtualHost.removeWebApplication(VirtualHost.java:296)
at com.ibm.ws.webcontainer.VirtualHostImpl.removeWebApplication(VirtualHostImpl.java:211)
at com.ibm.ws.webcontainer.WSWebContainer.removeWebApplication(WSWebContainer.java:820)
at com.ibm.ws.webcontainer.component.WebContainerImpl.uninstall(WebContainerImpl.java:458)
at com.ibm.ws.webcontainer.component.WebContainerImpl.stop(WebContainerImpl.java:729)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stop(ApplicationMgrImpl.java:1218)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStop(DeployedApplicationImpl.java:1375)
at com.ibm.ws.runtime.component.DeployedModuleImpl.stop(DeployedModuleImpl.java:671)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.stop(DeployedApplicationImpl.java:1149)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stopApplication(ApplicationMgrImpl.java:952)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stopApplicationImpl(ApplicationMgrImpl.java:1488)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stop(ApplicationMgrImpl.java:2249)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.stop(CompositionUnitMgrImpl.java:492)
at com.ibm.ws.runtime.component.CompositionUnitImpl.stop(CompositionUnitImpl.java:141)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$2.run(CompositionUnitMgrImpl.java:795)
at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5387)
at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5603)
at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.stopCompositionUnit(CompositionUnitMgrImpl.java:830)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.stopCompositionUnit(CompositionUnitMgrImpl.java:736)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stopApplication(ApplicationMgrImpl.java:1465)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:88)
at sun.reflect.GeneratedMethodAccessor133.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:292)
at javax.management.modelmbean.RequiredModelMBean$4.run(RequiredModelMBean.java:1261)
at java.security.AccessController.doPrivileged(AccessController.java:311)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:88)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1255)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:1093)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:832)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:814)
at com.ibm.ws.management.AdminServiceImpl$1.run(AdminServiceImpl.java:1335)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.management.AdminServiceImpl.invoke(AdminServiceImpl.java:1228)
at com.ibm.ws.management.connector.AdminServiceDelegator.invoke(AdminServiceDelegator.java:181)
at com.ibm.ws.management.connector.ipc.CallRouter.route(CallRouter.java:247)
at com.ibm.ws.management.connector.ipc.IPCConnectorInboundLink.doWork(IPCConnectorInboundLink.java:360)
at com.ibm.ws.management.connector.ipc.IPCConnectorInboundLink$IPCConnectorReadCallback.complete(IPCConnectorInboundLink.java:602)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)
Looks like your current Spring configuration is draining the whole connection pool. So either configuration is not correct or you are leaking connections somewhere.
==> Performing default dump from com.ibm.ejs.j2c.DiagnosticModuleForJ2C :Tue Aug 01 13:41:03 2017
Maximum number of connections has been reached, and the connection request has been waiting longer than:ConnectionWaitTime. Two possible solutions : increase the max number of connections, or increase the:ConnectionWaitTime.:
Maximum Connections = :10
Current number of connections = :10
Connection Wait Timout = :180
so either fix the configuration - maybe start with the simpler one and remove:
<property name ="maxConcurrentConsumers" value ="20"/>
<property name="idleTaskExecutionLimit" value="10" />
<property name="idleConsumerLimit" value="5" />
or just use normal Java EE MDB for reciving messages instead of this Spring.