Soap Request Error in Websphere Portal - websphere

The scenario is,
We created a Webservice (say WS1) using WebExperience factory V8.0.0.2 and the websphere application server is V8.0.0.10. The WS1 has an operation which calls another webservice (say WS2) for some authentication. All are soap services. When testing this scenario locally we were able to invoke the underlying webservice WS2 and was able to get the response.
I deployed it in Webphere Portal, wp_profile, but when I called the service WS1 it is showing some SOAP error. The error is happening when WS1 invokes WS2. We are using Portal 8.0 and the Application Server version is 8.0.0.10.
I changed the class loading to parent last in the application module. But it is still showing the error.
Stack trace is given
Category: bowstreet.system.server.logging.event.criterion.abnormal
Priority: ERROR
Msg: Component: SERVICE
Caught an exception processing a message style web service call.
The associated exception is: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.apache.axis.providers.java.MsgProvider.processMessage(MsgProvider.java:141)
at com.bowstreet.axis.providers.WebAppMSGProvider.processMessage(WebAppMSGProvider.java:46)
at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1214)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:774)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:456)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:125)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:77)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:926)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1023)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3703)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:304)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:962)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1662)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:195)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:452)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:511)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:305)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:83)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1659)
Caused by: java.rmi.RemoteException: Error in method BatchInvoke_al. Error in method getDetail_ws.invoke. Error in method getDetail_ws.invokeInternal. javax/xml/soap/MessageFactory.newInstance(Ljava/lang/String;)Ljavax/xml/soap/MessageFactory;
at com.bowstreet.axis.WebAppRPCInterface.aMethod(WebAppRPCInterface.java:68)
... 41 more
Can anyone help me on this issue..

The WARs you're deploying to the two servers (working and non-working) are different one is dev mode WAR built by deploying from designer (which is a dual mode WAR that supports the WEF standalone servlet mode plus portlets) and the other an exported portlet WAR. If so, then the two WARs would have different sets of JARs (less jars in the portlet WAR which defers to portal for some of those APIs).
Please cross checked the module class loader in the application module.
saaj-api.jar will be missing in dev mode war bulid.
Remove the saaj-api.jar from the lib and export the war and deploy.

Can you provide the libraries using in the Applications?
Please check whether any dependant jar files is loading twice after starting the application.
You can view the loaded class files in the application server console.

Related

Problems starting a WebSphere 9.0 app on developer's PC

We have a remote PC loaded with WebSphere 9.0 and we're trying to run an app that works in production, but we need to debug some things on the remote PC using Eclipse. We seem to have some kind of SSL problem with the app on the remote PC. We try to log in to the app, and get errors when it tries to get jquery - status 500. The jquery file it wants is definitely in the project.
Here is part of the logfile:
[1/18/23 12:02:30:787 EST] 000000fc webapp E com.ibm.ws.webcontainer.webapp.WebApp reportRecursiveError Error Page Exception:
[1/18/23 12:02:30:721 EST] 000000fc SystemErr R [/directory] - com.ait.security.servlets.ErrorS.displayError: (SSO artifact)
Request URI: GET /directory/lib/jquery-1.12.4.min.js
Remote addr: 0:0:0:0:0:0:0:1
Remote user: snagamalli
Referer: https://localhost:9443/directory/loginForm
Status code: 500
Status code: 500
Servlet name: Static File wrapper
Servlet error message: javax.servlet.ServletException: com.ibm.wsspi.webcontainer.ClosedConnectionException: OutputStream encountered error during write
Parameters:
_=1673975347000
javax.servlet.ServletException: com.ibm.wsspi.webcontainer.ClosedConnectionException: OutputStream encountered error during write
at com.ibm.ws.webcontainer.servlet.FileServletWrapper.service(FileServletWrapper.java:751)
at com.ibm.ws.webcontainer.servlet.FileServletWrapper.handleRequest(FileServletWrapper.java:543)
at com.ibm.ws.webcontainer.servlet.StaticFileServletWrapperImpl.handleRequest(StaticFileServletWrapperImpl.java:176)
at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.handleRequest(DefaultExtensionProcessor.java:780)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:143)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:96)
at com.ait.security.filters.SecurityCheckFilter.doFilter(SecurityCheckFilter.java:99)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:197)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:90)
at com.ait.security.filters.PerformanceFilter.doFilter(PerformanceFilter.java:78)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:197)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:90)
at com.ait.web.html.ImproveCachingFilter.doFilter(ImproveCachingFilter.java:87)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:197)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:90)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:979)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1119)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:4219)
at com.ibm.ws.webcontainer.webapp.WebAppImpl.handleRequest(WebAppImpl.java:2210)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:304)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:1030)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:382)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:465)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:532)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:318)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:88)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1833)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1909)
Caused by: com.ibm.wsspi.webcontainer.ClosedConnectionException: OutputStream encountered error during write
at com.ibm.ws.webcontainer.channel.WCCByteBufferOutputStream.write(WCCByteBufferOutputStream.java:192)
at com.ibm.ws.webcontainer.srt.SRTOutputStream.write(SRTOutputStream.java:106)
at com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream.writeOut(BufferedServletOutputStream.java:837)
at com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream.writeThis(BufferedServletOutputStream.java:529)
at com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream.write(BufferedServletOutputStream.java:438)
at com.ibm.ws.webcontainer.servlet.FileServletWrapper.writeByBytes(FileServletWrapper.java:1076)
at com.ibm.ws.webcontainer.servlet.FileServletWrapper._writeResponseToClient(FileServletWrapper.java:1035)
at com.ibm.ws.webcontainer.servlet.FileServletWrapper.writeResponseToClient(FileServletWrapper.java:944)
at com.ibm.ws.webcontainer.servlet.FileServletWrapper.service(FileServletWrapper.java:723)
... 35 more
Caused by: javax.net.ssl.SSLException: Fail to wrap application data
at com.ibm.jsse2.g.a(g.java:26)
at com.ibm.jsse2.bb.a(bb.java:102)
at com.ibm.jsse2.bb.a(bb.java:22)
at com.ibm.jsse2.bg.a(bg.java:308)
at com.ibm.jsse2.bg.wrap(bg.java:289)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:37)
at com.ibm.ws.ssl.channel.impl.SSLWriteServiceContext.encryptMessage(SSLWriteServiceContext.java:619)
at com.ibm.ws.ssl.channel.impl.SSLWriteServiceContext.encryptAndWriteAsync(SSLWriteServiceContext.java:400)
at com.ibm.ws.ssl.channel.impl.SSLWriteServiceContext.write(SSLWriteServiceContext.java:331)
at com.ibm.ws.ssl.channel.impl.SSLWriteServiceContext.write(SSLWriteServiceContext.java:177)
at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.asynchWrite(HttpServiceContextImpl.java:2523)
at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.sendOutgoing(HttpServiceContextImpl.java:2307)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundServiceContextImpl.sendResponseBody(HttpInboundServiceContextImpl.java:902)
at com.ibm.ws.webcontainer.channel.WCChannelLink.writeBufferAsynch(WCChannelLink.java:744)
at com.ibm.ws.webcontainer.channel.WCChannelLink.writeBufferResponse(WCChannelLink.java:718)
at com.ibm.ws.webcontainer.channel.WCChannelLink.writeBuffer(WCChannelLink.java:655)
at com.ibm.ws.webcontainer.channel.WCCByteBufferOutputStream.flushWriteBuffer(WCCByteBufferOutputStream.java:723)
at com.ibm.ws.webcontainer.channel.WCCByteBufferOutputStream.checkWriteArray(WCCByteBufferOutputStream.java:631)
at com.ibm.ws.webcontainer.channel.WCCByteBufferOutputStream.write(WCCByteBufferOutputStream.java:197)
... 43 more
Caused by: java.security.ProviderException: Could not determine buffer size
at javax.crypto.CipherSpi.a(Unknown Source)
at javax.crypto.CipherSpi.engineDoFinal(Unknown Source)
at javax.crypto.Cipher.doFinal(Unknown Source)
at com.ibm.jsse2.n$o$a.a(n$o$a.java:61)
at com.ibm.jsse2.d.e(d.java:14)
at com.ibm.jsse2.d.a(d.java:123)
at com.ibm.jsse2.e.a(e.java:162)
at com.ibm.jsse2.e.a(e.java:130)
at com.ibm.jsse2.bg.c(bg.java:126)
at com.ibm.jsse2.bg.b(bg.java:274)
at com.ibm.jsse2.bg.a(bg.java:239)
... 58 more
Caused by: java.lang.IllegalStateException: Cipher has not been initialized
at com.ibm.crypto.plus.provider.ChaCha20Poly1305Cipher.a(ChaCha20Poly1305Cipher.java:144)
at com.ibm.crypto.plus.provider.ChaCha20Poly1305Cipher.engineDoFinal(ChaCha20Poly1305Cipher.java:187)
... 69 more
Apps that don't use SSL seem to work. We checked the default certificates in the WebSphere administrative console, but they seem to be ok - not expired.
I've installed WebSphere on developer's PCs many times, so don't think I missed anything, but that's certainly possible. Googling the various error messages doesn't seem to give us anything helpful, but the error messages seem explicit enough that they should lead us to the problem:
OutputStream encountered error during write...
Fail to wrap application data...
Could not determine buffer size...
Cipher has not been initialized...
Does anyone have any ideas about what may be wrong?

Cannot deploy ear package when there is to many web modules inside

I've created ear app on payara server. There is an ejb package and some web modules. Everything was fine till I added a new web module.
I have an error message that says in my EJB jar there is no EJB class which is not true. When I have less than three web modules then everything works fine, but when I add new modules the error occurs again.
SEVERE: Exception while parsing file file:/Users/mikolaj/Documents/programowanie/NetBeansProjects/pstdio_services/pstdio_services-ear/target/gfdeploy/pstdio_services-ear/
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask#735ef671 rejected from java.util.concurrent.ThreadPoolExecutor#60f1aa16[Running, pool size = 20, active threads = 8, queued tasks = 30, completed tasks = 243]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
at org.glassfish.hk2.classmodel.reflect.Parser.parse(Parser.java:293)
at com.sun.enterprise.v3.server.ReadableArchiveScannerAdapter.handleJar(ReadableArchiveScannerAdapter.java:197)
at com.sun.enterprise.v3.server.ReadableArchiveScannerAdapter.onSelectedEntries(ReadableArchiveScannerAdapter.java:131)
at org.glassfish.hk2.classmodel.reflect.Parser.doJob(Parser.java:345)
at org.glassfish.hk2.classmodel.reflect.Parser.access$300(Parser.java:68)
at org.glassfish.hk2.classmodel.reflect.Parser$3.call(Parser.java:304)
at org.glassfish.hk2.classmodel.reflect.Parser$3.call(Parser.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
WARNING: result fault org.glassfish.hk2.classmodel.reflect.Parser$Result#1e051c19 Result for /Users/mikolaj/Documents/programowanie/NetBeansProjects/pstdio_services/pstdio_services-ear/target/gfdeploy/pstdio_services-ear/
SEVERE: Exception while deploying the app [pstdio_services-ear]
SEVERE: Exception during lifecycle processing
java.lang.IllegalArgumentException: Invalid ejb jar [pstdio_services-ejb-1.0.jar]: it contains zero ejb.
Note:
1. A valid ejb jar requires at least one session, entity (1.x/2.x style), or message-driven bean.
2. EJB3+ entity beans (#Entity) are POJOs and please package them as library jar.
3. If the jar file contains valid EJBs which are annotated with EJB component level annotations (#Stateless, #Stateful, #MessageDriven, #Singleton), please check server.log to see whether the annotations were processed properly.
at org.glassfish.ejb.deployment.util.EjbBundleValidator.accept(EjbBundleValidator.java:149)
at org.glassfish.ejb.deployment.util.EjbBundleValidator.accept(EjbBundleValidator.java:114)
at com.sun.enterprise.deployment.BundleDescriptor.visit(BundleDescriptor.java:643)
at org.glassfish.ejb.deployment.descriptor.EjbBundleDescriptorImpl.visit(EjbBundleDescriptorImpl.java:767)
at com.sun.enterprise.deployment.util.ApplicationValidator.accept(ApplicationValidator.java:121)
at com.sun.enterprise.deployment.BundleDescriptor.visit(BundleDescriptor.java:643)
at com.sun.enterprise.deployment.archivist.ApplicationArchivist.validate(ApplicationArchivist.java:723)
at com.sun.enterprise.deployment.archivist.ApplicationArchivist.openWith(ApplicationArchivist.java:268)
at com.sun.enterprise.deployment.archivist.ApplicationFactory.openWith(ApplicationFactory.java:232)
at org.glassfish.javaee.core.deployment.DolProvider.processDOL(DolProvider.java:189)
at org.glassfish.javaee.core.deployment.DolProvider.load(DolProvider.java:223)
at org.glassfish.javaee.core.deployment.DolProvider.load(DolProvider.java:91)
at com.sun.enterprise.v3.server.ApplicationLifecycle.loadDeployer(ApplicationLifecycle.java:934)
at com.sun.enterprise.v3.server.ApplicationLifecycle.setupContainerInfos(ApplicationLifecycle.java:874)
at com.sun.enterprise.v3.server.ApplicationLifecycle.prepare(ApplicationLifecycle.java:384)
at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:540)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:549)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:545)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:544)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:575)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:566)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1475)
at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1300(CommandRunnerImpl.java:111)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1857)
at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1733)
at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:564)
at com.sun.enterprise.v3.admin.AdminAdapter.onMissingResource(AdminAdapter.java:251)
at org.glassfish.grizzly.http.server.StaticHttpHandlerBase.service(StaticHttpHandlerBase.java:166)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:208)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:208)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.lang.Thread.run(Thread.java:748)
I think it's because Payara Server processes the EAR asynchronously in several threads using the Payara Executor Service. It looks like you're using Payara Server 5.184 or older which has the queue size limited to 30 by default and then it stops accepting more tasks, which results in the rejected exception. You should set the queue size to a higher value with the command set-payara-executor-service-configuration. The default value was set to 500 in version 5.191: https://github.com/payara/Payara/pull/3497

JAX-WS service cannot talk to another

We have a JAX-WS servce on one node of our WebSphere 8.5.5 box, and it needs to talk to a JAX-WS service on another node on the same box. It's been doing this in WebSpere 7 for months.
Yet suddenly, it won't. I wish I could say that ONLY the server version changed, but no, lots of code changed. But that code is now working on a WebSphere 7 box. So it seems like it's a version thing.
I can call the service that is being called by the other service with a unit test and it works fine.
The called service does not seem to report an exception. The calling service complains:
[8/29/14 14:59:49:222 EDT] 0000007d AxisEngine E org.apache.axis2.engine.AxisEngine receive The endpoint reference (EPR) for the Operation not found is {correct URL redacted) and the WSA Action = . If this EPR was previously reachable, please contact the server administrator.
org.apache.axis2.AxisFault: The endpoint reference (EPR) for the Operation not found is (correct URL redacted) and the WSA Action = . If this EPR was previously reachable, please contact the server administrator.
at org.apache.axis2.engine.DispatchPhase.checkPostConditions(DispatchPhase.java:102)
at org.apache.axis2.engine.Phase.invoke(Phase.java:329)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:360)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:195)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at com.ibm.ws.websvcs.transport.http.WASAxis2Servlet.doPost(WASAxis2Servlet.java:1583)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1230)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:779)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:478)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
at org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:960)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1064)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:909)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1662)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:459)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:526)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:312)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:283)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1862)
So here's what happened. Actually about half a dozen things went wrong at the same time which lead to people looking for solutions in the wrong places. But this particular error seemed to have come from a set of .class files that were not checked in in the .apt_generated folder after a recent change to the interface of the web service being called.
Once checked in, the deployment workspace got them, and all worked.

ejb2.0, log4j and EAR project configuration

I am posting one more question on EJB2.0. I thought my earlier question won't get any answer since EJB2.0 is pretty much outdated technology and nobody wants to work on it any more. But my first question got answered and I am trying my luck one more time.
I created one simple EJB 2.x entity bean. My entity bean uses another class (UniqueIdGenerator) to generate value of primary key for persistent primary key field. UniqueIdGenerator uses log4j and JDBC connection, so I configured appropriate build path references for my EJB project.
Added following JARS to buildpath for EJB project
log4j-1.2.16.jar
db2jcc.jar
db2jcc_license_cu.jar
Then I mapped persistent fields to database table fields. Generated code using Prepare for Deployment option in RSA (deploying on WAS).
Created EAR project and using Project References option added EJB project (HMS) to EAR project.
I did not set any other path/classpath.
Started WAS, deployed EAR on WAS and launched Universal Test Client to test entity bean. Using remote interface I tried to create my first bean, I entered the values for fields and clicked on submit on Universal test client. On server side it called ejbCreate() of entity bean but failed when loading UniqueIdGenerator class with error NoClassDefFoundError
Here is stack trace
java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
com.ibm.ejs.container.CreateFailureException: ; nested exception is:
java.lang.NoClassDefFoundError: org.apache.log4j.LogManager
at com.ibm.CORBA.iiop.UtilDelegateImpl.mapSystemException(UtilDelegateImpl.java:235)
at com.ibm.CORBA.iiop.UtilDelegateImpl.wrapException(UtilDelegateImpl.java:743)
at javax.rmi.CORBA.Util.wrapException(Util.java:296)
at ejbs._EJSRemoteCMPAddressHome_824957aa_Stub.create(_EJSRemoteCMPAddressHome_824957aa_Stub.java:258)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at com.ibm.etools.utc.model.ReflectionMethodModel.invoke(ReflectionMethodModel.java:65)
at com.ibm.etools.utc.servlet.InvokeServlet.invoke(InvokeServlet.java:113)
at com.ibm.etools.utc.servlet.InvokeServlet.doPost(InvokeServlet.java:374)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:738)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1657)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:939)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:502)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:179)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:91)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:864)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1583)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:186)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:452)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:511)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:305)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:83)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1604)
Caused by: com.ibm.ejs.container.CreateFailureException: ; nested exception is:
java.lang.NoClassDefFoundError: org.apache.log4j.LogManager
at ejbs.EJSCMPAddressHomeBean_824957aa.create(EJSCMPAddressHomeBean_824957aa.java:47)
at ejbs.EJSRemoteCMPAddressHome_824957aa.create(EJSRemoteCMPAddressHome_824957aa.java:28)
at ejbs._EJSRemoteCMPAddressHome_824957aa_Stub.create(_EJSRemoteCMPAddressHome_824957aa_Stub.java:251)
... 29 more
Caused by: java.lang.NoClassDefFoundError: org.apache.log4j.LogManager
at utils.UniqueLongIdGenerator.(UniqueLongIdGenerator.java:40)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
at utils.UniqueIdGenerator.(UniqueIdGenerator.java:28)
at utils.UniqueIdGenerator.(UniqueIdGenerator.java:24)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
at ejbs.AddressBean.ejbCreate(AddressBean.java:46)
at ejbs.ConcreteAddress_824957aa.ejbCreate(ConcreteAddress_824957aa.java:122)
at ejbs.EJSCMPAddressHomeBean_824957aa.create(EJSCMPAddressHomeBean_824957aa.java:33)
... 31 more
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.LogManager
at java.net.URLClassLoader.findClass(URLClassLoader.java:434)
at com.ibm.ws.bootstrap.ExtClassLoader.findClass(ExtClassLoader.java:191)
at java.lang.ClassLoader.loadClass(ClassLoader.java:660)
at com.ibm.ws.bootstrap.ExtClassLoader.loadClass(ExtClassLoader.java:111)
at java.lang.ClassLoader.loadClass(ClassLoader.java:626)
at com.ibm.ws.classloader.ProtectionClassLoader.loadClass(ProtectionClassLoader.java:62)
at com.ibm.ws.classloader.ProtectionClassLoader.loadClass(ProtectionClassLoader.java:58)
at com.ibm.ws.classloader.CompoundClassLoader.loadClass(CompoundClassLoader.java:509)
at java.lang.ClassLoader.loadClass(ClassLoader.java:626)
... 41 more
Have I missed anything in the configuration. Do I need to add all the jar files somewhere in EAR project. Isn't it sufficient to set them in the build path for EJB project?
I found answer for it on the following links
How to include external jar in ejb-jar
Developerworks Article
http://www.ibm.com/developerworks/rational/library/07/1211_schrag/index.html
You will need to import the third party jars into your EAR project and then setup JAR dependency in your EJB projects MANIFEST.MF file. Once you import external jars into your EAR project, they will be available in explorer for MANIFEST.MF

Class loading issue involving CXF on Websphere, parent last not helping

My problem concerns CXF and the wsdl4j_1.6.2.jar colliding with something, presumably the axis-wsdl4j.jar.
I have read: What's wrong with my Apache CXF client?
My project does not include the axis-wsdl4j.jar in the war. I have also asked our admin to switch Websphere to Parent Last. I still get this:
[12/13/12 16:01:28:378 CST] 0000001b ServletWrappe E SRVE0068E:
Uncaught exception thrown in one of the service methods of the
servlet: CXFServlet. Exceptio n thrown : java.lang.RuntimeException:
org.apache.cxf.interceptor.Fault:
javax/wsdl/xml/WSDLReader.readWSDL(Ljavax/wsdl/xml/WSDLLocator;Lorg/w3c/dom/Element;)
Ljavax/wsdl/Definition;
at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:116)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:322)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:122)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:211)
at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:213)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:154)
at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:129)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:187)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:110)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:166)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1143)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:591)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:481)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3453)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:267)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:815)
at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1466)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:119)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:458)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:387)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:267)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture$1.run(AsyncChannelFuture.java:205)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1473) Caused by: org.apache.cxf.interceptor.Fault:
javax/wsdl/xml/WSDLReader.readWSDL(Ljavax/wsdl/xml/WSDLLocator;Lorg/w3c/dom/Element;)Ljavax/wsdl/Definition;
at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:167)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:94)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:94)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)
... 26 more Caused by: java.lang.NoSuchMethodError: javax/wsdl/xml/WSDLReader.readWSDL(Ljavax/wsdl/xml/WSDLLocator;Lorg/w3c/dom/Element;)Ljavax/wsdl/Definition;
at org.apache.cxf.wsdl11.WSDLManagerImpl.loadDefinition(WSDLManagerImpl.java:242)
at org.apache.cxf.wsdl11.WSDLManagerImpl.getDefinition(WSDLManagerImpl.java:191)
at org.apache.cxf.wsdl11.WSDLServiceFactory.(WSDLServiceFactory.java:92)
at org.apache.cxf.jaxws.ServiceImpl.initializePorts(ServiceImpl.java:203)
at org.apache.cxf.jaxws.ServiceImpl.(ServiceImpl.java:147)
at org.apache.cxf.jaxws.spi.ProviderImpl.createServiceDelegate(ProviderImpl.java:97)
at javax.xml.ws.Service.(Service.java:35)
...
Project is built by Maven, deployed to Websphere via Continuum only; and all settings are changed via request. Application runs perfect on local Tomcat.
EDIT:
This was eventually fixed by:
Copy wsdl4j.jar to /usr/WebSphere/AppServer/appLibs/APPNAME directory.
Chmod 775 on previous.
In websphere admin console, environment (on left), Shared Libraries, and define a container wide shared library with name as APPNAME above.
Add the jar to the classpath of the shared library.
Servers (on left), APPNAME java, process management, Class Loader and make sure "Classes loaded with application class loader first"
Click on the class loader and assign the shared library to it.
Restart Server
i think you need to disable Jax ws engine
Try setting JVM property as below
com.ibm.websphere.webservices.DisableIBMJAXWSEngine=true
here is what i would do.
Turn on verbose class loading on WAS runtime and look at the JAR that contains WSDLReader.
Check if this is the JAR that you expected to be used in run-time (to start with) based on the verbose class loading.
HTH

Resources